Bug 508090 - Building the help with DITA makes the repository dirty
Summary: Building the help with DITA makes the repository dirty
Status: RESOLVED FIXED
Alias: None
Product: MAT
Classification: Tools
Component: Doc (show other bugs)
Version: 1.6   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: 1.14.0   Edit
Assignee: Krum Tsvetkov CLA
QA Contact:
URL: https://wiki.eclipse.org/MemoryAnalyz...
Whiteboard:
Keywords: Documentation
Depends on:
Blocks:
 
Reported: 2016-11-24 03:55 EST by Krum Tsvetkov CLA
Modified: 2023-01-26 07:06 EST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Krum Tsvetkov CLA 2016-11-24 03:55:48 EST
At the moment we store in the repository both the .dita files from which the documentation is generated, as well as the generated html files.
Some of the html files in the repository are built with different dita versions, which causes that every time the help is built, there are git changes shown.

I'd like to be able to rebuild the docs as often as I want without making the repository working. I'll try the following:
- document that the documentation should always be built with Dita 1.7.4 (I also tried the very latest version but it produced quite a lot of diffs and some files couldn't be parsed any longer).
- do another build with dita 1.7.4 and commit all touched files
- add <mat help>/plugins/ to .gitignore - some files get generated there

What I noticed is that some of the changes to the HTML files are IDs within table rows/columns ( tr / td ). I have no idea why and how they are generated. They do not change with every build, i.e. if I rebuild multiple times with the same dita version, I don't get modified files. Not sure if the IDs will change if someone else builds the help with the same dita version.

So what I'll try to do is a "quick fix", hoping that pinning the dita version helps. I guess a more reasonable solution would be not to store the generated files at all with the sources, but rather generate and package them with every build. I'll see if I can get the time to achieve the latter.
Comment 1 Eclipse Genie CLA 2016-11-24 04:03:08 EST
New Gerrit change created: https://git.eclipse.org/r/85661
Comment 3 Eclipse Genie CLA 2020-06-16 15:55:56 EDT
New Gerrit change created: https://git.eclipse.org/r/165020
Comment 5 Eclipse Genie CLA 2021-01-24 04:40:02 EST
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/175289
Comment 7 Andrew Johnson CLA 2022-02-25 13:24:43 EST
Going up to 1.7 still left some 'dirty' HTML files, changed when the original dita wasn't changed.
Perhaps it is time to move up to DITA 3.7 - I have some small changes which allow this, though all the HTML files will change 
as well, just this time.
Comment 8 Andrew Johnson CLA 2022-02-26 08:08:38 EST
The changes are a bit bigger to the ant script.
The id="d12e56789" and headers="d12e55645 " do still change if for example a row is added to a table.

Solution.
1. DITA 3.7 works best with the dita command not the ant task
2. The dita command then needs a properties file
3. Build dita to the 'out' directory
4. Copy html files from there to the outnew directory
5. Copy existing html files (except from doc,out) from the project to the outorig directory
6. Process those html files to convert the first part of the id to d?????
7. Delete the html files from outnew if they are the same as outorig
8. Copy the files back from the out directory to the project except for html files which do not have a corresponding file in outnew
9. Tidy up directories.

Full build
ant -f DitaBuild.xml
Build which doesn't update files which probably haven't really changed
ant -f DitaBuild.xml -Djustnew=true
Comment 9 Eclipse Genie CLA 2022-02-26 09:21:58 EST
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/191243
Comment 11 Andrew Johnson CLA 2022-02-26 10:56:08 EST
I don't see an existing project building Eclipse help at build time using DITA, but it might be possible using:
https://mvnrepository.com/artifact/org.dita-ot
https://www.dita-ot.org/3.7/reference/java-api.html
but I don't know if this includes the eclipsehelp plugin.
It looks like this would require some Java code to be compiled at build time to call dost.jar, but that should be possible.
Is it worth it though as it just saves us a little bit of time and has no impact on end users?
Comment 12 Eclipse Genie CLA 2022-02-28 02:29:29 EST
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/191251
Comment 14 Eclipse Genie CLA 2022-04-23 09:44:48 EDT
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/192881
Comment 16 Eclipse Genie CLA 2023-01-26 01:10:29 EST
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/199572
Comment 18 Eclipse Genie CLA 2023-01-26 06:38:40 EST
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/199575
Comment 20 Andrew Johnson CLA 2023-01-26 07:05:23 EST
Now by following the instructions at https://wiki.eclipse.org/MemoryAnalyzer/Contributor_Reference#Build_Help_with_DITA the DITA build should only generate new versions of files from files which have been modified.

I think this is now fixed. If we want to do a DITA build in CI then that should be another bug.