Bug 573421 - Local history created for derived files
Summary: Local history created for derived files
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 4.21   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 4.21 RC1   Edit
Assignee: Jörg Kubitz CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
: 411012 571622 (view as bug list)
Depends on:
Blocks: 571430
  Show dependency tree
 
Reported: 2021-05-07 02:04 EDT by Jörg Kubitz CLA
Modified: 2021-09-18 05:45 EDT (History)
4 users (show)

See Also:


Attachments
Screenshot of proposed Configuration Dialog.png (16.93 KB, image/png)
2021-05-07 08:52 EDT, Jörg Kubitz CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jörg Kubitz CLA 2021-05-07 02:04:31 EDT
This is a generalisation of JDT Bug 571622 for platform.resources.

I observed that currently both JDT and Xtext do write history files for files they derived. Both Projects already voted that this should not happen.

Can we change it in generally in platform.resources instead of microfixing every project? 
That would mean we would ignore IResource.KEEP_HISTORY if IResource.isDerived().
Note that KEEP_HISTORY is already sometimes ignored which is decided in 
 HistoryStore2.isValid(IFileStore, IFileInfo).
Comment 1 Eclipse Genie CLA 2021-05-07 08:47:19 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/180356
Comment 2 Eclipse Genie CLA 2021-05-07 08:48:31 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.ui/+/180357
Comment 3 Jörg Kubitz CLA 2021-05-07 08:52:09 EDT
Created attachment 286338 [details]
Screenshot of proposed Configuration Dialog.png

added Screenshot of proposed Configuration Dialog.
Comment 5 Lars Vogel CLA 2021-05-21 03:35:56 EDT
Jörg, please add to N&N.

Did you measure the performance improvements? If yes, please share here in the bug report.
Comment 6 Jörg Kubitz CLA 2021-05-21 04:25:18 EDT
(In reply to Lars Vogel from comment #5)
> Jörg, please add to N&N.
Is there some guide how to do so? I did not do such yet.
Comment 7 Lars Vogel CLA 2021-05-21 04:33:23 EDT
(In reply to Jörg Kubitz from comment #6)
> (In reply to Lars Vogel from comment #5)
> > Jörg, please add to N&N.
> Is there some guide how to do so? I did not do such yet.

The repo contains a guide, see https://git.eclipse.org/r/c/www.eclipse.org/eclipse/news/+/180702 for an example.


Can you also answer the second question:

Did you measure the performance improvements? If yes, please share here in the bug report.
Comment 8 Jörg Kubitz CLA 2021-05-21 04:44:32 EDT
(In reply to Lars Vogel from comment #7)
> Did you measure the performance improvements? 

I did not do any measurement yet. I know the HistoryStore2.addState() used to 
take 9sec of our full builds. That SHOULD be totaly gone now.
I will try a measurement again as soon as the IBuild of the UI is available to verify if that is solved and configurable.
However this change is not only for performance. The waste amount of history files does also either flood the disk or - if limited - will reduce the size of useful history.
Comment 9 Eclipse Genie CLA 2021-05-21 08:33:30 EDT
New Gerrit change created: https://git.eclipse.org/r/c/www.eclipse.org/eclipse/news/+/180874
Comment 11 Christian Dietrich CLA 2021-05-23 08:19:19 EDT
interestingly this now shows up as failing Xtext/Xtend tests
https://github.com/eclipse/xtext-xtend/issues/1198
Comment 12 Christian Dietrich CLA 2021-05-23 08:56:36 EDT
how to change this programmatically

org.eclipse.core.internal.resources.WorkspaceDescription.isKeepDerivedState()
org.eclipse.core.internal.resources.WorkspaceDescription.setKeepDerivedState(boolean)

seems to be asymetrically overriden in 
org.eclipse.core.internal.resources.WorkspacePreferences.setKeepDerivedState(boolean)
Comment 13 Jörg Kubitz CLA 2021-05-23 11:41:06 EDT
(In reply to Christian Dietrich from comment #12)
> how to change this programmatically

The UI in FileStatesPage.performOk() uses
ResourcesPlugin.getWorkspace().getDescription().setKeepDerivedState(boolean);

Later the actual WorkspaceDescription.keepDerivedState is updated via org.eclipse.core.internal.resources.WorkspacePreferences.synchronizeWithPreferences(String) which is registered as listener in WorkspacePreferences constructor.
Comment 14 Christian Dietrich CLA 2021-05-23 11:43:24 EDT
Somehow this does not fix out test.
Comment 15 Jörg Kubitz CLA 2021-05-23 11:46:13 EDT
(In reply to Christian Dietrich from comment #14)
> Somehow this does not fix out test.

Where can i see the commit which tries to fix it?
Comment 16 Christian Dietrich CLA 2021-05-23 11:49:48 EDT
https://github.com/eclipse/xtext-xtend/compare/cd_issue1198

i dont see the WorkspacePreferences.synchronizeWithPreferences
called
Comment 17 Christian Dietrich CLA 2021-05-23 11:51:44 EDT
org.eclipse.core.internal.resources.WorkspaceDescription.setKeepDerivedState(boolean)

calls into the non preference one. so there also no listener will be called
Comment 18 Christian Dietrich CLA 2021-05-23 11:54:22 EDT
using the internal api == downcast to workspace and use internalGetDescription
seems to help

val d = (ResourcesPlugin.getWorkspace() as Workspace).internalGetDescription()
d.setKeepDerivedState(keepLocalHistory);
Comment 19 Jörg Kubitz CLA 2021-05-23 12:28:45 EDT
> using the internal api == downcast to workspace and use

ugly. This should work:

IWorkspaceDescription description =  ResourcesPlugin.getWorkspace().getDescription();
description.setKeepDerivedState(keepDerivedState);
try {
  //As it is only a copy save it back in
  ResourcesPlugin.getWorkspace().setDescription(description);
} catch (CoreException e) {
	throw new RuntimeException(e)
}
Comment 21 Jörg Kubitz CLA 2021-05-23 14:57:10 EDT
*** Bug 571622 has been marked as a duplicate of this bug. ***
Comment 22 Lars Vogel CLA 2021-05-26 02:58:52 EDT
*** Bug 411012 has been marked as a duplicate of this bug. ***
Comment 23 Holger Voormann CLA 2021-06-06 18:47:40 EDT
Currently, disabling the option "History derived files" prevents only that when directly editing a derived file, new states are stored in the local history (stored in .metadata/.plugins/org.eclipse.core.resources/.history/**).

It does not prevent that when copying a file from non-derived to derived (e.g. when the Java project builder copies non-Java text files from the source folder to the output folder) the derived file gets the local history of the source file (stored in .metadata/.plugins/org.eclipse.core.resources/.projects/**/history.index when exiting the IDE). So, derived files still have a local history referring to file states created by the non-derived source files when they are copied from non-derived files (which I guess is the most common case) and not edit directly (which I guess is a corner case).

Maybe it would be a memory/speed improvement if disabling the option would also prevent copying of the history table when copying a non-derived file to a derived file.

Files not having the derived attribute but located in a folder with the derived attribute, should also be handled as derived (which is currently not the case and might be an issue when generating code in the output folder).
Comment 24 Jörg Kubitz CLA 2021-06-07 02:50:17 EDT
(In reply to Holger Voormann from comment #23)
> It does not prevent that when copying a file from non-derived to derived
> (e.g. when the Java project builder copies non-Java text files from the
> source folder to the output folder) the derived file gets the local history
> of the source file

I'd even say that it is a bug if copies get their history from source. Please file a separate bug about that.
Comment 25 Eclipse Genie CLA 2021-09-18 04:35:37 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.ui/+/185575