Bug 407161 - [incrementalBuild] build Xtend files only if changed Java file has been changed structurally
Summary: [incrementalBuild] build Xtend files only if changed Java file has been chang...
Status: CLOSED FIXED
Alias: None
Product: Xtend
Classification: Tools
Component: Core (show other bugs)
Version: 2.4.0   Edit
Hardware: PC Mac OS X
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard: v2.5
Keywords:
Depends on:
Blocks:
 
Reported: 2013-05-03 08:02 EDT by Moritz Eysholdt CLA
Modified: 2022-03-07 14:36 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Moritz Eysholdt CLA 2013-05-03 08:02:49 EDT
Current behavior:

If an Xtend file has a reference to an element defined in a Java file, the Xtend file will be rebuild if the Java file changes. 

With this behavior, however, the Xtend file is being rebuild more often than necessary: When a Java file is changed but the change did not affect the structure of Java file (i.e. externally visible API), then the Xtend file would not need to be rebuild.
Comment 1 Moritz Eysholdt CLA 2013-05-03 08:11:18 EDT
The JDT's implementation is quite straight forward:

in 
org.eclipse.jdt.internal.core.builder.IncrementalImageBuilder.build(SimpleLookupTable)
there is this loop:

while (this.sourceFiles.size() > 0) { // added to in acceptResult
  ...
  compile(allSourceFiles);
  ...
  addAffectedSourceFiles();
}

and in 
org.eclipse.jdt.internal.core.builder.IncrementalImageBuilder.writeClassFileCheck(IFile, String, byte[])
there is

if (!(reader.isLocal() || reader.isAnonymous()) && reader.hasStructuralChanges(newBytes)) {
  addDependentsOf(new Path(fileName), true);
  this.newState.wasStructurallyChanged(fileName);
}


In summary:
During incremental build, the JDT compares the byte code of newly compiled class files with the old class files before it writes them to disk. Based on this comparison it decides weather a class file has changed structurally or not. If a class file has changed structurally, the known dependees of the class file are being recompiled as well.
Comment 2 Sebastian Zarnekow CLA 2013-05-03 08:21:57 EDT
So we could probably exploit org.eclipse.jdt.internal.core.JavaModelManager.getLastBuiltState(IProject, IProgressMonitor) to get hold on the the list of structurally changed types
Comment 3 Moritz Eysholdt CLA 2013-05-03 09:18:51 EDT
(In reply to comment #2)
> So we could probably exploit
> org.eclipse.jdt.internal.core.JavaModelManager.getLastBuiltState(IProject,
> IProgressMonitor) to get hold on the the list of structurally changed types

yes, I just checked that.

we can find a list of the names of all structurally changed types in the internal private field ((org.eclipse.jdt.internal.core.builder.State)JavaModelManager.getJavaModelManager().getLastBuiltState(this.currentProject, null)).structurallyChangedTypes after every run of the JavaBuilder.

However, there seems to be a bug: If no structural changes occur, the set contains the name of the a type from a previous build.
Comment 4 Sebastian Zarnekow CLA 2013-05-03 09:25:46 EDT
I won't consider that a bug since there is no contract described.

We probably have to check the various time stamp things, too.
Comment 5 Moritz Eysholdt CLA 2013-05-03 10:24:40 EDT
(In reply to comment #4)
> I won't consider that a bug since there is no contract described.
> 
> We probably have to check the various time stamp things, too.

true, checking the time stamps fixes the issue.
Comment 6 Moritz Eysholdt CLA 2013-05-03 10:27:21 EDT
I've pushed a prototype to
https://git.eclipse.org/r/#/c/12487/

This prototype does logging only.
We get events for classes when
- field or methods change, regardless of their visibility.
- cross references become resolvable or unresolvable

We do not get events for a class if
- the classes super type changes
- referenced types if the class change
Comment 7 Moritz Eysholdt CLA 2013-05-07 11:32:44 EDT
I've updated the changes https://git.eclipse.org/r/#/c/12487/ (PatchSet 2).

This PatchSet extends QueuedBuildData with a list of "unconfirmedChanges":
IJavaElementChanges are considered to be unconfirmed if they describe a modification (not an "add" or "delete"). After the JavaBuilder has run, its state is consulted to check if a class file has changed structurally during the build. Based on the names of structurally changed Java types the changes are then confirmed/rejected.

If an Xtext build runs and there are unconfirmed changes in the queue, needsRebuild() is invoked because the JavaBuilder typically runs after the XtextBuilder and we need to ensure the XtextBuilder runs one more time after the JavaBuilder has confirmed/rejected pending changes.

Open tasks:
Write a test?
Propagate changes over indirect references (e.g. compile clients of a subclass if the superclass changes).
Comment 8 Moritz Eysholdt CLA 2013-05-08 10:12:23 EDT
(In reply to comment #7)
> Open tasks:
> Write a test?

see bug 407529

(In reply to comment #7)
> Propagate changes over indirect references (e.g. compile clients of a
> subclass if the superclass changes).

I tested if 

---- Java --------
class Foo { String foo }
class Bar extends Foo
------------------

---- Xtend -------
class Baz { val v = new Bar().foo }
------------------

gets rebuild if I change Foo. It does.
Comment 10 Sven Efftinge CLA 2013-05-14 05:43:29 EDT
I have reverted the commit, since it didn't work reliable for all cases.
When renaming a top level type, the structural changes are not reflected in the Java Builder State, nor does the event contain the necessary information.

A promising alternative approach could be to register for POST_RECONCILE events additionally and track when an editor gets opened. The reconcile events contain the needed structural information. We need to track them and commit them on save (i.e. when a POST_CHANGE event for that file occurs).
Comment 11 Sven Efftinge CLA 2013-05-14 05:43:55 EDT
see comment #10
Comment 12 Anton Kosyakov CLA 2013-09-24 08:32:08 EDT
pushed to review: https://git.eclipse.org/r/16722
Comment 13 Anton Kosyakov CLA 2013-09-26 08:17:07 EDT
pushed to review: https://git.eclipse.org/r/#/c/16793/

This patch provides support for nested and secondary top level types.

But it seems that it is not possible to handle all cases using IJavaElementDelta. For example:
1. If a container (a type or a compilation unit) was removed than it is not possible to identify which nested types were removed.
2. If a content of a compilation unit was completely replaced than it is not possible to identify which nested types were removed.

In such cases we can only guess that a primary type was removed or changed using a compilation unit's name.
Comment 14 Anton Kosyakov CLA 2013-09-30 04:40:35 EDT
pushed to review: https://git.eclipse.org/r/16869
Comment 15 Eclipse Webmaster CLA 2017-10-31 11:25:09 EDT
Requested via bug 522520.

-M.