Bug 412305 - Repeated Resource Updates when changing branches
Summary: Repeated Resource Updates when changing branches
Status: CLOSED WORKSFORME
Alias: None
Product: TMF
Classification: Modeling
Component: Xtext (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard: v2.4.3
Keywords:
Depends on:
Blocks:
 
Reported: 2013-07-04 08:16 EDT by Laurent Goubet CLA
Modified: 2017-10-31 10:59 EDT (History)
4 users (show)

See Also:


Attachments
Instrumentation log (282.83 KB, text/plain)
2013-07-04 10:26 EDT, Ed Willink CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Laurent Goubet CLA 2013-07-04 08:16:45 EDT
I have absolutely no idea why, and I cannot reproduce this with any other URI I tried, but trying to open a ".ocl" file importing the "compare" URI in the CompleteOCL editor fails in what seems to be an infinite loop (unresponsive UI, but I haven't waited more than two minutes before killing the process).

Here's all I did :

- File > New > Other..., select "empty OCL project", Finish without any change
- right-click oclsrc > New > File
- name it test.ocl
- open it in the CompleteOCL editor
- paste "import c : 'http://www.eclipse.org/emf/compare'"
- save

The save completes successfully, but a few jobs are stuck (in the progress view) :
- "building workspace" is stuck on "Building <project> : Create resource descriptions"
- a job "Xtext validation" is shown... without progress but seemingly runnning ("cancel" button available though not working)

Re-opening the test.ocl file in the CompleteOCL editor will freeze the whole IDE. Restarting Eclipse will help. If I restart the IDE, open the file in the "text" editor and change the imported URI in any way, the CompleteOCL editor will be available again.

This was tested in a brand new Kepler SDK, I only installed "OCL End-user SDK" and "OCL examples and editors" from the kepler update site (along with their dependencies, automatically pulled by p2). EMF Compare is not installed and the compare URI is not registered in any way in this Eclipse.
Comment 1 Laurent Goubet CLA 2013-07-04 08:18:31 EDT
> Re-opening the test.ocl file in the CompleteOCL editor will freeze the whole
> IDE. Restarting Eclipse will help.

That was obviously "Restarting Eclipse will not help", sorry about that.
Comment 2 Ed Willink CLA 2013-07-04 10:00:32 EDT
Excellent. I have a workspace already set up to try to uncover EGIT/Xtext "Create resource descriptions" loops.

Using my development OCL code it fails to reproduce.

But using the Kepler_SR0 tag it does. The Xtext builder is just building and building and building, but not quite forever.

Worker-1 57.313 doUpdate done
Worker-6 7.743 doUpdate done
Worker-9 0.8280000000000001 doUpdate done
Worker-8 7.885 doUpdate done
Worker-5 8.433 doUpdate done
Worker-7 7.462 doUpdate done
Worker-3 8.322000000000001 doUpdate done
Worker-3 7.309 doUpdate done
Worker-9 8.088000000000001 doUpdate done
Worker-0 10.757 doUpdate done
Worker-3 11.022 doUpdate done
Worker-0 12.506 doUpdate done
Worker-4 15.344000000000001 doUpdate done
Worker-0 18.793 doUpdate done
Worker-8 19.057000000000002 doUpdate done

16 builds at about 10 seconds per build.

My current code hasn't changed in relevant areas, so my suspicion is that regenerating an editor has made things happier!
Comment 3 Ed Willink CLA 2013-07-04 10:17:13 EDT
(In reply to comment #2)
> My current code hasn't changed in relevant areas, so my suspicion is that
> regenerating an editor has made things happier!

No. It's a transition problem. No problem if editor is closed and reopened.

Change branch to my development code and the problem appears, so it seems that

Xtext has a major bug in the update of its resource descriptions. It correctly detects their invalidity, but fails to recompute in one or even two passes.

So perhaps EGIT isn't part of the problem. It's just that many EGIT activities invalidate the Xtext resource descriptions triggering the very inadequate update.

Conclusion: for now be patient because you're going to have to wait for Xtext one day. Killing it early may just require it to be done again.
Comment 4 Ed Willink CLA 2013-07-04 10:26:42 EDT
Created attachment 233084 [details]
Instrumentation log

Attached log shows the many update resource descriptions that occurred when starting a nested Eclipse aginast a changed branch, with an Xtext editor open.

Each log line shows thread, time in seconds since start of update, file being updated.

All resources are on the class path of an open project, but not the project containg an open editor.
Comment 5 Laurent Goubet CLA 2013-07-04 10:32:10 EDT
In my case, EGit is not installed (I use an Eclipse SDK bundle as my base). All I installed are the two OCL features provided to Kepler along with their automatically fetched dependencies.

OCL End User SDK 4.1.0.v20130611-1511
OCL Examples and Editors 3.3.0.v20130611-1511

The versions of xtext that were pulled along were "2.4.2.v201306120542" :
o.e.xtext
o.e.xtext.builder
o.e.xtext.common.types
o.e.xtext.common.types.edit
o.e.xtext.common.types.ui
o.e.xtext.smap
o.e.xtext.ui
o.e.xtext.ui.shared
o.e.xtext.util
o.e.xtext.xbase.lib
Comment 6 Laurent Goubet CLA 2013-07-04 10:34:05 EDT
Ha, and do note that I have but a single file in my workspace, located in a single project (as outlined by the reproductions steps in comment 0).
Comment 7 Ed Willink CLA 2013-07-04 10:53:10 EDT
(In reply to comment #6)
> Ha, and do note that I have but a single file in my workspace, located in a
> single project (as outlined by the reproductions steps in comment 0).

Surely you are aware that the purveyors of global indexes (Windows Disk Index, Acceleo Model Registry) tend to regard the index as the most important thing in the world and so require users to wait for it. No matter that users are not actually using it. Your single file workspace has many plugins and they no doubt 'need' resource description updates when you first start.

[It is only after rewriting all my Acceleo templates that I am able to work without installing Acceleo and so am able to move on from the pain inflicted by Bug 405089 to the pain inflicted by Xtext.]

This problem has been around for quite possibly over a year. It has just been masked for me because Acceleo was always on the stack trace whenever I went in with a remote debugger.
Comment 8 Sven Efftinge CLA 2013-07-24 06:30:15 EDT
With what Eclipse (which plugins) can I reproduce the behavior described in comment #0 ?
Comment 9 Sven Efftinge CLA 2013-07-24 06:56:58 EDT
Sorry, I missed the last paragraph.
I could reproduce the issue described in comment #0 and it hangs because it tries to resolve http URIs (http://www.w3.org/MarkUp/DTD/xhtml-datatypes-1.mod) using urlConnection.
Comment 10 Ed Willink CLA 2013-07-24 07:04:56 EDT
(In reply to comment #8)
> With what Eclipse (which plugins) can I reproduce the behavior described in
> comment #0 ?

For the original report, it looks like OCL Examples and Editors, EMF Compare, as the leaf install. EMF, UML, EMFv, MWE2, Xtend, Xtext will be pulled in as dependencies.

For the instrumented log, EGIT of course. QVTo was contributing extra Ecores. And org.eclipse.ocl.examples.build (from GIT) was contributing a mega-dependency tree across a 5 Xtext editor hierarchy.

(In reply to comment #9)
> Sorry, I missed the last paragraph.
> I could reproduce the issue described in comment #0 and it hangs because it
> tries to resolve http URIs
> (http://www.w3.org/MarkUp/DTD/xhtml-datatypes-1.mod) using urlConnection.

There may be two different issues.

The EMF Compare problem may be due to a bad URI registration.

The instrumented problem is a severe inefficiency in the Update Resource Descriptions algorithm.
Comment 11 Sven Efftinge CLA 2013-07-24 07:10:38 EDT
(In reply to comment #10)
> (In reply to comment #9)
> > Sorry, I missed the last paragraph.
> > I could reproduce the issue described in comment #0 and it hangs because it
> > tries to resolve http URIs
> > (http://www.w3.org/MarkUp/DTD/xhtml-datatypes-1.mod) using urlConnection.
> 
> There may be two different issues.

Yes, it seems so.

> 
> The EMF Compare problem may be due to a bad URI registration.

Yes. The if the ecore package 'http://www.eclipse.org/emf/compare' is not in the registry the XML content handler seem to load the website and tries to resolve entities in it...

> 
> The instrumented problem is a severe inefficiency in the Update Resource
> Descriptions algorithm.

Could you please list the steps needed to reproduce this?
Comment 12 Sven Efftinge CLA 2013-07-24 08:03:26 EDT
I have tried (with no success) to reproduce the second issue by

1) Using Kepler with latest Xtext and OCL
2) checking out ocl and importing everything into a workspace
3) I tried with clean builds and branch switches (through Egit as well as on command line)
Comment 13 Sven Efftinge CLA 2013-07-26 07:29:10 EDT
I still cannot reproduce it.
Comment 14 Ed Willink CLA 2013-07-26 07:45:05 EDT
Thanks for trying. Maybe there's some commonality with Bug 412634.

I'll try to produce a start from clean workspace repro, otherwise a mega ZIP and start from saved workspace.
Comment 15 Mikaël Barbero CLA 2013-08-06 10:41:34 EDT
>> The EMF Compare problem may be due to a bad URI registration.

> Yes. The if the ecore package 'http://www.eclipse.org/emf/compare' is not in the registry the XML content handler seem to load the website and tries to resolve entities in it...

This is not a bad URI registration. As Laurent said in comment #0: "EMF Compare is not installed and the compare URI is not registered in any way in this Eclipse." The leaf install is only composed of: 

OCL End User SDK 4.1.0.v20130611-1511
OCL Examples and Editors 3.3.0.v20130611-1511
Comment 16 Sven Efftinge CLA 2013-08-06 10:49:29 EDT
(In reply to comment #15)
> >> The EMF Compare problem may be due to a bad URI registration.
> 
> > Yes. The if the ecore package 'http://www.eclipse.org/emf/compare' is not in the registry the XML content handler seem to load the website and tries to resolve entities in it...
> 
> This is not a bad URI registration. As Laurent said in comment #0: "EMF
> Compare is not installed and the compare URI is not registered in any way in
> this Eclipse." The leaf install is only composed of: 
> 
> OCL End User SDK 4.1.0.v20130611-1511
> OCL Examples and Editors 3.3.0.v20130611-1511

In anyway, what happens is that a nsURI is not found and OCL's metamodel registry is trying to load it by opening the URI via http, which returns a web page (containing entities) ... 

I'd say that you need to somehow make sure you have the referenced EPackage in that registry and for OCL one could maybe consider configuring the used resource set such that this kind of experience doesn't happen. EMF has recently introduced a timeout option which seems to be useful to prevent such cases.
Comment 17 Ed Willink CLA 2013-08-06 11:32:58 EDT
(In reply to comment #16)
> In anyway, what happens is that a nsURI is not found and OCL's metamodel
> registry is trying to load it by opening the URI via http, which returns a
> web page (containing entities) ... 

See Bug 364797 where we had a similar problem if http://www.eclipse.org/ocl/1.1.0/oclstdlib.ecore was not mapped/in the package registry. The web masters reported that poor standalone applications were triggering 7800 'hits' a day.
Comment 18 Sven Efftinge CLA 2013-08-27 03:10:36 EDT
I'm a bit confused what this ticket is about.
The issue described by Laurent is clearly due to the attempt to obtain an inputstream for an http URI (the compare ns UR).
This is not Xtext related but happens within OCL's meta model registry.

Regarding the bit Ed mentioned (repeated build triggers). Could you please provide we a step by step recipe to reproduce that?
Comment 19 Ed Willink CLA 2013-08-27 04:28:15 EDT
(In reply to comment #18)
> I'm a bit confused what this ticket is about.
> The issue described by Laurent is clearly due to the attempt to obtain an
> inputstream for an http URI (the compare ns UR).
> This is not Xtext related but happens within OCL's meta model registry.

The EMF http: access used to be a major disaster; every reference provoked its own timeout. The failure is now cached so that there is now (? >= EMF 2.7) only one timeout, so Laurent's two minutes may be the repeated trigger.

> Regarding the bit Ed mentioned (repeated build triggers). Could you please
> provide we a step by step recipe to reproduce that?

As part of Bug 412634, I have been running with a massive Console Log that has eventually identified a significant problem. Gaps in the log suggest times when the Xtext Builder decides to work very hard; probably this bug. Are there any logging options that would enable me to understand the Xtext builder's triggers and so maybe produce a repro? (Guice version checks discourage me from trying to create my own modified builder.)

[When the Bug 412634 multi-save problem appears and the progress view shows the Xtext builder updating and blocking saves, it seems clear that the Xtext builder is not adequately cancelable.]
Comment 20 Sven Efftinge CLA 2013-08-27 04:44:27 EDT
(In reply to comment #19)
> Are there any logging options that would enable me to understand the Xtext
> builder's triggers and so maybe produce a repro? (Guice version checks
> discourage me from trying to create my own modified builder.)

The builder is triggered by resource changes. It looks like multiple non-batched resource changes trigger the observed behavior.
This is not to be fixed within builder implementations, but resource changes should take care of that (e.g. the multi-save problem you mentioned).

> [When the Bug 412634 multi-save problem appears and the progress view shows
> the Xtext builder updating and blocking saves, it seems clear that the Xtext
> builder is not adequately cancelable.]

It is cancelable in general, but there might be clients who don't check for the cancelation often enough.
Comment 21 Ed Willink CLA 2013-08-27 05:52:44 EDT
(In reply to comment #20)
> The builder is triggered by resource changes. It looks like multiple
> non-batched resource changes trigger the observed behavior.
...
> It is cancelable in general, but there might be clients who don't check for
> the cancelation often enough.

With luck, the problem might go away when Bug 412634 is fixed, however it would nonetheless be very helpful if the Xtext Builder had some ".options" control so that any further refinements can be instrumented more easily. My original instrumentation showed what was being updated; this provided only limited insight. Much more interesting would be a log of why an update commences and when it completes. If the clients are also identified we can see who isn't adequately cancelable.
Comment 22 Sven Efftinge CLA 2013-08-27 08:10:25 EDT
(In reply to comment #21)
> (In reply to comment #20)
> > The builder is triggered by resource changes. It looks like multiple
> > non-batched resource changes trigger the observed behavior.
> ...
> > It is cancelable in general, but there might be clients who don't check for
> > the cancelation often enough.
> 
> With luck, the problem might go away when Bug 412634 is fixed, however it
> would nonetheless be very helpful if the Xtext Builder had some ".options"
> control so that any further refinements can be instrumented more easily. My
> original instrumentation showed what was being updated; this provided only
> limited insight. Much more interesting would be a log of why an update
> commences and when it completes. If the clients are also identified we can
> see who isn't adequately cancelable.

There is some debugging and tracing in place already but I doubt that this is a good tool to track down this kind of problems. If you don't want to debug you may want to try Yourkit Probes : 
http://blog.moritz.eysholdt.de/2013/04/probes-or-how-to-analyze-application.html

They hand out Open-Source licenses for free :-)

This ticket has become a bit too vague. If there are any concrete issues (despite adding tracing and logging statements to help your use case) please open a new bugzilla.
Comment 23 Ed Willink CLA 2017-06-26 10:20:05 EDT
Perhaps Bug 486083
Comment 24 Eclipse Webmaster CLA 2017-10-31 10:48:44 EDT
Requested via bug 522520.

-M.
Comment 25 Eclipse Webmaster CLA 2017-10-31 10:59:46 EDT
Requested via bug 522520.

-M.