Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [m2e-dev] M2Eclipse Performance (Was: [m2e-users] Having a Bad Week - eclipse sick)

or if you want to be very evil import jbosstools via tycho maven integration ;)

/max

On Apr 12, 2011, at 16:10, Igor Fedorenko wrote:

> clean local repo, slow/congested internal connection, jboss-as or any
> other project with large dependency trail.
> 
> --
> Regards,
> Igor
> 
> On 11-04-12 10:02 AM, Snjezana Peco wrote:
>> I have solved all the described issues (except #10 that I haven't fixed
>> yet) to one branch. Will try to separate them and analyze them
>> individually. Could you point me to any project which build takes more
>> than 10 min?
>> 
>> Snjeza
>> 
>> Igor Fedorenko wrote:
>>> Thank you for doing this analysis, Snjezana. I'll comment on individual
>>> items inline, but generally please submit git-format-patch patches for
>>> separate problems as separate bugzillas. For each bugzilla/patch please
>>> provide scenario that demonstrates the problem and before/after
>>> performance metrics. Where practical, please provide corresponding
>>> test(s) as a patch against m2e-core-test.
>>> 
>>> --
>>> Regards,
>>> Igor
>>> 
>>> On 11-04-11 07:59 PM, Snjezana Peco wrote:
>>>> I have tested m2eclipse performance.
>>>> These are my first impressions:
>>>> 
>>>> 1) ProjectRegistryRefreshJob changes a workspace and fires a lot of
>>>> resource change listeners.
>>>> When creating a marker, for instance, this job fires five resource
>>>> change listeners (createMarker and four the setAttribute methods).
>>>> This job should be WorkspaceJob.
>>>> 
>>> 
>>> Workspace project "refresh" is a two-stage process. First, m2e resolves
>>> dependencies of all affected workspace projects. Depending on network
>>> connection speed and number/size of dependencies, this can take
>>> arbitrary long time (I saw builds that took well over 30 minutes) so m2e
>>> does not acquire any workspace locks during this stage. After all
>>> dependencies are resolved, m2e acquires workspace lock and broadcasts
>>> MavenProjectChangeEvents to all registered listeners. m2e.jdt, one of
>>> the listeners, updates classpath and triggers (re)build of affected
>>> projects upon reception of MavenProjectChangeEvents.
>>> 
>>> Frankly, I am not convinced we need to keep workspace lock for
>>> entire time of project refresh but I am open for discussion. To help me
>>> understand pros and cons better
>>> 
>>> 1a) Is marker creation the only cause of resource change events fired by
>>> the job or there are other sources?
>>> 
>>> 1b) If marker creation is the only cause, what is the real overhead of
>>> firing them as separate events vs one bulk event?
>>> 
>>> Depending on answers to these two questions, delaying marker creation
>>> until the second stage of refresh may be a better solution.
>>> 
>>>> 2) ProjectRegistryRefreshJob and MavenBuilder (a builder job) aren't
>>>> synchronized what often causes StaleMutableProjectRegistryException
>>>> (ProjectRegistryManager, line 319) and that is the cause of starting
>>>> this job again.
>>> 
>>> I need to think some more about this, it looks like can disable
>>> ProjectRegistryRefreshJob altogether when workspace autobuild is
>>> enabled...
>>> 
>>> We still need to run the job when autobuild is off, but this should only
>>> cause StaleMutableProjectRegistryException if the user manually runs the
>>> build when refresh job is running, so should not be a big deal
>>> 
>>>> 3) LifecycleMappingFactory.getBundleMetadataSources is too often called.
>>>> Every time it is called it reads the extension point registry and
>>>> lifecycle mapping xml files. Caching would speed up the overall
>>>> performance.
>>> 
>>> We *guessed* that proper cache implementation, which deals with dynamic
>>> bundle installation/uninstallation, is not worth the performance
>>> benefits, but we did not actually measure it. Please provide
>>> before/after numbers that demonstrate performance benefits, proper cache
>>> implementation with adequate test coverage and I will be happy to review
>>> and merge this change (and admit we guessed wrong ;-) ).
>>> 
>>>> 4) The BuildPathManager.configureAttchedSourcesAndJavadoc method is very
>>>> slow. I have improved it a little (DownloadSourcesJob isn't started when
>>>> the download preferences are off), but it is possible to improve it
>>>> more.
>>> 
>>> DownloadSourcesJob should not be started if download sources and javadoc
>>> are disabled in maven configuration (see #getAttachedSourcesAndJavadoc
>>> implementation and how its return value is used in scheduleDownload
>>> around line 817). If you see DownloadSourcesJob start when
>>> source/javadoc download is off, please provide a patch and corresponding
>>> unit test(s) and I will review and merge it.
>>> 
>>> As for BuildPathManager.configureAttchedSourcesAndJavadoc performance in
>>> general, I briefly looked at it about a month ago. At that time Yourkit
>>> was telling me that MavenImpl.getSettings was the problem, but I did not
>>> investigate it further. Do you know what makes
>>> #configureAttchedSourcesAndJavadoc slow in your case?
>>> 
>>>> 
>>>> 5) NexusIndexManager.mavenProjectChanged(...) always removes and adds
>>>> back an artifact to the index repository. Performance will be much
>>>> better if the index repository is updated only when the artifact doesn't
>>>> exist or is changed. Searching the index repository is much faster than
>>>> removing and adding artifacts.
>>> 
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>> 
>>>> 6) MarkerLocationService.addEditorHintMarkers calls two methods that
>>>> acquire the WTP's IDOMModel
>>>> (StructuredModelManager.getModelManager().getModelForRead(...)) and
>>>> release it. WTP's methods can be slow for larger files and it would be
>>>> better to acquire/release the model once for
>>>> MarkerLocationService.addEditorHintMarkers.
>>>> MarkerLocationService.addEditorHintMarkers is used only once.
>>>> 
>>> 
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>> 
>>>> 7) ProjectRegistryManager.applyMutableProjectRegistry calls
>>>> stateReader.writeWorkspaceState(projectRegistry) when refreshing a
>>>> project. Since the workspaceState.ser file is only used when starting a
>>>> workspace, this method can be used only when stopping the
>>>> org.eclipse.m2e.core bundle
>>> 
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>> 
>>>> 8) BuildPathManager.mavenProjectChanged updates Maven classpath
>>>> containers of all projects no matter they are changed or not. They
>>>> should be updated only for relevant events (event.getFlags() != 0).
>>> 
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>> 
>>>> 9) ProjectRegistryManager.refresh(MutableProjectRegistry newState,
>>>> DependencyResolutionContext context, IProgressMonitor monitor)
>>>> aggressively
>>>> read Maven projects.
>>>> I have tested JBoss AS 7 (https://github.com/jbossas/jboss-as). When
>>>> calling Maven>Update Project Configuration on the jboss-as-parent
>>>> project, the MavenImpl.readProject is called for about 2-6 times for
>>>> each project in the workspace. Sometimes this method lasts short time
>>>> (Maven cache is active), but sometimes takes a longer time.
>>>> I have tried to cache MavenProjectFacade in each of the two phases so
>>>> that MavenImpl.readProject is called two times for every project (once
>>>> per each phase). Not sure if it is possible to make this method to be
>>>> run only once.
>>> 
>>> Let me put it this way. I am not aware of any cases when m2e unnecessary
>>> calls MavenImpl.readProject during project refresh... "Extra"
>>> MavenImpl.readProject invocations are needed to support workspace
>>> dependency resolution properly. When workspace project project changes,
>>> everything that directly and indirectly depends on this project needs to
>>> refresh. For parent pom project, this also includes all child modules.
>>> 
>>> Unit tests coverage should be pretty good for this part of the code, so
>>> if you are able to reduce number of MavenImpl.readProject invocations
>>> without breaking any tests, I review and apply your patch.
>>> 
>>>> 10) The Maven Pom Editor has a huge memory leak. Just opening/closing
>>>> jboss-as-parent/pom.xml will cause the JVM heap size to increase by
>>>> 2-10MB which can cause Eclipse to crash (OOM).
>>> 
>>> Please provide a patch with before/after heap size numbers and I
>>> will review and merge it.
>>> 
>>> 
>>>> 
>>>> I have implemented 1-9 enhancements. The Maven>Update Project
>>>> Configuration action on the jboss-as-parent project is much faster (2-3
>>>> times). Editing the pom.xml file of the jboss-as-parent project is 2-3
>>>> times faster. The Maven builder is improved 2-3 times.
>>>> 
>>>> Would you like me to create a patch?
>>>> 
>>>> Snjeza
>>>> 
>>>> Snjezana Peco wrote:
>>>>> Hello Steve,
>>>>> 
>>>>> I would like to test this a bit more. Could you provide me with more
>>>>> details?
>>>>> 
>>>>> Steve Cohen wrote:
>>>>>> and, sad to say, it's mostly because of the combination of m2eclipse,
>>>>>> helios, subversive and the whole fracking mess.
>>>>>> 
>>>>>> Eclipse has become less and less usable. I don't know who's to blame,
>>>>>> which plugin is the cause or whether it's the whole platform or some
>>>>>> combination. Rare is the day of heavy development where Eclipse
>>>>>> doesn't crash many times.
>>>>>> 
>>>>>> Every crash is different. Sometimes I have to delete the .lock file.
>>>>>> Other times I don't. Other times I can't do it without killing
>>>>>> processes.
>>>>> 
>>>>> Could you check if there are files named hs_err_pidXXXX.log in your
>>>>> working directory (your ECLIPSE_HOME probably)?
>>>>> If so, could you attach any of them?
>>>>> 
>>>>>> eclipse-jee-helios-SR2
>>>>>> m2eclipse, whatever the latest that's available today
>>>>>> subversive
>>>>>> hibernate plugin
>>>>> 
>>>>> Could you attach your configuration : Help>About Eclipse
>>>>> SDK>Installation Details>Configuration
>>>>> 
>>>>>> There are still user interface actions that provoke crashes.
>>>>> 
>>>>> What actions are causing the crash? Editing pom.xml, cleaning the
>>>>> project, svn checkout/merge or something else.
>>>>> 
>>>>> Thanks,
>>>>> Snjeza
>>> _______________________________________________
>>> m2e-dev mailing list
>>> m2e-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
>> 
>> _______________________________________________
>> m2e-dev mailing list
>> m2e-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
> _______________________________________________
> m2e-dev mailing list
> m2e-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/m2e-dev

/max
http://about.me/maxandersen





Back to the top