Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [m2e-dev] M2Eclipse Performance (Was: [m2e-users] Having a Bad Week - eclipse sick)

ah yes, would be different usecase yes - but its still not fast ;)

I never actually see it complete....reason why I don't run m2eclipse when doing plugindevelopment ;(

But this is m2e pre-0.13 so might have been fixed with the less aggressive 0.13 core.

/max

> How long does this actually take? m2e/tycho integration is only expected
> to resolve parent projects and tycho itself, everything else should be
> delegated to PDE... which does pretty much nothing.
> 
> --
> Regards,
> Igor
> 
> On 11-04-12 10:51 AM, Max Rydahl Andersen wrote:
>> or if you want to be very evil import jbosstools via tycho maven integration ;)
>> 
>> /max
>> 
>> On Apr 12, 2011, at 16:10, Igor Fedorenko wrote:
>> 
>>> clean local repo, slow/congested internal connection, jboss-as or any
>>> other project with large dependency trail.
>>> 
>>> --
>>> Regards,
>>> Igor
>>> 
>>> On 11-04-12 10:02 AM, Snjezana Peco wrote:
>>>> I have solved all the described issues (except #10 that I haven't fixed
>>>> yet) to one branch. Will try to separate them and analyze them
>>>> individually. Could you point me to any project which build takes more
>>>> than 10 min?
>>>> 
>>>> Snjeza
>>>> 
>>>> Igor Fedorenko wrote:
>>>>> Thank you for doing this analysis, Snjezana. I'll comment on individual
>>>>> items inline, but generally please submit git-format-patch patches for
>>>>> separate problems as separate bugzillas. For each bugzilla/patch please
>>>>> provide scenario that demonstrates the problem and before/after
>>>>> performance metrics. Where practical, please provide corresponding
>>>>> test(s) as a patch against m2e-core-test.
>>>>> 
>>>>> --
>>>>> Regards,
>>>>> Igor
>>>>> 
>>>>> On 11-04-11 07:59 PM, Snjezana Peco wrote:
>>>>>> I have tested m2eclipse performance.
>>>>>> These are my first impressions:
>>>>>> 
>>>>>> 1) ProjectRegistryRefreshJob changes a workspace and fires a lot of
>>>>>> resource change listeners.
>>>>>> When creating a marker, for instance, this job fires five resource
>>>>>> change listeners (createMarker and four the setAttribute methods).
>>>>>> This job should be WorkspaceJob.
>>>>>> 
>>>>> 
>>>>> Workspace project "refresh" is a two-stage process. First, m2e resolves
>>>>> dependencies of all affected workspace projects. Depending on network
>>>>> connection speed and number/size of dependencies, this can take
>>>>> arbitrary long time (I saw builds that took well over 30 minutes) so m2e
>>>>> does not acquire any workspace locks during this stage. After all
>>>>> dependencies are resolved, m2e acquires workspace lock and broadcasts
>>>>> MavenProjectChangeEvents to all registered listeners. m2e.jdt, one of
>>>>> the listeners, updates classpath and triggers (re)build of affected
>>>>> projects upon reception of MavenProjectChangeEvents.
>>>>> 
>>>>> Frankly, I am not convinced we need to keep workspace lock for
>>>>> entire time of project refresh but I am open for discussion. To help me
>>>>> understand pros and cons better
>>>>> 
>>>>> 1a) Is marker creation the only cause of resource change events fired by
>>>>> the job or there are other sources?
>>>>> 
>>>>> 1b) If marker creation is the only cause, what is the real overhead of
>>>>> firing them as separate events vs one bulk event?
>>>>> 
>>>>> Depending on answers to these two questions, delaying marker creation
>>>>> until the second stage of refresh may be a better solution.
>>>>> 
>>>>>> 2) ProjectRegistryRefreshJob and MavenBuilder (a builder job) aren't
>>>>>> synchronized what often causes StaleMutableProjectRegistryException
>>>>>> (ProjectRegistryManager, line 319) and that is the cause of starting
>>>>>> this job again.
>>>>> 
>>>>> I need to think some more about this, it looks like can disable
>>>>> ProjectRegistryRefreshJob altogether when workspace autobuild is
>>>>> enabled...
>>>>> 
>>>>> We still need to run the job when autobuild is off, but this should only
>>>>> cause StaleMutableProjectRegistryException if the user manually runs the
>>>>> build when refresh job is running, so should not be a big deal
>>>>> 
>>>>>> 3) LifecycleMappingFactory.getBundleMetadataSources is too often called.
>>>>>> Every time it is called it reads the extension point registry and
>>>>>> lifecycle mapping xml files. Caching would speed up the overall
>>>>>> performance.
>>>>> 
>>>>> We *guessed* that proper cache implementation, which deals with dynamic
>>>>> bundle installation/uninstallation, is not worth the performance
>>>>> benefits, but we did not actually measure it. Please provide
>>>>> before/after numbers that demonstrate performance benefits, proper cache
>>>>> implementation with adequate test coverage and I will be happy to review
>>>>> and merge this change (and admit we guessed wrong ;-) ).
>>>>> 
>>>>>> 4) The BuildPathManager.configureAttchedSourcesAndJavadoc method is very
>>>>>> slow. I have improved it a little (DownloadSourcesJob isn't started when
>>>>>> the download preferences are off), but it is possible to improve it
>>>>>> more.
>>>>> 
>>>>> DownloadSourcesJob should not be started if download sources and javadoc
>>>>> are disabled in maven configuration (see #getAttachedSourcesAndJavadoc
>>>>> implementation and how its return value is used in scheduleDownload
>>>>> around line 817). If you see DownloadSourcesJob start when
>>>>> source/javadoc download is off, please provide a patch and corresponding
>>>>> unit test(s) and I will review and merge it.
>>>>> 
>>>>> As for BuildPathManager.configureAttchedSourcesAndJavadoc performance in
>>>>> general, I briefly looked at it about a month ago. At that time Yourkit
>>>>> was telling me that MavenImpl.getSettings was the problem, but I did not
>>>>> investigate it further. Do you know what makes
>>>>> #configureAttchedSourcesAndJavadoc slow in your case?
>>>>> 
>>>>>> 
>>>>>> 5) NexusIndexManager.mavenProjectChanged(...) always removes and adds
>>>>>> back an artifact to the index repository. Performance will be much
>>>>>> better if the index repository is updated only when the artifact doesn't
>>>>>> exist or is changed. Searching the index repository is much faster than
>>>>>> removing and adding artifacts.
>>>>> 
>>>>> Please provide a patch with before/after performance numbers and I
>>>>> will review and merge it.
>>>>> 
>>>>>> 6) MarkerLocationService.addEditorHintMarkers calls two methods that
>>>>>> acquire the WTP's IDOMModel
>>>>>> (StructuredModelManager.getModelManager().getModelForRead(...)) and
>>>>>> release it. WTP's methods can be slow for larger files and it would be
>>>>>> better to acquire/release the model once for
>>>>>> MarkerLocationService.addEditorHintMarkers.
>>>>>> MarkerLocationService.addEditorHintMarkers is used only once.
>>>>>> 
>>>>> 
>>>>> Please provide a patch with before/after performance numbers and I
>>>>> will review and merge it.
>>>>> 
>>>>>> 7) ProjectRegistryManager.applyMutableProjectRegistry calls
>>>>>> stateReader.writeWorkspaceState(projectRegistry) when refreshing a
>>>>>> project. Since the workspaceState.ser file is only used when starting a
>>>>>> workspace, this method can be used only when stopping the
>>>>>> org.eclipse.m2e.core bundle
>>>>> 
>>>>> Please provide a patch with before/after performance numbers and I
>>>>> will review and merge it.
>>>>> 
>>>>>> 8) BuildPathManager.mavenProjectChanged updates Maven classpath
>>>>>> containers of all projects no matter they are changed or not. They
>>>>>> should be updated only for relevant events (event.getFlags() != 0).
>>>>> 
>>>>> Please provide a patch with before/after performance numbers and I
>>>>> will review and merge it.
>>>>> 
>>>>>> 9) ProjectRegistryManager.refresh(MutableProjectRegistry newState,
>>>>>> DependencyResolutionContext context, IProgressMonitor monitor)
>>>>>> aggressively
>>>>>> read Maven projects.
>>>>>> I have tested JBoss AS 7 (https://github.com/jbossas/jboss-as). When
>>>>>> calling Maven>Update Project Configuration on the jboss-as-parent
>>>>>> project, the MavenImpl.readProject is called for about 2-6 times for
>>>>>> each project in the workspace. Sometimes this method lasts short time
>>>>>> (Maven cache is active), but sometimes takes a longer time.
>>>>>> I have tried to cache MavenProjectFacade in each of the two phases so
>>>>>> that MavenImpl.readProject is called two times for every project (once
>>>>>> per each phase). Not sure if it is possible to make this method to be
>>>>>> run only once.
>>>>> 
>>>>> Let me put it this way. I am not aware of any cases when m2e unnecessary
>>>>> calls MavenImpl.readProject during project refresh... "Extra"
>>>>> MavenImpl.readProject invocations are needed to support workspace
>>>>> dependency resolution properly. When workspace project project changes,
>>>>> everything that directly and indirectly depends on this project needs to
>>>>> refresh. For parent pom project, this also includes all child modules.
>>>>> 
>>>>> Unit tests coverage should be pretty good for this part of the code, so
>>>>> if you are able to reduce number of MavenImpl.readProject invocations
>>>>> without breaking any tests, I review and apply your patch.
>>>>> 
>>>>>> 10) The Maven Pom Editor has a huge memory leak. Just opening/closing
>>>>>> jboss-as-parent/pom.xml will cause the JVM heap size to increase by
>>>>>> 2-10MB which can cause Eclipse to crash (OOM).
>>>>> 
>>>>> Please provide a patch with before/after heap size numbers and I
>>>>> will review and merge it.
>>>>> 
>>>>> 
>>>>>> 
>>>>>> I have implemented 1-9 enhancements. The Maven>Update Project
>>>>>> Configuration action on the jboss-as-parent project is much faster (2-3
>>>>>> times). Editing the pom.xml file of the jboss-as-parent project is 2-3
>>>>>> times faster. The Maven builder is improved 2-3 times.
>>>>>> 
>>>>>> Would you like me to create a patch?
>>>>>> 
>>>>>> Snjeza
>>>>>> 
>>>>>> Snjezana Peco wrote:
>>>>>>> Hello Steve,
>>>>>>> 
>>>>>>> I would like to test this a bit more. Could you provide me with more
>>>>>>> details?
>>>>>>> 
>>>>>>> Steve Cohen wrote:
>>>>>>>> and, sad to say, it's mostly because of the combination of m2eclipse,
>>>>>>>> helios, subversive and the whole fracking mess.
>>>>>>>> 
>>>>>>>> Eclipse has become less and less usable. I don't know who's to blame,
>>>>>>>> which plugin is the cause or whether it's the whole platform or some
>>>>>>>> combination. Rare is the day of heavy development where Eclipse
>>>>>>>> doesn't crash many times.
>>>>>>>> 
>>>>>>>> Every crash is different. Sometimes I have to delete the .lock file.
>>>>>>>> Other times I don't. Other times I can't do it without killing
>>>>>>>> processes.
>>>>>>> 
>>>>>>> Could you check if there are files named hs_err_pidXXXX.log in your
>>>>>>> working directory (your ECLIPSE_HOME probably)?
>>>>>>> If so, could you attach any of them?
>>>>>>> 
>>>>>>>> eclipse-jee-helios-SR2
>>>>>>>> m2eclipse, whatever the latest that's available today
>>>>>>>> subversive
>>>>>>>> hibernate plugin
>>>>>>> 
>>>>>>> Could you attach your configuration : Help>About Eclipse
>>>>>>> SDK>Installation Details>Configuration
>>>>>>> 
>>>>>>>> There are still user interface actions that provoke crashes.
>>>>>>> 
>>>>>>> What actions are causing the crash? Editing pom.xml, cleaning the
>>>>>>> project, svn checkout/merge or something else.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Snjeza
>>>>> _______________________________________________
>>>>> m2e-dev mailing list
>>>>> m2e-dev@xxxxxxxxxxx
>>>>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
>>>> 
>>>> _______________________________________________
>>>> m2e-dev mailing list
>>>> m2e-dev@xxxxxxxxxxx
>>>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
>>> _______________________________________________
>>> m2e-dev mailing list
>>> m2e-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
>> 
>> /max
>> http://about.me/maxandersen
>> 
>> 
>> 
>> _______________________________________________
>> m2e-dev mailing list
>> m2e-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
> _______________________________________________
> m2e-dev mailing list
> m2e-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/m2e-dev

/max
http://about.me/maxandersen





Back to the top