[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [m2e-dev] M2Eclipse Performance (Was: [m2e-users] Having a Bad Week - eclipse sick)
|
or if you want to be very evil import jbosstools via tycho maven integration ;)
/max
On Apr 12, 2011, at 16:10, Igor Fedorenko wrote:
> clean local repo, slow/congested internal connection, jboss-as or any
> other project with large dependency trail.
>
> --
> Regards,
> Igor
>
> On 11-04-12 10:02 AM, Snjezana Peco wrote:
>> I have solved all the described issues (except #10 that I haven't fixed
>> yet) to one branch. Will try to separate them and analyze them
>> individually. Could you point me to any project which build takes more
>> than 10 min?
>>
>> Snjeza
>>
>> Igor Fedorenko wrote:
>>> Thank you for doing this analysis, Snjezana. I'll comment on individual
>>> items inline, but generally please submit git-format-patch patches for
>>> separate problems as separate bugzillas. For each bugzilla/patch please
>>> provide scenario that demonstrates the problem and before/after
>>> performance metrics. Where practical, please provide corresponding
>>> test(s) as a patch against m2e-core-test.
>>>
>>> --
>>> Regards,
>>> Igor
>>>
>>> On 11-04-11 07:59 PM, Snjezana Peco wrote:
>>>> I have tested m2eclipse performance.
>>>> These are my first impressions:
>>>>
>>>> 1) ProjectRegistryRefreshJob changes a workspace and fires a lot of
>>>> resource change listeners.
>>>> When creating a marker, for instance, this job fires five resource
>>>> change listeners (createMarker and four the setAttribute methods).
>>>> This job should be WorkspaceJob.
>>>>
>>>
>>> Workspace project "refresh" is a two-stage process. First, m2e resolves
>>> dependencies of all affected workspace projects. Depending on network
>>> connection speed and number/size of dependencies, this can take
>>> arbitrary long time (I saw builds that took well over 30 minutes) so m2e
>>> does not acquire any workspace locks during this stage. After all
>>> dependencies are resolved, m2e acquires workspace lock and broadcasts
>>> MavenProjectChangeEvents to all registered listeners. m2e.jdt, one of
>>> the listeners, updates classpath and triggers (re)build of affected
>>> projects upon reception of MavenProjectChangeEvents.
>>>
>>> Frankly, I am not convinced we need to keep workspace lock for
>>> entire time of project refresh but I am open for discussion. To help me
>>> understand pros and cons better
>>>
>>> 1a) Is marker creation the only cause of resource change events fired by
>>> the job or there are other sources?
>>>
>>> 1b) If marker creation is the only cause, what is the real overhead of
>>> firing them as separate events vs one bulk event?
>>>
>>> Depending on answers to these two questions, delaying marker creation
>>> until the second stage of refresh may be a better solution.
>>>
>>>> 2) ProjectRegistryRefreshJob and MavenBuilder (a builder job) aren't
>>>> synchronized what often causes StaleMutableProjectRegistryException
>>>> (ProjectRegistryManager, line 319) and that is the cause of starting
>>>> this job again.
>>>
>>> I need to think some more about this, it looks like can disable
>>> ProjectRegistryRefreshJob altogether when workspace autobuild is
>>> enabled...
>>>
>>> We still need to run the job when autobuild is off, but this should only
>>> cause StaleMutableProjectRegistryException if the user manually runs the
>>> build when refresh job is running, so should not be a big deal
>>>
>>>> 3) LifecycleMappingFactory.getBundleMetadataSources is too often called.
>>>> Every time it is called it reads the extension point registry and
>>>> lifecycle mapping xml files. Caching would speed up the overall
>>>> performance.
>>>
>>> We *guessed* that proper cache implementation, which deals with dynamic
>>> bundle installation/uninstallation, is not worth the performance
>>> benefits, but we did not actually measure it. Please provide
>>> before/after numbers that demonstrate performance benefits, proper cache
>>> implementation with adequate test coverage and I will be happy to review
>>> and merge this change (and admit we guessed wrong ;-) ).
>>>
>>>> 4) The BuildPathManager.configureAttchedSourcesAndJavadoc method is very
>>>> slow. I have improved it a little (DownloadSourcesJob isn't started when
>>>> the download preferences are off), but it is possible to improve it
>>>> more.
>>>
>>> DownloadSourcesJob should not be started if download sources and javadoc
>>> are disabled in maven configuration (see #getAttachedSourcesAndJavadoc
>>> implementation and how its return value is used in scheduleDownload
>>> around line 817). If you see DownloadSourcesJob start when
>>> source/javadoc download is off, please provide a patch and corresponding
>>> unit test(s) and I will review and merge it.
>>>
>>> As for BuildPathManager.configureAttchedSourcesAndJavadoc performance in
>>> general, I briefly looked at it about a month ago. At that time Yourkit
>>> was telling me that MavenImpl.getSettings was the problem, but I did not
>>> investigate it further. Do you know what makes
>>> #configureAttchedSourcesAndJavadoc slow in your case?
>>>
>>>>
>>>> 5) NexusIndexManager.mavenProjectChanged(...) always removes and adds
>>>> back an artifact to the index repository. Performance will be much
>>>> better if the index repository is updated only when the artifact doesn't
>>>> exist or is changed. Searching the index repository is much faster than
>>>> removing and adding artifacts.
>>>
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>>
>>>> 6) MarkerLocationService.addEditorHintMarkers calls two methods that
>>>> acquire the WTP's IDOMModel
>>>> (StructuredModelManager.getModelManager().getModelForRead(...)) and
>>>> release it. WTP's methods can be slow for larger files and it would be
>>>> better to acquire/release the model once for
>>>> MarkerLocationService.addEditorHintMarkers.
>>>> MarkerLocationService.addEditorHintMarkers is used only once.
>>>>
>>>
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>>
>>>> 7) ProjectRegistryManager.applyMutableProjectRegistry calls
>>>> stateReader.writeWorkspaceState(projectRegistry) when refreshing a
>>>> project. Since the workspaceState.ser file is only used when starting a
>>>> workspace, this method can be used only when stopping the
>>>> org.eclipse.m2e.core bundle
>>>
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>>
>>>> 8) BuildPathManager.mavenProjectChanged updates Maven classpath
>>>> containers of all projects no matter they are changed or not. They
>>>> should be updated only for relevant events (event.getFlags() != 0).
>>>
>>> Please provide a patch with before/after performance numbers and I
>>> will review and merge it.
>>>
>>>> 9) ProjectRegistryManager.refresh(MutableProjectRegistry newState,
>>>> DependencyResolutionContext context, IProgressMonitor monitor)
>>>> aggressively
>>>> read Maven projects.
>>>> I have tested JBoss AS 7 (https://github.com/jbossas/jboss-as). When
>>>> calling Maven>Update Project Configuration on the jboss-as-parent
>>>> project, the MavenImpl.readProject is called for about 2-6 times for
>>>> each project in the workspace. Sometimes this method lasts short time
>>>> (Maven cache is active), but sometimes takes a longer time.
>>>> I have tried to cache MavenProjectFacade in each of the two phases so
>>>> that MavenImpl.readProject is called two times for every project (once
>>>> per each phase). Not sure if it is possible to make this method to be
>>>> run only once.
>>>
>>> Let me put it this way. I am not aware of any cases when m2e unnecessary
>>> calls MavenImpl.readProject during project refresh... "Extra"
>>> MavenImpl.readProject invocations are needed to support workspace
>>> dependency resolution properly. When workspace project project changes,
>>> everything that directly and indirectly depends on this project needs to
>>> refresh. For parent pom project, this also includes all child modules.
>>>
>>> Unit tests coverage should be pretty good for this part of the code, so
>>> if you are able to reduce number of MavenImpl.readProject invocations
>>> without breaking any tests, I review and apply your patch.
>>>
>>>> 10) The Maven Pom Editor has a huge memory leak. Just opening/closing
>>>> jboss-as-parent/pom.xml will cause the JVM heap size to increase by
>>>> 2-10MB which can cause Eclipse to crash (OOM).
>>>
>>> Please provide a patch with before/after heap size numbers and I
>>> will review and merge it.
>>>
>>>
>>>>
>>>> I have implemented 1-9 enhancements. The Maven>Update Project
>>>> Configuration action on the jboss-as-parent project is much faster (2-3
>>>> times). Editing the pom.xml file of the jboss-as-parent project is 2-3
>>>> times faster. The Maven builder is improved 2-3 times.
>>>>
>>>> Would you like me to create a patch?
>>>>
>>>> Snjeza
>>>>
>>>> Snjezana Peco wrote:
>>>>> Hello Steve,
>>>>>
>>>>> I would like to test this a bit more. Could you provide me with more
>>>>> details?
>>>>>
>>>>> Steve Cohen wrote:
>>>>>> and, sad to say, it's mostly because of the combination of m2eclipse,
>>>>>> helios, subversive and the whole fracking mess.
>>>>>>
>>>>>> Eclipse has become less and less usable. I don't know who's to blame,
>>>>>> which plugin is the cause or whether it's the whole platform or some
>>>>>> combination. Rare is the day of heavy development where Eclipse
>>>>>> doesn't crash many times.
>>>>>>
>>>>>> Every crash is different. Sometimes I have to delete the .lock file.
>>>>>> Other times I don't. Other times I can't do it without killing
>>>>>> processes.
>>>>>
>>>>> Could you check if there are files named hs_err_pidXXXX.log in your
>>>>> working directory (your ECLIPSE_HOME probably)?
>>>>> If so, could you attach any of them?
>>>>>
>>>>>> eclipse-jee-helios-SR2
>>>>>> m2eclipse, whatever the latest that's available today
>>>>>> subversive
>>>>>> hibernate plugin
>>>>>
>>>>> Could you attach your configuration : Help>About Eclipse
>>>>> SDK>Installation Details>Configuration
>>>>>
>>>>>> There are still user interface actions that provoke crashes.
>>>>>
>>>>> What actions are causing the crash? Editing pom.xml, cleaning the
>>>>> project, svn checkout/merge or something else.
>>>>>
>>>>> Thanks,
>>>>> Snjeza
>>> _______________________________________________
>>> m2e-dev mailing list
>>> m2e-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
>>
>> _______________________________________________
>> m2e-dev mailing list
>> m2e-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/m2e-dev
> _______________________________________________
> m2e-dev mailing list
> m2e-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/m2e-dev
/max
http://about.me/maxandersen