Community
Participate
Working Groups
The Mylyn JIRA Conenctor needs to scale to work well with large JIRA instances (many projects). Performance is central to this, both: 1) Quick responsiveness of the Mylyn JIRA client 2) Low impact on JIRA server load The 'Refresh Attributes' action from Mylyn JIRA Task Editor, JIRA Repository Connection dialog, and JIRA Query Dialog can take a VERY LONG time to run against a large JIAR install. !!! It takes over 15 minutes for me !!! * Several people have sited his as a reason not to use Mylyn with JIRA A great fix would be to only do a deep refresh of project attributes, for a single project at a time, when it is really necessary. Being more lazy about refreshing project attributes would have also mean memory isn't used for JIRA projects that haven't been refreshed. * Most users work and tasks are confined to a much smaller subset of projects than the full list on a large JIRA install. -- 1) The quickest win would be for the 'Refresh Attributes' button on the JIRA Task Editor to only refresh attribues for the JIAR project of that JIRA issue. This would let me quickly refresh attributes for one project so Mylyn is aware of changes to JIRA Components and Versions. (( I actually hacked that up on a Mylyn 2, and it had dramatic impact. But that hack is dead and deleetd now, I never had cycles to update it for Mylyn 3. Outline: - JiraClientCache.refreshDetails() to only do shallow refresh, or take boolean arg: isRefreshAllProjects - Add JiraClientCache.refreshProjectDetails( proj ) - Update UI to use )) 2) Refresh Attributes on the Task reposiotory and query dilogs could just refresh the list of projects, WITHOUT recursing to do a deep refresh of EVERY project. Then refresh the projects when you need deeper information on them (components, versions, workflow etc). * For example when: ** A project is selected in Query for more than a certain period (so doesn't refresh too much if you cursor down in Project list) ** When a query returns an JIRA issue which Mylyn hasn't already got the Project attributes for.
*** Bug 196914 has been marked as a duplicate of this bug. ***
That sounds like a good suggestion. My only concern is that it could break the user expectation in terms of Mylyn's offline support. Creating offline issues would only work for projects for which the configuration is known. The same would apply to the query dialog where clicking a project would trigger a download of project details making the interaction in the dialog less smooth. Overall I think there is a benefit to the suggested approach of making configuration retrieval more lazy since it's an expensive operation in JIRA. I have added this is an item to next week's meeting agenda to discuss implications in more detail.
Created attachment 145543 [details] JIRA connector lazy loading metadata for projects Here's a lazy load version for JIRA. It will download only global metadata for the repository, then project details will be downloaded only if they are accessed (and cached). The patch dramatically improves user experience when defining the JIRA repository for the first time - initial download takes less than 2 minutes for https://jira.atlassian.com (which is quite big). Then clicking each project downloads additional data which for one project takes less than a minute.
That's a good start Pawel. Here are some notes from me (I only tried the functionality, did not look at the code itself yet): The lazy retrieval seems to work fine while network is available. I agree with Steffen's concern that the query dialog is now less smooth. An idea would be to have a certain timer while downloading the projects, and if it takes long (some fixed timespan) we could ask the user if he wants to lazily download the rest of the configuration, or all at once. With some hints towards offline support the user then might be able to decide if he needs all or not. Maybe even letting him select certain projects for which he wants configuration immediately and for which lazily could be possible. As you mentioned Pawel, offline support is a problem now. I observed a case where I got the list of projects, and when creating a new task and double clicking on of the listed projects, I got an error dialog popping up saying the host is not reachable. So I could not get to the task editor. I closed this wizard and opened the query wizard, where it did not show the list of projects and displayed a similar error message. Going back to the new task wizard now did not show the projects anymore. I am failing in reproducing that case again, so I cannot say what triggered it. It seemed like there was maybe a previous configuration cached and the initial opening of the new task wizard tried to use it, not sure. What I am seeing most of the time now is that after having the projects, creating a new task works, the task editor opens (of course with fields like component being empty), but that is OK I would say. The tricky thing is to make sure that all clients have to most recent repository configuration. The problem I was seeing (and I am failing to reproduce now) might be due to some inconsistency there. Therefore I think it is important that 277254: use a single JiraClient instance per repository https://bugs.eclipse.org/bugs/show_bug.cgi?id=277254 is addressed before this task can be finished. Finally, I noticed 7 new failing tests after the patch is applied. We will work on getting the eclipse.org test server up and running again so the test suite can be run against them again. You can track the progress on 287730: [releng] setup JIRA test environment on mylyn.eclipse.org https://bugs.eclipse.org/bugs/show_bug.cgi?id=287730 Just fyi, here is the list of new failing tests: * JiraClientTest ** testCreateSubTask ** testProjectSecurityLevelAccessible * FilterDefinitionConverterTest ** testConversionWithFilterDefinitionConverter * JiraCustomQueryTest ** testJiraCustomQuery ** testJiraCustomQueryIncompleteDateFilter * JiraTaskDataHandler ** testCreateTaskData ** testGetTaskDataSubTasks ** testPostTaskDataSubTask
*** Bug 289375 has been marked as a duplicate of this bug. ***
Today's idea that I think that would totally revolutionize user experience here. Let's make projects a selection that user has to do when configuring the repository. The same way as it is for Bamboo - let the user decide what projects she wants to use and download metadata only for the selected ones. I think most of the time users work only on a small amount of projects. So most of the data Mylyn downloads is not needed at all! We would also enable lazy initialization - for example if someone users a query or filter that returns unknown projects we will happily download them (I know there's another issue for this). That would mean that offline mode still works as expected. But the initialization time is down to seconds. That would also fix another problem that some customers are facing - some JIRA instances are really, really big and synchronization takes even an hour. With the scenario suggested here we would decrease their wasted time significantly.
I have discussed this with Mik and Shawn. We want to avoid extra configuration UI and make this is as transparent as possible. With bug 277254 mostly resolved we should be able to pursue your original approach of postponing configuration download to the point when a project is first selected in the query dialog or a new task is created for a project that is not yet cached. As part of this bug 189689 will need to get resolved to automatically update missing configuration details as part of query and task synchronization. One of the key challenges will be to fail with a good error message if a user is offline and attempts to create a task for a project that is not cached. I think if caching configuration data for all projects that are tracked in the task list or have been explicitly selected should cover the most common case and provide the necessary caching for working offline. Let's resolve bug 290490 first and then let's figure out how to break down this task before we dive down into the implementation.
Pawel, would you be interested in working on bug 290860?
Steffen, I'm working on it.
That's already fixed by 290860
Thanks for getting the different bits and pieces tracked on the subtasks done for 3.3 Pawel! A few further optimizations are left for 3.4 which are tracked on bug 280834.
Thanks Steffen, I'll work on 280834 after you release 3.3. I don't see point doing it now since you have a code freeze anyway.
