Bug 272621 - create Mylyn Connector discovery mechanism
Summary: create Mylyn Connector discovery mechanism
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P2 enhancement (vote)
Target Milestone: 3.2   Edit
Assignee: David Green CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy, plan
Depends on: 272626 275029 275203 275204 275205 275224 275513 275777 276012 276213 276217 276229 276230 276231 276232 276233 277160 277191 277194 277259 277275 277344 277360 277361 277889 277910 278315 278316 278317 278872 279544 280376
Blocks:
  Show dependency tree
 
Reported: 2009-04-16 23:25 EDT by Mik Kersten CLA
Modified: 2009-06-17 14:22 EDT (History)
7 users (show)

See Also:


Attachments
mylyn/context/zip (10.46 KB, application/octet-stream)
2009-04-20 22:59 EDT, David Green CLA
no flags Details
_very_ preliminary draft of the UI (104.75 KB, image/png)
2009-04-23 13:18 EDT, David Green CLA
no flags Details
screenshot sequence (340.68 KB, application/octet-stream)
2009-04-28 00:06 EDT, David Green CLA
no flags Details
screenshot of discovery ui with filters (115.25 KB, image/png)
2009-04-28 00:07 EDT, David Green CLA
no flags Details
screenshot with no matching connectors (67.80 KB, image/png)
2009-04-28 00:08 EDT, David Green CLA
no flags Details
mylyn/context/zip (3.19 KB, application/octet-stream)
2009-05-26 20:50 EDT, David Green CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mik Kersten CLA 2009-04-16 23:25:57 EDT
One of the key challenges that we have had with Mylyn is making it easy for users to access the connectors that they need to use Mylyn.  The connectors that we bundle have high adoption, but discovering and installing additional connectors has been painful, and bundling third party components is not an option.  I propose that we take a similar approach to what WTP has done with enabling server discovery (screenshot attached).  We can create a similar UI that uses a web-based listing of connectors or connector portals that have passed some kind of transparent approval process.  

The goal of this initiative will be to make it easy to get high quality connectors that help the growth and adoption of Mylyn into the hands of Mylyn users.  The proposed approach is to closely follow both the UI and interaction style (e.g., license acceptance of the install) of WTP.
Comment 1 David Green CLA 2009-04-20 22:59:00 EDT
initial code contribution created in the sandbox
Comment 2 David Green CLA 2009-04-20 22:59:04 EDT
Created attachment 132534 [details]
mylyn/context/zip
Comment 3 David Green CLA 2009-04-23 13:16:39 EDT
from Mik:
> The main question I have is about the interaction for the install.  Should
> we use the checkbox style selection, or have "Install" and "90 day Trial"
> type buttons like Firefox?  

This type of requirement might be difficult, since each connector presented in the UI represents a single installable unit.  We will be using P2 to do the installation -- and there's no way that to tell P2 about options like that.
The alternative is to allow for multiple mutually-exclusive features per connectorDescriptor.  This would allow for multiple install buttons with different meaning.  It's doable but is definitely a new requirement that we haven't yet discussed.

> I guess either way we need to go with a checkbox
> style model because we want to support multiple simultaneous installs?  

Not quite.  Firefox uses such an interaction, showing selected items at the to of the dialog.  Deselecting them is a matter of clicking the 'x' next to a selected item.  To me the Firefox interaction style is a little less intuitive, and there is no precedent in other Eclipse UIs.  Elsewhere in Eclipse a shuttle control is used.  Personally I would prefer a checkbox if each connector can only be installed one way.  If we go with multiple buttons as discussed above, then the Firefox interaction style may be better.

> Is it easy enough to pop up the multiple licenses?

That should be handled by P2.
Comment 4 David Green CLA 2009-04-23 13:18:15 EDT
Created attachment 132979 [details]
_very_ preliminary draft of the UI
Comment 5 David Green CLA 2009-04-28 00:06:06 EDT
Created attachment 133476 [details]
screenshot sequence

a series of screenshots that show discovery and p2 in action
Comment 6 David Green CLA 2009-04-28 00:07:01 EDT
Created attachment 133477 [details]
screenshot of discovery ui with filters
Comment 7 David Green CLA 2009-04-28 00:08:19 EDT
Created attachment 133478 [details]
screenshot with no matching connectors
Comment 8 maarten meijer CLA 2009-04-28 13:20:12 EDT
Maybe the filtering mechanism can be extended using an Extension point. Also it should be possible to have a combo element selector in addition to a checkbox.
Use Case: for the generic SQL connector I would like to be able to search for ready made fragments for certain brand of database: example search for MySQ based connector fragments.
Comment 9 David Green CLA 2009-04-28 13:25:32 EDT
(In reply to comment #8)
> Maybe the filtering mechanism can be extended using an Extension point. 

What would be the purpose of such an extension point?

> Also it should be possible to have a combo element selector in addition to a checkbox.
> Use Case: for the generic SQL connector I would like to be able to search for
> ready made fragments for certain brand of database: example search for MySQ
> based connector fragments.

Can you elaborate on what you mean by "combo element selector" (do you mean a combo box?).
What would be in the "combo element selector" and how do you see it working?

Comment 10 maarten meijer CLA 2009-04-28 16:58:14 EDT
(In reply to comment #9)
If you want to apply a filter I assume you want connectors to belong to some sort of category, because you will use the category to filter on.
So the mechanism for categories and filtering on them should be extensible, because you will not be able to determine all categories beforehand.
Maybe you may even want the filtering to be multidimensional (Free/Shareware/Commercial or based on license or for what system, etc)

> What would be the purpose of such an extension point?
Such an extension point would allow an already installed connector to contribute to the discovery filtering, so that when you have one part of a set of connectors (clearcase, or jira) you can by filtering discover the related connectors (clearquest, other atlassian stuff). In the instance of the generic SQL connector, you may search for extension fragments for particular database types, as these must include an appririate driver and config files.

For the same reason I think that the Bugzilla connector Query Type Wizard should be extensible, so I can add my own custom type of bugzilla query that is far simpler for access to a company bugzilla.
I will also do something like that in the generic SQL connector Query Type Wizard Page.
> Can you elaborate on what you mean by "combo element selector" (do you mean a
> combo box?).
> What would be in the "combo element selector" and how do you see it working?
Yes combo box is the SWT class. I need that so I can add a selector to search for connectors for MySQL database that are also free (see example above) , so instead of a checkbox for commercial/free I would pick from a combo.
Comment 11 Mik Kersten CLA 2009-04-30 19:28:48 EDT
David: This is looking great.  I think it's all the UI we need for a first cut at a fully working versions.
Comment 12 David Green CLA 2009-04-30 21:11:43 EDT
(In reply to comment #10)

One of the key goals of the discover UI is simplicity: it has to be easy to use and very, very simple.  Excessive filtering options could prove intimidating for users, especially if they're novices.  On the other hand, if the UI is too simple then it could make it unweildy to use as well, since too much scrolling/reading may be required for people to find what they need.  These are conflicting requirements.  I think that the current design is a good compromise for a first cut.

Something that might achieve what you want without adding too much complexity to the UI is a simple search text field.  The Eclipse preferences dialog is an example of such a search field.    Would that serve your needs Maarten?  Users would be able to type in 'MySQL' to find your connectors.  I'd be interested to hear what others think about that idea.
Comment 13 David Green CLA 2009-05-02 22:57:21 EDT
(In reply to comment #11)
> David: This is looking great.  I think it's all the UI we need for a first cut at a fully working versions.

Mik, sounds great.  I'm going to hold off on doing more until we can sync up.  Once we've done that I'll clean up the code, take care of NLS and add some more unit tests.  It would be great to have a 'real' connector directory and at least one update site available for testing purposes.
Comment 14 Mik Kersten CLA 2009-05-05 13:04:24 EDT
David: Please do create a new bug for the search field, I think it's worth doing since we're expecting to support stores with a large number of listings.  Please take the "Find" UI approach of the Task List rather than the "type filter text" approach that the Preferences dialog uses.
Comment 15 Peter Stibrany CLA 2009-05-10 15:52:06 EDT
Hello,

what is the plan for including 3rd party connectors on discovery site? If I would be interested in including my connector, what steps should I take? I understand that this is probably very early to ask, so I would just like to know if you have any plan here.

I tried to create custom "discovery site" plugin, similar to org.eclipse.mylyn.discovery.site.trac, it works fine locally (modulo few minor problems) ... but obviously, I'd like to distribute it in the future ;-)

Thank you very much,
-Peter
Comment 16 Steffen Pingel CLA 2009-05-11 22:12:28 EDT
A few style nits from a first review:
- Code should be reformatted to pick up the project settings
- Plug-in versions should changed to 3.2.0.qualifier
- Bundle-name should not have a Plug-in postfix
- Cleanup all source code warning (NON-NLS tags etc.)
- The core sources should be internationalized or NON-NLS comments added
- All uses of MessageFormat.format() need to be replaced by NLS.bind()
- The test plug-in needs to be renamed to tests (we can do that when moving the plug-in out of the sandbox)
- Unused NLS messages need to be cleaned up

When I tried to use the discovery wizard installed into Eclipse 3.5M7 I noticed the following:
- The Link did not show up in the Add Task Repository wizard. I might have broken that with my change for bug 275204, comment 13. I used Ctrl+3 to trigger the wizard directly.
- After the install of the Trac connector finished only the feature ended up on disk but no plug-ins rendering the connector unusable. Installing the Trac connector through the standard P2 UI worked without any problems.
- Trying install the already installed connector through the discovery wizard gave me a P2 error.

We can review the UI in more details tomorrow. One thing I was missing is a Refresh button in the connector wizard (e.g. in case network was temporarily not available).
Comment 17 David Green CLA 2009-05-11 23:37:19 EDT
Thanks for the code review!  Some comments below:

(In reply to comment #16)
> A few style nits from a first review:
> - Code should be reformatted to pick up the project settings
> - Plug-in versions should changed to 3.2.0.qualifier
> - Bundle-name should not have a Plug-in postfix
> - Cleanup all source code warning (NON-NLS tags etc.)
> - The core sources should be internationalized or NON-NLS comments added

Delay in externalizing strings, source formatting etc. was intentional to help keep things agile.  After Tuesday's review it'll make sense to do that.

> - All uses of MessageFormat.format() need to be replaced by NLS.bind()

I only just discovered NLS.bind() today.  Thanks for pointing that out!

> - The test plug-in needs to be renamed to tests (we can do that when moving the
> plug-in out of the sandbox)
> - Unused NLS messages need to be cleaned up
> 
> When I tried to use the discovery wizard installed into Eclipse 3.5M7 I noticed
> the following:
> - The Link did not show up in the Add Task Repository wizard. I might have
> broken that with my change for bug 275204, comment 13. I used Ctrl+3 to trigger
> the wizard directly.

Yes, that code that was commented out is important.  Fixed in CVS.

> - After the install of the Trac connector finished only the feature ended up on
> disk but no plug-ins rendering the connector unusable. Installing the Trac
> connector through the standard P2 UI worked without any problems.

That's disappointing.  You weren't self-hosting were you?  Perhaps Rob could shed some light.
BTW, what do you mean when you say 'unusable'?

> - Trying install the already installed connector through the discovery wizard
> gave me a P2 error.

Perhaps we should be filtering already-installed bundles from the UI.

> We can review the UI in more details tomorrow. One thing I was missing is a
> Refresh button in the connector wizard (e.g. in case network was temporarily not
> available).

Agreed

Thanks again for the detailed code review.
Comment 18 Steffen Pingel CLA 2009-05-11 23:48:31 EDT
> > - After the install of the Trac connector finished only the feature ended up
> on
> > disk but no plug-ins rendering the connector unusable. Installing the Trac
> > connector through the standard P2 UI worked without any problems.
> 
> That's disappointing.  You weren't self-hosting were you?  Perhaps Rob could
> shed some light.
> BTW, what do you mean when you say 'unusable'?

I was using Eclipse 3.5M7 with discovery plug-ins in the dropins folder that I had manually exported from my workspace.

Since the wizard installed no plug-ins (only the feature) the Trac connector did not load.
Comment 19 Robert Elves CLA 2009-05-12 19:08:42 EDT
I seem to recall something similar happening when the metadata repo is loaded but not the artifact repository. The fact that the features got installed though seems to indicate a slightly different problem so I'll need to investigate this further.
Comment 20 Robert Elves CLA 2009-05-12 19:56:08 EDT
Just having looked at the source, the MatchQuery.isMatch code in InstallConnectorsJob looks suspect.  Are we getting the full group of IUs for the feature or just the feature themselves here? Might want to set a break point in there to see what is getting collected.  
Comment 21 Mik Kersten CLA 2009-05-12 20:53:55 EDT
(In reply to comment #15)
> what is the plan for including 3rd party connectors on discovery site? If I
> would be interested in including my connector, what steps should I take? I
> understand that this is probably very early to ask, so I would just like to know
> if you have any plan here.

Peter: We are in the process of figuring out the policy, which will boil down to a compromise between making it easy to get working connectors and ensuring that we don't overly burden the Mylyn project with support issues for third party connectors.  We will be discussing it on this Thursday's Mylyn call and should have more information for you next week.
Comment 22 Mik Kersten CLA 2009-05-12 23:44:53 EDT
David: Here are the notes that Steffen collected during our UI review.  Please turn these into bugs as needed.

h3. UI Review

h4. Add Task Repository wizard:

* Convert link to button with icon and move to the left
* Capital case download more connectors

h4. Connector Discovery wizard:

* Only show tooltip when hovering over additional question mark icon
* Use gradient for background of tooltip
* Change layout of tooltip to text left, image right: size 640x240+ padding
* Add Learn More link on the bottom right corner of the tooltip
* Remove More Information link to table
* Use gradient to distinguish categories in table (polish)
* Limit length of description  (polish)
* Connector list flickers when categories are toggled, consider hiding
instead of re-creating controls
* Blank space at the end of the connectors list
* Keyboard navigation doesn't work on Mac
* Ensure focus control is always visible in connector list
* Hide installed features
* Add context sensitive help
* Make wizard taller
* Add "Find" label to search, move search box on same line as category
check boxes
* Categories: Tasks, Documents, Version Control
* Detailed description for categories should be in tooltip
* Add policy for declaring categories

h4. Download

* Jars are located on Eclipse download mirrors
* Retry download once to recover from mirror failure
* Rename directory to directory.xml
* Check for content.jar on update site, disable installs for
unavailable connectors
* Add capability (Mik's note: Not sure if this is needed?)

h4. Needed Icons:

* Download button icon
* Wizard banner icon
Comment 23 David Green CLA 2009-05-13 23:17:24 EDT
(In reply to comment #22)
> David: Here are the notes that Steffen collected during our UI review.  Please
> turn these into bugs as needed.

Excellent, done.  Nice usage of Textile markup, btw.

> * Add capability (Mik's note: Not sure if this is needed?)

What do you mean by 'Add capability'?

Comment 24 Steffen Pingel CLA 2009-05-14 01:06:36 EDT
> > * Add capability (Mik's note: Not sure if this is needed?)
> 
> What do you mean by 'Add capability'?

I am not sure. It is probably an incomplete sentence for the feature covered on bug 276231.
Comment 25 Shawn Minto CLA 2009-05-14 12:54:13 EDT
(In reply to comment #24)
> > > * Add capability (Mik's note: Not sure if this is needed?)
> >
> > What do you mean by 'Add capability'?
> 
> I am not sure. It is probably an incomplete sentence for the feature covered on
> bug 276231.

Could this be the actual capabilities provided by eclipse (e.g. o.e.m.capability plugin)?
Comment 26 Steffen Pingel CLA 2009-05-14 14:42:09 EDT
> Could this be the actual capabilities provided by eclipse (e.g. o.e.m.capability
> plugin)?

No that should be already covered by our current capabilities.
Comment 27 Mik Kersten CLA 2009-05-20 12:02:39 EDT
(In reply to comment #23)
> Excellent, done.  Nice usage of Textile markup, btw.

And I was annoyed because the line break from pasting plain text broke my formatting ;)

> > * Add capability (Mik's note: Not sure if this is needed?)
> 
> What do you mean by 'Add capability'?

I didn't know what it meant either, so let's just ignore it.
Comment 28 Mik Kersten CLA 2009-05-21 13:40:29 EDT
Quick UI review from call:
* Put license string in light and small text to the right of the name
* Try to right-align the question mark, give it a tooltip called "Show Overview"
Comment 29 Mik Kersten CLA 2009-05-26 12:59:25 EDT
David, I just took a quick pass through the UI and here are some small nits to fix.

* When the initial download is happening, could you change the message to say something about the connection being made?  I.e.: "Retrieving listing from eclipse.org"
* Button in "Add Task Repository" wizard: "Install More Connectors..."
* Dialog shell: "Install Connectors"
* Dialog title: "Mylyn Connector Discover"
* Dialog description: "Click the question mark for a popup description and link for more information. Clicking Finish will open the install wizard."
Comment 30 David Green CLA 2009-05-26 20:50:46 EDT
(In reply to comment #29)

done.

> * Dialog description: "Click the question mark for a popup description and link
> for more information. Clicking Finish will open the install wizard."

'question mark' will be confusing to the user since the help button also displays a question mark.  Also we talked about providing an (i) information image instead of a question mark.  How about something like this:  "Select connectors to install.  Press Finish to proceed with installation.  Press the information button to see a detailed overview and a link to more information."
Comment 31 David Green CLA 2009-05-26 20:50:49 EDT
Created attachment 137250 [details]
mylyn/context/zip
Comment 32 Mik Kersten CLA 2009-05-27 00:22:56 EDT
Yup, your text is better so use that.
Comment 33 Mik Kersten CLA 2009-06-09 23:07:09 EDT
David: One issue we're currently seeing is that the "Community" listing is ranked first, above the Eclipse.org listing.  Do we have a bug for adding a "rank" attribute that breaks alphabetical sorting accordingly?  I think that's the top outstanding priority now.
Comment 34 Steffen Pingel CLA 2009-06-09 23:54:13 EDT
I have specified relevances for the categories which should fix the sorting.
Comment 35 Mik Kersten CLA 2009-06-10 11:59:37 EDT
Excellent.  I'll try to update the listing now.
Comment 36 Steffen Pingel CLA 2009-06-16 23:32:08 EDT
Great work on this David! I have removed and rescheduled the unresolved subtasks since we are past the commit window for this release. Please mark as resolved if we are done here so we can complete the last plan item :).
Comment 37 Mik Kersten CLA 2009-06-17 08:49:32 EDT
Great work David!
Comment 38 David Green CLA 2009-06-17 14:22:51 EDT
Thanks!  As usual, a team effort.  Great work on everyone's part.