Bug 317890 - [hudson][build] support auto discovery of Hudson servers
Summary: [hudson][build] support auto discovery of Hudson servers
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: 0.7   Edit
Assignee: Torkild Resheim CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed, noteworthy
Depends on: 325079 326738
Blocks:
  Show dependency tree
 
Reported: 2010-06-24 19:00 EDT by Steffen Pingel CLA
Modified: 2011-01-05 14:44 EST (History)
3 users (show)

See Also:


Attachments
service message (13.33 KB, image/png)
2010-09-11 17:20 EDT, Steffen Pingel CLA
no flags Details
Patch to fix bug 317890 (9.34 KB, patch)
2010-09-23 09:51 EDT, Torkild Resheim CLA
no flags Details | Diff
Updated patch (10.33 KB, patch)
2010-09-27 07:26 EDT, Torkild Resheim CLA
no flags Details | Diff
Updated patch (10.42 KB, patch)
2010-10-04 09:13 EDT, Torkild Resheim CLA
steffen.pingel: iplog+
Details | Diff
mylyn/context/zip (3.78 KB, application/octet-stream)
2010-10-04 09:13 EDT, Torkild Resheim CLA
no flags Details
auto discovery (13.61 KB, image/png)
2010-10-08 19:39 EDT, Steffen Pingel CLA
no flags Details
Revised patch (9.88 KB, patch)
2010-10-18 02:25 EDT, Markus Kuppe CLA
no flags Details | Diff
mylyn/context/zip (95.77 KB, application/octet-stream)
2010-10-18 02:25 EDT, Markus Kuppe CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Pingel CLA 2010-06-24 19:00:36 EDT
Hudson instances listen on UDP port 33848. Whenever a UDP packet is received, it will respond with a short XML fragment that shows the connection information: http://wiki.hudson-ci.org/display/HUDSON/Remote+access+API

It would be nice if the Mylyn connector supported auto discovery and configuration of Hudson servers.
Comment 1 Markus Knittig CLA 2010-06-26 12:32:00 EDT
Doesn't look too hard and definitely a useful function. I question is where to place that. I think that belongs to the settings page, but that cannot be simply overriden, because this would expose task-related classes. A delegate might be a solution, but on long term overriding the settings page should be possible...
Comment 2 Steffen Pingel CLA 2010-06-26 16:36:12 EDT
Yes, the framework would need to provide some UI for scanning the network and an extension for that. It's a  nice to have feature but very low priority. I would consider that as part of generalizing the team repositories view so that users can scan for any type of server when creating team repositories (bug 318097).
Comment 3 Torkild Resheim CLA 2010-09-06 08:34:13 EDT
I've already implemented discovery through Multicast DNS for my "Buildmonitor" project. If you like I could port that code.
Comment 4 Steffen Pingel CLA 2010-09-06 19:03:21 EDT
Sounds great. I'll assign this bug to you.
Comment 5 Torkild Resheim CLA 2010-09-09 16:02:20 EDT
Thanks Steffen. I've got some working code in org.eclipse.mylyn.hudson.ui but I rely on ECF for the MDNS implementation. I also need org.eclipse.emf.ecore. So I'm wondering if these additional dependencies are OK.
Comment 6 Torkild Resheim CLA 2010-09-09 16:17:28 EDT
I'm also wondering what to when a server has been discovered. The alternatives I came up with are:

1. silently add the newly discovered server to repositories
2. pop up a notification and add the server
3. show the new Hudson server wizard with some information filled in

Personally I would go for item two. What do you guys think?
Comment 7 Torkild Resheim CLA 2010-09-09 16:19:16 EDT
Come to think of it; a combination of 2 and 3 would also be nice. E.g. add "yes", "no", "configure" buttons to the notification and show the wizard when the latter alternative is selected.
Comment 8 Steffen Pingel CLA 2010-09-11 17:19:53 EDT
(In reply to comment #5)
> Thanks Steffen. I've got some working code in org.eclipse.mylyn.hudson.ui but I
> rely on ECF for the MDNS implementation. I also need org.eclipse.emf.ecore. So
> I'm wondering if these additional dependencies are OK.

Yes, that sounds reasonable to me. We haven't used ECF in the past but I'd be curious to learn more about it and I don't see any problem with adding that and EMF as a dependency. 

(In reply to comment #7)
> Come to think of it; a combination of 2 and 3 would also be nice. E.g. add
> "yes", "no", "configure" buttons to the notification and show the wizard when
> the latter alternative is selected.

I have been thinking of this as an aid when creating new servers, i.e. the user explicitly triggers the action for creating a server and then has the option to auto scan the network. My concern with automatic popup notifications is that they interrupt the workflow. 

How about showing a message in the builds view when a server is discovered similar to how the Tasklist shows service messages (screenshot)? Clicking the message could open the Hudson settings dialog with fields pre-populated.
Comment 9 Steffen Pingel CLA 2010-09-11 17:20:16 EDT
Created attachment 178690 [details]
service message
Comment 10 Torkild Resheim CLA 2010-09-13 03:03:01 EDT
(In reply to comment #8)
> Yes, that sounds reasonable to me. We haven't used ECF in the past but I'd be
> curious to learn more about it and I don't see any problem with adding that and
> EMF as a dependency.
OK, good.
> 
> 
> I have been thinking of this as an aid when creating new servers, i.e. the user
> explicitly triggers the action for creating a server and then has the option to
> auto scan the network. My concern with automatic popup notifications is that
> they interrupt the workflow.
> 
> How about showing a message in the builds view when a server is discovered
> similar to how the Tasklist shows service messages (screenshot)? Clicking the
> message could open the Hudson settings dialog with fields pre-populated.
Looks nice. I'll get started on that.
Comment 11 Torkild Resheim CLA 2010-09-23 09:51:06 EDT
Created attachment 179457 [details]
Patch to fix bug 317890

Adds a discovery mechanism for Hudson servers.
Comment 12 Steffen Pingel CLA 2010-09-23 12:34:04 EDT
Impressive how this is with ECF. The patch looks. Some suggestions for improvements: 
* Instead of using a startup extension I would recommend coupling the listener to the life cycle of the Builds view. If a user has never opened the Builds view it is not likely that they intend to use the integration and are interested in notifications about new build server so we should minimize plug-in loading.
* Replace usages of MessageFormat.format() with NLS.bind() which is generally used in Eclipse.
* Exceptions should be logged to the error log and not dumped to the console.

The manifest lists a lot of ECF plug-ins as dependencies. Are all of those required?
Comment 13 Steffen Pingel CLA 2010-09-23 12:38:07 EDT
(In reply to comment #12)
> * Instead of using a startup extension I would recommend coupling the listener
> to the life cycle of the Builds view. If a user has never opened the Builds view
> it is not likely that they intend to use the integration and are interested in
> notifications about new build server so we should minimize plug-in loading.

We'll probably an extension point in the builds framework to support this since the Hudson connector doesn't know anything about the Builds view. Also, the necessary classes should be in API/SPI packages to avoid coupling the Hudson connector to internals.
Comment 14 Torkild Resheim CLA 2010-09-27 07:26:19 EDT
Created attachment 179622 [details]
Updated patch

Using NLS.bin() and logs exceptions. List of required dependencies has been shortened. I've not stopped using the startup extension as the replacement mechanism is not available yet.
Comment 15 Torkild Resheim CLA 2010-10-04 09:13:40 EDT
Created attachment 180159 [details]
Updated patch

Updated code to use latest notification mechanisms.
Comment 16 Torkild Resheim CLA 2010-10-04 09:13:46 EDT
Created attachment 180160 [details]
mylyn/context/zip
Comment 17 Steffen Pingel CLA 2010-10-08 19:39:29 EDT
Created attachment 180530 [details]
auto discovery
Comment 18 Steffen Pingel CLA 2010-10-08 19:39:48 EDT
Thanks. That's great stuff. I have applied the patch. Can you let me know what feature dependencies we should add for Hudson? It seems that we need the ECF Discovery and JmDNS support?

!ENTRY org.eclipse.mylyn.hudson.ui 4 0 2010-10-08 13:58:24.167
!MESSAGE Could not start Hudson automatic discovery service
!STACK 1
org.eclipse.ecf.core.ContainerCreateException: Unknown Container Name
	at org.eclipse.ecf.provider.discovery.SingletonDiscoveryContainerInstantiator.getInstance(SingletonDiscoveryContainerInstantiator.java:40)
	at org.eclipse.ecf.provider.discovery.SingletonDiscoveryContainerInstantiator.createInstance(SingletonDiscoveryContainerInstantiator.java:52)
	at org.eclipse.ecf.core.ContainerFactory.createContainer(ContainerFactory.java:288)
	at org.eclipse.ecf.core.ContainerFactory.createContainer(ContainerFactory.java:307)
Comment 19 Torkild Resheim CLA 2010-10-11 04:05:11 EDT
Yes, looks like we need:

  * o.e.e.discovery
  * o.e.e.provider.jmdns
  * o.e.e.provider.discovery
  
These should probably be added to the manifest.
Comment 20 Steffen Pingel CLA 2010-10-12 01:02:07 EDT
As far as I could tell ECF didn't split components into separate features until the latest Helios release. Since we should maintain backwards compatibility with Eclipse Galileo (3.5) I was wondering how to resolve this? Maybe it's best to leave out the feature dependency and let P2 find the required plug-ins instead.
Comment 21 Torkild Resheim CLA 2010-10-14 08:19:11 EDT
I don't know what would be the best approach here. Maybe you could give us a pointer Scott?
Comment 22 Scott Lewis CLA 2010-10-14 09:57:15 EDT
I'm going to add Markus Kuppe to this discussion...he and I have both done a lot of work on ecf discovery...and recently he's been making changes to the discovery feature structure as well (i.e. bug 327755)...in an effort to make discovery and other parts of ECF more easily consumable.

A few more words about our feature structure...we are struggling a little bit in dealing with the platform's convention of specifying exact versions (rather than relaxing version range checking) for feature includes.  Thus the source of bug 327755.  What this means is that the feature structure for ECF discovery is likely to change.

And some explanation...with any/all of ECF we strive to have a clear api-provider separation...at the bundle level...meaning that with, say, ECF discovery, you need the plugins that expose the discovery api (org.eclipse.ecf.discovery), and then to use zeroconf discovery you also need the org.eclipse.ecf.provider.jmdns bundle (plus any dependencies).  So no matter how things are consumed for this Hudson enhancement (i.e. via bundles or via features) one ultimately needs both the api bundle(s) and one or more provider bundles.  Not having the provider present (jmdns in this case) is what would likely cause the output from comment 18.

So in answer to the questions in comment 20 and comment 21...I suggest that you leave out the feature dependencies...and we'll try to deal with the feature structure relaxed version range stuff as soon as we can...to make the feature-level consumption easier in the future.
Comment 23 Markus Kuppe CLA 2010-10-14 10:23:39 EDT
(In reply to comment #22)
> So in answer to the questions in comment 20 and comment 21...I suggest that you
> leave out the feature dependencies...and we'll try to deal with the feature
> structure relaxed version range stuff as soon as we can...to make the
> feature-level consumption easier in the future.

FWIW: bug #327755 might be of interest
Comment 24 Steffen Pingel CLA 2010-10-15 14:24:53 EDT
Thanks for the input. We'll go with plug-in dependencies for now and ask users to install ECF if they want the full discovery UI or update to the latest ECF release.
Comment 25 Markus Kuppe CLA 2010-10-18 02:25:22 EDT
Created attachment 181065 [details]
Revised patch

Hi, this patch changes the current implementation in following ways:

- Make it discovey provider agnostic (hudson.ui should not care if the service gets discovered by mDNS, SLP, DNS-SD...)
- Do not match "local" domains (a Hudson instance might be located in a different network/subnet)
- Use ECF 3.4 whiteboard IServiceListener registration to simplify impl
Comment 26 Markus Kuppe CLA 2010-10-18 02:25:28 EDT
Created attachment 181066 [details]
mylyn/context/zip
Comment 27 Markus Kuppe CLA 2010-10-18 02:36:52 EDT
(In reply to comment #25)

> - Use ECF 3.4 whiteboard IServiceListener registration to simplify impl

The patch contains a glitch WRT ECF 3.4. Please change MANIFEST.MF's Require-Bundle stmt to 4.0.0 (instead of 4.1.0) for bundle "org.eclipse.ecf.discovery".
Comment 28 Torkild Resheim CLA 2010-10-18 09:12:00 EDT
Nice. Thanks Markus :-)
Comment 29 Markus Kuppe CLA 2010-10-18 09:20:30 EDT
(In reply to comment #27)
> (In reply to comment #25)
> 
> > - Use ECF 3.4 whiteboard IServiceListener registration to simplify impl
> 
> The patch contains a glitch WRT ECF 3.4. Please change MANIFEST.MF's
> Require-Bundle stmt to 4.0.0 (instead of 4.1.0) for bundle
> "org.eclipse.ecf.discovery".

Code-wise you might as well lower the dependency to 3.0.0 as the API has not changed between 3.x and 4.x in this area. It's just, that with discovery 3.x the listener will never be notified.
Comment 30 Steffen Pingel CLA 2010-10-18 14:18:58 EDT
Thanks Markus, I like those changes. How does the service discovery get activated? Is registering the service sufficient or does ECF need to a nudge to start listening to the network?
Comment 31 Markus Kuppe CLA 2010-10-18 15:42:53 EDT
(In reply to comment #30)
> How does the service discovery get
> activated? Is registering the service sufficient or does ECF need to a nudge to
> start listening to the network?

Registering the listener does not start discovery. Starting a provider (e.g. discovery) is a deployment issue for ECF. Obviously this has to be addressed somehow for this enhancement, but I'm not sure it's the best approach to do this from the hudson.ui bundle.
Comment 32 Steffen Pingel CLA 2010-10-18 16:42:32 EDT
We have a startup extension point for the builds UI that we can use to trigger discovery initialization in ECF. What would be the best ECF API way to do that? 

I would also like to do the service registration lazily. With the patch the Hudson bundle is always activated on startup which is what we were trying to avoid by introducing the builds UI startup. The goal is that there should be minimal cost in having the Hudson connector installed unless Builds View is actually used.

And if you would like to get credit for the patch, Markus, you should open a separate bug :).
Comment 33 Markus Kuppe CLA 2010-10-19 02:33:29 EDT
(In reply to comment #32)
> We have a startup extension point for the builds UI that we can use to trigger
> discovery initialization in ECF. What would be the best ECF API way to do that? 

IMO the best way to do this would be to declaratively have discovery started during IDE startup (e.g. config.ini). Is it possible to modify config.ini via p2?

Alternatively we might want to add a preference page where the user can select if Hudson servers should be discovered upon startup. This could be generalized into a ECF provider pref page though (bug # 217981).

A button in the build UI might also make sense that starts discovery once pressed.

> I would also like to do the service registration lazily. With the patch the
> Hudson bundle is always activated on startup which is what we were trying to
> avoid by introducing the builds UI startup. The goal is that there should be
> minimal cost in having the Hudson connector installed unless Builds View is
> actually used.

Can't you change OSGI-INF/component.xml to lazily start the bundle if classes are loaded?

> And if you would like to get credit for the patch, Markus, you should open a
> separate bug :).

Nah, then I will be held responsible for bugs introduced by my patch. ;-)
Comment 34 Markus Kuppe CLA 2010-10-19 08:48:58 EDT
(In reply to comment #33)
> (In reply to comment #32)
> > We have a startup extension point for the builds UI that we can use to trigger
> > discovery initialization in ECF. What would be the best ECF API way to do that? 
> 
> IMO the best way to do this would be to declaratively have discovery started
> during IDE startup (e.g. config.ini). Is it possible to modify config.ini via
> p2?
> 
> Alternatively we might want to add a preference page where the user can select
> if Hudson servers should be discovered upon startup. This could be generalized
> into a ECF provider pref page though (bug # 217981).
> 
> A button in the build UI might also make sense that starts discovery once
> pressed.

Thinking about it, Bundle-ActivationPolicy: lazy will trigger discovery to be started including all available providers. No need to do anything besides registering the IServiceListener with the OSGi service registry.
Comment 35 Steffen Pingel CLA 2010-10-19 18:44:23 EDT
(In reply to comment #33)
> IMO the best way to do this would be to declaratively have discovery started
> during IDE startup (e.g. config.ini). Is it possible to modify config.ini via
> p2?

I explicitly don't want to do that to avoid bundle activation.

> Alternatively we might want to add a preference page where the user can select
> if Hudson servers should be discovered upon startup. This could be generalized
> into a ECF provider pref page though (bug # 217981).

I think the workbench startup extensions are sufficient to handle that.

> A button in the build UI might also make sense that starts discovery once
> pressed.

Yes, I think that makes sense. Also to explicitly search for servers on the network.

> > I would also like to do the service registration lazily. With the patch the
> > Hudson bundle is always activated on startup which is what we were trying to
> > avoid by introducing the builds UI startup. The goal is that there should be
> > minimal cost in having the Hudson connector installed unless Builds View is
> > actually used.
> 
> Can't you change OSGI-INF/component.xml to lazily start the bundle if classes
> are loaded?

Yes, that should probably work.

> > And if you would like to get credit for the patch, Markus, you should open a
> > separate bug :).
> 
> Nah, then I will be held responsible for bugs introduced by my patch. ;-)

Were you able to auto discovery Hudson servers with the patch? I haven't been able to make it work so far but I suspect it's because discovery is never actually started.
Comment 36 Steffen Pingel CLA 2010-10-19 19:55:59 EDT
Markus, can you file a separate bug with instructions how to make this work? I am unable to get the HudsonService.serviceDiscovered() event to fire although the Service Discovery view shows Hudson services.
Comment 37 Markus Kuppe CLA 2010-10-20 04:45:26 EDT
(In reply to comment #36)
> Markus, can you file a separate bug with instructions how to make this work? I
> am unable to get the HudsonService.serviceDiscovered() event to fire although
> the Service Discovery view shows Hudson services.

You have to use a recent ECF build (or master/HEAD). Anyway, bug #328218 has been filed as requested.