[ecf-dev] remote service callbacks

[Scott Lewis <slewis@xxxxxxxxxxxxx>]

Hi Folks,

I want to describe a little work that I've been doing with OSGi remote 
services.

One of the things that remoting makes somewhat complicated is 
callbacks.   In local/non-remote programs callbacks are easy...e.g. in 
Eclipse, for example, the IProgressMonitor is used as a callback to many 
methods that have the potential to block...e.g.:

IResource.copy(IPath destination,  boolean force, IProgressMonitor monitor)

The IProgressMonitor is a callback whose methods will be called as 
progress is made during the file/directory/workspace copy.

Another use of callbacks is the OSGi 'whiteboard' service pattern, where 
services are registered and then called by some caller under the 
appropriate conditions.  A good example of this is the OSGi EventHandler 
interface [1], where receivers of events implement and register an 
EventHandler, and then the EventAdmin delivers a method (handleEvent) 
when an appropriate event is present.

I've recently been working on using OSGi RS for remote management [2].  
This API can be used to remotely manage many aspects of an OSGi runtime 
(e.g. the framework, bundles, services, scr, applications, p2, ecf rsa, 
ecf core, others...).    There are many cases, however, where it would 
be useful to be able allow a callback to be used to update the remote 
state of a managed framework.  For example, consider remotely managing 
the set and state of the bundles running in a framework.  First, one 
needs to get the current state of the remote bundles so that it can be 
presented.   Soon however, that state can/could be made obsolete (e.g a 
bundle(s) get started and change their state from resolved to active).   
At that point the remote management agent's presentation of the state of 
at least some of the bundles is out of date and has to be refreshed.   
One way to refresh is to periodically poll the remote framework for it's 
current state, and update the state information once received.   This 
has disadvantages, however, as polling can be both inaccurate and 
inefficient when the state is changing relatively rapidly.

Another way is to allow the framework to send 'update' events to the 
remote manager when the state changes.   For example, the BundleContext 
allows one to iterate over all of the bundles and figure out their 
state.   It's also possible, however, to register a BundleListener and 
be called back by the framework when a bundle changes it's state.

For remote management, it would be nice if a remote manager could get 
the initial state of all the bundles in a remote framework, and then 
receive events as the bundles changed state.  This would allow it to 
keep it's information about the remotely running framework synchronized 
with the framework's actual state, without having to constantly poll for 
the current state.  It would also be a much more efficient way (in terms 
of bandwidth usage) to keep the  distributed state synchronized with 
theb actual state.

Now, in the org.eclipse.ecf.mgmt.framework bundle is an IBundleManager 
service [3].  This service allows a remote consumer to get the current 
state of all (or some) of the bundles in a framework.    I've been 
working on the mechanism for associating this service with a 'callback' 
service called IBundleEventHandler [4].   Using RSA, it's possible to 
associate the import (by consumer) of an IBundleManager service with the 
export (by consumer) of an IBundleEventHandler...along with subsequent 
import of the IBundleEventHandler by the IBundleManager service host.

The sequencing is this:

1.  Framework A (being remotely managed) exports an IBundleManager

2.  Framework B (remote manager) imports IBundleManager and calls remote 
method to get initial state

3.  Framework B *exports* IBundleEventHandler

4.  Framework A *imports* IBundleEventHandler and calls 
handleBundleEvent to send state updates as they occur


This results in the following two remote service relationships

A B

IBundleManager host           <------- IBundleManager consumer

IBundleEventHandler consumer  ---->      IBundleEventHandler host

In effect the IBundleEventHandler is a callback for the IBundleManager 
remote service, allowing bundle state change events to be delivered to 
the remote manager as they occur allowing the bundlemanager consumer to 
keep its state up-to-date.

To implement this association, I've created two classes, one for the 'A' 
side called ServiceExporterCallbackImporter and one for the 'B' side 
called ServiceImporterCallbackExporter [5].   Note that these aren't 
bound in any way to IBundleManager/IBundleEventHandler specifically, and 
so can be used for other kinds of remote service<->callback 
relationships.  But I have been testing the use of these classes 
specifically with IBundleManager and IBundleEventHandler for monitoring 
and dynamically updating a 'remote bundles view'...i.e. an Eclipse view 
that allows one to monitor the set of bundles in a remote framework, 
start/stop bundles, etc.   It's all working quite well so far.

The use of this callback relationship between remote services can only 
be done if *both* A and B are able to both import and export remote 
services.  This isn't by default provided with some Remote Service 
providers, however, because they might be asymmetric and unable to be 
both an importer and an exporter.  For example, most rest-based clients 
(e.g. Jax-RS, etc [7]) are unable to be remote service exporters, as 
they are only http clients, and not http servers.   This makes it 
impossible to create the callback relationship between two remote 
services as above with these rest providers.

But many of ECF's providers *do* support both A and B being able to 
export and import remote services (e.g. rosgi, generic, jms, mqtt, 
jgroups, others), and so can support this notion of remote service 
callbacks without modification (I've been testing on ECF generic and 
mqtt providers to start).

I'm going to next use this codebase to try a remote 
IServiceManager/IServiceEventHandler structure for managing services.   
Then my intention is to also use it for managing scr/ds, and others.  
Also testing it out on other appropriate distribution providers (jms, 
jgroups, etc).

I believe that this approach will be very helpful with remotely managing 
IoT frameworks in particular...as IoT frameworks (e.g. Kura) need some 
way to remotely monitor and manage the various parts and services 
dynamically, efficiently without a required binding to a particular 
networking transport.

Any thoughts/comments appreciated.   I'm going to write something more 
in the wiki about this (with demo/pictures, etc) as soon as I am able.

Scott

[1] https://osgi.org/javadoc/r4v42/org/osgi/service/event/EventHandler.html

[2] https://github.com/ECF/OSGIRemoteManagement

[3] 
https://github.com/ECF/OSGIRemoteManagement/blob/master/bundles/org.eclipse.ecf.mgmt.framework/src/org/eclipse/ecf/mgmt/framework/IBundleManager.java

[4] 
https://github.com/ECF/OSGIRemoteManagement/blob/master/bundles/org.eclipse.ecf.mgmt.framework/src/org/eclipse/ecf/mgmt/framework/IBundleEventHandler.java

[5] 
https://github.com/ECF/OSGIRemoteManagement/tree/master/bundles/org.eclipse.ecf.osgi.services.remoteserviceadmin.callback/src/org/eclipse/ecf/osgi/services/remoteserviceadmin/callback

[6] 
https://github.com/ECF/OSGIRemoteManagement/tree/master/plugins/org.eclipse.ecf.remoteservice.ui.bundleview

[7] https://wiki.eclipse.org/Distribution_Providers