Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [iot-wg] IoT backend server platform proposal
In my view, Henryk's examples detail the need to maintain 'low coupling and high cohesion' when designing micro services.  Put another way, when defining, implementing, and exposing services, it's very helpful to make clear the distinction between service contract (API) and implementation...i.e. to decouple API from implementation.

OSGi services provides a number of mechanisms to support low coupling and high cohesion in design of services.  For example, a service API can be as simple as a single interface with a single method.   The instance creation/configuration/starting/stopping of the service is left to implementations, and only the service contract is used to discover and access the underlying implementation.   The service API and implementations are separated into distinct bundles, allowing for a very clear (and enforced) separation between service contract and implementations.  Both service API and implementations can be versioned separately.  Service instances can appear, disappear, and reappear at runtime and there are OSGi mechanisms to support this reality (e.g. DS).   As well, services can be distributed on a network (remote) via a variety of distribution mechanisms.

Also, as Henryk describes in his example, there's frequently needs to extend/enhance either the service contract and/or implementation over time, and there are frequently inter-service dependencies (e.g. DeviceManagementService->DeviceEventCallback) as well.

For those interested, here are slides from a presentation discussing coupling and OSGi services [1].

But as Henryk's examples suggests, it's not necessary that OSGi be used to achieve low coupling in services...it's quite possible to have and maintain a clear separation between service API and service implementation.   But I would argue that it's frequently harder to do so without the OSGi services registry and framework because you lose support for a clear lifecycle and notification, versioning, a less-clear notion of 'component' and 'module', etc.

I think it makes sense to define fine-grained (micro) IoT service contracts independently from the implementation/configuration/extension and then provide test and full implementations in as many environments/frameworks as possible to test and incrementally refine or enhance the service contract.   Henryk seems to refer to this as 'bootstrap API' in his examples...unless I am misinterpreting his usage, I would personally tend to refer to this as 'service contract' or 'service API'.

I think it's instructive to look at popular OSGi services that have been specified/standardized:  e.g. the log service, http service, eventadmin service, config mgmt, device service, etc).  The service API for these services is in org.eclipse.osgi.service bundle.

Scott

[1] http://www.slideshare.net/bjhargrave/servicesfirst-migration-to-osgi?related=1

On 10/27/2015 1:30 PM, Henryk Konsek wrote:
Hi Kai,

The boostrap API and the abstraction it provides is something we need to introduce in order to make our services framework independent and avoid a vendor lock-in.

To be more exact, let me use the example. Imagine that we implement a device management service based on Leshan (kind of thing we are doing in Rhiot). The service should be provided in a form of the library, which can be started and stopped.

DeviceManagementService deviceService = new DeviceManagementService();
deviceService.start();
deviceService.stop();

"Start" method starts embedded Leshan server, initialized database connection pool (used by Leshan to store devices in the registry), connects to the cache cluser (like Hazelcast or Infinispan, used to cache devices information and metrics, for performance reasons) and performs all the other kind of an initialization needed by the device service. When a service is supposed to be shut down, we call the "stop" method. It makes sure that all of those resources have been properly closed. Also the start/stop sequence has to be properly ordered, not accidental. Our DeviceManagementService can be used out of the box without any additional configuration - it will use default settings, default persistence providers, etc.

Now imagine that the company using our DeviceManagementService as a part of their IoT cloud offering, would like to add some non-trivial extension to it. Our platform be easily extensible as this is required for its wider adoption. So for example:
- company Foo would like create an extra module based on Apache Camel and its SAP connector, so the information about the devices is immediately available in the SAP system
- company Bar would like to provide its own Leshan devices registry implementation
- company Baz would like to connect all the events related to the devices to its Apache Spark cluster and perform real-time analysis on it, then perform some actions on the Leshan server based on the data analysis

In all those cases we need to: 
a) create proper callback interfaces in DeviceManagementService that could be implemented to achieve customization on the level of the client code
b) client has to provide the customized callback code to the classpath of the application
c) somehow wire this custom code to the core of our service

The thing is that c) is very different depending on what kind of application framework we would like to use. For example if we are using Spring Boot, the complete customized application for company Baz could look like:

public class BigDataAwareDeviceManagement {

  @Bean(init = "start", destroy = "stop")
  DeviceManagementService deviceManagementService(DeviceEventCallback deviceEventCallback) {
    return new DeviceManagementService(deviceEventCallback);
  }

  @Component
  static class SparkDeviceEventCallback implements DeviceEventCallback {...}

} 

However the same application written in OSGi will be much different. For example DeviceManagementService could be deployed as a one OSGi bundle, while the SparkDeviceEventCallback could be a second bundle with OSGi service exposed.

If we would like to make DeviceManagementService framework-agnostic, we should relay on the vendor-neutral API when accessing the collaborators (like DeviceEventCallback implementations). We could name it BeanRegistry and it could be used inside DeviceManagementService as follows:

class DeviceManagementService {

  DeviceEventCallback deviceEventCallback;

  ...

  void start() {
    deviceEventCallback = beanRegistry.beanByType(DeviceEventCallback.clas);
  }

}

Then we could provide various implementations of the BeanRegistry, like SpringBeanRegistry or OsgiServiceBeanRegistry. With this approach, we can access beans implementing the extensions to our services in a framework-agnostic way.

The same principle applies to the configuration management. OSGi uses Configuration Admin Service, Spring Boot uses its properties/YAML files, Vert.x system properties and so forth. If we would like to make our services configurable, we should provide configuration access abstraction, which in turn should delegate to the proper configuration management system (OSGi, Spring, etc):

class DeviceManagementService {

private static final int DEFAULT_TIMEOUT = 5000; 

  ...

  void start() {
    int maxDevicesCacheSize = configurationResolver.intProperty("devices.cache.size.max", DEFAULT_TIMEOUT); 
  }

}

We can also decide to create a library, not a service implementation runnable out-of-the-box. In such case it will be an end-user responsibility to assemble the service using our library with the gazillions of setters and getters. Which considering all the possible object collaborators that will be included in service like device management, will soon become a configuration nightmare for the end-users. I'm strong believer that the developer adoption is one of the key factors when designing the middleware, so I would prefer to avoid following pure-library path and provide a "boostrap API" instead.

Does it make sense to you?

Cheers!


wt., 27.10.2015 o 11:43 użytkownik Hudalla Kai (INST/ESY) <Kai.Hudalla@xxxxxxxxxxxx> napisał:

Hi Henryk,

 

thanks for picking up the discussion of the IoT Server Platform again J

I have updated the content of our IoT Server Platform wiki [1] to reflect the results of our most recent discussions regarding this topic. In particular, I have added the whiteboard diagram we have been discussing and have updated the black-box view of the components involved. This now reflects the state-of-affairs which I would like to also use for our discussion during the Unconference of the upcoming EclipseCon 2015  in Ludwigsburg.

 

Regarding your points:

 

I think we agree regarding the relevance and purpose of the “re-usable software services” .  From my point of view, creating these will be the main subject of the IoT Server Platform effort and I think it is these components that we should start to collaborate on. I also agree that Rhiot’s Device Management Cloudlet would be a good candidate for serving as the basis for the LWM2M Protocol Adapter J

 

So far I haven’t been aware of the importance of the “bootstrap API” as you called it. I would be interested to learn a little more about its relevance for the overall effort. Could you maybe give an example where such a “bootstrap API” provides the “needed layer of abstraction” as you put it? Or, to put it the other way around, what should we try to avoid when implementing a “re-usable software service” of the IoT Server Platform to make sure that we can deploy them to arbitrary execution environments? Are you thinking about something like the implementation guidelines described by “The Twelve-Factor App” [2]? From my point of view, the advice given there could serve as our guiding principles for implementing our “re-usable software services”.

 

[1] https://wiki.eclipse.org/IoT/IoTServerPlatform

[2] http://12factor.net/

 

Regards,

Kai

 

From: Henryk Konsek [mailto:hekonsek@xxxxxxxxx]
Sent: Friday, October 23, 2015 9:05 AM
To: IoT Working Group mailing list
Cc: Evers Steffen (INST/ESY); Kristan Johannes (INST/ESY); Frank Karsten (INST/ESW2-Be); Maier Daniel (INST/ESY3); Guggemos Dominik (INST/ESW1-Imb); Pellmann Marc (INST/ESW2-Be); Hudalla Kai (INST/ESY); Zimmermann Kai (INST/ESY3)
Subject: IoT backend server platform proposal

 

Hi,

 

I'd like to return to the topic of the server platform that has been initially raised by the Bosch team. I've been thinking about the proposal, about the things we already got in Rhiot, and about the things we would like to have in Rhiot in the future.

 

At this point we would be most interested in the codebase that could be used on the "right" side of the architecture diagram presented by the Bosch team. In particular in:

 

a) reusable software services

b) custom solution services

 

By a) I understand reusable software services that can be easily deployed into the PaaS of choice. Runnable out-of-the-box without need for extra code. The example of such service could be a device management service based on Leshan (which is something [2] we are already working on in Rhiot). The other example could be software provisioning service based on Hawkbit (something I'd love to have). In my understanding these services should be designed with horizontal scalability in mind and PaaS- (and Docker-) friendliness. In general by a) I'm thinking about something I can take, deploy into the cloud of my choice, configure using some properties provider and run. Run without extra coding (until some more significant customizations are needed).

 

By b) I understand a service that will be deployed as the addition to the standard a) services. For us (Red Hat) what would usually be the custom services written by our customers, whom will be using standard a) services (like device management and software provisioning services), but the majority of their application will be custom. While standard a) services should be PaaS-friendly, these b) services can be really anything that uses our a) API.

 

I believe that the very first thing we need to create in order to start a foundation for our further development efforts, is the "bootstrap API" that would be responsible for the service life-cycle in general, and in particular responsible for:

 

A. providing configuration access abstraction

B. handling star/stop of the service (aka service life-cycle) 

C. providing the "beans registry" i.e. way to access Java objects regardless of the dependency injection engine used by the service developer

 

Those ABC points are needed to provide the layer of abstraction that would decouple our further efforts from the IoC technology used. This way end-users could use OSGi, Spring Boot, JEE/JNDI, CDI or any other IoC technology of their choice when working with our services and API.

 

I believe that working together on such "bootstap API" would be the best first step towards the platform, as that would allow us to build the proper abstraction which could be used later for working together on reuseable services (like device management).

 

As the second step I would propose to discuss the "IoT connector". That should be the second step IMHO, because in the first place we need the proper abstraction (bootstrap API) that will be used to configure and access the IoT connector in the framework-agnostic manner.

 

What do you think about this approach? As a prototype is worth 1000 words, I will be happy to contribute the initial code for this kind of bootstrap API. Of course if you also believe that it is the right action for the first step.

 

Cheers!

 

--

Henryk Konsek

_______________________________________________
iot-wg mailing list
iot-wg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/iot-wg
--
Henryk Konsek


_______________________________________________
iot-wg mailing list
iot-wg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/iot-wg

Back to the top