[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [ecf-dev] Remote service question

Hi,

A comment on async remote calls: There are two sides to this, the caller side and callee side. The caller side is nicely handled with the CompletableFuture, since it allows you to register a one-time callback. But what if the callee side also needs to be async?

In my case, the service is implemented on a server side Eclipse, and its task is to update one or more Eclipse resources and return the problem markers that appear after the build. The method is something like the following:
Collection<Problems> updateResources(Collection<ResourceUpdates)).
The service implementation needs to update the resources and then wait for a listener to be notified about the changes to the problem markers attached to the same resources. Hence, the service implementation is also async, since it needs to wait for a value that will later be made available from a different thread. I can use wait and notify to handle this, but would rather use a CompletableFuture. I.e. the service method would be changed to CompletableFuture<Collection<Problems>> updateResources(Collection<ResourceUpdates)) and then the service implementation could immediately return a CompletableFuture and later call complete. My question is, can the callee end of a remote service handle this, i.e. receive a CompletableFuture from a service implementation method and wait for the value to appear?


Hallvard

On 19.10.14 23:06, Wim Jongman wrote:
Hi Scott,

Thanks for your great insights. You really know this stuff. Instead of
trying to invent something new I should acknowledge the current
"constraints" and work with that.

I guess I could solve my use case with the following oversimplified call:

void  fakeListener(int pTime)  {
         System.out.println("Remote time is:"  +  pTime)
         // Get the CompletableFuture...no blocking here
	CompletableFuture<Long>  cf=  timeService.getCurrentTimeBlockedAsync();
	// print out time when done...no blocking anywhere!
	cf.thenAccept((time)  ->  fakeListener(time));
}

timeService.getCurrentTimeBlocked does not return until a new time is
available but this is all taken care of by the CompletableFuture.

I think this is what Hallvard is also looking for.

Cheers,

Wim







On Sun, Oct 19, 2014 at 10:10 PM, Scott Lewis <slewis@xxxxxxxxxxxxx
<mailto:slewis@xxxxxxxxxxxxx>> wrote:

    Oops.  Forgot to add

    [1]
    http://wiki.eclipse.org/Tutorial:_Building_your_first_Asynchronous_OSGi_Remote_Service
    [2] http://wiki.eclipse.org/ECF/Asynchronous_Remote_Services



On 10/19/2014 1:05 PM, Scott Lewis wrote:
On 10/19/2014 11:18 AM, Wim Jongman wrote:
    I guess I forgot what the question was ;)

    On Sun, Oct 19, 2014 at 3:42 PM, Wim Jongman
    <wim.jongman@xxxxxxxxx <mailto:wim.jongman@xxxxxxxxx>> wrote:

        Hi,

        Suppose I want to stream data from a remote service to
        myself. I am now thinking that this is polling only.  I call
        the remote service to get any available data. However, if the
        remote service does not have data this leads to either
        locking (the remote service waits until data comes available)
        or unwanted network traffic (the remote service says "I have
        nothing"). I can use Async calls but this still requires me
        to poll the Future to find out if there is data available.

        To continuously push data from the remote server when it
        becomes available I need to "reverse" e.g. I have to make a
        service available that the data provider can consume and it
        can then push whenever new data is ready without me having to
        poll (either through a Future or any other primitive way).

        I wish I could consume a remote service and then set a
        callback that it could call whenever data is available.


Yes. A few thoughts/ideas here below. I wish I could say to you that your use case is trivially supported with OSGi Remote Services (any implementation...or specifically ECF's), but in truth I think that at the root is a slight mismatch between the 'remote procedure call' (RPC) model underneath OSGi Remote Services and what you are looking to do.

    First, with ECF's support for CompletableFuture, it is possible to
    easy to define an asynchronous/non-blocking remote method
    invocation, e.g. see [1].  But this isn't exactly what you are
    looking for, at least partially because it only completes *once*.
    But it does provide what I think of as a very nice way to express
    asynchronous/non-blocking remote services.

    Let's say we have declared a service API that looks like this:

    void myMethodCall(String s, SomeCallback callback)

    with the behavior being that 'callback can/is'  called potentially
    many times.  An example is all the Eclipse methods that have
    IProgressMonitor as the final argument in the method call.

    The question is:  How to implement this remotely/in a distributed
    app?   As you know, it's possible to pass method arguments and
    return values either by reference or by value.   For most remote
    method calls passing by value is the norm (e.g. 's' argument in
    the above), but it is possible to implement remote references,
    allowing 'callback' instances/object references to be referred
    to/called from out of process (e.g. a remote service host).

    For example, Java's RMI is an implementation that has call by
    reference.   And it would be possible using RMI and/or other
    call-by-reference distribution to implement a provider that
    supports the above callback structure.  For example, an RMI-based
    provider could (pretty) easily be created for OSGi remote services.

    But call-by-reference can/does end up being very complex in the
    real (distributed) world...for a number of reasons.  One reason is
    that unlike local/memory object references, remote references
    frequently fail and when they do it's often unclear what should
    happen.   Another reason is that remote references make garbage
    collection particularly complex (since not all object references
    are in local memory).   A third reason is that the network can
    introduce blocking that is problematic (for example, with Eclipse
    EFS if a remote file system is used the calls to the
    IProgressMonitor can block/be very slow...meaning that Eclipse
    can/does just 'freeze up' for long periods).   So in any event,
    the distributed systems world has kind of moved away from fully
    pass-by-reference for RPC systems,  because of these and other
    complexities.

    Another common way to implement this sort of desired behavior is
    with event-driven approaches/asynchronous messaging.   One example
    is the ECF filetransfer API...another is the ECF datashare and
    shared object APIs, as well as JMS, vert.x, akka, zeromq, mqtt,
    javagroups, etc...all of the asynchronous/messaging-based middleware.

    More than an implementation difference, I think what you are
    asking for implies a different sort of API than the 'normal' RPC,
    which I summarize as:    A thread calls a method synchronously and
    method returns a single value to same thread.   What I think you
    are really asking for is a simple way to express a behavior that
    isn't clearly expressed with a single method call (RPC), e.g. send
    a message and some data, return from this
    immediately/non-blocking, and receive one or more (remote)
    callbacks (with arbitrary data).    IMHO this is awkward for the
    remote procedure call/OSGi remote service model in general, and it
    gets really complicated when you factor in the distributed systems
    issues of (e.g.) partial failure (what should happen if one of n
    callbacks fails?).  And then things get *really* complex when we
    are talking about > 2 group members.

    There are of course other ways to deal with this sort of behavior
    e.g. callbacks can be 'client-exported' remote services., the
    'remote whiteboard pattern' like the chat example that you and
    Markus created, etc.   IMHO these approaches have their own
    complexities (e.g. how to dynamically associate the callback
    remote service instance with the service interface, how to do
    reliable discovery of the 'whole/associated thing', what happens
    when things fail because of network unreliability, etc.).  But
    even so, there could probably be more/better standardized API
    and/or meta-data specifically for doing the 'remote whiteboard
    pattern'.

    It's my hope that we and others can/could eventually create
    standardized API (e.g. MultiCompletableFuture?, remote streams,
    standardized event/messaging models, replicated data structures,
    or collectors, etc  :) that would help deal with this and similar
    use cases.  ECF has much of the underlying support for different
    approaches (e.g. asynchronous messaging, replicated data models,
    etc), meaning that we can easily try out/implement a variety of
    approaches.

    My $0.03.

    Scott




_______________________________________________ ecf-dev mailing list ecf-dev@xxxxxxxxxxx <mailto:ecf-dev@xxxxxxxxxxx> To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/ecf-dev


    _______________________________________________
    ecf-dev mailing list
    ecf-dev@xxxxxxxxxxx <mailto:ecf-dev@xxxxxxxxxxx>
    To change your delivery options, retrieve your password, or
    unsubscribe from this list, visit
    https://dev.eclipse.org/mailman/listinfo/ecf-dev



-- Hallvard Traetteberg, Associate Professor Dept. of Computer and Information Sciences (IDI) Norwegian University of Science and Technology (NTNU)