I didn't make the connection between
the CBFuture class Martin proposed and Java 5's Future, so I didn't mean
to compare the job API to the java.util.concurrent API. I agree that java.util.concurrent
does a very good job of separating concerns and providing reusable interfaces/classes
that we could incorporate into the Job API and other Eclipse APIs. I know
you're already aware of it Pawel, but for others who are interested there
is an existing bug report about making use of java.util.concurrent API
in the jobs API: https://bugs.eclipse.org/bugs/show_bug.cgi?id=178278.
I've referenced this thread in the bug so that the information is available
when the day comes that we can use Java 5 there.
John
Pawel Piech <pawel.piech@xxxxxxxxxxxxx> Sent by: eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx
10/31/2008 12:53 PM
Please respond to
E4 developer list <eclipse-incubator-e4-dev@xxxxxxxxxxx>
To
E4 developer list <eclipse-incubator-e4-dev@xxxxxxxxxxx>
cc
Subject
Re: [eclipse-incubator-e4-dev] Asynchronous
Infrastructure (was: EFS,
ECF and asynchronous)
Hi John,
I don't think it's fair to compare just the *Future* interface to the entire
jobs API. Future is a single purpose interface, used to synchronously
access a result of an asynchronous operation. Jobs API includes,
in some form, additional features found in the concurrent package:
*Executor* - In the concurrent package this is another single-purpose interface
which abstracts the task of running an operation using some facility. In
jobs API, this facility is implemented as a thread pool, or a UI dispatch
loop when using UIJob.
*Callable* - It's basically a runnable with a return value. In jobs
API, this is the Job.run():IStatus method.
IMO, the killer feature of the concurrent package is that you can pick
and choose the elements to use as is, re-implement other elements, and
invent new elements as needed. With the jobs API the choice is either
all or nothing. Looking forward in e4, or even 3.x, I hope that Eclipse
can support the complete java concurrent package as a layer either above
or below the jobs API.
If it is above, then all that would be needed is an Executor implementation
which is a proxy to the current jobs API, which would simply execute Runnables
and Callables using Jobs. If it is below, which I think is the more
work/more gain option, then the jobs API becomes a collection of components:
- JobManager becomes an Executor which accepts any Runnable or Callable,
where progress monitor and a scheduling rule are optional.
- A Job becomes just an implementation of Callable<IStatus>.
- UIJob is replaced by a separate implementation of an Executor.
- etc.
Supporting the java concurrent package would only make the current jobs
api functionality more convenient and extensible. The larger goal
for e4 should be to make asynchronous programming in Eclipse easier and
safer. To that end we'd need additional standard APIs for things
like callbacks and realms, and more importantly we'd need good tooling
for validating and debugging asynchronous systems.
Cheers,
Pawel
John Arthorne wrote:
There doesn't seem to be much difference between the future construct you
describe and the Job API. You can attach listeners to jobs which seems
to be the same as your Callback mechanism. Future.waitFor() is the same
as Job.join(), and Future.get() is similar to Job.getResult(). I did actually
have futures in mind when designing the jobs API, with the job's "result"
being the payload returned from the asynchronous operation. I initially
made this result of type Object so clients could pass back whatever return
value they wanted. I then perhaps mistakenly switched the result type to
IStatus, thinking that clients could return sub-types of Status containing
any result object they wanted. This is why I specified almost nothing for
the return value of Job#run and Job#getResult, leaving it as a mechanism
for clients to communicate whatever they want back to the caller. In
reality it didn't end up being used this way, because people fell into
the common coding patterns around IStatus and just returned the usual OK/ERROR
results.
So, I'm wondering if there's something fundamental missing from Jobs that
makes these asynchronous coding patterns difficult, and is there some incremental
improvement we can make to Jobs to make it as expressive and useful as
your Future construct? If not, the org.eclipse.core.jobs bundle could
still perhaps be a home for such an API, since it obviously needs a backing
thread pool implementation with support for progress monitors, etc.
John
Martin Oberhuber wrote on 10/30/2008 04:47:02 PM:
> Hi Scott, Pawel and all,
>
> it looks like this Thread has long left the resources/EFS
> aspect of things, and moved to a more general discussion
> about infrastructure for asynchronous coding patterns.
>
> I'd thus like to make the discussion more general. We
> seem to agree that there needs to be some base infrastructure
> for asynchronous coding patterns, and (perhaps even more
> important) API Documentation for how to properly use that
> infrastructure. If this base infrastructure is unified,
> we all win.
>
> Thanks Scott for volunteering to offer your expertise
> as well as contributions. What could be the next steps
> towards making it happen? I'm assuming that the base
> infrastructure should be in Equinox. Is anyone from
> the Equinox team listening and could guide through
> their process for contribution?
>
> Assuming that Equinox is right, we should perhaps first
> find a proper place for this discussion; then argue
> about good infrastructure/patterns; these need to be
> backed by some actual usage somewhere. Right now, it
> looks like what we'll want is at least
>
> Future (aka RequestMonitor, AsyncExecutionResult)
> Callback (aka Listener)
> Status/MultiStatus (async variant)
> Executor/Queue/Realm (for posting async Runnables/Callbacks
> in a well-known Thread)
>
> along with some well-documented Exception types (cancellation,
> timeout) as well as Threading paradigm.
>
> How to proceed from here? Potential clients of async
> certainly include DD/DSF and ECF, perhaps Resources/EFS;
> who else is interested in infrastructure for async?
>
> Cheers,
> --
> Martin Oberhuber, Senior Member of Technical Staff, Wind River
> Target Management Project Lead, DSDP PMC Member
> http://www.eclipse.org/dsdp/tm
>
>
>
> > -----Original Message-----
> > From: eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx
> > [mailto:eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx]
On
> > Behalf Of Scott Lewis
> > Sent: Wednesday, October 29, 2008 7:06 PM
> > To: E4 developer list
> > Subject: Re: [eclipse-incubator-e4-dev] [resources] EFS, ECF
> > and asynchronous
> >
> > Hi Martin,
> >
> > Oberhuber, Martin wrote:
> > > Hi Scott,
> > >
> > > to me, Futures and Listeners don't need to be a contradiction.
> > >
> >
> > Before my further comments...I don't believe they are in
> > conflict either
> > (that is, both can be used in some cases as described by Martin).
I
> > guess I sort of presented them as exclusive, but I didn't
> > really mean to
> > have it be so.
> >
> > > What's more interesting to me, is how to deal with Progress.
> > > When a Progress Monitor already exists for the client, then
> > > using it makes a lot of sense even if the result is obtained
> > > asynchronously:
> > >
> > > final CBFuture<IFileStore[]> childrenF =
> > myFileStore.list(myProgress);
> > > childrenF.chain(new Callback() {
> > > public void onDone(IStatus result) {
> > > if (result.isOK()) {
> > > handleResult(childrenF.get());
> > > }
> > > };
> > > });
> > >
> > > I'm using class "CBFuture" as an "enhanced
Future" that allows
> > > registering Callbacks. Using a Callback style of handling
things,
> > > or CBFuture.waitFor() remains up to the client. Note that
I'm
> > > using a "chain()" method to indicate that the
Framework/Future could
> > > allow chaining multiple callbacks such that one is exeucuted
after
> > > the other. Also note how the callback retrieves the result
of
> > > computation from the Future, and not from the callback itself.
> > >
> >
> > I agree that the general issue of how to handle progress monitors
is
> > tricky. Although I accept your ideas above as a possible
> > solution, I'm
> > not sure whether this is the 'right' mechanism or not for 'remote
> > progress monitoring'. I've been thinking about this for
some
> > time, but
> > still don't feel like I have a good general solution for supporting
> > IProgressMonitor for remote procedures.
> >
> > > The problems that I have seen with callbacks in our products
> > > in the past are listed on
> > >
> > http://wiki.eclipse.org/E4/Pervasive_Themes#Becoming_More_Asynchronous
> > >
> > > * Much boilerplate code - Closures would be nice to avoid
explosion
> > > of anonymous inner classes, which could cause bloat
> > >
> > > * Need clarification on what thread and in what context
the
> > > callback will be called
> > >
> > > * When debugging, it is very hard to trace back the flow
of
> > > operation across multiple callback invocations. It
can even
> > > make debuging close to impossible unless some Tracing
> > > functionality for the callbacks is built into the
Framework
> > > (we ended up doing this in our commercial product).
> > >
> > > * Exception handling needs to be clarified. Java6 Future
only
> > > provides Future#isCanceled(), that's not enough since
the
> > > result of an operation might also be an exception.
I'm
> > > introducint "Istatus result" above but
that's also not
> > > optimal.
> > >
> >
> > I agree these are other issues...thanks.
> >
> >
> > > The synchronous variant needs more verbosity writing it
than
> > > one would expect, because cancellation and errors (exceptions)
> > > need to be handled, wrapped and potentially re-wrapped with
> > > Futures:
> > >
> > > final CBFuture<IFileStore[]> childrenF =
> > myFileStore.list(myProgress);
> > > try {
> > > handleResult(childrenF.get());
> > > } catch(CancellationException e) {
> > > throw new OperationCancelledException(e);
> > > } catch(ExecutionExeption e) {
> > > throw new CoreException(new Status(/*.blabla*/));
> > > }
> > >
> > > although that could perhaps be simplified if we declared
some
> > > Eclipse specific implementation of Future which throws the
> > > kinds of Exceptions that we already know (like CoreException
> > > embedding an Istatus) instead of the JRE's ExecutionException
> > > that's really alien to our current code.
> > >
> >
> > Yes, I agree that these are issues. I also agree that it
would be
> > useful to have Equinox-specific impls of Future (which is really
what
> > the IAsyncResult interface was meant to be and can/will
> > change to be if
> > desired). Further, I've recently also realized that there
> > also should
> > probably be something like remote impls of
> > IStatus/MultiStatus, as I've
> > been doing some remote mgmt interfaces (i.e. accessing and managing
a
> > remote processes' OSGi framework, p2, etc)...and it's clear
> > to me that
> > it is going to be very helpful to support the usage of
> > IStatus/Multistatus as return values, as well as exceptions in
remote
> > service access. I agree that Future/IAsyncResult as well
as
> > IStatus/Multistatus and exception types should be widely
> > available (i.e.
> > in Equinox rather than redone/available in many locations above
> > Equinox). We (ECF) are willing to contribute (and modify
as desired)
> > what we've done in this area (e.g. IAsyncResult+impl,
> > RemoteStatus/RemoteMultiStatus, exception types) as desired.
> >
> > Scott
> >
> >
> >
> > > Cheers,
> > > --
> > > Martin Oberhuber, Senior Member of Technical Staff, Wind
River
> > > Target Management Project Lead, DSDP PMC Member
> > > http://www.eclipse.org/dsdp/tm
> > > _______________________________________________
> > > eclipse-incubator-e4-dev mailing list
> > > eclipse-incubator-e4-dev@xxxxxxxxxxx
> > > https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
> > >
> >
> > _______________________________________________
> > eclipse-incubator-e4-dev mailing list
> > eclipse-incubator-e4-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
> >
> _______________________________________________
> eclipse-incubator-e4-dev mailing list
> eclipse-incubator-e4-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
_______________________________________________
eclipse-incubator-e4-dev mailing list eclipse-incubator-e4-dev@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
_______________________________________________
eclipse-incubator-e4-dev mailing list
eclipse-incubator-e4-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev