Bug 309247 - graduate org.eclipse.equinox.concurrent.future API
Summary: graduate org.eclipse.equinox.concurrent.future API
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Components (show other bugs)
Version: 3.6   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: Kepler M7   Edit
Assignee: Thomas Watson CLA
QA Contact:
URL:
Whiteboard:
Keywords: api
Depends on:
Blocks:
 
Reported: 2010-04-14 19:33 EDT by Scott Lewis CLA
Modified: 2013-04-30 13:44 EDT (History)
9 users (show)

See Also:


Attachments
patch for adding generics support (25.30 KB, patch)
2010-04-15 15:31 EDT, Gunnar Wagenknecht CLA
no flags Details | Diff
listenable future (enh), generics, version, build (33.70 KB, patch)
2010-08-24 13:06 EDT, Scott Lewis CLA
no flags Details | Diff
listenable future, generics, version change (34.92 KB, patch)
2010-08-24 17:45 EDT, Scott Lewis CLA
no flags Details | Diff
new proposed patch: git repo patch, generics, listenable future (82.84 KB, patch)
2013-02-12 22:58 EST, Scott Lewis CLA
no flags Details | Diff
zip of updated concurrent api: generics, listenable future (22.22 KB, application/octet-stream)
2013-02-14 15:13 EST, Scott Lewis CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Scott Lewis CLA 2010-04-14 19:33:36 EDT
The org.eclipse.equinox.concurrent.future API is currently marked as 'internal' because since it's original contribution has been considered in the 'provisional' state.  I would like to request the graduation of this API.

Bug 253777 has the original discussion/review that led to the development of the API.
Comment 1 Gunnar Wagenknecht CLA 2010-04-15 03:29:15 EDT
I wonder if it would be possible to include support for Generics into the API while still maintaining compatibility to lower Java versions?
Comment 2 Thomas Watson CLA 2010-04-15 09:25:09 EDT
It is after API freeze and I don't think we will be able to get this in for the last two weeks of M7 that are left.  Marking for 3.7.
Comment 3 Thomas Watson CLA 2010-04-15 09:26:12 EDT
(In reply to comment #1)
> I wonder if it would be possible to include support for Generics into the API
> while still maintaining compatibility to lower Java versions?

Yes, we are doing similar things in p2 APIs in Helios.  We plan to do more in the framework APIs for the next release of the specification.
Comment 4 Scott Lewis CLA 2010-04-15 10:08:10 EDT
(In reply to comment #1)
> I wonder if it would be possible to include support for Generics into the API
> while still maintaining compatibility to lower Java versions?

The lack of use of generics is only because of Equinox constraint for running on EE miminimum of CDC 1.1/Foundation 1.1.  If this can be dealt with, then use of generics can easily be introduced.
Comment 5 John Arthorne CLA 2010-04-15 10:42:14 EDT
(In reply to comment #4)
> The lack of use of generics is only because of Equinox constraint for running
> on EE miminimum of CDC 1.1/Foundation 1.1.  If this can be dealt with, then use
> of generics can easily be introduced.

It is possible to use generics today while maintaining Foundation 1.1 class libraries. All of the p2 API uses generics for example while running on F1.1. The -jsr14 target can be used to down-compile generics to a java 1.4 compliant class file format.
Comment 6 Scott Lewis CLA 2010-04-15 10:47:19 EDT
(In reply to comment #5)
> (In reply to comment #4)
> > The lack of use of generics is only because of Equinox constraint for running
> > on EE miminimum of CDC 1.1/Foundation 1.1.  If this can be dealt with, then use
> > of generics can easily be introduced.
> 
> It is possible to use generics today while maintaining Foundation 1.1 class
> libraries. All of the p2 API uses generics for example while running on F1.1.
> The -jsr14 target can be used to down-compile generics to a java 1.4 compliant
> class file format.


Ok...so Gunnar if you want to propose patches to introduce generics into the IFuture API then we'll do so.  Since it's still provisional I believe we can do so right away.

And John I'm not sure what's necessary to properly build things under these conditions...is it just adding -jsr14 to the 1.5+ compiler?  or is there something else that has to be available at compile and/or run time?
Comment 7 Jeff McAffer CLA 2010-04-15 10:56:56 EDT
Please see http://wiki.eclipse.org/Provisional_API_Guidelines  In particular, the section about after the API freeze.
Comment 8 Scott Lewis CLA 2010-04-15 11:17:38 EDT
(In reply to comment #7)
> Please see http://wiki.eclipse.org/Provisional_API_Guidelines  In particular,
> the section about after the API freeze.

So are you saying that because of 

"After the API freeze, there is really no such thing as "provisional API" 

that this API cannot be genericized (i.e. adding generics) until after Helios...because Equinox is under API freeze?

This seems overly conservative and more than a little strange, since

1) nothing can be graduated until after the release
2) adding generics support is not your typical API change...as it
doesn't has limited runtime implications.
3) under the Changing Provisional APIs section it also says:

"Technically, a provisional API can change arbitrarily or be removed at any time without notice"

obviously that's not what we're proposing here (changing arbitrarily or removing)...but rather simply adding generics support to IFuture.

So again, this seems overly conservative to me...given this API's provisional status and the change that's being contemplated (i.e. adding generics support).
Comment 9 Thomas Watson CLA 2010-04-15 11:30:32 EDT
(In reply to comment #8)
> So again, this seems overly conservative to me...given this API's provisional
> status and the change that's being contemplated (i.e. adding generics support).

We are getting really close to M7, to me that implicitly means we get more and more conservative.  M7 is considered our first release candidate.  We should not be comfortable with changing (even provisional API) that close to a release candidate.  I don't think generics qualify as a must have.  And I also do not think adding generics is a trivial piece of work.
Comment 10 Scott Lewis CLA 2010-04-15 11:49:45 EDT
(In reply to comment #9)
> (In reply to comment #8)
> > So again, this seems overly conservative to me...given this API's provisional
> > status and the change that's being contemplated (i.e. adding generics support).
> 
> We are getting really close to M7, to me that implicitly means we get more and
> more conservative.  M7 is considered our first release candidate.  We should
> not be comfortable with changing (even provisional API) that close to a release
> candidate.  I don't think generics qualify as a must have.  

It seems that at least some in the consumer community disagree with this judgment Thomas.

>And I also do not
> think adding generics is a trivial piece of work.

So even though still provisional, we can't enhance it in response to specific, direct community input...even in ways that...although perhaps not trivial...are not in the least disruptive to the community (the same community that's asking for the change btw).  

That, to me, is *overly* conservative, and is a good way to prevent adoption...and stifle consumer-focused innovation at the platform level.

Also, I would ask:  why do the same freeze rules apply to both provisional and non-provisional API?
Comment 11 Thomas Watson CLA 2010-04-15 12:14:06 EDT
(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > So again, this seems overly conservative to me...given this API's provisional
> > > status and the change that's being contemplated (i.e. adding generics support).
> > 
> > We are getting really close to M7, to me that implicitly means we get more and
> > more conservative.  M7 is considered our first release candidate.  We should
> > not be comfortable with changing (even provisional API) that close to a release
> > candidate.  I don't think generics qualify as a must have.  
> 
> It seems that at least some in the consumer community disagree with this
> judgment Thomas.
> 
Hmmm... 

(In reply to comment #1)
> I wonder if it would be possible to include support for Generics into the API
> while still maintaining compatibility to lower Java versions?


I don't see how I can interpret this quote from comment 1 as something that is a must have, cannot live without.


> >And I also do not
> > think adding generics is a trivial piece of work.
> 
> So even though still provisional, we can't enhance it in response to specific,
> direct community input...even in ways that...although perhaps not trivial...are
> not in the least disruptive to the community (the same community that's asking
> for the change btw).  
> 
> That, to me, is *overly* conservative, and is a good way to prevent
> adoption...and stifle consumer-focused innovation at the platform level.

hmmm, other discussion on the mailing list seemed to indicate the current consumers are perfectly happy with the current shape and want it graduated in its current form.  Now we are saying that the lack of generics is going to prevent adoption and stifle innovation.  Really?  I cannot believe that is true.

> 
> Also, I would ask:  why do the same freeze rules apply to both provisional and
> non-provisional API?

Rampdown, rampdown, rampdown.  Did I mention rampdown.
Comment 12 Scott Lewis CLA 2010-04-15 12:29:02 EDT
(In reply to comment #11)
> 
> I don't see how I can interpret this quote from comment 1 as something that is
> a must have, cannot live without.

It's been brought up before by Gunnar and others...e.g. the (extended) discussion on bug 253777.  And it's been brought up directly with me before in other contexts (blogs, direct conversation, emails, etc) over the past year.

<stuff deleted>
> > 
> > That, to me, is *overly* conservative, and is a good way to prevent
> > adoption...and stifle consumer-focused innovation at the platform level.
> 
> hmmm, other discussion on the mailing list seemed to indicate the current
> consumers are perfectly happy with the current shape and want it graduated in
> its current form.  Now we are saying that the lack of generics is going to
> prevent adoption and stifle innovation.  Really?  I cannot believe that is
> true.

I'm the messenger reporting what I hear from the community.  ECF is a consumer of the future/concurrent work so far (although of course I don't know who others are), and I/we are able to use it as is/without generics.  I'm not personally a huge fan of generics, although I appreciate their usefulness in certain contexts.  

But within ECF team there have been folks that have asked repeatedly about generics support, and outside of ECF project there have been folks (through various channels) that have indicated directly to me that the lack of generics (i.e. IFuture.get()) makes the API much less attractive (Gunnar and others).

> 
> > 
> > Also, I would ask:  why do the same freeze rules apply to both provisional and
> > non-provisional API?
> 
> Rampdown, rampdown, rampdown.  Did I mention rampdown.


Yes, you did :).  But I why is the rampdown the same for provisional and non-provisional?...when their needs (for change vs. stability) are so obviously different?
Comment 13 John Arthorne CLA 2010-04-15 12:44:14 EDT
(In reply to comment #7)
> Please see http://wiki.eclipse.org/Provisional_API_Guidelines  In particular,
> the section about after the API freeze.

My reading of this document is that provisional API is not subject to freeze since it is equivalent to any other internal code after the API freeze. While graduating the API after the API freeze *is* an API change, changes to the provisional API itself are not. For example it says, "Clients ... cannot make any assumptions about the state of any provisional API between any two non-release builds."

However, Jeff's proposed new API Guidelines [2], are different, and say "After the API freeze, code designated as API and provisional API will remain binary backward compatible except by prior notification and approval of the relevant PMC and sufficient notice to the community."

Based on the fact that we're not following the package naming rules in the original guidelines, it looks like the Equinox project is actually using [2] as its API guidelines. At the least we need to make it clear which guidelines Equinox is using.


[2] http://wiki.eclipse.org/Provisional_API_Guidelines_Update_Proposal
Comment 14 Jeff McAffer CLA 2010-04-15 14:00:07 EDT
Sigh.  Thanks John, I put in the wrong link.  Sorry for the confusion.  Equinox should be following the updated guidelines (I seem to recall a vote on that in the not so distant past).

My general interpretation is that provisional API is something we want people to use. We are not committing to it for the long term.  However, in order to facilitate people doing their stuff, we ARE freezing it for the release along with the normal API.

Of course, this is a policy and is not written in stone. It is there as a reflection of what we the community have decided to do. I tis not Tom or me or anyone else.  It is US who have choosen to operate this way. If a case can be made that that change is safe, valuable, etc then such a change will be considered like any other.
Comment 15 Jeff McAffer CLA 2010-04-15 14:31:49 EDT
The relevant part of the guideline is:

"Note that while there are no restrictions on changing internal (aka non-API) code before or after the API freeze, it is strongly recommended that provisional API not be changed after the API freeze as the expectation is that there will actually be external consumers. Similarly, changing provisional API in a maintenance release is strongly discouraged as it will break the desired early adopters."

So, basically we are freezing the provisional stuff with the API as "strong" motivations must be present to drive change after that point.
Comment 16 Gunnar Wagenknecht CLA 2010-04-15 14:38:24 EDT
Just to clarify, I raised the question in comment #1 with an upcoming graduation in mind. I'd like to see this addressed before graduation. But it's not something which I consider urgent enough to justify possible instabilities in 3.6 builds. It's good to defer this to an early 3.7 milestone.
Comment 17 Scott Lewis CLA 2010-04-15 14:41:13 EDT
(In reply to comment #14)
<stuff deleted>
> 
> My general interpretation is that provisional API is something we want people
> to use. We are not committing to it for the long term.  However, in order to
> facilitate people doing their stuff, we ARE freezing it for the release along
> with the normal API.
> 
> Of course, this is a policy and is not written in stone. It is there as a
> reflection of what we the community have decided to do. I tis not Tom or me or
> anyone else.  It is US who have choosen to operate this way. If a case can be
> made that that change is safe, valuable, etc then such a change will be
> considered like any other.


Then I strongly suggest a change be made to this policy...specifically the
policy of treating provisional and non-provisional API as the same WRT the API
freeze.

Although I would guess that not many realize it (excepting all on this bug),
this policy can/will/does slow down change...and therefore it can/will/does
prevent both innovation and adoption of new things at the platform level.  

I would assert that for *any* new API to get long-term adoption, it has to go
through some process of usage...and then modification/refinement in response to
community input...via timely response from the API designers and implementers. 
In today's environment, without such flexiblity consumers will turn
elsewhere to meet their sw platform needs.

Of course there is/will be other parts of Equinox (and any other EF projects),
where the API is established and perhaps even ultimately
standardized...naturally I'm not suggesting that those APIs (and the API
freeze) should have the same rules as new/provisional APIs.  But it's my understanding that that's why we have the concept of provisional/non-provisional APIs...in order to recognize the fact that some APIs are stable (and should be subject to freeze) and others are provisional/not stable (and therefore should have different rules that recognize that fact).  

I'm again *not* saying that having 'different' rules for provisional API means
'no rules'...that would also not be great for consumers.  But I think that the
uniform application of these freeze rules for provisional API is indeed overly
conservative, and overly focused on the needs of existing consumers of Equinox
(which favor stability) over needs of new consumers (which favor
innovation/new).

(btw, by 'community' above I mean the actual consuming community...which is
quite different from the PMC, the Board, or the EF company membership, or even
the EF committers).
Comment 18 Gunnar Wagenknecht CLA 2010-04-15 15:31:46 EDT
Created attachment 165007 [details]
patch for adding generics support

BTW, this is how it could look like. There are some glitches with the default implementations that this patch does not cover correctly. Also, it's just some thoughts ... nothing seriously tested.
Comment 19 Jeff McAffer CLA 2010-04-15 17:26:55 EDT
re comment 17
Provisional stuff is already considered differently wrt API lifecycle.
API : Must be binary compatible going forward forever or until deprecated or next major release
Provisional : Good for this release (and service releases) but may change in the next minor release
Internal : all bets are off

As for the freeze at M6 of a given release, this captures a narrow gap in the overall evolution of an API where the community as a whole is ramping down development in the interest of getting a coherent and working release out the door.  With the idea that we want people to adopt provisional APIs to try them and give feedback, we need to encourage and support them in their adoption. To support that we need to lock down at some point so they can get on with their development. We can be a little more liberal with changes to Provisional API after the freeze perhaps but still must be aware of the risk/reward equation and the ripples caused.

In this particular case, it seem that the function is there and working and being used.  Yes, it is marked as provisional but there are no problems per se and the package names are in what would be their final form (i.e., no internal).  The addition of generics is interesting but not AFAICT causing bugs or preventing adoption.
Comment 20 Scott Lewis CLA 2010-04-15 17:39:00 EDT
(In reply to comment #19)
> re comment 17
> Provisional stuff is already considered differently wrt API lifecycle.
> API : Must be binary compatible going forward forever or until deprecated or
> next major release
> Provisional : Good for this release (and service releases) but may change in
> the next minor release
> Internal : all bets are off

Yeah, I'm aware of this.

> 
> As for the freeze at M6 of a given release, this captures a narrow gap in the
> overall evolution of an API where the community as a whole is ramping down
> development in the interest of getting a coherent and working release out the
> door.  With the idea that we want people to adopt provisional APIs to try them
> and give feedback, we need to encourage and support them in their adoption. To
> support that we need to lock down at some point so they can get on with their
> development. 


Fine, but what good does early adoption and feedback do if you can't respond to the feedback by improving the API?...at least after the freeze date (which applies to non-provisional and provisional the same...this is the policy that I believe is too conservative).


>We can be a little more liberal with changes to Provisional API
> after the freeze perhaps but still must be aware of the risk/reward equation
> and the ripples caused.

Of course.  But as I tried to say, 'different rules' != 'no rules'.

> 
> In this particular case, it seem that the function is there and working and
> being used.  Yes, it is marked as provisional but there are no problems per se
> and the package names are in what would be their final form (i.e., no
> internal).  The addition of generics is interesting but not AFAICT causing bugs
> or preventing adoption.

My point was that for some, the use of Object rather than generics makes an API harder to use and understand.  That's what people have told me.  Like I said, I don't happen to share all of that perspective, but I'm not representing only my own design/API sensibilities.  Rather I'm trying to respond to what the community seems to be saying.
Comment 21 Jeff McAffer CLA 2010-04-15 20:38:05 EDT
(In reply to comment #20)
> Fine, but what good does early adoption and feedback do if you can't respond to
> the feedback by improving the API?...at least after the freeze date (which
> applies to non-provisional and provisional the same...this is the policy that I
> believe is too conservative).

Fair enough. For next release please surface your opinions when the rampdown policy is put in place so the community can discuss and expectations set.

> My point was that for some, the use of Object rather than generics makes an API
> harder to use and understand.  That's what people have told me.  Like I said, I
> don't happen to share all of that perspective, but I'm not representing only my
> own design/API sensibilities.  Rather I'm trying to respond to what the
> community seems to be saying.

I don't think anyone is arguing against generics here. The challenge is changing things at this point. If someone can make a proposal that enumerates the risks (and is relatively risk free) and is high value then it certainly would be considered just like any other API change.
Comment 22 Scott Lewis CLA 2010-04-16 18:17:33 EDT
(In reply to comment #18)
> Created an attachment (id=165007) [details]
> patch for adding generics support
> 
> BTW, this is how it could look like. There are some glitches with the default
> implementations that this patch does not cover correctly. Also, it's just some
> thoughts ... nothing seriously tested.

The ideas look fine/sound to me...at least upon casual observation.  What Gunnar has proposed is what I've been contemplating WRT generics for some time now.

I believe the changes are quite safe (I don't believe they would actually affect any existing clients *at all*, but I'm a little rusty on what generics + the jsr14 switch actually do with the byte code...if anything).  Any thought/comments on this John, or Jeff, or someone that's more familiar with the details of the generics support in compiler?

In any event, I guess since the freeze is on we'll revisit this in 2.5 months.
Comment 23 Scott Lewis CLA 2010-04-16 18:29:47 EDT
(In reply to comment #5)
> (In reply to comment #4)
> > The lack of use of generics is only because of Equinox constraint for running
> > on EE miminimum of CDC 1.1/Foundation 1.1.  If this can be dealt with, then use
> > of generics can easily be introduced.
> 
> It is possible to use generics today while maintaining Foundation 1.1 class
> libraries. All of the p2 API uses generics for example while running on F1.1.
> The -jsr14 target can be used to down-compile generics to a java 1.4 compliant
> class file format.

John I'm not completely clear on how this is done.  Is this just setting the -target jsr14 in the javac compiler options?
Comment 24 John Arthorne CLA 2010-04-17 21:34:15 EDT
(In reply to comment #23)
> John I'm not completely clear on how this is done.  Is this just setting the
> -target jsr14 in the javac compiler options?

To have it down-compile in the IDE, you need to edit .settings/org.eclipse.jdt.core.prefs, and set the following property:

  org.eclipse.jdt.core.compiler.codegen.targetPlatform=jsr14

For more details, including setting up to run with PDE Build, see:

http://wiki.eclipse.org/Equinox_p2_down_compilation
Comment 25 John Arthorne CLA 2010-04-17 21:57:40 EDT
(In reply to comment #22)
> I believe the changes are quite safe (I don't believe they would actually
> affect any existing clients *at all*, but I'm a little rusty on what generics +
> the jsr14 switch actually do with the byte code...if anything).  Any
> thought/comments on this John, or Jeff, or someone that's more familiar with
> the details of the generics support in compiler?

Generics are erased by the compiler so they don't have any effect on the byte code. So, from a binary compatibility viewpoint introducing generics has no effect. Once generics have been introduced, you can create source compatibility problems if you make further changes (add/remove generic types, change types of parameters, etc). So, you want to be careful that you get it right when the generics are first introduced (although since this is provisional it could be changed in a later release anyway).
Comment 26 Scott Lewis CLA 2010-04-18 22:54:39 EDT
(In reply to comment #25)
> (In reply to comment #22)
> > I believe the changes are quite safe (I don't believe they would actually
> > affect any existing clients *at all*, but I'm a little rusty on what generics +
> > the jsr14 switch actually do with the byte code...if anything).  Any
> > thought/comments on this John, or Jeff, or someone that's more familiar with
> > the details of the generics support in compiler?
> 
> Generics are erased by the compiler so they don't have any effect on the byte
> code. So, from a binary compatibility viewpoint introducing generics has no
> effect. Once generics have been introduced, you can create source compatibility
> problems if you make further changes (add/remove generic types, change types of
> parameters, etc). So, you want to be careful that you get it right when the
> generics are first introduced (although since this is provisional it could be
> changed in a later release anyway).


Given that there is no affect on the byte code (minimal/no risk of breaking clients), and little effect on the source (given some further review of Gunnar's attachment/proposal), perhaps we should apply for a waiver...however that's done.
Comment 27 Scott Lewis CLA 2010-08-24 13:06:02 EDT
Created attachment 177349 [details]
listenable future (enh), generics, version, build

This patch adds:

1) IListenableFuture interface.  This interface and supporting implementation allow an IProgressRunnable to be executed upon completion of a future operation.  This supports the use case of having clients be called back (i.e. code block executed), upon completion of an asynchronous operation.  Several people have expressed desire for these semantics, and so they are now present.  See IListenableFuture.

2) The support for generics provided in attachment 165007 [details] have been incorporated into this patch.  Thanks to Gunnar for providing the patch and adding the generics support.

3) The manifest has been updated to increase the version to 1.1.0 (from 1.0.100).  The EE changed from jre 1.3 to 1.4 (now has 1.5, cdc 1.0, 1.4). The exported package version was changed from 1.0 to 1.1.  If these version changes are not appropriate then they can be downshifted, but they seemed appropriate given the enhancements/changes, etc.

4) The build.properties and .settings files have been updated to have the compile target be 1.4, while having the source compatibility be 1.5.  This allows people to use generics support on 1.5 runtimes, but still use 1.4 as well.
Comment 28 Scott Lewis CLA 2010-08-24 17:45:13 EDT
Created attachment 177370 [details]
listenable future, generics, version change

This is an updated patch with small fixes/changes.  See comment 27
Comment 29 John Arthorne CLA 2010-09-27 17:21:43 EDT
Scott, I did explore using equinox.concurrent in e4 but found some missing pieces in the API. I'm not asking for any changes and have no current plans to use it, but here are some things I ran into, FWIW:

1) The IExecutor had no mechanism to influence scheduling. For example, run now, run later, run repeatedly, etc.

2) Passing a progress monitor into the executor felt wrong. Typically in Eclipse APIs only blocking operations take progress monitors, and #execute typically does not block. The caller will usually create some progress reporting widget, but since they don't know when the task will run, they don't know when to create and display that widget to their user. In the end this means no existing IProgressMonitor implementations work properly here. The client ends up needing to pass in some wrapper that hooks beginTask/endTask to create/destroy the concrete progress widget.

3) There is no way to hook a callback to know when a task has finished executing. To do this, you need to poll the IFuture in a loop until a result comes out, which defaults the value of futures. It felt like there needed to be something like this:

IFuture f = executor.execute(task);
f.onDone(someOtherTask);

4) IFuture#get is a blocking call, but there is no mechanism for the caller to obtain progress or request cancelation.

When I played around with adding some of these things, I found I was just converging with the Job API which obviously is not the goal here. This is probably just my bias with respect to level of functionality I was looking for when using it from the UI. For simpler use cases or lower level libraries this simpler API is probably more appropriate.

As a more general point I'm a bit concerned that the distinction between java.util.concurrent and org.eclipse.equinox.concurrent will be confusing for clients. The API of IExecutor and IFuture are nearly identical to the java.util versions, with some subtle differences. If a client wanted to use futures in their own API I suspect they would be tempted to just use the Java one that is functionally similar and more likely to interoperate with standard Java libraries. This is becoming a more viable option as there are less and less libraries out there that run on JRE < Java SE 5.
Comment 30 Scott Lewis CLA 2010-09-27 18:32:27 EDT
(In reply to comment #29)
> Scott, I did explore using equinox.concurrent in e4 but found some missing
> pieces in the API. I'm not asking for any changes and have no current plans to
> use it, but here are some things I ran into, FWIW:
> 
> 1) The IExecutor had no mechanism to influence scheduling. For example, run
> now, run later, run repeatedly, etc.

Why is this needed...if you don't have any plans to use it then what is the use case you are thinking of here?

> 
> 2) Passing a progress monitor into the executor felt wrong. Typically in
> Eclipse APIs only blocking operations take progress monitors, and #execute
> typically does not block. The caller will usually create some progress
> reporting widget, but since they don't know when the task will run, they don't
> know when to create and display that widget to their user. In the end this
> means no existing IProgressMonitor implementations work properly here. The
> client ends up needing to pass in some wrapper that hooks beginTask/endTask to
> create/destroy the concrete progress widget.


I'm not sure I agree about your feeling, but if this is more than a matter of your taste then it could be changed.

> 
> 3) There is no way to hook a callback to know when a task has finished
> executing. To do this, you need to poll the IFuture in a loop until a result
> comes out, which defaults the value of futures. It felt like there needed to be
> something like this:
> 
> IFuture f = executor.execute(task);
> f.onDone(someOtherTask);


The support for a post-completion callback is why in I introduced the IListenableFuture.  This interface and associated implementation is in attachment 177370 [details].


> 
> 4) IFuture#get is a blocking call, but there is no mechanism for the caller to
> obtain progress or request cancelation.

I'm not sure I understand what would be desired here.  The get() calls are (like the Future in the jre concurrent API) are designed to block for result.  And there is already an IFuture.cancel().  

> 
> When I played around with adding some of these things, I found I was just
> converging with the Job API which obviously is not the goal here. This is
> probably just my bias with respect to level of functionality I was looking for
> when using it from the UI. For simpler use cases or lower level libraries this
> simpler API is probably more appropriate.

Right.  I've already created JobsExecutor...and this can/could be easily added to this API (but obviously it depends upon both jobs and this API...and so can't go in either one without introducing such a dependency).

> 
> As a more general point I'm a bit concerned that the distinction between
> java.util.concurrent and org.eclipse.equinox.concurrent will be confusing for
> clients. The API of IExecutor and IFuture are nearly identical to the java.util
> versions, with some subtle differences. 

All of the differences basically have to do with the integration of the equinox idioms here...i.e. IStatus (for more complex status instead of a simple exception), IProgressMonitor (for progress monitoring and cancellation that is familiar to Equinox users).

As you may recall (I can't remember the bug right now), these were introduced into the API at your request (I think)...in order to consistently use IStatus, IProgressMonitor.  The (long) discussion about this resulted in the introduction of the references to these concepts (IStatus and IProgressMonitor).


>If a client wanted to use futures in
> their own API I suspect they would be tempted to just use the Java one that is
> functionally similar and more likely to interoperate with standard Java
> libraries. This is becoming a more viable option as there are less and less
> libraries out there that run on JRE < Java SE 5.

Perhaps true...but the JRE libraries (obviously) don't have any integration with the IStatus mechanism (which is a useful, significant generalization for IFuture...allowing futures to represent multiple operations, with MultiStatus for example.

And I think that the similarity to the concurrent API has some nice advantages...for one, people familiar with that API will be able to easily use this one (there is little interference...if that's your concern).  It does use the IStatus and IProgressMonitor idioms...which is very useful for integration with the jobs API...but this doesn't make it terribly different from the concurrent API.  This one does run on JRE < 1.5...and it's actually quite possible that concurrent API can/should be used to implement this API.

So I contend that the value created by integration with IStatus, IProgressMonitor and jobs API (all things integral to Equinox), along with support for < 1.5 JREs for these core concepts justify this very small API.
Comment 31 Scott Lewis CLA 2010-09-27 19:01:55 EDT
(In reply to comment #30)
<stuff deleted>
> All of the differences basically have to do with the integration of the equinox
> idioms here...i.e. IStatus (for more complex status instead of a simple
> exception), IProgressMonitor (for progress monitoring and cancellation that is
> familiar to Equinox users).
> 
> As you may recall (I can't remember the bug right now), these were introduced
> into the API at your request (I think)...in order to consistently use IStatus,
> IProgressMonitor.  The (long) discussion about this resulted in the
> introduction of the references to these concepts (IStatus and
> IProgressMonitor).

For reference, see the API comments, contributions, and use cases on bug 253777.
Comment 32 John Arthorne CLA 2010-09-28 11:40:33 EDT
I'm not asking you to make any changes. I'm just saying I explored using equinox.concurrent as an abstraction to be used by the e4 UI rather than referencing a concrete library such as core.jobs directly. The UI had need for things like delayed scheduling, repeating scheduling, progress during a blocking get(), etc, so it didn't turn out to be useful. That's fine - it's a simpler API and I'm sure it is valuable for other clients (clearly it seems useful to you in ECF). I just thought I would report my experience with the API in case you found the feedback useful.
Comment 33 Scott Lewis CLA 2010-09-28 12:37:48 EDT
(In reply to comment #32)
> I'm not asking you to make any changes. I'm just saying I explored using
> equinox.concurrent as an abstraction to be used by the e4 UI rather than
> referencing a concrete library such as core.jobs directly. The UI had need for
> things like delayed scheduling, repeating scheduling, progress during a
> blocking get(), etc, so it didn't turn out to be useful. That's fine - it's a
> simpler API and I'm sure it is valuable for other clients (clearly it seems
> useful to you in ECF). I just thought I would report my experience with the API
> in case you found the feedback useful.


I'm not at all opposed to making additions to support other use cases (e.g. e4 UI)...and I believe scheduling additions (delayed and repeating, etc) would/could be easily added...either to IExecutor or to some new interface.  But in my own/ECF project's usage this hasn't been needed (so far), so I've taken the general strategy of leaving it out for simplicity/size, etc.  

Since others have a better handle than I on the use cases/needs here...for e4 UI and perhaps other things, I would like some details about what would be desired...and ideally some proposed API additions/changes, etc. that would meet these desires.
Comment 34 Thomas Watson CLA 2011-06-08 11:29:04 EDT
Move all 3.8 bugs to Juno.
Comment 35 Scott Lewis CLA 2012-04-30 11:56:17 EDT
Hi Thomas.

Question:  Why is it necessary to delay this maturation beyond Juno?  There's still a pending patch from 1.5 years ago that hasn't yet been adopted...and extending things out further won't likely help move this forward any faster.  

How can moving it forward be done?  The concurrent API is integral to ECF's implementation of the remote services spec, and actually adds a great deal...support for asynchronous remote services.

Thanks.
Comment 36 Thomas Watson CLA 2012-04-30 13:45:09 EDT
(In reply to comment #35)
> Hi Thomas.
> 
> Question:  Why is it necessary to delay this maturation beyond Juno?  There's
> still a pending patch from 1.5 years ago that hasn't yet been adopted...and
> extending things out further won't likely help move this forward any faster.  
> 
> How can moving it forward be done?  The concurrent API is integral to ECF's
> implementation of the remote services spec, and actually adds a great
> deal...support for asynchronous remote services.
> 
> Thanks.

Sorry, I was doing a bunch of deferring of bugs since we are basically done with enhancements for Juno since M7 is this week and this was one of them.  But I think it is fine time to call the API 1.0 version done and  I am fine with removing the x-internal from the exported package since this "API" obviously has remained stable.  Unfortunately I don't have the time to do much more than that for Juno.

Can we move the other parts, generics etc out to kepler?
Comment 37 Scott Lewis CLA 2012-04-30 13:59:59 EDT
(In reply to comment #36)
<stuff deleted>
> 
> Sorry, I was doing a bunch of deferring of bugs since we are basically done
> with enhancements for Juno since M7 is this week and this was one of them.  But
> I think it is fine time to call the API 1.0 version done and  I am fine with
> removing the x-internal from the exported package since this "API" obviously
> has remained stable.  Unfortunately I don't have the time to do much more than
> that for Juno.
> 
> Can we move the other parts, generics etc out to kepler?

Is there any way to include the generics for Juno?  The generics support is pretty clear/obvious IMHO...i.e. specifying types for (e.g.) methods that currently return Objects, etc.

If generics can't make it into Juno, then I would prefer to keep the API provisional until at least generics could be added (along with any other API changes...e.g. those discussed in comments)...in order to simplify moving to a new API.

Another aspect to this...fwiw...is that there has been discussion...of late...on adding some asynchronous invocation methods (e.g. future) to the remote services part of OSGi spec (as there are plenty of use cases for asynchronous/non-blocking remote service invocation).
Comment 38 Dani Megert CLA 2013-01-08 03:16:56 EST
SR1 has shipped and SR2 is at the door. What's the state of this bug?
Comment 39 Scott Lewis CLA 2013-02-11 16:49:57 EST
(In reply to comment #38)
> SR1 has shipped and SR2 is at the door. What's the state of this bug?

I'm wondering the same thing...s if the Kepler target is going to be reached I believe something would have to begin pretty soon.

BTW...just so it's clear, as an Equinox incubator committer I will be willing to support/maintain this API, once released...assuming that fits the rules for the Equinox project.
Comment 40 Thomas Watson CLA 2013-02-12 08:21:14 EST
(In reply to comment #39)
> (In reply to comment #38)
> > SR1 has shipped and SR2 is at the door. What's the state of this bug?
> 
> I'm wondering the same thing...s if the Kepler target is going to be reached
> I believe something would have to begin pretty soon.

Yes, now is the time.

> 
> BTW...just so it's clear, as an Equinox incubator committer I will be
> willing to support/maintain this API, once released...assuming that fits the
> rules for the Equinox project.

Great, the bundle is already out of the incubator git repo, in rt.equinox.bundles repository.  All that is needed now is any finalizations you want to make Scott.  If you get me a patch I can work with you to get it released and also nominate you to do ongoing maintenance, if required.
Comment 41 Scott Lewis CLA 2013-02-12 13:55:03 EST
(In reply to comment #40)
> (In reply to comment #39)
> > (In reply to comment #38)
> > > SR1 has shipped and SR2 is at the door. What's the state of this bug?
> > 
> > I'm wondering the same thing...s if the Kepler target is going to be reached
> > I believe something would have to begin pretty soon.
> 
> Yes, now is the time.
> 
> > 
> > BTW...just so it's clear, as an Equinox incubator committer I will be
> > willing to support/maintain this API, once released...assuming that fits the
> > rules for the Equinox project.
> 
> Great, the bundle is already out of the incubator git repo, in
> rt.equinox.bundles repository.  All that is needed now is any finalizations
> you want to make Scott.  If you get me a patch I can work with you to get it
> released and also nominate you to do ongoing maintenance, if required.

Ok, sounds good.  I don't have any other changes...other than the ones in attachment 177370 [details] (which have Gunnar's generics additions from comment 18 in CVS patch format) but I will go over that patches again, create a new git patch, and attach the new patch to this bug.

Gunnar...if you have any desired changes WRT generics support and/or any others (changes to listenable future, etc)...please let all know via this bug over the next week.
Comment 42 Thomas Watson CLA 2013-02-12 16:55:31 EST
Lets finalize this in M6.
Comment 43 Scott Lewis CLA 2013-02-12 22:58:58 EST
Created attachment 226976 [details]
new proposed patch:  git repo patch, generics, listenable future

An updated git-based patch for the org.eclipse.equinox.concurrent plugin.  Adds support for generics, but maintains compile target of 1.4.  Adds IListenableFuture and implementation, to allow callbacks executed upon future completion.

Please review, and if changes are needed will make them.  Would appreciate review by Gunnar, if at all possible.
Comment 44 Thomas Watson CLA 2013-02-13 08:55:15 EST
Because of the issues with jsr14 and the Java 7 compiler we have been moving off of using it.  Would it be ok to target Java 5 for the concurrent bundle?
Comment 45 Scott Lewis CLA 2013-02-13 10:37:39 EST
(In reply to comment #44)
> Because of the issues with jsr14 and the Java 7 compiler we have been moving
> off of using it.  Would it be ok to target Java 5 for the concurrent bundle?

It's fine with me.
Comment 46 Thomas Watson CLA 2013-02-14 13:59:54 EST
(In reply to comment #45)
> (In reply to comment #44)
> > Because of the issues with jsr14 and the Java 7 compiler we have been moving
> > off of using it.  Would it be ok to target Java 5 for the concurrent bundle?
> 
> It's fine with me.

I was going to look to moving this up to J2SE-1.5 BREE and started with your patch, but I am having a hard time applying it to master.  Is this patch against master or some other branch you have you your own local repo?

The patch is attempting to modify files that do not exist such as AbstractListenableFuture, IListenableFuture, SingleOperationListenableFuture.
Comment 47 Scott Lewis CLA 2013-02-14 14:16:38 EST
(In reply to comment #46)
> (In reply to comment #45)
> > (In reply to comment #44)
> > > Because of the issues with jsr14 and the Java 7 compiler we have been moving
> > > off of using it.  Would it be ok to target Java 5 for the concurrent bundle?
> > 
> > It's fine with me.
> 
> I was going to look to moving this up to J2SE-1.5 BREE and started with your
> patch, but I am having a hard time applying it to master.  Is this patch
> against master or some other branch you have you your own local repo?

No, they are off of master.

> 
> The patch is attempting to modify files that do not exist such as
> AbstractListenableFuture, IListenableFuture, SingleOperationListenableFuture.

Hmmm.  Yes, these are new files.   I thought the patch I created would add them...since I added to local index, but obviously haven't committed or pushed.

If desired I could just create a zip of the entire new contents...but I thought using patch would be easier.  Just let me know what would be best.
Comment 48 Thomas Watson CLA 2013-02-14 14:39:55 EST
(In reply to comment #47)
> If desired I could just create a zip of the entire new contents...but I
> thought using patch would be easier.  Just let me know what would be best.

I preferred patch, but if you want to just zip up the project that is fine also.  Out of curiosity, how did you create the patch?
Comment 49 Scott Lewis CLA 2013-02-14 14:56:16 EST
(In reply to comment #48)
> (In reply to comment #47)
> > If desired I could just create a zip of the entire new contents...but I
> > thought using patch would be easier.  Just let me know what would be best.
> 
> I preferred patch, but if you want to just zip up the project that is fine
> also.  Out of curiosity, how did you create the patch?

egit 

ok, I'll attach zip shortly.
Comment 50 Scott Lewis CLA 2013-02-14 15:13:02 EST
Created attachment 227103 [details]
zip of updated concurrent api: generics, listenable future

Zip with entire contents of org.eclipse.equinox.concurrent bundle.  Includes updates as in description, also includes change to min 1.5 ee.
Comment 51 Thomas Watson CLA 2013-02-14 16:17:12 EST
Thanks Scott.  I updated the copyright dates and I updated the pom.xml to the correct version (so glad we have to maintain that in two spots now):

http://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/?id=ce9a763c246cc24ea504a41a1af624229a385c20

Leaving this open for review by you and Gunnar to make sure I did not mess anything up.
Comment 52 Scott Lewis CLA 2013-02-14 16:53:14 EST
(In reply to comment #51)
> Thanks Scott.  I updated the copyright dates and I updated the pom.xml to
> the correct version (so glad we have to maintain that in two spots now):
> 
> http://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/
> ?id=ce9a763c246cc24ea504a41a1af624229a385c20
> 
> Leaving this open for review by you and Gunnar to make sure I did not mess
> anything up.

Thanks Thomas.  Looks good to me.  +1 for resolving.  I've contacted Gunnar in a private email, so hopefully he will take a look soon.

One question:  when will this appear in a build (first in integration build, I assume)?
Comment 53 Thomas Watson CLA 2013-02-14 17:22:26 EST
Should happen in the nightly build tonight.  But yes, the first integration build with this will be next tuesday.
Comment 54 Scott Lewis CLA 2013-03-18 14:47:41 EDT
Hi Thomas.

It appears to me (from inspection of Kepler M6 binaries) that the new version of concurrent is in there now (M6).  Is that wrong?
Comment 55 Thomas Watson CLA 2013-03-18 15:00:05 EDT
(In reply to comment #54)
> Hi Thomas.
> 
> It appears to me (from inspection of Kepler M6 binaries) that the new
> version of concurrent is in there now (M6).  Is that wrong?

No you are correct, I was leaving open for review by Gunnar and give any last minute API changes that we may want before M7 is done.  If Gunnar is happy with the current state of the API then we can close this one out (but I was planning to harass Gunnar at EclipseCon to review) :)
Comment 56 Scott Lewis CLA 2013-03-18 15:05:20 EDT
(In reply to comment #55)
> (In reply to comment #54)
> > Hi Thomas.
> > 
> > It appears to me (from inspection of Kepler M6 binaries) that the new
> > version of concurrent is in there now (M6).  Is that wrong?
> 
> No you are correct, I was leaving open for review by Gunnar and give any
> last minute API changes that we may want before M7 is done.  If Gunnar is
> happy with the current state of the API then we can close this one out (but
> I was planning to harass Gunnar at EclipseCon to review) :)

Good idea.
Comment 57 Thomas Watson CLA 2013-04-18 08:53:33 EDT
I'm going to close as fixed.  Gunnar, please have a look at the final API and let us know if there are any issues.  We are getting close to the end for Kepler.
Comment 58 Scott Lewis CLA 2013-04-19 15:51:00 EDT
(In reply to comment #57)
> I'm going to close as fixed.  Gunnar, please have a look at the final API
> and let us know if there are any issues.  We are getting close to the end
> for Kepler.

Thanks Thomas.

If you wish, two new aspects of this API can/could be included in a new and noteworthy:

1) Support for generics (originally done by Gunnar)
2) Addition of IListenableFuture api and implementation (me)