[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [p2-dev] iterators, queries, and collectors....oh my!!

Susan and others,

I am going to try and re-write some of the complex collectors in terms of Queries.  If this is done, then you're right, we can simplify the collectors.

I have also been thinking about asynchronous data collection, and there are two things that may impede this:

1. Queries that cannot be done asynchronously (because of their implementation).  For example, if a query needs all the results to make decisions, (get the IU with the greatest version).
2. IQueryables that do not support asynchronous behaviour.  This happens, for example, if the multiple read - single write problem is not protected against.

What do you think about changing the signature of IQueryable to be:

public Collector query(Query query, Collector collector, IProgressMonitor monitor, boolean asynchronous);

(note the boolean). The asynchronous is a hint that says, if both the IQuerable and Query supports asynchronous, then the data should be retrieved asynchronously, and the query method should be "quick". If one of these do not support this behaviour , then the data is retrieved synchronously.

Note: From a user of this API not much changes (just the performance of the call).  In both cases, a caller will get the data back from the collector.  

Anyways, before any of this can be considered, the existing "complex" collectors should be simplified.  I will start on that.


On Tue, Dec 23, 2008 at 8:58 AM, Susan Franklin McCourt <susan_franklin@xxxxxxxxxx> wrote:

Hi, Ian, Thomas and others:

I've been commenting in various bugs on this topic, but I feel like my intent gets lost with comments sprinkled around bugs that are discussing very specific parts of the API.
I think we need to look collectively at the problem of remote queries, large data sets, filtering, etc.

Specifically, I think we need to define and implement queries for the routine tasks that are causing complexities in the collectors themselves, before defining a solution for async collectors.
This also helps more generally with the ability to optimize queries later on, as Thomas points out in https://bugs.eclipse.org/bugs/show_bug.cgi?id=256412.
I think having done this work, most collectors will simplify to doing nothing or possibly just wrapping the results (as the UI does). At that point, a pluggable acceptor might work (not sure).

Ideally, I'd hope that a collector would not need to operate in a special async mode.

>From the UI point of view, these queries are needed:
- latest version of an IU
- categories (this is really just a property query)
- IU's in a particular category (that also satisfy a UI visibility property query)
- IU's that are not categorized (that also satisfy a UI visiblity property query)
- IU's that pattern match a string

That last one would solve most of my pain with regards to lazy repo loading and filtering. If we can push pattern matching to the queryable side where it belongs, the UI would have no need to force a full iteration of all repos in special cases.


p2-dev mailing list

R. Ian Bull, PhD
Software Developer, EclipseSource