[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [p2-dev] iterators, queries, and collectors....oh my!!

Hi, Ian...

>1. Queries that cannot be done asynchronously (because of their implementation). For example, if a query needs all the results to make decisions, (get the IU with the greatest version).

I think this is where Thomas was asking for a finite set of non-opaque queries. If we have a LatestIUVersionQuery, then a naive implementation (SimpleMetadataRepository) could simply fetch everything and then check the versions, but a repository with server-side logic could actually do the work on the server side. Same with filtering. Imagine a PatternMatchQuery. If I query all IU's with "*rel" in the name, a naive implementation will fetch everything and pattern match the results, but a smarter repo would search on the server side and accumulate the results.

Another lifecycle thing to consider is how/when results are available. For LatestIUVersionQuery, any implementation couldn't possibly know the results until it had looked at everything. For PatternMatchQuery, the results are cumulative and it would be nice if some results could be made available incrementally so that the UI could show a subset of results faster. For example, I wouldn't want to make the user wait until every IU with "*rel" were found before showing anything (as we do today). And I wouldn't want to block on hasNext() if the queryable was still working, I'd want a way to know if the repo is still working on the next one, or is truly finished. For that reason, I'm not sure if Iterator is enough. In the UI I imagine a pattern where a job would periodically fetch the results and populate the view, and keep rescheduling the job until the collector was really done. Something like

public IStatus run() (IProgressMonitor monitor) {
while (collector.hasNext()) {
// collect for the UI
if (collector.mightHaveNext()) {
return somePartiallyDoneInfoStatus;
} else
return Status.OK;

For these reasons I don't see that the collector itself or the caller knows how fast or slow the query will be, as it's a combination of the queryable and the query itself that determine how the work is to proceed.

In the long run I think the UI should be prepared for the random performance characteristics of a query, it's just a matter of time whether we could do so for 3.5 (ironing out all these queryable issues first). Handling pattern matching in the query would remove the first major road block in this area.

One more thing - the nasty "uncategorized" query might be made to go away by us mandating that only categorized things are shown by the UI when working in category mode (vs. assembling the uncategorized info). That's probably the best way to make that problem go away rather than making it work in a query. (See https://bugs.eclipse.org/bugs/show_bug.cgi?id=227675#c16).

Inactive hide details for "Ian Bull" <irbull@xxxxxxxxxxxxxxxxx>"Ian Bull" <irbull@xxxxxxxxxxxxxxxxx>

          "Ian Bull" <irbull@xxxxxxxxxxxxxxxxx>
          Sent by: p2-dev-bounces@xxxxxxxxxxx

          12/23/2008 10:13 AM
          Please respond to P2 developer discussions

To: "P2 developer discussions" <p2-dev@xxxxxxxxxxx>
Subject: Re: [p2-dev] iterators, queries, and collectors....oh my!!

Susan and others,

I am going to try and re-write some of the complex collectors in terms of Queries. If this is done, then you're right, we can simplify the collectors.

I have also been thinking about asynchronous data collection, and there are two things that may impede this:

1. Queries that cannot be done asynchronously (because of their implementation). For example, if a query needs all the results to make decisions, (get the IU with the greatest version).
2. IQueryables that do not support asynchronous behaviour. This happens, for example, if the multiple read - single write problem is not protected against.

What do you think about changing the signature of IQueryable to be:

public Collector query(Query query, Collector collector, IProgressMonitor monitor, boolean asynchronous);

(note the boolean). The asynchronous is a hint that says, if both the IQuerable and Query supports asynchronous, then the data should be retrieved asynchronously, and the query method should be "quick". If one of these do not support this behaviour , then the data is retrieved synchronously.

Note: From a user of this API not much changes (just the performance of the call). In both cases, a caller will get the data back from the collector.

Anyways, before any of this can be considered, the existing "complex" collectors should be simplified. I will start on that.


On Tue, Dec 23, 2008 at 8:58 AM, Susan Franklin McCourt <susan_franklin@xxxxxxxxxx> wrote:

R. Ian Bull, PhD
Software Developer, EclipseSource

p2-dev mailing list

GIF image

GIF image

GIF image