Bug 263600 - [repo] make progress reporting smoother - "contacting sites" shows no progress on first use
Summary: [repo] make progress reporting smoother - "contacting sites" shows no progres...
Status: CLOSED DUPLICATE of bug 285308
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.5   Edit
Hardware: Macintosh Mac OS X
: P3 normal (vote)
Target Milestone: 3.5   Edit
Assignee: Henrik Lindberg CLA
QA Contact:
URL:
Whiteboard:
Keywords: api, polish, usability
: 280744 (view as bug list)
Depends on:
Blocks: 285797
  Show dependency tree
 
Reported: 2009-02-04 03:28 EST by Neil Bartlett CLA
Modified: 2010-02-26 22:38 EST (History)
5 users (show)

See Also:


Attachments
Dialog showing 0% progress (27.35 KB, image/png)
2009-02-04 03:28 EST, Neil Bartlett CLA
no flags Details
Same dialog, with details expanded (35.10 KB, image/png)
2009-02-04 03:31 EST, Neil Bartlett CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Neil Bartlett CLA 2009-02-04 03:28:55 EST
Created attachment 124649 [details]
Dialog showing 0% progress

Just downloaded 3.5M5 and used P2 UI to install an update. The first time this is done, the "Contacting Software Sites" dialog appears to show zero progress for several minutes, leading me to suspect it had hung.

Actually clicking Details reveals that it is downloading content.jar. But the top-level dialog should not show 0% progress while this is done.
Comment 1 Neil Bartlett CLA 2009-02-04 03:31:21 EST
Created attachment 124650 [details]
Same dialog, with details expanded

Attached image of the same dialog with details expanded, showing that there is actually some progress being made.
Comment 2 Susan McCourt CLA 2009-02-04 12:41:24 EST
this sounds a lot like bug 252446, there may be some variant of that happening.  
Comment 3 Susan McCourt CLA 2009-02-23 12:34:44 EST
This is a recent regression.

This was the exact use case we fixed in bug 252446, so I know that the UI and the SubProgressMonitor were checked over.  I recall verifying at the time that Ganymede incrementally updated the progress monitor while downloading the file.  (the problem was in SubProgressMonitor).

My best guess here is that it's some kind of change on the repo, side.  DJ, could this be composite repos (was Ganymede a composite repo when this bug was reported?)
Comment 4 DJ Houghton CLA 2009-02-23 14:02:57 EST
Susan were you able to replicate this problem? I just tried it and it appeared to show progress ok, but granted it didn't take a long time because of our local mirror. I can try this at home to see what the story is.

I also visually looked at the composite repo code and didn't see any glaring errors in the progress monitoring.
Comment 5 Susan McCourt CLA 2009-02-23 16:57:46 EST
Yes, I can reproduce this.
As Neil says, you can expand the details and see the ganymede content.jar downloading, and there is no progress above. 

I'm really stabbing in the dark by mentioning composite repo (does ganymede use one?)...I was trying to figure out what could have changed to cause this to reappear.
Comment 6 Susan McCourt CLA 2009-04-30 01:19:34 EDT
I noticed today while installing releng tools from the I-build site that Install showed no progress and then was suddenly done.  However I had a very poor connection, so it's possible that it was just waiting.  but it seems like there's something fishy with composite repos and progress.
Comment 7 Henrik Lindberg CLA 2009-05-07 21:59:50 EDT
I tracked it down so far that I think it is a scaling issue. After having reported 100% progress on a content.jar of 500k, the progress bar has only moved about 1/10th of the way. Will continue tomorrow.
Comment 8 DJ Houghton CLA 2009-05-08 08:10:27 EDT
I came to similar conclusions. I was looking in the area of the ProgressStatistics class, if that helps.
Comment 9 Henrik Lindberg CLA 2009-05-08 13:22:13 EDT
(In reply to comment #8)
> I came to similar conclusions. I was looking in the area of the
> ProgressStatistics class, if that helps.
> 
I have gone through all of that code. And I even poked a 100% done to see what happened. It works as it is suposed to at the lowest level.  I found a couple of small things as calls are reaching transport - but I am not sure if they are of any consequence.

Basically, the problem is a scaling issue - for instance - the entire headroom is divided per repo, then per suffix, compressed or not etc, Suffixes alone divides the space in 6 slots. Since compressed jar is the first, it will consume 1/6th, and then be divided again - so it ends up being a 1/12 of the allocation. This corresponds with observations.

Now if there are 10 repositories to contact - and the first has all the work, it will then only get 1/10 of the total to begin with - or about 1% of the total length.

So, yeah - there will be a big jump from 1% to all done in this case. 

I will try to fix some of these - for instance when there are 6 choices to try, and only one has any real work - here all the remaining work should be allocated to the next trial, not 1/6th. But have a feeling it is going to get tricky - the lower level consumes their ticks (with monitor.done) even on a file not found. I am not intimatly famliar with submonitors inner working, but I think that the sequence:
sub = SubMonitor,concert(monitor,1000);
sub.newChild(100);
sub.setWorkRemaining(1000);

Will always consume 100.

Greatful for hints...
Comment 10 Henrik Lindberg CLA 2009-05-08 15:54:42 EDT
I hacked away to test what the behavior would be with code that looks like:

for( each attempt where one should succeed )
{
   submonitor.setWorkRemaining(1000);
   doDaDing(submonitor.newChild(1000));
}

Which works when no receiver of the monitor under doDaDing calls 'done' on the received monitor, as setWorkRemaining then gets to reallocate any unused headroom. It is ok for a 'doDaDing' to consume a few ticks before giving up.

The effect of changing this was that progress moved at least 2/3d across before the process was done.

I think a new and concise policy should be used throughout the code. I can think of:
a) Use of SubMonitor in method signatures, and use setWorkRemaining, newChild, and worked, never call done.
b) Use IProgressMonitor in method signatures, always convert to SubMonitor, when "done", a check should  be made if the monitor is an instance of submonitor, and if that is the case, do not call 'done'. 

Option a) works well when api is all p2 internal. Option b) works both internally, and for an api, enabling callers that are not using SubMonitor to still get civil behavior (although not benefiting from fully using progress bar headroom.
Comment 11 Henrik Lindberg CLA 2009-05-08 16:22:22 EDT
This snowballs and many files needs to be touched in several places, and involves rearranging intricate logic in some cases - I think the risk is too high.
Moving this to 3.6.

Comment 12 Henrik Lindberg CLA 2009-05-14 15:04:09 EDT
Modified the summary to better reflect that this is a general issue.
Comment 13 Susan McCourt CLA 2009-06-18 11:34:53 EDT
*** Bug 280744 has been marked as a duplicate of this bug. ***
Comment 14 Henrik Lindberg CLA 2010-01-28 18:56:44 EST
I know other work has been taking place on similar issues (progress reporting), and I wonder if this is still an issue.
Comment 15 Pascal Rapicault CLA 2010-02-26 22:38:35 EST

*** This bug has been marked as a duplicate of bug 258308 ***
Comment 16 Pascal Rapicault CLA 2010-02-26 22:38:50 EST

*** This bug has been marked as a duplicate of bug 285308 ***