Re: [cross-project-issues-dev] p2 download statistics broken or biased?

[David M Williams <david_williams@xxxxxxxxxx>]

Martin, thanks for these summaries, and
questions. I don't know many answers, but I know one, and will offer some
speculations, and confirm one of your conclusions. 

> The
ID’s for WTP/WST stat counters are not exactly consistent in SR2, though
they were in SR1:[but one of the ones you site, is actually from TPTP,
not WTP]. 

Yes. I simplified some of our WTP promotion
scripts, so they'd not need so much attention or "customizing"
each release, build, or promotion. I may have simplified too much. But,
I am just explaining this change wasn't a surprise. 

> I still
find the statistics highly suspicious, especially because daily stats show
downloads of “Helios SR0” content up until today … which is unlikely
correct since everyone should be getting SR1 by now I’d assume.

Yes. suspicious. But, one explanation
of some of the anomalies _might_ be accounted for by builds. That is, some
people (Eclipse projects, or even adopters) might be building against a
specific version, that is not necessarily the latest (and, from my small
experience, this isn't always by some great design  ... they just
don't remember its important to build against the latest). Also, some anomalies
might be explainable if the artifact in question did not actually change
from SR0, to SR1. The way the composite artifacts repositories are currently
searched in order  (and, the way they are currently ordered in compositeArtifacts.xml
files), if an artifact of same id and version exists in SR0 and SR1, it
will be retrieved (and tracked) from SR0. 

> I
have since changed our contribution to reference the “release” site rather
than interim, but I don’t expect a re-aggregation to be realistic?  
 

Yes, I can confirm, that is correct.
No rebuilds for stats. (You know how it goes ... any rebuild is risky,
even if "nothing changed"). But, I applaud you asking here on
this list. Would you believe you are not alone? A few other people have
asked me this week if reasonable to request a respin for stat data values,
but when pressed, they all admitted it was not worth it.  I am not
sure what the root cause of several people forgetting or otherwise not
able to attend to stats until it was too late, but since more than one
person, seems some thought might be given to how to document a checklist,
or something, to help remind us all what needs to be done each release.
Any volunteers? 

I hope others can offer some better
explanations for some of the anomalies you've been seeing, and how to tell
if it is an anomaly in use/behavior or if data is being "lost"
somehow? 

Thanks again, 











From:      
 "Oberhuber, Martin"
<Martin.Oberhuber@xxxxxxxxxxxxx>
To:      
 Cross project issues
<cross-project-issues-dev@xxxxxxxxxxx>
Date:      
 02/24/2011 06:18 PM
Subject:    
   Re: [cross-project-issues-dev]
p2 download statistics broken        or  
     biased?
Sent by:    
   cross-project-issues-dev-bounces@xxxxxxxxxxx

---

Hi again,
 
Here are some final observations
about download stats.
 
·        
Helios stats are found
under /stats/releases/helios in the portal
·        
The number of stat counters
in Helios has gone down from 84 in Helios to 23 in SR1, then 27 in SR2:
moberhuber@build:~/x> unzip -p /home/data/httpd/download.eclipse.org/releases/helios/201006230900/aggregate/artifacts.jar
| grep download.stats | wc
·        
My TM project doesn’t
count stats in Helios SR1 or SR2 since we add stats only to “release”
sites but Helios has been aggregated from “interim” sites
o   I
have since changed our contribution to reference the “release” site rather
than interim, but I don’t expect a re-aggregation to be realistic?  
 
·        
The ID’s for WTP/WST
stat counters are not exactly consistent in SR2, though they were in SR1:
moberhuber@build:~/x> unzip -p /home/data/httpd/download.eclipse.org/releases/helios/201102250900/aggregate/artifacts.jar
| grep download.stats | egrep 'wtp|wst'
       
<property name='download.stats' value='org.eclipse.wst.web_ui.feature'/>
       
<property name='download.stats' value='org.eclipse.wst.xml_ui.feature'/>
       
<property name='download.stats' value='org.eclipse.wst.jsdt.feature'/>
       
<property name='download.stats' value='org.eclipse.tptp.wtp.feature_helios_sr2'/>
·        
The top 6 downloads from
Helios in terms of stats so far are:
/stats/releases/helios/org.eclipse.wst.xml_ui.feature_helios_SR1
196879
/stats/releases/helios/org.eclipse.wst.jsdt.feature_helios_SR1
158391
/stats/releases/helios/org.eclipse.wst.web_ui.feature_helios_SR1
150220
/stats/releases/helios/org.eclipse.emf.ecore
64884
/stats/releases/helios/org.eclipse.jst.enterprise_ui.feature_helios_SR1
58687
/stats/releases/helios/org.eclipse.cdt.core-5.2.0.201006141710
49689
 
I still find the statistics
highly suspicious, especially because daily stats show downloads of “Helios
SR0” content up until today … which is unlikely correct since everyone
should be getting SR1 by now I’d assume.
 
As I mentioned, I have fixed
the stat contribution for my TM project, re-aggregating this somehow would
be interesting for us but it’s not a must have.
 
Thanks,
Martin
--
Martin Oberhuber, SMTS
/ Product Architect – Development Tools, Wind
River
direct +43.662.457915.85  fax
+43.662.457915.6
 
From: cross-project-issues-dev-bounces@xxxxxxxxxxx
[mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx]
On Behalf Of Oberhuber, Martin
Sent: Thursday, February 24, 2011 11:28 PM
To: Cross project issues
Subject: Re: [cross-project-issues-dev] p2 download statistics broken
or biased?
 
Digging a little deeper and
looking at stats by day and by country, I see that
 
1. By country distribution
is as expected, with a clear maximum in the US
 
2. By day distribution shows
many “_tm320” downloads (which would indicate Helios non-SR1)of the examples
up until today.
I have no explanation for
this, since on our TM project site we ONLY distribute the _tm321 bits,
and Helios never had the “examples” bits.
 
Does this ring a bell for
anyone?
-        
 Could these “_tm320” downloads come from any mirrors that I do
not know?
-        
 Could these be generated by clients updating from 320 to 321, even
if only 321 bits are being downloaded?
-        
 Could it be a bug in the p2 code or in the stats counter at eclipse.org?
 
Ideas welcome…
 
Thanks,
Martin
--
Martin Oberhuber, SMTS
/ Product Architect – Development Tools, Wind
River
direct +43.662.457915.85  fax
+43.662.457915.6
 
From: cross-project-issues-dev-bounces@xxxxxxxxxxx
[mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx]
On Behalf Of Oberhuber, Martin
Sent: Thursday, February 24, 2011 9:14 PM
To: Cross project issues (cross-project-issues-dev@xxxxxxxxxxx)
Subject: [cross-project-issues-dev] p2 download statistics broken or
biased?
 
Hi all,
 
Preparing our TM Helios SR2 release, I
was also testing our p2 download stats.
Along those lines I retrieved the existing
stats for Helios and Helios SR1.
 
What I see is making me wonder whether
there’s any problem collecting the stats, or any automated download agents
leading to broken results:
 
·         Some
of our “esoteric” offerings like examples or unittests get very low numbers
as ZIP downloads but very high numbers from the p2 repo.
o   These
offerings are NOT available from Helios, but only our project site, and
relatively hard to get to.
·         Between
Helios Release and SR1, our ZIP downloads are approx. the same number,
but SR1 update site stats are less than half of the Release?
·         Our
SDK offering should “include” the runtime offering but yet the runtime
stats are lower than the SDK stats ?!?
 
It’s completely possible and likely that
I’ve been doing something wrong on the stats preparation, retrieval or
data interpretation… but I’m wondering if anybody else is seeing the
same? FWIW, I’m relatively certain that the results are not tainted by
Milestone downloads, these are for the release only.
 
FYI, full stats with more data are at http://dev.eclipse.org/mhonarc/lists/tm-dev/msg00015.html
 
Thanks,
Martin
--
Martin Oberhuber, SMTS
/ Product Architect – Development Tools, Wind
River
direct +43.662.457915.85  fax
+43.662.457915.6
 _______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev