Bjorn is not
subscribed to the PMC list. Here are his comments. We should use
dash-dev for future comments, I think.
Martin, Doug, (and the DSDP
PMC and dash-dev)
In answer to your email:
1. The
project dashboard has been totally changed recently, without any notice
to committers. I liked the old dashboard, and I had put a link to it on
the dsdp/tm homepage. So my #1 feedback to Bjorn is: if you are
planning any changes to publicly visible URLs, let committer know what
you're up to. Don't we want to have open
processes? I hate broken links like http://www.eclipse.org/projects/dashboard/dashboard_detail.php?project=dsdp.tm that
used to work fine.
The project dashboard was a
prototype (that's what the big red box warning on the page was about)
and was being misused by (among other people) the press. The data was
not valid. Rather than continue to tell people that, I just removed it.
It was a public beta. I waffle between two positions: providing public
betas and then changing them, or not providing any information at all.
We look forward to your input via Bugzilla and your help in writing the
dashboard code (it's all in the Project Dash CVS).
2. Projects should not
define the metrics themselves if they are
publicly visible. The dashboard gets unusable if it's not totally clear
what's visible. Projects could be enabled to use the DASH databases to
make their own computations and publish them on their own homepages.
But the common dashboard should work the same for all projects, or it
gets totally confusing.
3. I agree
that different
metrics would be useful in different phases
of the project, but there should be ONE common definition of what is
visible.
I think the best solution is
to come up with a community-defined single formula. The original
formula was a pseudo-random invention by me. Far better would be to
have the community experiment with the raw data and come up with a
consensus about the formula or formulas. The old prototype dashboard
provided raw data for such experiments, but there was no conversation
about the formula other than to say "it's wrong". I'm not sure what to
do here.
My next proposal is to allow projects to use project-info.xml to define
a formula and then to have dashboard pages that show all the projects
"the world as seen by BIRT", "the world as seen by DSDP", etc.
4. Yes,
being explicit
about the formulas used is important. I
don't think that SQL statements are sufficient. There should be some
plaintext explanation of what's visible on a report.
There was a whole page about
how the old dashboard was computed, and there were links from the
dashboard pages, and yet people didn't seem to find it. That page
explained the formula, the raw data, everything. The new
project-defined or community-defined formulas can do the same.
5. Regarding
the new metrics:
5a) dsdp.tm project is
missing totally.
5b) I liked
the metrics on mailinglist, newsgroup and bugzilla activity, I'm
missing those. I'm not sure that commits only is a good indicator.
We are working hard to recreate the code that
extracts that data. We would be happy to have your assistance. We agree
that commits alone is not a good measure.
- Bjorn