Sorry maybe I did not explain myself correctly.
I think we first need to agree what we want our web site to accomplish, then
we need to attach metrics. Do you think we have agreement on what we want our
web site to accomplish? I was just putting some suggestions forward. I agree
it is obviously not about conversions and counting downloads is not meaningful
by itself.
From: phoenix-dev-bounces@xxxxxxxxxxx
[mailto:phoenix-dev-bounces@xxxxxxxxxxx] On
Behalf Of Bjorn Freeman-Benson
Sent: Thursday, April 17, 2008
11:59 AM
To: For developers on the new
Eclipse.org website project.
Subject: [phoenix-dev] Re: First
steps towards a better eclipse.org website
The problem with vague
things like "objectives" / "promote ecosystem" /
"provide infrastructure" is that it leads to "he said / she
said" kinds of arguments about improving the website. My experience at
companies like Amazon is that numbers are much more accurate than gut feelings.
If we don't pick a good set of conversion measures to improve, then we're not
going to be able to improve the website - instead we'll end up with random
changes and arguments
about why they are good or bad.
In addition to some of
your ideas, I would suggest we could also measure.
1. Number of unique users to our site (new and returning users).
We want to encourage returning users.
Why? Don't we really want
to encouraging something else? Eclipse usage maybe? Click throughs to our
members? ... It seems to me that if we could find a way to achieve our goals
without serving any web pages at all, that would be perfect, right? So
just encouraging returning users is (to me) not a good measure of success.
1.
2. Length of time they spend on the site - Longer means
they are getting the useful information or they are struggling to find stuff
. This might be dependant on the pages they are visiting. For instance,
if they go to the download page it should be quick. If they go to
project pages (ie they are looking for information) they it might be longer. We
might want to think of use cases for this.
Again, I think this is
the wrong measure. We want to measure whether they are *getting* useful
information, not whether they are hanging around - for exactly the reason you
state: time on site could be good, could be bad and thus it's not a useful
optimization number.
1.
2. Number of pages visited.
Again, why? What
value is that providing to our members and our users? We want to be measuring
the end value, not the symptoms of that value or lack of value.
1.
2. Overall satisfaction with the site (yes we don’t measure
conversions but I would hope the goal of the site is to be a useful information
source.)
Ah, the holy grail of all
websites - I don't know of a way to measure that, do you? Hence the only
things we can measure are things like conversion (downloads, articles, mailing
lists, etc). I suppose we could have those annoying useless "user
satisfaction surveys" that pop-up and we all ignore (I say in jest).
There's a science to build a good website. The first step in science to figure
out what you want to measure.
- Bjorn