I actually think Mik has made some very
valid observations.
1) The ‘cost’ of voting on
EPIC is too low. We seem to be trading off quantity vs quality. In
the long term I think this will hurt us. Eric raised the point of
requiring registrations to do a vote; maybe this would be a ‘cost’.
I also believe registration raises the profiles of individuals (even
though they might be aliases) that take part in inappropriate behaviour. Another
alternative is that a vote that includes a well articulate comment (maybe
moderated) receives extra weight, maybe 10x votes?
2) We don’t factor in time. Opinions
will change over time and we don’t reflect it here. Maybe the home
page should reflect the previous 3 months but we also have a view for the past
12 months and since inception.
Ian
From: Mike Taylor
[mailto:mike_taylor@xxxxxxxxxxxxxxxxxx]
Sent: Monday, September 18, 2006 10:01 AM
To: The EPIC component of
Phoenix
Cc: 'Mik Kersten'
Subject: Re: [phoenix-epic-dev]
FW: bogus rankings on EPIC
I read with concern the message from Mik Kersten.
As all of you on the EPIC Council know, I am very concerned with the legitimacy
of the ranking system and am among those pushing most strongly for making it
better.
At Instantiations we have an explicit, written policy that prohibits
our employees from ranking either our own products or those of others on EPIC.
We do this although we are fully aware that it is common practice in the
industry for those with a vested interest to vote for their own products and
against competitive products. We do this because a) we believe our products can
win on their own merit, and b) we know how angry it makes us when a competitor
unjustly slams our ratings (an unfortunately regular occurrence).
That said, we have a couple of dozen employees and its entirely possible that
the trickle of votes Mik references came from our IP address. We vigilantly
watch for this and constantly reinforce our company policy...but violations of
that policy have certainly occurred. In every case where we could identify the
source we have appropriately disciplined the person responsible. We will
continue to do this and if Mik or anyone else can point us to an offender we
will enforce our company policy.
Instantiations' internal policies aside, I wonder what makes these
votes inherently "bad"? Are plug-in vendors officially
restricted from voting for their own products on EPIC? Not that I know of. What
alerted Mik to these votes? Maybe if we knew we could watch that data source
and use it to enforce our own company policies.
In Mik's data I count 17 votes over a 5 month period involving about a dozen
different products. Certainly not enough to swing ratings much if at all.
WindowBuilder, our highest volume product (and the only one in the EPIC top
10), has literally thousands of votes, so the one in the questionable data set
had no practical effect. Our other products in the data set have a smaller vote
count, but there are still not enough votes to affect the ratings in a
meaningful way.
I have to say that even given the many discussions we've had, and are
continuing to have, at the EPIC Council about improving the ranking system and
protecting it from those who would game it...these votes wouldn't make it onto
our radar screen. What the Council is trying to do is remove the possibility of
significant, illegitimate ranking. We have to rely on the community to alert us
to small, infrequent misuses of the ranking system, and for that I thank Mik.
I'm curious, why Mik's previous emails didn't get the visibility they deserve?
Instantiations would like to be alerted anytime any gaming is going on with our
listed products or if it is suspected that our employees are voting in an
unsavory manner.
Regards,
MikeT
PS. We would be perfectly happy if all the votes referenced by Mik were
removed from the database.
At 9:51 AM -0400 9/18/06, Ian Skerrett
wrote:
All,
Mik Kersten sent me an e-mail complaining about some of the rankings that have
been made on EPIC. I think we need to decide how we want to respond to
Mik's concerns.
Btw, Mik agreed for me to post his
original e-mail on the mailing list. Please copy him on any replies,
since he has not subscribed to this list.
Thanks
Ian
Ian Skerrett
Director of Marketing
Eclipse Foundation, Inc.
Tel: 613-224-9461 ext. 227
Fax: 613-224-5172
ian.skerrett@xxxxxxxxxxx
http://www.eclipse.org
Blog: http://ianskerrett.blogspot.com/
From: Mik
Kersten [mailto:beatmik@xxxxxxx]
Sent: September 15, 2006 2:38 PM
To: 'Ian Skerrett'
Subject: bogus rankings on EPIC
Hi Ian,
Could you give me the email address of someone
responsible for EPIC? I have submitted 3 email/web complaints of bogus
rankings trying to bring up Instantiations products and bump others near them
on the top-10 down. I still have received no response and the bogus
ranking is continuing, e.g.:
http://www.eclipseplugincentral.com/modules.php?op=modload&name=Web_Links&file=index&req=ShowRaterDetails&userid=&hostname=68.178.73.218
If these kinds of problems with anonymous rakings are
not addressed pro-actively by EPIC abuse like this is likely to continue, and
someone could blog about this and make EPIC look untrustworthy.
Mik
--
Mik
Kersten, http://kerstens.org/mik
Mylar Project Lead, http://eclipse.org/mylar
_______________________________________________
phoenix-epic-dev mailing list
phoenix-epic-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/phoenix-epic-dev
--
----------------------------------------
Mike Taylor
President and CEO
Instantiations, Inc.
Power Tools for Professional Software Developers
Voice: (503) 598-4911
mike_taylor@xxxxxxxxxxxxxxxxxx
http://www.instantiations.com