[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [phoenix-epic-dev] FW: bogus rankings on EPIC
|
Title: RE: [phoenix-epic-dev] FW: bogus rankings on
EPIC
I, in no way see this as an Instantiations issue.
It is very unfortunate that the example Mik forward might make it seem
that way but I don't think it was the intention. I know you
have a lot of passion about EPIC and want it to be a legitimate
resource for the Eclipse community.
Thanks, Ian, but I can't help but take it a bit personally.
Instantiations has been on the forefront of improving and policing the
rating system since before we transferred EPIC to the Foundation.
There have been numerous examples of unsavory voting, but the one that
got widespread visibility was ours. I'm not sure why this particular
complaint had to get the wide attention it did, but that's water under
the bridge and its stimulating good discussion.
One of my employees who knows how strongly we feel about this
suggested we put out an internal bounty for anyone inside the company
who is involved in EPIC voting...I think we'll hold off on that for
now. ;-)
I think there are a couple of things that we need to
resolve: 1) what is our policy for handling these individual
requests,
I would suggest that the complaints be logged by Nathan and then
forwarded to the accused company...unless the questionable voting
appears to be "automated" in which case I would immediately
call it to the attention of the EPIC Council for possible action. If
the accused company can't refute the accusation and it happens again,
then maybe we should talk about blocking IP addresses from the rating
process.
2) do we believe this is a legitimate voting and who
makes the decision and
Speaking legalistically, there are no rules against this type of
voting (at least that I know of) so there is no real punishable
"offense", and it follows that there's no decision to
make.
I suggest that we use the process I described above and try
letting EPIC "listers" police themselves. Otherwise we have
to try to get into the minds of the rater and that's very difficult.
For example, in the list of votes Mik submitted is one for his
project, Mylar. No one at Instantiations sees Mylar as competitive, so
there's really no reason to consider that vote (a "4")
anything but a legitimate comment from one of our employees who looked
at Mylar...but who knows? The ""10's" and "1's"
might be easier to interpret negatively, but...?
3) is there a way to restrict this type of voting in
the future.
Very difficult, as we've discussed before. We want to encourage
many votes and not do anything to discourage a legitimate rater. Both
goals I fully support, but they are a bit in opposition to each
other.
I would prefer we don't just delete the votes from
this specific incident, since I don't think it solves the real
issue.
I can understand the "purity" of your position, but if
the goal is to have credible ratings and the company responsible for
creating a particular vote or set of votes agrees to have them removed
then I think we should consider doing it. It should be harder for the
company/plug-in being rated to get votes thrown out, but there should
be a process by which the EPIC Council looks at such requests and
makes a determination.
My two cents,
MikeT
From:
phoenix-epic-dev-bounces@xxxxxxxxxxx
[mailto:phoenix-epic-dev-bounces@xxxxxxxxxxx] On Behalf Of Mike
Taylor
Sent: September 18, 2006 1:01 PM
To: The EPIC component of Phoenix
Cc: 'Mik Kersten'
Subject: Re: [phoenix-epic-dev] FW: bogus rankings on
EPIC
All,
I read with concern the message from Mik Kersten. As all of you on the
EPIC Council know, I am very concerned with the legitimacy of the
ranking system and am among those pushing most strongly for making it
better.
At
Instantiations we have an explicit, written policy that prohibits our
employees from ranking either our own products or those of others on
EPIC. We do this although we are fully aware that it is common
practice in the industry for those with a vested interest to vote for
their own products and against competitive products. We do this
because a) we believe our products can win on their own merit, and b)
we know how angry it makes us when a competitor unjustly slams our
ratings (an unfortunately regular occurrence).
That said, we have a couple of dozen employees and its entirely
possible that the trickle of votes Mik references came from our IP
address. We vigilantly watch for this and constantly reinforce our
company policy...but violations of that policy have certainly
occurred. In every case where we could identify the source we have
appropriately disciplined the person responsible. We will continue to
do this and if Mik or anyone else can point us to an offender we will
enforce our company policy.
Instantiations' internal policies aside, I
wonder what makes these votes inherently "bad"? Are
plug-in vendors officially restricted from voting for their own
products on EPIC? Not that I know of. What alerted Mik to these votes?
Maybe if we knew we could watch that data source and use it to enforce
our own company policies.
In Mik's
data I count 17 votes over a 5 month period involving about a dozen
different products. Certainly not enough to swing ratings much if at
all. WindowBuilder, our highest volume product (and the only one in
the EPIC top 10), has literally thousands of votes, so the one in the
questionable data set had no practical effect. Our other products in
the data set have a smaller vote count, but there are still not enough
votes to affect the ratings in a meaningful way.
I have to
say that even given the many discussions we've had, and are continuing
to have, at the EPIC Council about improving the ranking system and
protecting it from those who would game it...these votes wouldn't make
it onto our radar screen. What the Council is trying to do is remove
the possibility of significant, illegitimate ranking. We have to rely
on the community to alert us to small, infrequent misuses of the
ranking system, and for that I thank Mik.
I'm curious, why Mik's previous emails didn't get the visibility they
deserve? Instantiations would like to be alerted anytime any gaming is
going on with our listed products or if it is suspected that our
employees are voting in an unsavory manner.
Regards,
MikeT
PS. We would
be perfectly happy if all the votes referenced by Mik were removed
from the database.
At 9:51 AM
-0400 9/18/06, Ian Skerrett wrote:
All,
Mik Kersten sent me an e-mail complaining about some
of the rankings that have been made on EPIC. I think we need to
decide how we want to respond to Mik's concerns.
Btw, Mik agreed for me to post his original e-mail on
the mailing list. Please copy him on any replies, since he has
not subscribed to this list.
Thanks
Ian
Ian Skerrett
Director of Marketing
Eclipse Foundation, Inc.
Tel: 613-224-9461 ext. 227
Fax: 613-224-5172
ian.skerrett@xxxxxxxxxxx
http://www.eclipse.org
Blog: http://ianskerrett.blogspot.com/
From: Mik Kersten
[mailto:beatmik@xxxxxxx]
Sent: September 15, 2006 2:38 PM
To: 'Ian Skerrett'
Subject: bogus rankings on EPIC
Hi
Ian,
Could
you give me the email address of someone responsible for EPIC? I
have submitted 3 email/web complaints of bogus rankings trying to
bring up Instantiations products and bump others near them on the
top-10 down. I still have received no response and the bogus
ranking is continuing, e.g.:
http://www.eclipseplugincentral.com/modules.php?op=modload&name=Web_Links&file=index&req=ShowRaterDetails&userid=&hostname=68.178.73.218
If
these kinds of problems with anonymous rakings are not addressed
pro-actively by EPIC abuse like this is likely to continue, and
someone could blog about this and make EPIC look
untrustworthy.
Mik
--
Mik Kersten, http://kerstens.org/mik
Mylar
Project Lead, http://eclipse.org/mylar
_______________________________________________
phoenix-epic-dev mailing list
phoenix-epic-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/phoenix-epic-dev
--
----------------------------------------
Mike
Taylor
President and CEO
Instantiations, Inc.
Power Tools for Professional Software Developers
Voice: (503) 598-4911
mike_taylor@xxxxxxxxxxxxxxxxxx
http://www.instantiations.com
_______________________________________________
phoenix-epic-dev mailing list
phoenix-epic-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/phoenix-epic-dev
--
----------------------------------------
Mike Taylor
President and CEO
Instantiations, Inc.
Power Tools for Professional Software Developers
Voice: (503) 598-4911
mike_taylor@xxxxxxxxxxxxxxxxxx
http://www.instantiations.com
