Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [phoenix-epic-dev] FW: bogus rankings on EPIC
Title: RE: [phoenix-epic-dev] FW: bogus rankings on EPIC

Yikes, I no way intended to direct this complaint at Instantiations, or to imply any deliberate involvement.  I actually made that clear in one of my original complaints, but was less careful in the message to Ian.  The voting policy that Mike has outlined for his organization is exemplary, and the problem that I’m trying to point out is that we can’t count on other companies being as exemplary.

 

Mike: note that I would never attribute something like this to Instantiations without explicit evidence which there isn’t in this case, and I was actually assuming that Instantiations would never do something like this deliberately because of history with EPIC and Eclipse.  I also agree that such complaints should be forwarded to the company for the reasons you outline.  I raised is that while my response after having complaints ignored is to figure out how else to raise this issue or to ignore it, others in the community would have flamed on their blogs and made EPIC and/or Instantiations look bad. This is why I think it’s important to address the ranking problems sooner than later.

 

Also note that my motivation for getting into this discussion here was not the ranking of Mylar, because our visibility as an eclipse.org project is high enough for the current stage of adoption.  The reason I feel strongly about this is that having an accurate ranking that’s ambivalent to whether the projects are eclipse.org/commercial/volunteer efforts is key to encouraging and supporting innovation.

 

Mik

 

 

From: phoenix-epic-dev-bounces@xxxxxxxxxxx [mailto:phoenix-epic-dev-bounces@xxxxxxxxxxx] On Behalf Of Mike Taylor
Sent: Monday, September 18, 2006 2:16 PM
To: The EPIC component of Phoenix
Subject: RE: [phoenix-epic-dev] FW: bogus rankings on EPIC

 

I, in no way see this as an Instantiations issue.  It is very unfortunate that the example Mik forward might make it seem that way but I don't think it was the intention.  I know you have a lot of passion about EPIC and want it to be a legitimate resource for the Eclipse community.

 

Thanks, Ian, but I can't help but take it a bit personally. Instantiations has been on the forefront of improving and policing the rating system since before we transferred EPIC to the Foundation. There have been numerous examples of unsavory voting, but the one that got widespread visibility was ours. I'm not sure why this particular complaint had to get the wide attention it did, but that's water under the bridge and its stimulating good discussion.

 

One of my employees who knows how strongly we feel about this suggested we put out an internal bounty for anyone inside the company who is involved in EPIC voting...I think we'll hold off on that for now. ;-)

 

 

I think there are a couple of things that we need to resolve: 1) what is our policy for handling these individual requests,

 

I would suggest that the complaints be logged by Nathan and then forwarded to the accused company...unless the questionable voting appears to be "automated" in which case I would immediately call it to the attention of the EPIC Council for possible action. If the accused company can't refute the accusation and it happens again, then maybe we should talk about blocking IP addresses from the rating process.

 

2) do we believe this is a legitimate voting and who makes the decision and

 

Speaking legalistically, there are no rules against this type of voting (at least that I know of) so there is no real punishable "offense", and it follows that there's no decision to make.

 

I suggest that we use the process I described above and try letting EPIC "listers" police themselves. Otherwise we have to try to get into the minds of the rater and that's very difficult. For example, in the list of votes Mik submitted is one for his project, Mylar. No one at Instantiations sees Mylar as competitive, so there's really no reason to consider that vote (a "4") anything but a legitimate comment from one of our employees who looked at Mylar...but who knows? The ""10's" and "1's" might be easier to interpret negatively, but...?

 

3) is there a way to restrict this type of voting in the future.

 

Very difficult, as we've discussed before. We want to encourage many votes and not do anything to discourage a legitimate rater. Both goals I fully support, but they are a bit in opposition to each other.

 

 

I would prefer we don't just delete the votes from this specific incident, since I don't think it solves the real issue.   

 

 

I can understand the "purity" of your position, but if the goal is to have credible ratings and the company responsible for creating a particular vote or set of votes agrees to have them removed then I think we should consider doing it. It should be harder for the company/plug-in being rated to get votes thrown out, but there should be a process by which the EPIC Council looks at such requests and makes a determination.

 

My two cents,

MikeT

 

 

From: phoenix-epic-dev-bounces@xxxxxxxxxxx [mailto:phoenix-epic-dev-bounces@xxxxxxxxxxx] On Behalf Of Mike Taylor
Sent: September 18, 2006 1:01 PM
To: The EPIC component of Phoenix
Cc: 'Mik Kersten'
Subject: Re: [phoenix-epic-dev] FW: bogus rankings on EPIC

 

All,


I read with concern the message from Mik Kersten. As all of you on the EPIC Council know, I am very concerned with the legitimacy of the ranking system and am among those pushing most strongly for making it better.

At Instantiations we have an explicit, written policy that prohibits our employees from ranking either our own products or those of others on EPIC. We do this although we are fully aware that it is common practice in the industry for those with a vested interest to vote for their own products and against competitive products. We do this because a) we believe our products can win on their own merit, and b) we know how angry it makes us when a competitor unjustly slams our ratings (an unfortunately regular occurrence).


That said, we have a couple of dozen employees and its entirely possible that the trickle of votes Mik references came from our IP address. We vigilantly watch for this and constantly reinforce our company policy...but violations of that policy have certainly occurred. In every case where we could identify the source we have appropriately disciplined the person responsible. We will continue to do this and if Mik or anyone else can point us to an offender we will enforce our company policy.

Instantiations' internal policies aside, I wonder what makes these votes inherently "bad"?  Are plug-in vendors officially restricted from voting for their own products on EPIC? Not that I know of. What alerted Mik to these votes? Maybe if we knew we could watch that data source and use it to enforce our own company policies.

 

In Mik's data I count 17 votes over a 5 month period involving about a dozen different products. Certainly not enough to swing ratings much if at all. WindowBuilder, our highest volume product (and the only one in the EPIC top 10), has literally thousands of votes, so the one in the questionable data set had no practical effect. Our other products in the data set have a smaller vote count, but there are still not enough votes to affect the ratings in a meaningful way.

 

I have to say that even given the many discussions we've had, and are continuing to have, at the EPIC Council about improving the ranking system and protecting it from those who would game it...these votes wouldn't make it onto our radar screen. What the Council is trying to do is remove the possibility of significant, illegitimate ranking. We have to rely on the community to alert us to small, infrequent misuses of the ranking system, and for that I thank Mik.


I'm curious, why Mik's previous emails didn't get the visibility they deserve? Instantiations would like to be alerted anytime any gaming is going on with our listed products or if it is suspected that our employees are voting in an unsavory manner.

Regards,
MikeT

PS. We would be perfectly happy if all the votes referenced by Mik were removed from the database.

 

At 9:51 AM -0400 9/18/06, Ian Skerrett wrote:

All,

 

Mik Kersten sent me an e-mail complaining about some of the rankings that have been made on EPIC.  I think we need to decide how we want to respond to Mik's concerns.

 

Btw, Mik agreed for me to post his original e-mail on the mailing list.  Please copy him on any replies, since he has not subscribed to this list.

 

Thanks

Ian

 

 

 

Ian Skerrett

Director of Marketing

Eclipse Foundation, Inc.

Tel: 613-224-9461  ext. 227

Fax: 613-224-5172

ian.skerrett@xxxxxxxxxxx

http://www.eclipse.org

Blog: http://ianskerrett.blogspot.com/ 

 

 

 

 

From: Mik Kersten [mailto:beatmik@xxxxxxx]
Sent: September 15, 2006 2:38 PM
To: 'Ian Skerrett'
Subject: bogus rankings on EPIC

 

Hi Ian,

 

Could you give me the email address of someone responsible for EPIC?  I have submitted 3 email/web complaints of bogus rankings trying to bring up Instantiations products and bump others near them on the top-10 down.  I still have received no response and the bogus ranking is continuing, e.g.:

 

http://www.eclipseplugincentral.com/modules.php?op=modload&name=Web_Links&file=index&req=ShowRaterDetails&userid=&hostname=68.178.73.218

 

If these kinds of problems with anonymous rakings are not addressed pro-actively by EPIC abuse like this is likely to continue, and someone could blog about this and make EPIC look untrustworthy. 

 

Mik

 

--

Mik Kersten, http://kerstens.org/mik

Mylar Project Lead, http://eclipse.org/mylar

 


_______________________________________________
phoenix-epic-dev mailing list
phoenix-epic-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/phoenix-epic-dev

 

 

--

----------------------------------------

Mike Taylor
President and CEO
Instantiations, Inc.
Power Tools for Professional Software Developers

Voice: (503) 598-4911
mike_taylor@xxxxxxxxxxxxxxxxxx
http://www.instantiations.com


_______________________________________________
phoenix-epic-dev mailing list
phoenix-epic-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/phoenix-epic-dev

 

 

--

----------------------------------------
Mike Taylor
President and CEO
Instantiations, Inc.
Power Tools for Professional Software Developers

Voice: (503) 598-4911
mike_taylor@xxxxxxxxxxxxxxxxxx
http://www.instantiations.com

Back to the top