Bug 459905 - Create an Eclipse Project Market
Summary: Create an Eclipse Project Market
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Marketplace (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Management Organization CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 418865
Blocks: 355716 457087 460564 461603
  Show dependency tree
 
Reported: 2015-02-13 11:30 EST by Mickael Istria CLA
Modified: 2015-12-16 17:11 EST (History)
13 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mickael Istria CLA 2015-02-13 11:30:05 EST
It would be convenient that Eclipse projects would be automatically listed on MarketPlace using the information they put on the PMI.
Comment 1 Matthias Sohn CLA 2015-02-13 11:40:37 EST
+1
Comment 2 Wayne Beaton CLA 2015-02-13 12:30:44 EST
I'm not at all convinced that just copying existing stuff from the PMI is going to work. We can't just copy the project or release description. We may be able to assume the simultaneous release repository as a software source, but the feature identity is going to have to be specified. Some manual intervention is going to be required. Why not just have project members maintain the marketplace record directly.

I believe that Marketplace has a means of creating a separate marketplace for the simultaneous release. Instead of burying project entries in with everybody else's stuff, does it make sense to instead pursue the option of creating a separate marketplace that only contains stuff from the simultaneous release?

We can add specific metadata entries in the PMI as necessary. We can assume that the simultaneous release repository is the source of the software, but projects will have to specify the feature(s), provide a suitable description (which will likely be different from the project or release description), screenshots, etc.

Inclusion in the "Project" marketplace should be a benefit of participation in the simultaneous release.

Do we create a marketplace for each instance of the simultaneous release, or just one for all of them. Can we combine different versions of features into one marketplace entry?
Comment 3 Ian Skerrett CLA 2015-02-13 12:46:11 EST
(In reply to Wayne Beaton from comment #2)
> I'm not at all convinced that just copying existing stuff from the PMI is
> going to work. We can't just copy the project or release description. We may
> be able to assume the simultaneous release repository as a software source,
> but the feature identity is going to have to be specified. Some manual
> intervention is going to be required. Why not just have project members
> maintain the marketplace record directly.

I agree the feature IDs was the problem we encountered the last time we tried this.

> 
> I believe that Marketplace has a means of creating a separate marketplace
> for the simultaneous release. Instead of burying project entries in with
> everybody else's stuff, does it make sense to instead pursue the option of
> creating a separate marketplace that only contains stuff from the
> simultaneous release?

We can do this but unless we get critical mass it might actually make the existing projects less visible.

> 
> We can add specific metadata entries in the PMI as necessary. We can assume
> that the simultaneous release repository is the source of the software, but
> projects will have to specify the feature(s), provide a suitable description
> (which will likely be different from the project or release description),
> screenshots, etc.
> 
> Inclusion in the "Project" marketplace should be a benefit of participation
> in the simultaneous release.
> 
> Do we create a marketplace for each instance of the simultaneous release, or
> just one for all of them. Can we combine different versions of features into
> one marketplace entry?

It should be just one Marketplace.  We are adding a feature so each listing can specifying which version of Eclipse is compatible.
Comment 4 Wayne Beaton CLA 2015-02-13 12:53:06 EST
(In reply to Ian Skerrett from comment #3)
> We can do this but unless we get critical mass it might actually make the
> existing projects less visible.

My SWAG is that we'll get 60% participation. For some projects, a marketplace entry just doesn't make sense. Others will have multiple entries. Some will complain that we're asking them to do more work.

TL;DR is that this needs to be opt in, but I think that we'll get pretty good adoption. 

Do you a specific target in mind? How do you define critical mass?

> It should be just one Marketplace.  We are adding a feature so each listing
> can specifying which version of Eclipse is compatible.

Theoretically, can we just serve an "Eclipse Projects" marketplace directly out of the PMI? 

I guess that I'm asking what the Marketplace Client needs. Is it just a feed?
Comment 5 Ian Skerrett CLA 2015-02-13 13:03:25 EST
(In reply to Wayne Beaton from comment #4)
> (In reply to Ian Skerrett from comment #3)
> > We can do this but unless we get critical mass it might actually make the
> > existing projects less visible.
> 
> My SWAG is that we'll get 60% participation. For some projects, a
> marketplace entry just doesn't make sense. Others will have multiple
> entries. Some will complain that we're asking them to do more work.

60% would be good. We could also look at having Eclipse projects be in the existing Eclipse Marketplace and the new Eclipse Projects Marketplace. This way they wouldn't lose visibility.


> 
> Theoretically, can we just serve an "Eclipse Projects" marketplace directly
> out of the PMI? 
> 
> I guess that I'm asking what the Marketplace Client needs. Is it just a feed?

We have an API. https://wiki.eclipse.org/Marketplace/REST   btw, the API is changing for Mars.
Comment 6 Marcel Bruch CLA 2015-02-13 13:28:16 EST
I just thinking out loud...

We manually add/update a "changelog" for every version in the marketplace entry (using bugzilla queries). The PMI has a much more neat way of generating such a list. 

Maybe there is a way how to integrate this into the Marketplace entry?


[1] http://marketplace.eclipse.org/content/eclipse-code-recommenders
Comment 7 Carsten Reckord CLA 2015-02-13 15:57:08 EST
(In reply to Ian Skerrett from comment #3)
> > Do we create a marketplace for each instance of the simultaneous release, or
> > just one for all of them. Can we combine different versions of features into
> > one marketplace entry?
> 
> It should be just one Marketplace.  We are adding a feature so each listing
> can specifying which version of Eclipse is compatible.

Chris and I are working on supporting multiple releases for different Eclipse versions in a single Marketplace entry. The right one is chosen transparently for the user by MPC based on the Eclipse version they're running. See bug 418865 and children for details.

So we just need one Market with one entry per participating project.

(In reply to Wayne Beaton from comment #4)
> My SWAG is that we'll get 60% participation. 

60% sounds good :)

> TL;DR is that this needs to be opt in, but I think that we'll get pretty
> good adoption. 

I agree that it should be opt-in, not a simrel requirement - but maybe encouraged somewhere in the simrel docs.

> Theoretically, can we just serve an "Eclipse Projects" marketplace directly
> out of the PMI? 
> 
> I guess that I'm asking what the Marketplace Client needs. Is it just a feed?

Just so I don't answer about the wrong thing here: we have Marketplaces (as in servers, like http://marketplace.eclipse.org and those run by strategic members) and those in turn can have Markets (like for Eclipse currently "Tools", "RCP Applications", "Training & Consulting", "Long Term Support").

Marketplaces are the horizontal list of icons at the bottom in MPC, Markets can be chosen in the search bar to see their listings. Marketplaces are very isolated from one another (i.e. you cannot search/browse across Marketplaces and you have to do a separate install session for each Marketplace). Markets can be used to see content specific to them, but without a specific Market selected, I can search across all Markets and entries on a Marketplace.

That said, I don't think it's a good idea to have a separate Marketplace for Eclipse projects, because that's still a degree of separation too much in my opinion. A separate market on the Eclipse marketplace would be great and should be sufficient for all intents and purposes.

I'm not sure if it's technically feasible to serve the data for an Eclipse Projects market from a different source, i.e. directly from PMI. But as Konstantin mentioned on the ide-dev list, those Marketplace entries require very little maintenance, so it's probably not worth the effort. 

If we wanna be fancy with the PMI data, we could look into something like Marcel suggested:

(In reply to Marcel Bruch from comment #6)
> We manually add/update a "changelog" for every version in the marketplace
> entry (using bugzilla queries). The PMI has a much more neat way of
> generating such a list. 
> 
> Maybe there is a way how to integrate this into the Marketplace entry?

The lowest-tech solution for that might be to have a ready-to-paste HTML block generated by some javascript in PMI. And if there's sufficient interest we could still look into better automation for the including those parts of the PMI data that have proven useful.
Comment 8 Wayne Beaton CLA 2015-02-27 11:55:31 EST
According to a note recently posted on mpc-dev [1], the new API is in place. If I understand correctly, this means that Bug 418865 will be resolved soon. It's time to start planning what we need to have in place to generate a catalogue for the simultaneous release.

[1] https://dev.eclipse.org/mhonarc/lists/mpc-dev/msg00392.html
Comment 9 Ian Skerrett CLA 2015-02-27 14:23:06 EST
(In reply to Wayne Beaton from comment #8)
> According to a note recently posted on mpc-dev [1], the new API is in place.
> If I understand correctly, this means that Bug 418865 will be resolved soon.
> It's time to start planning what we need to have in place to generate a
> catalogue for the simultaneous release.
> 
> [1] https://dev.eclipse.org/mhonarc/lists/mpc-dev/msg00392.html

There are actually two approaches we could take:
1) create a new marketplace for projects
2) use the existing marketplace but create a new market on Eclipse Marketplace.

(Yes I know this is not obvious terminology)

I would recommend a new Market on Eclipse Marketplace.  This would allow for the following:
- on the Marketplace web site the Market would be visible. 
- from the Marketplace Client, there would be a Eclipse Project market. This wouldn't be that obvious from the client but we might look at extending MPC to have a special Eclipse projects tab.  Just a thought.
Comment 10 Wayne Beaton CLA 2015-03-06 08:55:37 EST
(In reply to Ian Skerrett from comment #9)
> I would recommend a new Market on Eclipse Marketplace.  This would allow for
> the following:
> - on the Marketplace web site the Market would be visible. 
> - from the Marketplace Client, there would be a Eclipse Project market. This
> wouldn't be that obvious from the client but we might look at extending MPC
> to have a special Eclipse projects tab.  Just a thought.

Is there a means of encouraging the MPC to open on an "Eclipse Projects" tab?

e.g. could we maybe add an icon to the welcome/intro screen to "Add Eclipse Project Features" (or something way better than that) that opens the MPC in the Eclipse Project Marketplace?

I'm thinking that adding yet another "install" item to the Help menu isn't the right approach.
Comment 11 Carsten Reckord CLA 2015-03-06 09:24:46 EST
I like the idea. Yes, there already is API to open the MPC on a specific tab. I can create an "Eclipse Projects" tab fairly easily and add a welcome contribution to MPC to open it from the intro screen, if we want to go that way.
Comment 12 Wayne Beaton CLA 2015-03-06 10:23:11 EST
(In reply to Carsten Reckord from comment #11)
> I like the idea. Yes, there already is API to open the MPC on a specific
> tab. I can create an "Eclipse Projects" tab fairly easily and add a welcome
> contribution to MPC to open it from the intro screen, if we want to go that
> way.

I think that this is a reasonable step in the right direction.

To summarize:

* There's really not enough data in the correct format to automatically generate a marketplace from the PMI; we'd have to add additional fields to capture additional data to make this work.
* Adding fields to capture data necessary to generate a marketplace would effectively require that we replicate functionality already present in the Eclipse Marketplace.
* Rather than reimplement what we already have, let's just use what we already have.
* We'll create an "Eclipse Projects" market in the Eclipse Marketplace.
* Only simultaneous release projects may be included in Eclipse Projects market.
* Only features that might reasonably be installed into an Eclipse-based IDE will be present in the Eclipse Projects market.
* Content will be curated to ensure that it is presented in a consistent manner.
* We'll add an Intro item that opens the MPC on the Eclipse Project market.
* The Eclipse Projects market will be relatively easy to find in the MPC.
* This is the first step in a transition that might eventually have the MPC take over the p2 installer menu option.

EF staff will assist with creating and curating Marketplace content.

Does all of this make sense? 

I'm moving this bug to Community/Marketplace; I think that this is the right place for it.
Comment 13 Ian Skerrett CLA 2015-03-06 11:52:21 EST
(In reply to Wayne Beaton from comment #12)
> (In reply to Carsten Reckord from comment #11)
> > I like the idea. Yes, there already is API to open the MPC on a specific
> > tab. I can create an "Eclipse Projects" tab fairly easily and add a welcome
> > contribution to MPC to open it from the intro screen, if we want to go that
> > way.
> 
> I think that this is a reasonable step in the right direction.
> 
> To summarize:
> 
> * There's really not enough data in the correct format to automatically
> generate a marketplace from the PMI; we'd have to add additional fields to
> capture additional data to make this work.
> * Adding fields to capture data necessary to generate a marketplace would
> effectively require that we replicate functionality already present in the
> Eclipse Marketplace.
> * Rather than reimplement what we already have, let's just use what we
> already have.
> * We'll create an "Eclipse Projects" market in the Eclipse Marketplace.
> * Only simultaneous release projects may be included in Eclipse Projects
> market.
> * Only features that might reasonably be installed into an Eclipse-based IDE
> will be present in the Eclipse Projects market.
> * Content will be curated to ensure that it is presented in a consistent
> manner.
> * We'll add an Intro item that opens the MPC on the Eclipse Project market.
> * The Eclipse Projects market will be relatively easy to find in the MPC.
> * This is the first step in a transition that might eventually have the MPC
> take over the p2 installer menu option.
> 
> EF staff will assist with creating and curating Marketplace content.
> 
> Does all of this make sense? 

Yes this makes sense but it is not clear how the projects get added to the Market? Is this manual or automated.
Comment 14 Wayne Beaton CLA 2015-03-06 14:02:48 EST
(In reply to Ian Skerrett from comment #13)
> Yes this makes sense but it is not clear how the projects get added to the
> Market? Is this manual or automated.

At this point, I think it's manual. 

I'm assuming this because I'm not quite sure how we'd automate it. Bullets two and three, I think, speak to my assumptions.

I am certainly interested in exploring options for automating any part of the process. e.g. can we do an initial import of all Mars projects that self-identify as "Tools"?
Comment 15 Wayne Beaton CLA 2015-03-30 15:10:24 EDT
Ian and I discussed this today. Here is our proposal...

We believe that the best course of action is to create a new market within the Eclipse Marketplace. This will ideally manifest as a tab labeled "Eclipse Projects" alongside "Search", "Recent", "Popular", etc. It will also manifest as a filter condition in the search form.

Eclipse project teams will be encouraged to work with cross-purpose to create entries in the Eclipse Marketplace for reasonable combinations their projects.

Constraints for participation in the Eclipse Project Market:
* Must be a n Eclipse projects that participate in the simultaneous release;
* Must be tools that are appropriate for installation into an IDE;
* Must provide a reasonable description of the feature (we need to define reasonable);
* Must cite the simultaneous release repository as the software source; and
* Must be complete (i.e. actually installable) (e.g must include feature ids and versions)

Note that marketplace now supports versions, so a single entry can include information for Luna, Mars, etc.

We need to keep the granularity of entries high. It is possible to specify multiple features in a single marketplace entry and mark some of them as optional; the user gets to decide which ones to install. Mylyn does this already.

We should, for example, be able to have a single entry for CDT that lists a number of optional features for each of the different compilers/tool chains supported. We should be able to to create a single entry for modeling stuff that lists EMF, CDO, EMF Compare, ... And so forth.

I'm thinking that we need have the following entries:
* Java development tools
* C/C++ Development Tools
* PHP Development Tools
* Java EE Development Tools
* Plug-in Development Tools
* Modeling tools
* Mylyn

I'm thinking that we should have around the same number of entries as we have packages. It might, however, make sense to have a separate entry for something like TM or Data Tools. IMHO, Code Recommenders (for example) belongs in the Java entry. Correct me if I'm wrong.

SDKs and sources are not generally included. Users looking for SDKs and sources will be sophisticated enough to just use the p2 installer or access the source repositories via the workspace.

I believe that the value is obvious. For the user, it should be _very_ easy to quickly sort out what they want/need to install. Projects can reference the marketplace entry on their websites (and other places) to make drag and drop installs dirt-simple (for Windows users, anyway). For projects, this will be an easy way to get your stuff into people's hands. 

Seeding this will require input from the project teams (perhaps with help from the package maintainers). Since some of these entries are likely cross-project, coordination will be required. Keeping the market current will be the biggest challenge.

It may also be a challenge to keep this useful. The end user is the primary focus. If the Eclipse Project market ends up with more than a dozen entries, then we've failed. 

Finally... I will be the initial curator of the "Eclipse Projects" market. This will require that I review entries for that market with every SR of the simultaneous release and decide what conforms to rules and initiate mitigating steps. I will, for example, actively look for opportunities to merge entries with an eye to keeping this as simple as possible for the end user. I will seek advice from the Planning Council where necessary.

Poke holes.
Comment 16 Wayne Beaton CLA 2015-05-29 12:27:19 EDT
I've build a prototype market:

http://marketplace.eclipse.org/category/markets/eclipse-project/title

I started by making a small number of entries representing some of the packages. I then started experimenting with finer-grained entries that are not already represented as packages (e.g. SQL tools).

Unfortunately, I think that rather than making it easier for users to install functionality, this may just make it more confusing. With this market in place, the user now has three different ways to get package content: package downloads, the installer, and now a special market place. Further, while the notion of having an EF staff member curate the page seemed like a good idea at the time, I'm less convinced that this is the right thing for the community.

What we're essentially trying to do with this is duplicate the package space in the Eclipse Marketplace; maintaining stuff in two different places is doomed to fail. The better long-term solution, I think, is to leverage the packages directly via the installer rather than the marketplace. I think that what we need to have is the ability to combine packages in the installer (Bug 468830).

In the meantime, we should encourage Eclipse Projects to make and maintain their own marketplace entries.

I'd like to let this set for a bit before taking any specific action, and am naturally interested to hear your take on it.

The action that I think we should take is to shut down the Eclipse Project market and close this bug as WONTFIX.
Comment 17 Doug Schaefer CLA 2015-05-29 20:01:50 EDT
+1 Makes the most sense. You really need to be a product manager to get this right and we really have been treating the packages as product so that activity is already happening. Would be best to leverage that.

And, yes, I always hoped the installer would combine packages for me.

I think additional questions come up as we consider other important segments of our ecosystem that aren't hosted at Eclipse. The Nodeclipse and Pydev projects come to mind. Projects like them are pretty important if we want to be the IDE for everything which is one of the few competitive advantages that we still have.
Comment 18 Wayne Beaton CLA 2015-12-16 17:11:27 EST
I changed my mind again.

Any Marketplace record that uses a p2 software repository with download.eclipse.org in the URL will automatically be added to the Eclipse Project Market.

This automatic association only happens when the record is saved, so existing entries will need to be revisited and updated to appear in the list.

We'll decide how we're going to use this after we see the quality of the records that appear there.

I think that we've done all that we can with this bug, so I'm marking it as fixed.