[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [cross-project-issues-dev] [WARNING] SimRel Headed Off the Tracks
|
Comments below.
On 29.01.2020 10:48, Mickael Istria
wrote:
It seems mostly based on the the principle/assumption
that moving the problem will simplify the problem or
make existing problems disappear by magic.
Indeed, I believe that moving the problem to a better
functioning project according to EDP practices, and by the
way getting rid of a lot of questionable effort, will make
many projects disappear.
Making project disappears will definitely help make their
contributed problems disappears. Also, centralization of
responsibilities seems helpful.
Currently the train repo is a prerequisite for
producing the packages and it composes a large set of
repositories into a single aggregate at which point a
high level of consistency is checked and ensured. In the
end, ensuring that all the artifacts that comprise each
package are coherent (and stable) does not go away even
if somehow the packages were produced by directly
pulling content from something other than the train
repository.
With EPP, we ensure that the packages are consistent and
of good enough quality to be released. I don't think that
would change much, and EPP could also add some tests
verifying all features can install in the same IDE.
The user's often install additional things the consistency of which
is also important it seems to me. The train has helped a great
deal with that. E.g., you can be sure that when you install Xtext
you also install the corresponding EMF version...
Nothing changes with regard to ensuring consistent
licenses, signed content, proper inter-operation, stable
repositories, and mutual instability.
Right, but at least, if it's in EPP, then we're sure the
projects that are integrated have at least 1 person (the EPP
maintainer for this package) caring about those issues and
dealing with them. While with current push model in SimRel,
some project push outdated stuff and aren't reachable.
Yes, though it's not clear that this point what subset remains based
on the transitive closure of all EPP package dependencies. Often
there are surprises. E.g., I tried to remove BIRT, but there remain
charting dependencies so I could only reduce the subset aggregated
from BIRT...
In the end, I'm not even sure if you're suggesting that
there needs to be no aggregation at all, but simply a
very large set of direct references to various project
repositories. But I can assure you that loading 50
repositories instead of 2 when doing an install will not
improve the experience, and that getting n projects to
maintain long-term stable sites is a new problem that
will also turn into yet another cat herding exercise and
when it fails (as all things do on occasion), the users
will notice immediately.
I suggest EPP builds the aggregated and categorized p2
repository, containing only stuff that are included in
packages.
We'd need to understand the transitive closure of all packages
and presume that consistency for anything in a package is just not
important to anyone, not even users.
As such, I did quick test, resolve all EPP IUs for 2020-03 into a
target platform, which suggests that only 10% of the IUs are in
packages. It's not clear which cross section of projects this
covers. But it's clear looking at which EMF things are in the TP
that only a very small subset of what EMF contributes to the
aggregate is transitively required by the EPP packages. I'd not
be happy with a train aggregate that did not include all the
things I currently contribute to the train. One stop shopping
from the aggregate, an aggregate that's bigger than what's
transitively pre-installed in the union of all EPP packages, seems
to me to have significant value...
To build. EPP references different source p2
repositories, just like SimRel reference other repositories.
So how will the exact repositories to use be managed? I see later
you suggest a *.target file, but it's not clear if that's a
per-package *.target, in which case keeping it consistent across
packages is important, or if it's shared single *.target, in which
case multiple packages need to main the union of IUs to pull from
the URLs.
It feel as if you've joined the discussion years after
all the reasons for having a train the first place were
had, and that you assume there really are no good
reasons because you were not part of those discussions.
So all the reasons need to be reiterated, at which point
you are highly inclined to try to shoot each one down
because they don't fit you current conclusion.
I know the reasons, and participated in integration of a
project in SimRel 11 years ago and was very excited about
SimRel and invested in it. I think it was interesting back
then, but times have changed: many project are almost
inactive in SimRel, SimRel build has lost maintenance
workforce and there doesn't seem to be much hope for more,
here is Marketplace, there are EPP packages, there is an
installer... So I think it's just time to question again
whether we can collectively afford SimRel as it is now, and
even whether this is still the appropriate solution for past
problems.
So at best you are suggesting the transitive closure of the EPP
dependencies is much smaller and will be easier to manage by a
yet-to-be determined someone. That seems not unlikely but it
presumes there is no other value for the aggregate.
Those who need more need to invest more; but if no-one
wishes to invest more despite the urge, then it's that there
is actually no strong enough need to keep things as they are
now.
Certainly centralized cohesive management seems a necessary part of
any solution, but distributed management of contributed content (and
organizing it) will remain a problem for any solution.
In any case, no matter exactly all the concrete details
of what you are suggesting, the question of who does
that work remains the same one.
If we keep separated SimRel and EPP and keep projects
that are irrelevant to EPP in a push mode, it's a lot of
work and no-one will want to do this work.
In the end though, the projects will need to push what's intended
to be pushed. E.g., a bug like this is opened every few weeks
asking me which EMF repo to push into the platform's build:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=559488
This is still work for me, and I imagine someone needing to
manage what ends up in EPP packages will need to do this type of
thing for a significantly large number of projects, which seems
incredibly painful to me. I think that whatever the process, it
needs further automated to make this all less painful. E.g., why
can't we have staging EPP repos/packages daily?
If we make things simpler and better address current
issues instead of sticking to older ones, then it's less
work and it's more interesting, some people will continue
it.
Which issues will be simpler and simpler for whom? I already easily
push a new repo into simrel and that for me is simpler than being
prompted by Bug 559488. For anyone producing an EPP package it's
very simple to find what's aggregated into the train repo because
all the providers just push content into it... I would not want to
manage finding what should be pulled... I would expect a push
process to make those available. And I would not want to be faced
with resolution failures because someone pushed something
incompatible with something someone else pushed; this something the
aggregator prevents right now.
This is far from being top-priority, I have more valuable
work in the backlog (like lobbying for the end of SimRel as
we know it ;)
Apparently a consistent marketplace is no one's priority, so the
users end up with a perception that much of it is crap, and
therefore Eclipse is crap.
That hasn't happened and the fully 1/3 of the
marketplace entries are completely broken or somewhat
broken. Consistent/correct marketplace listings is yet
another exercise of cat herding.
There are stats about installation issues in Marketplace
already, and IIRC there is even a system to notify owner in
case of too many install failures. That seems far enough to
me, and my current usage of Marketplace is pretty pleasant
and leads to good experience. (while I never use the SimRel
site...)
There is also a full report:
https://www.eclipse.org/setups/marketplace/?style=all
It suggests that randomly choosing something from the marketplace
will result in 1/3 of those things failing. Do doubt many
marketplace contributors do an excellent job maintaining their
listings, but those that don't reflect poorly on all of those that
do.
* packages would still be available -> no loss
for users
So at least we agree that packages are needed.
Unfortunately there's no one to produce them.
On the epp-dev mailing-lists, some questions are pending
for others to evaluate how much the can invest.
But EPP packages are a trivial-ish Tycho project, that
builds itself on CI on top of active technologies and
reactive, communities. It's not a too big deal (compared to
SimRel), and once the actual maintenance steps are
clarified, there will be some people to do the work.
It seems a lot of +1 activities around 2013-03 M1 are absent.
* installer would still be available -? no loss
for users
Here the question is: Which repositories will contain all
the artifacts?
So there will still be aggregation, it will just be implemented
differently. Presumably the aggregate will be a smaller set of
projects so some project-specific problems will disappear, which
would be good. But overall the same nature of problems will remain
for whatever projects remain, there might well be fewer of those
projects. Someone will just need to implement a new processes and
manage contributing repositories differently.
How much work will I personally (Oomph) have because of
a complete change in design in EPP structure?
Hopefully none.
* SimRel and its strange governance and all the
discussions that have emerged with it disappear
-> time saved
It will only disappear from view, but the identical
technical problems will simply migrate somewhere else.
Somewhere decentralized? Somewhere with no central
oversight? This doesn't not sound like the problems will
go away, but rather will become invisible to most of us,
but not for the users.
Indeed, some checks need to remain; but most of them can
be automated, and in EPP, much is already done by package
testers.
Yes, automation is key, including automation of more problem
detection and automation of problem reporting...
* EPP starts handing everything, and EPP
governance is working well -> EDP used
efficiently.
The person who does not exist will restructure everything
and will manage everything personally and none of us will
have to do anything at all anymore to help that person.
That sounds great in principle, except for that person.
And I'm often that person, and I can assure you it's not
great at all; I often cannot solve problems that come from
elsewhere.
That's fair, however, I'm convinced the proposed change
is profitable. But it indeed requires investment; but it's
always the same, if we can convince people there is a return
on investment (and IMO there is as SimRel day-to-day
maintenance will be drastically reduced), then we'll have
them more likely to help.
I think the key driving force for you is to see the simrel
aggregator go away, correct? But I still feel that there needs to
be an automatic process by which contributing projects make their
repositories known. The categorization of the features also
remains. So a number of things that the aggregator solves well
already needs to be rearchitected via a new process that will have
new problems as well as many of the same old ones. And, someone
must step up to do that.
* Active contributors like you stop spending
effort on projects that are not worth it (to you):
many projects are in SimRel just by the force of
habit, but the value for the user community is
arguable and the cost for maintainers is present
(more things to check, more bugs to open, more files
to watch, longer build time....).
Removing projects from the train will be somewhat helpful
in reducing the overhead. We could start with EMF and
finish with Oomph; that would save me personally one hell
of a lot of work. Or did you have specific projects in
mind that are not worth the effort?
Every project that is not part of an EPP package and not
investing in testing the EPP package is IMO not worth the
effort.
That seems a reasonable position.
With this proposal, the maintenance cost would be
drastically reduced and the process be made more
streamlined with typical EDP. Hopefully this will
become simple enough for the few active contributors
on SimRel (build & infra, not contributions) and
EPP to be able to cope with this for some years.
Are you volunteering to step up to prototype and
demonstrate all the new infrastructure that would be
involved in your proposal so that we may concretely assess
how that alternative would work in detail rather than in
the abstract?
:)
Yes and no. I may try it at some point, but cannot
guarantee it nor estimate when. But as I'm convinced of the
ROI and my suggestion is mostly about adding a .target file
into EPP build that happens to be a simple Tycho build, then
I may find some time for it when I'm bored with other
things.
I.e., implement something that captures the same information as in
captured by the aggr/aggrecon model. The URLs in that *.target file
will need to be maintained by a distributed set of
projects/people. And you'll need a site.xml for categorization
which also must be managed in distributed way.
About enforcing or checking SimRel rules, then
they are not really SimRel rules and checking that
or declaring compatibility should be handled by
projects, as part of their releases; not by a
downstream consumption.
I've seen that projects are so very very responsive in
addressing the issues raised for them, not!
Then it becomes the responsibility and choice of the ones
who "pull" the project: if i maintain an EPP and see a
dependency in a bad state (let's say Mylyn), it becomes my
duty to get it fixed or get it removed from the EPP
packages/aggregation. But as long as Mylyn doesn't react and
comply with the necessary rules, they'll just be removed
from EPP and aggregated site.
This is effectively what I've been doing for the past months.
Mostly hunting down problems and contributing fixes for them
whenever possible. Between that and contributing my own projects,
it's been a full time job.
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev