Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] [URGENT] Indigo bug causes framework to keep bundle file handles open for lifetime of framework instance.

I'm sort of 50/50 on this one. Granted, it is a very bad regression, but, there is a work around ... increasing ulimit -n ... but, on the other hand, if someone doesn't know that (or, until they do) they can be "blocked" from installing anything large. And, to make it worse, in some of the install-and-restart failures the error messages do not always say "too many files open" in the console or configuration log ... but other, worse sounding, more esoteric messages .... yes, making it sound like p2 is at fault and messed up the install :)  On the third hand, adopters with large distro's could include a patch if we can not easily respin ... and SR1 is only 3 months away :/

Hence, my advice is to take it "one step at a time". Let the platform rebuild, and re-contribute to the common repo aggregation build, and if that respin goes well then we could move on to rebuilding the EPP packages (granted, at least RAP runtime will probably need to re-contribute ... maybe others?). (We'd save restore points along the way, to be able to back out, if necessary.)

The risk is that the common repo may not be easily reproducible. If other projects changed URLs, or accidentally changed content, then it might cause several days of a lot of churn, adding even more risk. If that seems to be happening over Tuesday/Wednesday, we might want to reconsider and fall back to Plan B: well publicize the work around and, perhaps, Equinox could provide an official "patch feature"?

It is low cost and low risk to try to rebuild the common repo, and see how it goes. We can throw away our attempts if it doesn't go well. As long as effected projects can react and other participating projects have stayed constant, then a rebuild would go relatively smoothly ... assuming we don't burn out another power supply. :)  If the re-builds/re-spins do not go well, then we could reconsider. We'd know more by Tuesday afternoon, then final decision Wednesday afternoon. As part of this "let's try it and see" approach, I will turn back on the aggregation builds, and see if we can rebuild right now, as-is, even before the platform re-contributes, so if anyone has moved content and forgotten to keep b3aggrcon files in synch, we'll know by Tuesday morning.

We are in sort of a catch-22 ... being in a low-level bundle makes it risker to patch/rebuild, but precisely because it is a low-level, central function it is important that it be right.

Bottom line, I'm fine going either direction, and willing to go along with the judgment of the RT PMC (and other effected parties).






From:        John Arthorne <John_Arthorne@xxxxxxxxxx>
To:        Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>
Date:        06/13/2011 10:09 PM
Subject:        Re: [cross-project-issues-dev] [URGENT] Indigo bug causes framework to keep bundle file handles open for lifetime of framework instance.
Sent by:        cross-project-issues-dev-bounces@xxxxxxxxxxx




> Wayne Beaton <wayne@xxxxxxxxxxx>
> 06/13/2011 06:11 PM

>
> This sounds like a stop-ship kind of bug to me. Or am I over reacting?


It definitely falls into that category. The only argument I can imagine against fixing it, is that this problem existed in previous releases (3.3 and earlier I believe). In 3.4/3.5 you had to set a system property to enable this cache so in those releases there was an easy workaround for the problem. The workaround at this point is increasing your OS per-process file limit, but that means Eclipse will be consuming large amounts of OS resources in these cases. Of course, Eclipse and EPP packages have increased their bundle counts since then so the problem will be hit more frequently.


> Do any of the EPP packages have more than 1024 bundles?


I don't think so, but you can hit this problem by attempting to install "all of indigo" as David was doing when he discovered it.


> I assume that if a package gets close to the 1024 count, there'd
> still be problems as files are opened by the running instance. If I
> understand correctly, this would only affect the first run and
> restarting would clear things up, right?


Well, if the user figures out the workaround of increasing their ulimit, they can restart. If the user doesn't find this workaround they will not be able to restart.


> This would affect adopters/distributions that add a bunch of bundles, right?


Yes, any distro or large scale adopter would probably need to patch this if we didn't fix it.


Really, I think Tom only phrased it as an "if" because we are beyond the normal end of the release train. Projects/people that need to react and respin to accommodate this fix should be involved in making the decision to go ahead with it (EPP, RAP, David, possibly others).


John


>
> Wayne
>
> On 06/13/2011 05:42 PM, Thomas Watson wrote:
> Please see bug
https://bugs.eclipse.org/bugs/show_bug.cgi?id=349105
>
> There has been a regression in the Equinox framework with respect to
> the file handles it keeps open for each installed bundle. In Helios
> the framework defaulted to keeping at most 100 bundle jar files open
> at a time. In Indigo a bug has been introduced that basically
> disables this functionality and causes the framework to keep every
> bundle jar file which gets something loaded from it to be left open
> until the framework is shutdown.
>
> This mostly effects platforms like Linux where the ulimit (number of
> files a process can have open at a time) is limited. I have been
> told that some common Linux distributions limit this to 1024 by
> default. The result is that the platform will not be able to start
> successfully the first time if there are over 1000 bundles
> installed. This is because the framework must open every bundle jar
> to parse its manifest and then the jar file is left open until the
> framework shuts down. The bug is rather severe, but there is a work
> around on some systems by increasing the ulimit. I am not sure if
> that is always feasible and it is far from obvious to the typical user.
>
> We have started a build of the eclipse/equinox in preparation incase
> we decide that this must be fixed for the Indigo release. Let me
> emphasize the statement that "we must decide" if this is important
> enough to fix for Indigo. The Equinox and Eclipse project leadership
> is asking for input on the need to fix this in Indigo and the
> ability for others to react in time to consume the fix. For example,
> I know all the EPPs will have to be respun. I am fairly certain
> other projects will have to respin if they have prebuilt targets
> that they deliver with their tools. For example, I think RAP falls
> into this category and perhaps Riena. If these projects cannot react
> in time then we may have no choice but to wait for SR1 to fix.
>
> Tom


>
>
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
>
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
>
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev


Back to the top