Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] [URGENT] Indigo bug causes framework to keep bundle file handles open for lifetime of framework instance.

I agree with John, I suggest we fix this if the rest of the projects on the release can react in time. But we need to hear from the project this effects and that need to react. EPP, RAP, others?

The two scenarios this bug hits the most are:

1) Any application/product for which the number of bundles approaches the file limit of the OS. First start will fail because the framework must open every bundle installed on first start. Also any restart with -clean option will fail since the manifest have to be re-parsed in this case also. A variation of this is any initial launch of eclipse from PDE will also fail in similar ways if the target includes a number of bundles approaching the OS limit.

2) Any restart of eclipse after performing a p2 operation which updates or installs a number of bundles approaching the OS limit. On restart the framework must open every newly installed/updated bundle to parse the manifest again.

Another thing to consider is the extender pattern which has become popular in OSGi. Extenders typically look for some metadata within each installed bundle to determine if they should act upon or "extend" the bundle. If an extender is installed which does not properly cache these results (for example, the extension registry does extensive caching and is considered an extender) then they may be loading resources from every installed bundle on restart. Such a bundle would also cause issues when the number of installed bundle approaches the OS open file limit.

Tom



Inactive hide details for John Arthorne ---06/13/2011 09:08:59 PM---> Wayne Beaton <wayne@xxxxxxxxxxx>John Arthorne ---06/13/2011 09:08:59 PM---> Wayne Beaton <wayne@xxxxxxxxxxx>


From:

John Arthorne <John_Arthorne@xxxxxxxxxx>

To:

Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>

Date:

06/13/2011 09:08 PM

Subject:

Re: [cross-project-issues-dev] [URGENT] Indigo bug causes framework to keep bundle file handles open for lifetime of framework instance.




> Wayne Beaton <wayne@xxxxxxxxxxx>
> 06/13/2011 06:11 PM

>
> This sounds like a stop-ship kind of bug to me. Or am I over reacting?


It definitely falls into that category. The only argument I can imagine against fixing it, is that this problem existed in previous releases (3.3 and earlier I believe). In 3.4/3.5 you had to set a system property to enable this cache so in those releases there was an easy workaround for the problem. The workaround at this point is increasing your OS per-process file limit, but that means Eclipse will be consuming large amounts of OS resources in these cases. Of course, Eclipse and EPP packages have increased their bundle counts since then so the problem will be hit more frequently.


> Do any of the EPP packages have more than 1024 bundles?


I don't think so, but you can hit this problem by attempting to install "all of indigo" as David was doing when he discovered it.


> I assume that if a package gets close to the 1024 count, there'd
> still be problems as files are opened by the running instance. If I
> understand correctly, this would only affect the first run and
> restarting would clear things up, right?


Well, if the user figures out the workaround of increasing their ulimit, they can restart. If the user doesn't find this workaround they will not be able to restart.


> This would affect adopters/distributions that add a bunch of bundles, right?


Yes, any distro or large scale adopter would probably need to patch this if we didn't fix it.


Really, I think Tom only phrased it as an "if" because we are beyond the normal end of the release train. Projects/people that need to react and respin to accommodate this fix should be involved in making the decision to go ahead with it (EPP, RAP, David, possibly others).


John


>
> Wayne
>
> On 06/13/2011 05:42 PM, Thomas Watson wrote:
> Please see bug
https://bugs.eclipse.org/bugs/show_bug.cgi?id=349105
>
> There has been a regression in the Equinox framework with respect to
> the file handles it keeps open for each installed bundle. In Helios
> the framework defaulted to keeping at most 100 bundle jar files open
> at a time. In Indigo a bug has been introduced that basically
> disables this functionality and causes the framework to keep every
> bundle jar file which gets something loaded from it to be left open
> until the framework is shutdown.
>
> This mostly effects platforms like Linux where the ulimit (number of
> files a process can have open at a time) is limited. I have been
> told that some common Linux distributions limit this to 1024 by
> default. The result is that the platform will not be able to start
> successfully the first time if there are over 1000 bundles
> installed. This is because the framework must open every bundle jar
> to parse its manifest and then the jar file is left open until the
> framework shuts down. The bug is rather severe, but there is a work
> around on some systems by increasing the ulimit. I am not sure if
> that is always feasible and it is far from obvious to the typical user.
>
> We have started a build of the eclipse/equinox in preparation incase
> we decide that this must be fixed for the Indigo release. Let me
> emphasize the statement that "we must decide" if this is important
> enough to fix for Indigo. The Equinox and Eclipse project leadership
> is asking for input on the need to fix this in Indigo and the
> ability for others to react in time to consume the fix. For example,
> I know all the EPPs will have to be respun. I am fairly certain
> other projects will have to respin if they have prebuilt targets
> that they deliver with their tools. For example, I think RAP falls
> into this category and perhaps Riena. If these projects cannot react
> in time then we may have no choice but to wait for SR1 to fix.
>
> Tom


>
>
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
>
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
>
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev


GIF image

GIF image


Back to the top