Bug 276190 - EMF Interim update site broken
Summary: EMF Interim update site broken
Status: RESOLVED WONTFIX
Alias: None
Product: EMF
Classification: Modeling
Component: Releng (show other bugs)
Version: 2.5.0   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Nick Boldt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 212203 271486
Blocks: 250525
  Show dependency tree
 
Reported: 2009-05-13 16:36 EDT by Anthony Hunter CLA
Modified: 2018-01-22 11:58 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Anthony Hunter CLA 2009-05-13 16:36:43 EDT
I promoted an EMF Query, EMF Transaction and EMF Validation build to downloads.eclipse.org.

I thought this broke http://download.eclipse.org/modeling/emf/updates/interim/ , but it would appear that the interim download site was broken before this.

When you try to access the update site from P2, you get the error:

java.io.IOException: The element type "copyright" must be terminated by the matching end-tag "</copyright>".

and p2 refuses to use the update site.

Note that milestones site is fine.
Comment 1 Anthony Hunter CLA 2009-05-13 16:41:45 EDT
Nick thought it was:

> So, the question becomes... is someone sticking incomplete HTML in 
> feature such that
>       <copyright>
>         %copyright
>       </copyright>
> becomes unparseable XML?
> And if so... which of the 5 EMF components contributes this problem?

But these exist in the milestones site as well and it is fine.
Comment 2 Anthony Hunter CLA 2009-05-13 17:15:18 EDT
I loaded the contents.xml from the p2 contents.jar from both the interim and milestones site. There is some "fishy" xml in the interim site, as we know.

Do you know of any way to rebuild this and remove the old entries?

We really only need to integration builds since M6, there are I builds from Galileo in there. 

Comment 3 Nick Boldt CLA 2009-05-13 17:52:25 EDT
Another case where metadata gets hosed: bug 250525. Perhaps the issue here is that the metadata generator runs out of space/ram/permgen/whatever and crashes, producing garbage but not reporting an error in the log?
Comment 4 Nick Boldt CLA 2009-05-13 18:30:45 EDT
To ensure less metadata is needed per site, I've implemented the suggested fix in bug 271486.
Comment 5 Thomas Hallgren CLA 2009-05-18 04:11:03 EDT
(In reply to comment #3)
> ... Perhaps the issue here is
> that the metadata generator runs out of space/ram/permgen/whatever and crashes,
> producing garbage but not reporting an error in the log?
> 
I think it would be very valuable if a bugzilla with a patch that triggers the problem were submitted to P2. Silently ignoring OutOfMemoryErrors would indeed be very serious. 
Comment 6 Nick Boldt CLA 2009-05-18 10:52:20 EDT
Agreed, but the EMF update site is 903M atm, which is a little big to stick on a bugzilla, and therefore difficult to reproduce. 

Has anyone ever tested to see how large a content.xml can be produced before the metadata generator OOMs out? 

Or why these files compress so well? Seems like there's a lot of duplicated information if:

$ du -shc /tmp/content.xml /tmp/artifacts.xml 
17M	/tmp/content.xml
1.3M	/tmp/artifacts.xml
18M	total

$ du -shc content.jar artifacts.jar 
665K	content.jar
68K	artifacts.jar
733K	total

That's compression of 94-96%! Obviously not something we can fix for 3.5, but perhaps there's a better way to store this information so the XML files are smaller and less OOM-prone?

I suppose I could switch over to use the Publisher, now that it's available and semi-documented.
Comment 7 Thomas Hallgren CLA 2009-05-18 11:49:56 EDT
(In reply to comment #6)
> Agreed, but the EMF update site is 903M atm, which is a little big to stick on
> a bugzilla, and therefore difficult to reproduce. 
> 
Perhaps the attached sample can make assumptions about where to find the update site rather then actually including it?
Comment 8 Jacek Pospychala CLA 2009-05-19 07:19:17 EDT
To exclude potential memory issue, you can:
1. check what happens if you increase heap/perm memory usage? (still fails?)
2. switch JVM to generate coredump on out of memory. That's by default in IBM JVM, and you'd have to check in docs for SUN jvm.

on a side note, There are some adopter products that on daily basis generate p2 repositories xmls of 20MB size. p2 is then a bit slow but generated xmls are correct. See bug 251561 for example.
Comment 9 Ed Merks CLA 2018-01-22 11:58:06 EST
I'm resolving all old releng bugs because the build has been replaced with a Tycho build:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=529487

And the update site has been superseded by:

http://download.eclipse.org/modeling/emf/emf/builds/

If there are outstanding issues, please report a new bug based on the new build infrastructure and the new update site.