Re: [tycho-user] OutOfMemory

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]

Re: [tycho-user] OutOfMemory - while packing.

From: "Sievers, Jan" <jan.sievers@xxxxxxx>
Date: Tue, 19 Aug 2014 09:00:41 +0000
Accept-language: en-US
Delivered-to: tycho-user@xxxxxxxxxxx
List-archive: <https://dev.eclipse.org/mailman/private/tycho-user>
List-help: <mailto:tycho-user-request@eclipse.org?subject=help>
List-subscribe: <https://dev.eclipse.org/mailman/listinfo/tycho-user>, <mailto:tycho-user-request@eclipse.org?subject=subscribe>
List-unsubscribe: <https://dev.eclipse.org/mailman/options/tycho-user>, <mailto:tycho-user-request@eclipse.org?subject=unsubscribe>
Thread-index: AQHPu4dpJ4vDaFUQk0qkv82tjYq+l5vXmxkA
Thread-topic: [tycho-user] OutOfMemory - while packing.

Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory

        at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)

this doesn't look like a typical java OOM. It rather looks like the plexus archiver trying to spawn a /bin/sh process runs out of resources.

I doubt increasing -Xmx will help, actually it may worsen as the JVM may take more memory from the overall available memory.

See http://stackoverflow.com/questions/1124771/how-to-solve-java-io-ioexception-error-12-cannot-allocate-memory-calling-run

The reason why it forks a shell is to get file permission info (which is not available via Java API prior to Java 7)

We did some work in Tycho 0.21 to create tar archives in-process [1] (using Java 7, avoiding to fork shell processes just to get the file permissions information).

It seems you are experiencing the problem while creating a zip archive. File permission info in zip archives is generally not needed/only supported via proprietary zip file format extensions. One should rather use tar if one needs to preserve file permission info in the archive. So we could think about giving up on file permission info (or introduce a switch) in zip archives if it's causing problems.

Regards,

Jan

[1] https://wiki.eclipse.org/Tycho/Release_Notes/0.21

From: tycho-user-bounces@xxxxxxxxxxx [mailto:tycho-user-bounces@xxxxxxxxxxx] On Behalf Of Christophe Bouhier
Sent: Dienstag, 19. August 2014 10:27
To: Tycho user list
Subject: [tycho-user] OutOfMemory - while packing.

Hi Folks,

I am succesfully using Tycho with various products. One of them (The biggest one), however consistently fails with an Out of memory. See the full-stack-trace [1].

I have tried to increase the memory with MAVEN_OPTS (-Xmx2048m -XX:MaxPermSize=128m), which didn't help. It a VPS server I rent, expanding the phy. memory would cost me.

It fails on the tycho modules which process the .product files, it fails zipping the files.  (Long running time of 43 sec.) Is this seen before? Any ideas on how to tackle this?

Thank You! 
Christophe

[1] Excepetion

 
[INFO] Reactor Summary:

[INFO]

[INFO] releng.netxstudio ................................. SUCCESS [3.320s]

[INFO] netxstudio-ui-plugins ............................. SUCCESS [0.018s]

[INFO] com.netxforge.netxstudio.common ................... SUCCESS [6.344s]

[INFO] com.netxforge.netxstudio.data ..................... SUCCESS [3.600s]

[INFO] com.netxforge.netxscript .......................... SUCCESS [5.672s]

[INFO] com.netxforge.netxscript.ui ....................... SUCCESS [4.775s]

[INFO] com.netxforge.netxstudio.data.fixtures ............ SUCCESS [3.450s]

[INFO] com.netxforge.netxstudio.webservice ............... SUCCESS [3.270s]

[INFO] com.netxforge.netxstudio.data.import .............. SUCCESS [3.470s]

[INFO] com.netxforge.netxstudio.data.ui .................. SUCCESS [3.775s]

[INFO] com.netxforge.netxstudio.help ..................... SUCCESS [3.975s]

[INFO] com.netxforge.netxstudio.screens.editing .......... SUCCESS [5.094s]

[INFO] com.netxforge.netxstudio.screens.xtext ............ SUCCESS [4.204s]

[INFO] com.netxforge.netxstudio.screens .................. SUCCESS [7.087s]

[INFO] com.netxforge.netxstudio.screens.activities ....... SUCCESS [3.460s]

[INFO] netxstudio-server-plugins ......................... SUCCESS [0.051s]

[INFO] com.netxforge.netxstudio.server ................... SUCCESS [3.959s]

[INFO] com.netxforge.netxstudio.server.job ............... SUCCESS [3.431s]

[INFO] com.netxforge.netxstudio.server.logic ............. SUCCESS [3.958s]

[INFO] com.netxforge.netxstudio.server.metrics ........... SUCCESS [3.724s]

[INFO] com.netxforge.netxstudio.server.jsp ............... SUCCESS [3.603s]

[INFO] com.netxforge.netxstudio.log4j .................... SUCCESS [3.147s]

[INFO] netxstudio-features ............................... SUCCESS [0.074s]

[INFO] com.netxforge.netxscript.feature .................. SUCCESS [3.211s]

[INFO] com.netxforge.netxscript.ui.feature ............... SUCCESS [3.302s]

[INFO] netxstudio-product ................................ SUCCESS [0.016s]

[INFO] com.netxforge.netxstudio.client.product ........... SUCCESS [4.230s]

[INFO] com.netxforge.netxstudio.help.feature ............. SUCCESS [3.547s]

[INFO] com.netxforge.netxstudio.screens.feature .......... SUCCESS [3.398s]

[INFO] com.netxforge.netxstudio.screens.xtext.feature .... SUCCESS [3.585s]

[INFO] com.netxforge.netxstudio.data.feature ............. SUCCESS [3.258s]

[INFO] com.netxforge.netxstudio.common.feature ........... SUCCESS [3.468s]

[INFO] com.netxforge.netxstudio.data.import.feature ...... SUCCESS [3.290s]

[INFO] com.netxforge.netxstudio.client.product.feature ... SUCCESS [3.457s]

[INFO] com.netxforge.netxstudio.data.ui.feature .......... SUCCESS [3.276s]

[INFO] com.netxforge.netxstudio.screens.editing.feature .. SUCCESS [3.401s]

[INFO] com.netxforge.netxstudio.server.db.feature ........ SUCCESS [3.315s]

[INFO] com.netxforge.netxstudio.server.db.mysql.product .. SUCCESS [2.857s]

[INFO] com.netxforge.netxstudio.server.db.mysql.product.feature  SUCCESS [3.253s]

[INFO] com.netxforge.releng.products.netxstudio.cdo ...... FAILURE [42.039s]

[INFO] com.netxforge.releng.products.netxserver .......... SKIPPED

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 4:27.453s

[INFO] Finished at: Tue Aug 19 10:16:23 CEST 2014

[INFO] Final Memory: 82M/684M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-p2-director-plugin:0.19.0:archive-products (archive-products) on project com.netxforge.releng.products.netxstudio.cdo: Error packing product: Failed to quote directory: '/var/lib/jenkins/workspace/netxstudio/products/netxstudio/com.netxforge.releng.products.netxstudio.cdo/target/products/com.netxforge.netxstudio/win32/win32/x86': Error while executing process. Cannot run program "/bin/sh": java.io.IOException: error=12, Cannot allocate memory -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.eclipse.tycho:tycho-p2-director-plugin:0.19.0:archive-products (archive-products) on project com.netxforge.releng.products.netxstudio.cdo: Error packing product

        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)

        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)

        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)

        at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)

        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)

        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)

        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)

        at org.jvnet.hudson.maven3.launcher.Maven31Launcher.main(Maven31Launcher.java:132)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:616)

        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)

        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)

        at jenkins.maven3.agent.Maven31Main.launch(Maven31Main.java:181)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:616)

        at hudson.maven.Maven3Builder.call(Maven3Builder.java:134)

        at hudson.maven.Maven3Builder.call(Maven3Builder.java:69)

        at hudson.remoting.UserRequest.perform(UserRequest.java:118)

        at hudson.remoting.UserRequest.perform(UserRequest.java:48)

        at hudson.remoting.Request$2.run(Request.java:328)

        at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

        at java.util.concurrent.FutureTask.run(FutureTask.java:166)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

        at java.lang.Thread.run(Thread.java:679)

Caused by: org.apache.maven.plugin.MojoExecutionException: Error packing product

        at org.eclipse.tycho.plugins.p2.director.ProductArchiverMojo.execute(ProductArchiverMojo.java:143)

        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)

        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)

        ... 31 more

Caused by: org.codehaus.plexus.archiver.ArchiverException: Failed to quote directory: '/var/lib/jenkins/workspace/netxstudio/products/netxstudio/com.netxforge.releng.products.netxstudio.cdo/target/products/com.netxforge.netxstudio/win32/win32/x86'

        at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:472)

        at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.createArchiveMain(AbstractZipArchiver.java:244)

        at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.execute(AbstractZipArchiver.java:222)

        at org.codehaus.plexus.archiver.AbstractArchiver.createArchive(AbstractArchiver.java:910)

        at org.eclipse.tycho.plugins.p2.director.ProductArchiverMojo.execute(ProductArchiverMojo.java:141)

        ... 33 more

Caused by: java.io.IOException: Failed to quote directory: '/var/lib/jenkins/workspace/netxstudio/products/netxstudio/com.netxforge.releng.products.netxstudio.cdo/target/products/com.netxforge.netxstudio/win32/win32/x86'

        at org.codehaus.plexus.components.io.attributes.PlexusIoResourceAttributeUtils.getFileAttributesByPath(PlexusIoResourceAttributeUtils.java:254)

        at org.codehaus.plexus.components.io.attributes.PlexusIoResourceAttributeUtils.getFileAttributesByPath(PlexusIoResourceAttributeUtils.java:193)

        at org.codehaus.plexus.components.io.resources.PlexusIoFileResourceCollection.getResources(PlexusIoFileResourceCollection.java:189)

        at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:468)

        ... 37 more

Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.

        at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:656)

        at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLineAsCallable(CommandLineUtils.java:141)

        at org.codehaus.plexus.components.io.attributes.PlexusIoResourceAttributeUtils.getFileAttributesByPath(PlexusIoResourceAttributeUtils.java:245)

        ... 40 more

Caused by: java.io.IOException: Cannot run program "/bin/sh": java.io.IOException: error=12, Cannot allocate memory

        at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)

        at java.lang.Runtime.exec(Runtime.java:610)

        at java.lang.Runtime.exec(Runtime.java:526)

        at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:636)

        ... 42 more

Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory

        at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)

        at java.lang.ProcessImpl.start(ProcessImpl.java:81)

        at java.lang.ProcessBuilder.start(ProcessBuilder.java:468)

Follow-Ups:
- Re: [tycho-user] OutOfMemory - while packing.
  - From: Christophe Bouhier

References:
- [tycho-user] OutOfMemory - while packing.
  - From: Christophe Bouhier

Prev by Date: Re: [tycho-user] OutOfMemory - while packing.
Next by Date: [tycho-user] How to get pom dependencies in a Manifest-first project correctly?
Previous by thread: Re: [tycho-user] OutOfMemory - while packing.
Next by thread: Re: [tycho-user] OutOfMemory - while packing.
Index(es):
- Date
- Thread

Breadcrumbs