Re: [geclipse-dev] Timeout problem

[Romain <reuillon@xxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Mateusz,

Mateusz Pabis' a écrit :
> Hi,
> 
> sorry for the delay,

no problem I have implemented timeout at the application level it is working now ! :)

> 
> As far as I can see it locks when disconnecting. Globus has already some
> mechanisms to perform timeouts (that's why I introduced keep alive
> activity).
> 
> I have a question. Is it always last 5 jobs or is it always all 9995+
> threads that lock?

I can't answer that question, in fact. When I submit 10000 jobs, at least 5 block forever.

In my application I have 200 threads to manage all the jobs, refresh their status, get the
results, resubmit failling job, submit new jobs if some are taking too long.

At the end of an execution, some jobs that belong to the first submission wave (10000jobs)
are still not submitted or blocked in job state.

Rom.

> 
> Does this last 5 transfers succeed or failed?
> 
>> Hi guys,
>>
>> I think there is a problem with the file transfers having sometimes no timeout.
>>
>> After the sucessfull execution of 9995 jobs out of 10000 the last 5 ones stays blocked forever.
>>
>> The stack traces are :
>>
>> java.lang.Object.wait(Native Method)
>> java.lang.Thread.join(Thread.java:1143)
>> java.lang.Thread.join(Thread.java:1196)
>> org.globus.ftp.dc.TaskThread.join(TaskThread.java:88)
>> org.globus.ftp.dc.TransferThreadManager.stopTaskThread(TransferThreadManager.java:310)
>> org.globus.ftp.dc.TransferThreadManager.close(TransferThreadManager.java:316)
>> org.globus.ftp.extended.GridFTPServerFacade.close(GridFTPServerFacade.java:489)
>> org.globus.ftp.FTPClient.close(FTPClient.java:951)
>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.disconnect(GridFTPConnection.java:132)
>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.asyncPut(GridFTPConnection.java:725)
>> eu.geclipse.efs.gridftp.GridFile.openOutputStream(GridFile.java:437)
>> eu.geclipse.core.filesystem.internal.filesystem.GEclipseFileStore.openOutputStream(GEclipseFileStore.java:491)
>> eu.geclipse.glite.WMSClient.sendInputFileToSandbox(WMSClient.java:763)
>> eu.geclipse.glite.WMSClient.sendInputFilesToSandbox(WMSClient.java:717)
>> eu.geclipse.glite.WMSClient.submit(WMSClient.java:872)
>> eu.geclipse.glite.WMSClient.submitJob(WMSClient.java:1144)
>> eu.geclipse.glite.GliteJobAPI.submitJob(GliteJobAPI.java:124)
>> eu.geclipse.glite.resources.GliteWMSService.submitJob(GliteWMSService.java:75)
>> org.simexplorer.geclipse.GEclipseLimitedAccessJobService.submitJob(GEclipseLimitedAccessJobService.java:60)
>> org.simexplorer.geclipse.GEclipseLimitedAccessJobService.submitJob(GEclipseLimitedAccessJobService.java:83)
>> org.simexplorer.geclipse.GEclipseEnvironmentExecutionJob.submitJob(GEclipseEnvironmentExecutionJob.java:38)
>> org.simexplorer.geclipse.GEclipseExecutionJob.submit(GEclipseExecutionJob.java:187)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.submit(GEclipseSimExplorerExecutionJob.java:343)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.update(GEclipseSimExplorerExecutionJob.java:56)
>> org.simexplorer.updater.UpdaterTask.run(UpdaterTask.java:33)
>> org.simexplorer.tools.PoolExecutor.run(PoolExecutor.java:22)
>> java.lang.Thread.run(Thread.java:619)
>>
>> or :
>>
>> java.lang.Object.wait(Native Method)
>> java.lang.Thread.join(Thread.java:1143)
>> java.lang.Thread.join(Thread.java:1196)
>> org.globus.ftp.dc.TaskThread.join(TaskThread.java:88)
>> org.globus.ftp.dc.TransferThreadManager.stopTaskThread(TransferThreadManager.java:310)
>> org.globus.ftp.dc.TransferThreadManager.close(TransferThreadManager.java:316)
>> org.globus.ftp.extended.GridFTPServerFacade.close(GridFTPServerFacade.java:489)
>> org.globus.ftp.FTPClient.close(FTPClient.java:951)
>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.disconnect(GridFTPConnection.java:132)
>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.asyncPut(GridFTPConnection.java:725)
>> eu.geclipse.efs.gridftp.GridFile.openOutputStream(GridFile.java:437)
>> org.simexplorer.efsfile.EFSFile.openOutputStream(EFSFile.java:155)
>> org.simexplorer.efsfile.EFSFile.copy(EFSFile.java:228)
>> org.simexplorer.applicationloader.internal.EFSSimulationSerializerImpl.saveJar(EFSSimulationSerializerImpl.java:161)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.uploadInputFile(GEclipseSimExplorerExecutionJob.java:227)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.getInputFile(GEclipseSimExplorerExecutionJob.java:275)
>> org.simexplorer.geclipse.GEclipseJobBuilder.getJobDescription(GEclipseJobBuilder.java:37)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.getDescription(GEclipseSimExplorerExecutionJob.java:303)
>> org.simexplorer.geclipse.GEclipseEnvironmentExecutionJob.getDescription(GEclipseEnvironmentExecutionJob.java:27)
>> org.simexplorer.geclipse.GEclipseEnvironmentExecutionJob.submitJob(GEclipseEnvironmentExecutionJob.java:38)
>> org.simexplorer.geclipse.GEclipseExecutionJob.submit(GEclipseExecutionJob.java:187)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.submit(GEclipseSimExplorerExecutionJob.java:343)
>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.update(GEclipseSimExplorerExecutionJob.java:56)
>> org.simexplorer.updater.UpdaterTask.run(UpdaterTask.java:33)
>> org.simexplorer.tools.PoolExecutor.run(PoolExecutor.java:22)
>> java.lang.Thread.run(Thread.java:619)
>>
>>
>> I am implementing timeouts at the application level now, but I think it might be better to have it directlty in the gridftp layer.
>>
>> May you look into it?
>>
>> Romain. 
> _______________________________________________
> geclipse-dev mailing list
> geclipse-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/geclipse-dev
> 
> 
> _______________________________________________
> geclipse-dev mailing list
> geclipse-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/geclipse-dev

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkl3NCUACgkQbUN+hKKXqgsiUwCgzHLT5qiFo4sZOOmer47ijSzD
lVsAn3fJTR6E4vlxwI25wI5hbn/xU/z4
=aMER
-----END PGP SIGNATURE-----