Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
AW: [geclipse-dev] srm, gridftp upload problem 
Hi Romain,

Just a quick question. These potentially 10000 files are different files? Or are these just 10000 copies of the same file? In the second case you could upload them once to a SE and then use g-Eclipse to transfer it within the Grid from one SE to the others. Mateusz can give you more details on that if it covers your use case.

In any case the static buffer below is a serious bottleneck. We could do two things

1) File a bug against efs. We should definitely do so.
2) Provide a workaround within our gecl efs-wrapper. What do the gecl-guys think about that? Mateusz? Mariusz?

Cheers, Mathias


-----Ursprüngliche Nachricht-----
Von: geclipse-dev-bounces@xxxxxxxxxxx im Auftrag von Romain
Gesendet: Do 08.01.2009 08:35
An: Developer mailing list
Betreff: Re: [geclipse-dev] srm, gridftp upload problem
 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I think I have understood what goes wrong :

Here is the code of the method transferStreams of the class FileStore:

synchronized (buffer) {
	while (true) {
		int bytesRead = -1;
		try {
			bytesRead = source.read(buffer);
		} catch (IOException e) {
			String msg = NLS.bind(Messages.failedReadDuringWrite, path);
			Policy.error(EFS.ERROR_READ, msg, e);
		}
		if (bytesRead == -1)
			break;
		try {
			destination.write(buffer, 0, bytesRead);
		} catch (IOException e) {
		String msg = NLS.bind(Messages.couldNotWrite, path);
		Policy.error(EFS.ERROR_WRITE, msg, e);
	}
	monitor.worked(1);
}

The buffer object is final and static !!

private static final byte[] buffer = new byte[8192];

This means that only one file transfer appends at time on an instance of eclipse no matter what is the destination
file system.

In my case I have one very slow transfer happening, preventing all others from doing there jobs.. it sucks !

Romain


Romain a écrit :
> Hi Mateusz,
> 
> actually, you where right about the denial of service problem, with a better load and failure management (using more often reliable SE),
> I manage to avoid most of the thread spending long times in that state :
> 
>>> java.net.SocketInputStream.socketRead0(Native Method)
>>> java.net.SocketInputStream.read(SocketInputStream.java:129)
>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>>> java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>>> org.globus.ftp.extended.GridFTPInputStream.readLine(GridFTPInputStream.java:35)
>>> org.globus.ftp.extended.GridFTPInputStream.readMsg(GridFTPInputStream.java:103)
>>> org.globus.gsi.gssapi.net.GssInputStream.hasData(GssInputStream.java:86)
>>> org.globus.gsi.gssapi.net.GssInputStream.read(GssInputStream.java:60)
> 
> So my freezing problem was not because of that, but after a longer run all my threads are freezes here :
> 
> org.eclipse.core.filesystem.provider.FileStore.transferStreams(FileStore.java:70)
> org.eclipse.core.filesystem.provider.FileStore.copyFile(FileStore.java:219)
> org.eclipse.core.filesystem.provider.FileStore.copy(FileStore.java:140)
> org.eclipse.core.internal.filesystem.FileCache.cache(FileCache.java:89)
> org.eclipse.core.filesystem.provider.FileStore.toLocalFile(FileStore.java:468)
> org.simexplorer.efsfile.EFSFile.cache(EFSFile.java:186)
> org.simexplorer.applicationloader.internal.EFSSimulationSerializerImpl.loadJar(EFSSimulationSerializerImpl.java:45)
> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.tryToGetOutput(GEclipseSimExplorerExecutionJob.java:134)
> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.update(GEclipseSimExplorerExecutionJob.java:97)
> org.simexplorer.updater.UpdaterTask.run(UpdaterTask.java:33)
> org.simexplorer.tools.PoolExecutor.run(PoolExecutor.java:22)
> java.lang.Thread.run(Thread.java:619)
> 
> 
> It an eclipse function !!!
> 
> By chance, do anyone have any idea what's going on?
> 
> Rom
> 
> Mateusz Pabis' a écrit :
>> Hi Romain.
> 
>> Happy New Year.
> 
>> Yes. There is one extra thread for each connection to maintain timeouts
>> and keep alive activity. But this should not be a problem with freezing.
>>  Where do other threads come from I don't know. With this amount of
>> active threads you may cause a distributed denial of service attack.
> 
> 
>>> Hi guys,
>>>
>>> happy new year !!!
>>>
>>> I am trying to updload at least 10000 files on diverse storage elements of EGEE (srm and gridftp).
>>>
>>> To do that I use the geclipse implemetation of EFS (copy method) and 200 threads.
>>>
>>> At the begining everything is fine and uploads are very fast, but after a while my application
>>> get very slow and freeze.
>>>
>>> Many thread have been created: 1031 active threads instead of around 200.
>>>
>>> Most of them are waiting:
>>>
>>> **************************************
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:485)
>>> org.globus.ftp.dc.TaskThread$Buffer.get(TaskThread.java:122)
>>> org.globus.ftp.dc.TaskThread.getNextTask(TaskThread.java:55)
>>> org.globus.ftp.dc.TaskThread.run(TaskThread.java:66)
>>> java.lang.Thread.run(Thread.java:619)
>>> **************************************
>>>
>>> I have got also many timer threads (I suppose it is for the timout):
>>>
>>> **************************************
>>> java.lang.Object.wait(Native Method)
>>> java.util.TimerThread.mainLoop(Timer.java:509)
>>> java.util.TimerThread.run(Timer.java:462)
>>> **************************************
>>>
>>> And my user threads are waiting to :( also I have lot of network bandwidth available:
>>>
>>> **************************************
>>> java.net.SocketInputStream.socketRead0(Native Method)
>>> java.net.SocketInputStream.read(SocketInputStream.java:129)
>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>>> java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>>> org.globus.ftp.extended.GridFTPInputStream.readLine(GridFTPInputStream.java:35)
>>> org.globus.ftp.extended.GridFTPInputStream.readMsg(GridFTPInputStream.java:103)
>>> org.globus.gsi.gssapi.net.GssInputStream.hasData(GssInputStream.java:86)
>>> org.globus.gsi.gssapi.net.GssInputStream.read(GssInputStream.java:60)
>>> sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
>>> sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
>>> sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
>>> java.io.InputStreamReader.read(InputStreamReader.java:167)
>>> java.io.BufferedReader.fill(BufferedReader.java:136)
>>> java.io.BufferedReader.readLine(BufferedReader.java:299)
>>> java.io.BufferedReader.readLine(BufferedReader.java:362)
>>> org.globus.ftp.vanilla.Reply.<init>(Reply.java:71)
>>> org.globus.ftp.vanilla.FTPControlChannel.read(FTPControlChannel.java:342)
>>> org.globus.ftp.extended.GridFTPControlChannel.authenticate(GridFTPControlChannel.java:285)
>>> org.globus.ftp.GridFTPClient.authenticate(GridFTPClient.java:107)
>>> org.globus.ftp.GridFTPClient.authenticate(GridFTPClient.java:92)
>>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.connect(GridFTPConnection.java:187)
>>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.getClient(GridFTPConnection.java:476)
>>> eu.geclipse.efs.gridftp.internal.GridFTPConnection.asyncPut(GridFTPConnection.java:716)
>>> eu.geclipse.efs.gridftp.GridFile.openOutputStream(GridFile.java:437)
>>> org.eclipse.core.filesystem.provider.FileStore.copyFile(FileStore.java:218)
>>> org.eclipse.core.filesystem.provider.FileStore.copy(FileStore.java:140)
>>> org.eclipse.core.internal.filesystem.local.LocalFile.copy(LocalFile.java:109)
>>> org.simexplorer.efsfile.EFSFile.copy(EFSFile.java:215)
>>> org.simexplorer.applicationloader.internal.EFSSimulationSerializerImpl.saveJar(EFSSimulationSerializerImpl.java:145)
>>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.uploadInputFile(GEclipseSimExplorerExecutionJob.java:208)
>>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.getInputFile(GEclipseSimExplorerExecutionJob.java:244)
>>> org.simexplorer.geclipse.GEclipseJobBuilder.getJobDescription(GEclipseJobBuilder.java:37)
>>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.getDescription(GEclipseSimExplorerExecutionJob.java:265)
>>> org.simexplorer.geclipse.GEclipseEnvironmentExecutionJob.getDescription(GEclipseEnvironmentExecutionJob.java:33)
>>> org.simexplorer.geclipse.GEclipseEnvironmentExecutionJob.submitJob(GEclipseEnvironmentExecutionJob.java:39)
>>> org.simexplorer.geclipse.GEclipseExecutionJob.submit(GEclipseExecutionJob.java:190)
>>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.submit(GEclipseSimExplorerExecutionJob.java:305)
>>> org.simexplorer.geclipse.GEclipseSimExplorerExecutionJob.update(GEclipseSimExplorerExecutionJob.java:68)
>>> org.simexplorer.updater.UpdaterTask.run(UpdaterTask.java:33)
>>> org.simexplorer.tools.PoolExecutor.run(PoolExecutor.java:22)
>>> java.lang.Thread.run(Thread.java:619)
>>> **************************************
>>>
>>>
>>> Do you have any idea how to tackle this problem?
>>>
>>> Is it an implementation error in globus ftp implementation?
>>>
>>> Cheers,
>>> Romain.
>> _______________________________________________
>> geclipse-dev mailing list
>> geclipse-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/geclipse-dev
> 
>> _______________________________________________
>> geclipse-dev mailing list
>> geclipse-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/geclipse-dev
> 
_______________________________________________
geclipse-dev mailing list
geclipse-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/geclipse-dev

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkllrNAACgkQbUN+hKKXqgsLaQCfbrh4OkzPNDByGxKJhOyQD2Gl
9q0AoMq4rvL98LI3MOIF8agotvRowQlt
=Tasm
-----END PGP SIGNATURE-----
_______________________________________________
geclipse-dev mailing list
geclipse-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/geclipse-dev

Back to the top