Bug 248913 - [ssh] SSH subsystem loses connection
Summary: [ssh] SSH subsystem loses connection
Status: ASSIGNED
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: 3.0.1   Edit
Hardware: PC Linux
: P2 major (vote)
Target Milestone: ---   Edit
Assignee: Martin Oberhuber CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords:
Depends on: 256581
Blocks:
  Show dependency tree
 
Reported: 2008-09-29 06:09 EDT by Michiel van Leening CLA
Modified: 2012-11-19 04:50 EST (History)
1 user (show)

See Also:


Attachments
RSEF 1002 error message (11.70 KB, image/png)
2008-09-29 06:09 EDT, Michiel van Leening CLA
no flags Details
RSEF1002 error messages in the log (9.68 KB, text/plain)
2008-09-29 13:49 EDT, Michiel van Leening CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michiel van Leening CLA 2008-09-29 06:09:11 EDT
Created attachment 113716 [details]
RSEF 1002 error message

When working with the SSH connector the connection is dropped unexpectedly.

I have tried different settings for keepalive (300s, 10s, 1440s) but nothing seems to fix the problem. I'm connecting from Ubuntu (openssh-client 1:4.7p1-8ubuntu1.2) to SuSE (openssh-4.1p1-10.13)

I have even adjusted the server's sshd_config (ClientAliveCountMax + ClientAliveInterval). I've left TCPKeepAlive to the default (yes).

Steps to reproduce:

1. create a new connection to a linux server using SSH only
2. Use default values
3. start working on several files, opening and saving and closing  them
4. After a while (it varies in time) the message RSEF1002 is given (see attach)

Is there a way for me to debug this? Can i send logfiles? I've set the loglevel in the general preferences for RSE to include informational messages.
Comment 1 Martin Oberhuber CLA 2008-09-29 10:28:03 EDT
1. What version of Eclipse Platform are you running on? - The Keepalive feature
   works with Eclipse 3.4 (Ganymede) only, and not on 3.3 (Europa).

2. After what time do you see the connection drop? Does the time vary or is 
   it always the same?

3. When you use a different SSH client (e.g. your native Ubuntu SSH shell, or
   Ubuntu's sftp client), do you also see the connection drop or not?

4. What kind of work are you performing via RSE SSH? Shell, directory browsing,
   transfer of large numbers of files, Terminal?

5. "How remote" is your host? Is it over the internet, or on a local LAN?

One interesting thing to try out might be if you use the TM-terminal-SDK (Window > Show View > OTher > Terminal >Terminal), make an SSH Connection (with keepalive) but don't type in the Terminal. The "connected" status will show you if it's still connected. Does it disconnect automatically?
Comment 2 Michiel van Leening CLA 2008-09-29 13:24:58 EDT
1. Eclipse Version: 3.4.0, Build id: I20080617-2000

2. the time varies, i have a feeling it happens faster when i perform more actions, but that's hard to say. I generally save every 30-60 sec.

3. Using an Ubuntu ssh client the connection rarely drops, even letting it sit overnight doesn't drop it. Sftp performs the same.

4. I use RSE exclusively for files: browsing, opening and saving approx. 10 different files at a time, copying the occasional files (maybe 2 per session)

5. The host is not really remote, on the internet in a datacentre in the same city (tracepath says 5 hops, avereging 40ms)

The terminal session I created to the same host didn't go offline for 45 minutes. When I first "entered" in the terminal after 25 minutes it was still online. Then i started to traverse a little, vim here, ls there, but the connection stayed open.

I've noticed that RSE doesn't use sftp but ssh, or am I mistaken?

Would it be helpful if i posted some logs? I can clear the logfile and start working, then when i receive a disconnect i can set aside the logfile. Which file does the RSE log to?
Comment 3 Michiel van Leening CLA 2008-09-29 13:49:42 EDT
Created attachment 113776 [details]
RSEF1002 error messages in the log

More info: I found the logfile and am posting the relevant part of the log that contains the error message for the RSEF1002 error.
Comment 4 Martin Oberhuber CLA 2008-09-29 14:39:27 EDT
(In reply to comment #2)
> 4. I use RSE exclusively for files: browsing, opening and saving approx. 10
> different files at a time, copying the occasional files (maybe 2 per session)

I assume the problem is that typical SSH Servers allow 10 concurrent channels only and issue an error when you try opening the 11th channel. We have an existing bug for improving the way how RSE handles multiple Streams (we'd want to re-use existing Channels where possible, and/or open a new Session if needed). 

Additional investigations pending, this issue is likely a duplicate of that other one. Since the improved channel re-use is a bit more work, an easier workaround to fix this issue is probably better reacting to the error that occurs when opening the 11th channel.

If my assumptions are true, the issue should be easy to reproduce when you open 8 or 9 SSH Shells from RSE, and then try opening one or two files in the editor.

BTW, do you open your Editors directly from RSE, or through EFS?
Comment 5 Martin Oberhuber CLA 2008-09-29 14:40:10 EDT
PPS since it looks like you can tweak your SSH server, as a workaround you might be able to increase the number of channels that it accepts.
Comment 6 Michiel van Leening CLA 2008-09-29 15:07:09 EDT
I'll try your sugestion of opening 8 or 9 ssh shells tomorrow. I open all files through the RSE (I assume that's the Remote Systems view).
Comment 7 Michiel van Leening CLA 2008-09-30 05:09:39 EDT
I've tried opening 10 Terminal windows. That worked well. Then when i opened a file through RSE i could save the file without problems.

From what i read in the sshd manpage the maximum of 10 refers to MaxStartups
(Specifies the maximum number of concurrent unauthenticated connections to the SSH daemon)

I cannot find a channel setting.
Comment 8 Martin Oberhuber CLA 2008-09-30 07:25:12 EDT
The TM Terminal is not representative here, since each Terminal view opens a new SSH Session (see SshConnection#run() method).

Therefore, I asked you to open 8 RSE Command views (from RSE, select any folder on your SSH connection, and right-click > Launch Shell or right-click > Launch Terminal.

In RSE, each SSH Connection as seen in the RSE SystemView Tree uses only ONE session and each service (shell, terminal, files, download) uses one channel.

Following message seems related and might be interesting for you:
http://sourceforge.net/mailarchive/forum.php?thread_name=5a2cf1f60712240508w45a7d730v10e0597c5bb4df64%40mail.gmail.com&forum_name=jsch-users

This one may also be related:
https://bugzilla.mindrot.org/show_bug.cgi?id=1090
Comment 9 Martin Oberhuber CLA 2008-09-30 07:54:45 EDT
FYI, the related bug is bug 230831 -- especially read about the OpenSSH bug involved. Bug 209090 is also related.
Comment 10 Martin Oberhuber CLA 2008-11-26 04:47:29 EST
The enhancement requesting a channel pool to restrict the number of SSH channels per session is now being tracked by bug 256581.
Comment 11 Hemang Lavana CLA 2008-12-24 09:59:00 EST
Adding myself to CC list
Comment 12 Martin Oberhuber CLA 2010-02-26 19:11:22 EST
Bulk update of target milestones to 3.2
Comment 13 Martin Oberhuber CLA 2011-05-31 17:49:21 EDT
Bulk moving 3.3 deferred items to 3.3.1