Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [ptp-dev] New SFTP requirement for remote?

Greg,

Thanks for proposing this option.  Does the java program need to run on the intermediate or the final host?

Rocky

On Mon, Oct 9, 2017 at 7:24 PM, Greg Watson <g.watson@xxxxxxxxxxxx> wrote:
Rocky,

Sorry, too busy lately. There's a new proxy service that only requires a single ssh channel and no sftp channels. It does require java on the remote system, however, as it downloads a small java app that handles the multiplexing and file service at the remote end. Let me know if you'd like to give that a try. 

Regards,
Greg




On Sep 24, 2017, at 4:29 PM, Rocky Dunlap - NOAA Affiliate <rocky.dunlap@xxxxxxxx> wrote:

Greg,

Any word on this?  This is an outstanding issue for a number of folks who connect through an intermediate node with port forwarding disallowed.

For the particular system in question, NASA's Discover, there is a procedure here to set up a "master" ssh connection using "ssh -M".  This allows clients to share the master connection.


You would set up your ~/.ssh/config something like this:

Host dali.nccs.nasa.gov
    User <your user name>
    ControlPath ~/.ssh/master-%r%h:%p
    ControlMaster no
    ServerAliveInterval 900
    ProxyCommand ssh login.nccs.nasa.gov direct %h
    LogLevel Quiet
    Protocol 2
    ConnectTimeout 300

This does work fine for command line tools (e.g., git, rsync, etc.) but I'm not sure if this approach could be used with Eclipse.  The Eclipse connection would need to be set up to use an established master connection.  Who would know whether/how to set up Eclipse to be a client of a master SSH connection?

Rocky

On Wed, Sep 13, 2017 at 3:46 PM, Rocky Dunlap - NOAA Affiliate <rocky.dunlap@xxxxxxxx> wrote:
Greg,

I wanted to see if it we can resolve this issue we discussed a while ago about 2-hop connections. I think you were able to correctly identify that our situation is a 2-hop connection with no forwarding and a restricted shell (see below).  You had proposed modifications to the openMinimal() method that would allow us to establish a connection.

Do you recall if any work had been done on this?  I just tried in the latest Oxygen release and I still get a RemoteConnectionException trying to use the two connection ssh proxy approach in Eclipse.

Rocky

On Thu, Sep 8, 2016 at 11:47 AM, Greg Watson <g.watson@xxxxxxxxxxxx> wrote:
The openMinimal() method Roland added was to support proxy connections where there is a 2-hop process for accessing the target system. The connection to the intermediate machine is just a very basic ssh connection that is used to either set up port forwarding (if the intermediate host allows it) or run a command (usually another ssh command). Once a connection is established to the target system, this connection is opened “fully”, which would then (try to) start an sftp server channel, etc.

Here are the situations that I think need to be supported:

1-hop host:
a) ssh + sftp, >2 channels
b) ssh + sftp, 2 channels 
c) ssh, >1 channels
d) ssh, 1 channel

2-hop intermediate host (same target host options as for 1-hop):
e) forwarding
f) no forwarding, normal shell
g) no forwarding, restricted shell

I’m presuming your system is (g). I think the connection class will need to be changed to deal with this as it still assumes that commands like ‘printenv’ are allowed.

Greg

On Sep 1, 2016, at 9:29 PM, Rocky Dunlap - NOAA Affiliate <rocky.dunlap@xxxxxxxx> wrote:

Greg,

I just tried with Mars 4.5.0 and in also shows the SFTP error.  Apologies, I thought it had worked in Mars.  Then I tried the Luna SR1 and it is also showing the same issue!

Maybe what is happening here is that the system I am connecting to previously had sftp available and it has since been disabled, so now these connections are failing in Luna - Neon, although they previously worked.  (The system in question is NASA Discover.)

I looked back at my interactions with Roland.  It resulted in these changes:


Maybe that will shed light on what changed under the hood.

It must have been that the issue he resolved had more to do with the limited shell--i.e., you cannot run any commands to pull in environment information, but the connection is still established.

Basically, for some of these login nodes they are stripped down and you cannot assume much of anything.  However, it would be preferable if PTP made a "best effort" to establish a connection -- i.e., don't fail if sftp or env. commands are not available.  I think Roland had an openMinimal() in there which is maybe doing just this.

Rocky



On Thu, Sep 1, 2016 at 1:13 PM, Greg Watson <g.watson@xxxxxxxxxxxx> wrote:
Rocky,

I was able to take a look at this today. I’m puzzled however, as I don’t see any changes between Mars and Neon that would be causing this to fail. Can you try a Mars installation to verify that it is working?

Thanks,
Greg

On Aug 22, 2016, at 3:14 PM, Rocky Dunlap - NOAA Affiliate <rocky.dunlap@xxxxxxxx> wrote:

Greg,

I was able to test the change.  Interestingly, I am able to connect to a login node (this is NASA Discover), although I still get an exception:

Thread [pool-3-thread-1] (Suspended (exception JSchException))
ChannelExec(Channel).sendChannelOpen() line: 728
ChannelExec(Channel).connect(int) line: 151
ChannelExec(Channel).connect() line: 145
ExecCommand$1.call() line: 38
ExecCommand$1.call() line: 1
FutureTask<V>.run() line: 266 [local variables unavailable]
ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1142
ThreadPoolExecutor$Worker.run() line: 617
Thread.run() line: 745

This is happening when trying to run the command "printenv" on the connection.  I remember going through this with Roland.  The issue here is that these login nodes have a restricted shell so you cannot run arbitrary commands.  The way I have it set up is that the login connection is used only as a proxy.  Roland had it so that it would not fail even if unable to execute this initial set of commands.

Since the connection was actually established (despite the exception above) I tried to use it as a proxy for another connection.  However, this fails with the following:

Thread [Worker-21] (Suspended (exception JSchException))
owns: JSchConnectionProxyFactory$CommandProxy  (id=288)
ChannelExec(Channel).sendChannelOpen() line: 728
ChannelExec(Channel).connect(int) line: 151
ChannelExec(Channel).connect() line: 145
JSchProcessBuilder.start(int) line: 184
JSchProcessBuilder(AbstractRemoteProcessBuilder).start() line: 157
JSchConnectionProxyFactory$CommandProxy.connect(SocketFactory, String, int, int) line: 108
Session.connect(int) line: 231
JSchConnection.newSession(IProgressMonitor) line: 781
JSchConnection.open(IProgressMonitor, boolean) line: 831
JSchConnection.open(IProgressMonitor) line: 804
OpenConnectionHandler$1.run(IProgressMonitor) line: 51
Worker.run() line: 55

Rocky

On Mon, Aug 22, 2016 at 12:07 PM, Greg Watson <g.watson@xxxxxxxxxxxx> wrote:
Gerrit change set is here: https://git.eclipse.org/r/79393

Let me know if you have problems testing it.

Regards,
Greg

On Aug 12, 2016, at 2:49 PM, Rocky Dunlap - NOAA Affiliate <rocky.dunlap@xxxxxxxx> wrote:


On Fri, Aug 12, 2016 at 12:43 PM, Greg Watson <g.watson@xxxxxxxxxxxx> wrote:
Yes, I don’t think this should be a requirement for general remote ssh connections. Obviously there will be services (e.g. synchronized projects) that will not work in this case. Please open a bug and I’ll take a look.

Thanks
Greg

> On Aug 12, 2016, at 2:30 PM, Rocky Dunlap - NOAA Affiliate <rocky.dunlap@xxxxxxxx> wrote:
>
> We have a number of our customers on government machines without sftp support on login nodes.  Would it be possible to relax the sftp requirement back to how it worked in Mars?  It would be a big help for us.
>
> Thread [Worker-12] (Suspended (exception JSchException))
> ChannelSftp.start() line: 315
> ChannelSftp(Channel).connect(int) line: 152
> ChannelSftp(Channel).connect() line: 145
> JSchConnection.openSftpChannel(Session) line: 1023
> JSchConnection.checkConfiguration(Session, IProgressMonitor) line: 331
> JSchConnection.open(IProgressMonitor, boolean) line: 1008
> JSchConnection.open(IProgressMonitor) line: 970
> OpenConnectionHandler$1.run(IProgressMonitor) line: 51
> Worker.run() line: 55
>
> Thanks,
> Rocky
> _______________________________________________
> ptp-dev mailing list
> ptp-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/ptp-dev

_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev

_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev


_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev

_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev


_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev

_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev


_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev


_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev


_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ptp-dev


Back to the top