|
SSH disconnects with new server
#32073
06 Dec 19 10:49 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
Hi,
After replacing the server at one of our customers, we are experiencing an issue where users are sporadically receive a "SSH Rcv Status -20 (-43)" message and are disconnected. We have SBR=INFLD_KEEPALIVE specified in the miame.ini file. Prior to the upgrade, many users were using TUNNEL to connect to the server. Now, they all use SSH. We have done nothing special in the sshd_config file. Some clients are remote, but others are at the same location as the server.
Could anyone offer some advice of what configuration to change to try to solve?
Thanks, John Andreasen Diversified Data Solutions
|
|
|
Re: SSH disconnects with new server
[Re: John Andreasen]
#32074
06 Dec 19 11:14 PM
|
Joined: Jun 2001
Posts: 11,794
Jack McGregor
Member
|
Member
Joined: Jun 2001
Posts: 11,794 |
I think I've seen this problem on my own development laptop (W10 with CentOS7 running under VMware) after updating to VMware 15.5. The reason for updating was that Windows 10 refused to update any further due to an older version of VMware, so it's hard to say whether it was the W10 update or the VMware update (or something else coincidental). And it has only been a few days since I updated, and I'm often in the middle of several questionable activities when something goes bump, so I hadn't really given it much thought until you brought it up here. Of course now I can't get it to happen.
Those error codes aren't really helpful - that's the typical combination when the connection is lost for uncertain reasons. (For example, if connecting over a VPN, this is what you'll get if you disconnect the VPN while the connections are open.)
This is a difficult problem to debug, since it's infrequent, we don't know how to make it happen, and once it happens, there isn't much left to analyze. (Unless there happened to be something in the ashlog.)
I'm pretty sure that on my end it wasn't connected to an A-Shell update, but I'm not positive. It sounds though like you didn't update A-Shell or ATE, although you did change the connection protocol on some of them, all of which tends to suggest some kind of system-level SSH problem. But I know of several quite large CentOS7 sites that have not reported anything like this, and it's hard to believe that SSH got broken in a Linux update.
We might have to start from the bottom up collecting version information and other details/clues and see if we can't chip away at it.
|
|
|
Re: SSH disconnects with new server
[Re: John Andreasen]
#32075
07 Dec 19 06:15 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
In our development environment, we have been using VMware 15.5 for a while now (for the same reason you noted above.) I can't say that we have had this problem there though.
The server in question runs CentOS 8 as the host OS with the KVM hypervisor managed by Libvirt. The VM with A-Shell is a CentOS 7 guest running on top of that. Many of our servers are configured this way, except this is the only the second one with CentOS 8 on the host. Their former server was running VMware ESXi with a CentOS 6 VM.
The A-Shell version did change with the upgrade, but only from 6.5.1664.4 to 6.5.1665.4.
We'll take a look at the ashlog and the Linux auth/secure logs when it happens again.
Is there any traces you would recommend adding to the miame file? We have INOUT,BASERR,SIGHUP on currently.
Thanks, John
|
|
|
Re: SSH disconnects with new server
[Re: John Andreasen]
#32076
07 Dec 19 08:29 PM
|
Joined: Jun 2001
Posts: 11,794
Jack McGregor
Member
|
Member
Joined: Jun 2001
Posts: 11,794 |
I haven't been able to make the problem happen since yesterday either. I'll try to set up a CentOS 8 for further testing shortly.
As for traces, to the extent that the problem really is related to SSH (as opposed to something other software problem that then leads to the connection crashing), the only trace that I think would relate to the SSH channel on the server side would be DEBUG (SET TRACE DEBUG ON), but it's pretty verbose and also would only really help on the input side. (You didn't mention whether the connection loss seems to occur in the midst of input, output, or idleness; in my case, I think it was idleness.)
ATE maintains its own ashlog.log, and there are some possible transport-related traces that will appear (without having to set any flags), so I would check there as well.
|
|
|
Re: SSH disconnects with new server
[Re: John Andreasen]
#32080
08 Dec 19 08:32 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
OK, I have not been able to reproduce this myself yet. It may be that there is some obscure network problem, and we've been on a wild goose chase. I've been running a continuous ping from a machine in the building to the VM on the server for over a day, and not one packet has been dropped (which makes me question a network problem.) We'll see what happens Monday. Thanks, John
|
|
|
|
|
|