Open SSH character encoding issue

Question

I have been asked to replace an old insecure version of FreeSSH with OpenSSH on a couple of our servers, and am now having a problem that appears to related to character encoding types.

We have a batch file and python/fabric scripts which perform an installation of some software.

The script starts an ssh session, which in the case of this particular installation happens to be to a different user on the same server (but could be different as the same generic code is used to install other software which could be on different servers).

The script runs commands, which are returning garbage/corrupted characters e.g. at some point it runs a ver command to check what type of operating system it is running on and gets back [2J rather than Microsoft Windows ..

Getting the same issues on a windows 2016 server with open ssh 9.2.2.2.beta and on a 2019 server with open ssh for windows v 7.7.2.1

I thought this might fix the issue but it did not (i checked with the chcp command that it was set correctly) https://stackoverflow.com/questions/57131654/using-utf-8-encoding-chcp-65001-in-command-prompt-windows-powershell-window/57134096#57134096

Not sure why this worked in free ssh but not in open ssh or what I need to do to fix it.

If I need to provide any more details please advise, many thanks for any help.

Additional details:

There is a Jenkins server running on linux, on there is a job which runs a batch file on a windows machine to perform an installation of some software. The batch file runs a python script which runs a command which creates another ssh session, which when performing this particular installation happens to be to a different user on the same server.

To check which operating system it is running on, it initially assumes it is running on a windows server and runs a ver command which rather than returning Microsoft Windows.. returns some garbage characters. i.e. :

ver_command = runw(ctx, "ver", warn=True, hide=True, echo=False)
if ver_command.ok and ver_command.stdout.lstrip().startswith('Microsoft Windows'): // This contains garbage
 os_name = 'windows'
def runw(ctx, cmd, **kwargs):
 """ Run a remote command, assuming a remote windows (dos) environment """
 kwargs.setdefault("echo", True)
 kwargs.setdefault("shell", False)
 kwargs.setdefault("pty", True)
 return ctx.run("cmd /c " + cmd, **kwargs)

in the log file it shows the following when doing the run command : File "c:\python39\lib\site-packages\fabric\connection.py", line 634, in open self.client.connect(**kwargs) File "c:\python39\lib\site-packages\paramiko\client.py", line 368, in connect

Having made a change to temporarily hardcode that part the installation job then goes through successfully, though in the log entries shown in Jenkins again there are garbage characters e.g. [12;1HBuilding script... [13;1H Neither issue existed with FreeSSH.

• Community: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
• Kamil Maciorowski: (1) "replace FreeSSH with OpenSSH", "with open ssh 9.2.2.2.beta" etc. – It's not clear where you refer to the SSH client side and where you refer to the SSH server side. Even within a single machine you can use a client not related to the SSH server software in use. (2) Please post an example command and its actual unexpected output, and the expected output. More than one example maybe. (3) Is there anything non-standard in the configuration of the client? the server? What is it? (4) I wonder if this [2J is a remnant of the ANSI escape sequence to clear the screen: \e[2J.
• grawity: Which build of OpenSSH are you using – the one provided by Microsoft, or the one from Cygwin, or something else?
• Richard King: re. versions on the 2016 server i installed OpenSSH 9.2.2.2.beta from github.com/PowerShell/Win32-OpenSSH/releases, on the 2019 server i ran Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0, the ssh client and server are both version 0.0.1.0 as reported by this command : Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*'
• Richard King: Thanks for your replies, i have added some more details but will review that shortly to see if i can improve the description of the issue, i am not very familiar with ssh so any help is much appreciated.

Answer 1

gets back [2J rather than Microsoft Windows ..

...

again there are garbage characters e.g. [12;1HBuilding script... [13;1H

Kamil Maciorowski is correct, Those "garbage" characters are ANSI escape sequences.

Esc 2 J is asking your SSH client to clear the screen and move the cursor to top left.

Esc [ 12; 1 H is asking your SSH client to move the cursor to row 12 column 1.

So this is not a character encoding issue.

The problem is that some software, probably at the server, thinks your client is a character mode terminal (or terminal emulator) that supports ANSI control sequences. A wild guess is that this might be because the OpenSSH client sends a TERM value that suggests this but the old FreeSSH didn't.

I would diagnose this by trying commands such as ssh username@servername ver

From a Linux system you might then try TERM=dumb ssh username@servername ver to tell the server your ssh client doesn't understand ANSI.

See also https://serverfault.com/q/986847/55524

---

As Kenster noted in a comment regarding your kwargs.setdefault("pty", True) "Try running without a PTY. PTYs are for interactive sessions."

Programs that emit ANSI sequences often only do so if they believe they are connected to an interactive terminal session. They don't do so when connected to a pipeline, run as a scheduled cron command or script etc. A typical example might be ls - which often has a default behaviour equivalent to option -color=AUTO. Admittedly I'm surprised this applies to Microsoft Windows' ver command or cmd's prompt etc.

• Richard King: Okay thankyou for that useful info, I will try to diagnose based on your suggestion.
• Kenster: kwargs.setdefault("pty", True) Try running without a PTY. PTYs are for interactive sessions.
• RedGrittyBrick: @Kenster: That seems very plausible as a cause - I'll add that to the answer.
• Richard King: Of course I should have realised that this was not an encoding error, because otherwise all the text in the log messages would be unreadable not just the parts at the start and end. I tried running ssh username@servername ver from command prompt - that works okay, returns expected "microsoft windows.." text. Also tried setting Pty to false, but still doesn't work.
• Richard King: Hello, we got this working in the end. By setting "pty" to false but also changing the line ctx.run("cmd /c " + cmd, **kwargs) to simply ctx.run(cmd, **kwargs), then it works okay. Thanks again for your help and suggestions.