uptermd with websockets: "FATA[0006] EOF" and "error waiting for pipe" error since 0.13.1

[meise]

Buildung uptermd and running version 0.13.0 sessions are working. Using version 0.13.1 or higher results in FATA[0006] EOF errors:

Host:

:~$ upterm host --server ws://0.0.0.0:2223                                                                            2024-02-06 12:14:52
=== S7EAMGIBCCJ5L5OZRKSA                                                                                                                                                      
Command:                /usr/bin/zsh                                                                                                                                         
Force Command:          n/a                                                                                                                                                  
Host:                   ws://0.0.0.0:2223                                                                                                                                    
Authorized Keys:        n/a                                                                                                                                                  
SSH Session:            ssh -o ProxyCommand='upterm proxy ws://S7EAmGiBCCJ5L5ozRksA:MTI3LjAuMC4xOjIyMjI=@0.0.0.0:2223' S7EAmGiBCCJ5L5ozRksA:MTI3LjAuMC4xOjIyMjI=@0.0.0.0:2223

Run 'upterm session current' to display this screen again

Press <q> or <ctrl-c> to accept connections...
No matches.
:~$ Error: EOF
Usage:
  upterm host [flags]

Examples:
  # Host a terminal session running $SHELL, attaching client's IO to the host's:
  upterm host

  # Accept client connections automatically without prompts:
  upterm host --accept

  # Host a terminal session allowing only specified public key(s) to connect:
  upterm host --authorized-keys PATH_TO_AUTHORIZED_KEY_FILE

  # Host a session executing a custom command:
  upterm host -- docker run --rm -ti ubuntu bash

  # Host a 'tmux new -t pair-programming' session, forcing clients to join with 'tmux attach -t pair-programming':
  upterm host --force-command 'tmux attach -t pair-programming' -- tmux new -t pair-programming

  # Use a different Uptermd server, hosting a session via WebSocket:
  upterm host --server wss://YOUR_UPTERMD_SERVER -- YOUR_COMMAND

Flags:
      --accept                   Automatically accept client connections without prompts.
      --authorized-keys string   Specify a authorize_keys file listing authorized public keys for connection.
  -f, --force-command string     Enforce a specified command for clients to join, and link the command's input/output to the client's terminal.
      --github-user strings      Authorize specified GitHub users by allowing their public keys to connect. Configure GitHub CLI environment variables as needed; see https://cli.github.com/manual/gh_help_environment for details.
      --gitlab-user strings      Authorize specified GitLab users by allowing their public keys to connect.
  -h, --help                     help for host
      --known-hosts string       Specify a file containing known keys for remote hosts (required). (default "/home/user/.ssh/known_hosts")
  -i, --private-key strings      Specify private key files for public key authentication with the upterm server (required). (default [/home/user/.ssh/id_ed25519,/home/user/.ssh/id_rsa])
  -r, --read-only                Host a read-only session, preventing client interaction.
      --server string            Specify the upterm server address (required). Supported protocols: ssh, ws, wss. (default "ssh://uptermd.upterm.dev:22")
      --srht-user strings        Authorize specified SourceHut users by allowing their public keys to connect.

FATA[0006] EOF  

Upterm join:

:~$ ssh -o ProxyCommand='upterm proxy ws://S7EAmGiBCCJ5L5ozRksA:MTI3LjAuMC4xOjIyMjI=@0.0.0.0:2223' S7EAmGiBCCJ5L5ozRksA:MTI3LjAuMC4xOjIyMjI=@0.0.0.0:2223
No matches.
:~$ dError: websocket: close 1006 (abnormal closure): unexpected EOF                                                  2024-02-06 12:15:56
                                                                                        Usage:
                                                                                                upterm proxy [flags]
                                                                                                                    
                                                                                                                    Examples:
                                                                                                                               # Host shares a session running $SHELL over WebSocket:
                         upterm host --server wss://uptermd.upterm.dev -- YOUR_COMMAND
                                                                                      
                                                                                        # Client connects to the host session via WebSocket:
                                                                                                                                              ssh -o ProxyCommand='upterm proxy wss://TOKEN@uptermd.upterm.dev' TOKEN:uptermd.uptermd.dev:443
                                                                               
                                                                               Flags:
                                                                                       -h, --help   help for proxy
                                                                                                                  
                                                                                                                  FATA[0001] websocket: close 1006 (abnormal closure): unexpected EOF 
                        client_loop: send disconnect: Broken pipe
:~$ efefefefe  

uptermd log:

INFO[0000] starting server                               app=uptermd network=mem network-opt="[]" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"
INFO[0002] dialing sshproxy sshd                         app=uptermd com=ws-sshproxy-dialer host=user network=mem network-opt="[]" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" sshproxy-addr="127.0.0.1:2222" ws-addr="[::]:2223"
INFO[0002] dialing sshd                                  app=uptermd com=ssh-conn-dialer host=user network=mem network-opt="[]" node="127.0.0.1:2222" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"
INFO[0002] attempt to bind                               app=uptermd com=sshd network=mem network-opt="[]" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" tunnel-host=S7EAmGiBCCJ5L5ozRksA tunnel-port=0 ws-addr="[::]:2223"
INFO[0007] dialing sshproxy session                      addr="127.0.0.1:2222" app=uptermd com=ws-sshproxy-dialer network=mem network-opt="[]" node-addr="127.0.0.1:2222" session=S7EAmGiBCCJ5L5ozRksA ssh-addr="127.0.0.1:2222" sshproxy-addr="127.0.0.1:2222" ws-addr="[::]:2223"
INFO[0007] dialing session                               addr="127.0.0.1:2222" app=uptermd com=ssh-conn-dialer network=mem network-opt="[]" node="127.0.0.1:2222" node-addr="127.0.0.1:2222" session=S7EAmGiBCCJ5L5ozRksA ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"
DEBU[0009] error waiting for pipe                        addr="127.0.0.1:42878" app=uptermd com=ssh-proxy error=EOF network=mem network-opt="[]" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"
ERRO[0009] error piping                                  app=uptermd com=ws-proxy error="readfrom tcp 127.0.0.1:48936->127.0.0.1:2222: websocket: close 1006 (abnormal closure): unexpected EOF" network=mem network-opt="[]" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"
DEBU[0009] error waiting for pipe                        addr="127.0.0.1:48936" app=uptermd com=ssh-proxy error=EOF network=mem network-opt="[]" node-addr="127.0.0.1:2222" ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"
DEBU[0009] error handling ssh session                    app=uptermd com=stream-local-handler error=closed network=mem network-opt="[]" node-addr="127.0.0.1:2222" session-id=S7EAmGiBCCJ5L5ozRksA ssh-addr="127.0.0.1:2222" ws-addr="[::]:2223"

[owenthereal]

What's the output of upterm version and ssh -V? The error websocket: close 1006 (abnormal closure): unexpected EOF means the websocket connection was closed (for whatever reason)

[meise]

:~$ upterm version                                                                                           2024-02-06 23:35:41
Upterm version v0.13.0
:~$ ssh -V                                                                                                            2024-02-06 23:35:43
OpenSSH_9.6p1, OpenSSL 3.2.1 30 Jan 2024

[lukasjuhrich]

Observing the same thing starting with 0.13.1:

• setup: docker run --rm -p 8080:8080 -p 2222:2222 ghcr.io/owenthereal/upterm/uptermd:v0.13.1 --ws-addr 0.0.0.0:8080 --ssh-addr 0.0.0.0:2222
• host: either upterm host --server ws://0.0.0.0:8080 --accept or upterm host --server ssh://0.0.0.0:2222
• join: copy the command, add a -F /dev/null to the ssh command for reproducibility

Then, typing in the host window will be duplicated without problems, however once pressing a single char in the joined session, it exits with Broken pipe as described above.

A key need not be pressed if using -f tmux new-session -t test on the host command.

ssh -V: OpenSSH_9.5p1, OpenSSL 3.1.4 24 Oct 2023

Swapping the v0.13.1 with the v0.13.0 docker image will make it work.

[rjshrjndrn]

Hey @owenthereal Can you please take a look at the issue. Some remote instances, webproxy is the only option :(