Connecting Qwen Code to vLLM on SAS HPC clusters¶

This setup allows Qwen Code running in a local Singularity container to connect to a vLLM instance running on a GPU compute node of the Perun HPC cluster.

Communication between the workstation and the cluster is established through an SSH tunnel. The vLLM API is not exposed externally and listens exclusively on 127.0.0.1 on the allocated compute node.

Architecture¶

The connection consists of three components:

Workstation: Runs qwen_remote.sh and Qwen Code inside a local Singularity container.
Perun login node: Provides SSH access to the cluster, SLURM job discovery and an intermediate SSH tunnel.
Compute node: Runs the vLLM server inside a SLURM allocation.

The connection looks as follows:

Workstation                    Perun login node       Compute node
------------------------       ----------------       ------------
qwen_remote.sh                 SSH                    vLLM
Singularity container   --->   tunnel          --->   127.0.0.1:<port>
127.0.0.1:<local_port>

The SSH tunnel consists of two forwarding stages:

The workstation forwards its local port to a port on the Perun login node.
The login node forwards that port to the vLLM server on the allocated compute node.

All connection parameters are discovered at runtime. The client automatically determines:

Running vLLM SLURM job.
Allocated compute node.
vLLM server port.
Served model name.
API key.

Consequently, changing the model, restarting the job or receiving a different compute node allocation does not require modifying the workstation configuration.

1. Prerequisites¶

Workstation¶

The workstation requires:

SSH client.
Singularity/Apptainer.
curl.
Bash.
A local Qwen Code Singularity image.

Perun¶

The cluster requires:

SSH access to the Perun login node.
SLURM.
A running vLLM job.
Python 3 on the login node (used to select an available intermediate port).
Passwordless SSH connectivity from the login node to compute nodes.

File layout¶

The workstation-side files are organized as follows:

~/llm/
├── config.env
├── qwen_remote.sh
└── Containers/
    └── Qwen_code_x86_64_Nov2026.sif

The scripts can be stored elsewhere. By default, qwen_remote.sh looks for config.env in the same directory.

The configuration location can be overridden using:

export QWEN_REMOTE_CONFIG=/path/to/config.env

Singularity image and SLURM integration¶

The provided Singularity image is intended for running Qwen Code on a workstation and connecting to the remote vLLM service on Perun.

It does not necessarily include SLURM client utilities or the configuration required to communicate with a local HPC cluster.

If Qwen Code needs to interact with a local SLURM environment, additional configuration is required.

There are two approaches:

Bind existing SLURM installation: Bind the host's SLURM client binaries, libraries and configuration files into the container. The container must have compatible dependencies and access to the appropriate authentication mechanisms.
Build a dedicated image: Create a custom Singularity image containing the required SLURM client utilities, libraries and configuration.

The required setup depends on the local SLURM installation, authentication mechanism and cluster configuration.

For remote interaction with Perun through the provided SSH tunnel, SLURM does not need to be installed inside the workstation container. SLURM commands are executed remotely through SSH.

2. SSH configuration¶

Passwordless SSH access is required because the launcher uses BatchMode=yes for automated SSH operations.

2.1. Generate an SSH key¶

On the workstation, generate a dedicated SSH key:

ssh-keygen -t ed25519 -C "user@workstation -> perun" \
    -f ~/.ssh/id_ed25519_perun

A passphrase can be configured. If one is used, an SSH agent must be running and the key must be loaded before starting Qwen Code.

The command creates:

~/.ssh/id_ed25519_perun - private key.
~/.ssh/id_ed25519_perun.pub - public key.

The private key must remain on the workstation.

2.2. Install the public key¶

Copy the public key to the Perun login node:

ssh-copy-id -i ~/.ssh/id_ed25519_perun.pub \
    user@<perun-login-hostname>

Alternatively, install it manually:

cat ~/.ssh/id_ed25519_perun.pub | \
    ssh user@<perun-login-hostname> \
    'mkdir -p ~/.ssh && chmod 700 ~/.ssh &&
     cat >> ~/.ssh/authorized_keys &&
     chmod 600 ~/.ssh/authorized_keys'

2.3. Configure SSH¶

Add the following entry to ~/.ssh/config on the workstation:

Host perun
    HostName <perun-login-hostname>
    User <username>
    IdentityFile ~/.ssh/id_ed25519_perun
    IdentitiesOnly yes
    ServerAliveInterval 30
    ServerAliveCountMax 3

    # Reuse an existing SSH connection.
    ControlMaster auto
    ControlPath ~/.ssh/cm-%r@%h:%p
    ControlPersist 10m

Set the appropriate permissions:

chmod 600 ~/.ssh/config

Test the connection:

ssh perun true && echo "login ok"

The command must complete without requesting a password.

The SSH alias (perun in this example) is referenced by the workstation-side config.env.

Unlike a direct SSH ProxyJump connection, the current implementation uses the Perun login node to establish the second SSH connection.

Therefore, the login node must be able to connect to allocated compute nodes without requesting a password.

Test the connection:

ssh perun "ssh -o BatchMode=yes <compute-node> hostname"

If the command completes successfully, no additional configuration is necessary.

If a password is requested, configure a dedicated internal SSH key on Perun.

Log in to the cluster:

ssh perun

Generate an internal key:

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_internal -N ""

Install the public key:

cat ~/.ssh/id_ed25519_internal.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Configure SSH to use the internal key:

cat >> ~/.ssh/config <<'EOF'
Host *
    IdentityFile ~/.ssh/id_ed25519_internal
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null
EOF

chmod 600 ~/.ssh/config

Test again from the workstation:

ssh perun "ssh -o BatchMode=yes <compute-node> hostname"

The command must complete without requesting a password.

The compute nodes may regenerate their SSH host keys. The launcher therefore disables host-key verification for the internal connection.

3. API key configuration¶

The API key is shared between the vLLM server and the Qwen Code client.

It is stored on Perun in:

~/.llm_api_key

Create the key once:

openssl rand -hex 24 > ~/.llm_api_key
chmod 600 ~/.llm_api_key

The server-side run_vllm.sh reads this file when starting vLLM.

The workstation-side qwen_remote.sh retrieves the same key over SSH whenever it starts.

This provides a single source of truth and avoids storing API keys in the workstation configuration.

An optional local key file can be configured through LOCAL_KEY_FILE in config.env. When configured and readable, it takes precedence over the remotely retrieved key.

Important: The API key used by the client must match the key configured for the running vLLM server. Otherwise, vLLM rejects requests with HTTP 401.

4. Workstation configuration¶

The workstation configuration is stored in config.env.

It defines:

SSH connection parameters.
SLURM job name.
Remote runtime metadata location.
API key location.
Local port.
Local Singularity image.
Additional container bind mounts.
Terminal settings.

The configuration is independent of the served model. The model name is retrieved from the metadata generated by run_vllm.sh.

config.env¶

# Workstation-side configuration for connecting Qwen Code to a vLLM job
# running on Perun. Sourced by qwen_remote.sh.

# Nothing here is a secret: the API key is read from the cluster at runtime.

# ---- Cluster access ----------------------------------------------------

# LOGIN_ALIAS is the Host entry in ~/.ssh/config.
export LOGIN_ALIAS="perun"
export LOGIN_USER="username"

# Name of the SLURM job that runs vLLM.
export VLLM_JOB_NAME="vllm"

# Remote runtime metadata directory.
# Must match LLM_RUNTIME in the Perun-side config.env.
export REMOTE_RUNTIME="\$HOME/llm/run"

# Remote API key file.
export REMOTE_KEY_FILE="\$HOME/.llm_api_key"

# Optional local API key file.
# When readable, this takes precedence over the remote key.
# export LOCAL_KEY_FILE="$HOME/.llm_api_key"

# ---- Tunnel ------------------------------------------------------------

# Local port Qwen Code talks to.
# Set to 0 to let the script pick a free one.
export LOCAL_PORT="44044"

# SSH options used for the compute node.
# Compute nodes often regenerate their host keys.
export NODE_SSH_OPTS="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR"

# Maximum time to wait for the vLLM API endpoint.
export WAIT_TIMEOUT="60"

# ---- Model -------------------------------------------------------------

# The served model is retrieved from the job metadata.
# Set this variable only when an explicit override is required.
# export SERVED_MODEL="qwen3.8"

# ---- Local Qwen Code container -----------------------------------------

# Local Singularity image containing Qwen Code.
export QWEN_SIF_LOCAL="$HOME/llm/Containers/Qwen_code_x86_64_Nov2026.sif"

# Additional bind mounts (space-separated paths).
export QWEN_BINDS="/apps"

# ---- Terminal ----------------------------------------------------------

# Flickering bug fix.
export QWEN_CODE_FORCE_SYNCHRONIZED_OUTPUT=1
export TERM=xterm-256color

5. Starting the vLLM server¶

The vLLM server is started through SLURM using the cluster-side submission scripts.

For example:

ssh perun 'cd ~/llm && sbatch submit_llm.sh'

Check the job status:

ssh perun 'squeue -u username -n vllm'

Wait until the job reaches the R (RUNNING) state.

The first execution may take longer because the model weights need to be downloaded or loaded into GPU memory.

Monitor the job log:

ssh perun 'tail -f ~/llm/jobs/vllm-<jobid>.log'

Runtime metadata¶

The server publishes its runtime information in:

~/llm/run/<jobid>.job

The metadata file is created by run_vllm.sh when the server starts and removed when the job terminates.

It contains the following information:

Variable	Description
`JOB_ID`	SLURM job ID.
`NODE`	Allocated compute node.
`SERVER_PORT`	Port used by vLLM.
`SERVED_MODEL`	Model identifier exposed through the API.
`MODEL_NAME`	Model name used for logging and display.
`STARTED`	Server start timestamp.

The client uses this file as the source of truth for connection parameters.

The SLURM queue is only used to identify the running job ID.

6. Workstation launcher¶

The workstation launcher is qwen_remote.sh.

Its main responsibilities are:

Load the workstation configuration.
Identify the running vLLM SLURM job.
Retrieve the job metadata from Perun.
Retrieve the API key.
Select an available local port.
Allocate an intermediate port on Perun.
Establish the two-hop SSH tunnel.
Wait until the vLLM API becomes available.
Launch Qwen Code inside the local Singularity container.

The launcher does not require manual configuration of the compute node, remote port or served model.

qwen_remote.sh¶

#!/bin/bash
# Connect Qwen Code on this workstation to a vLLM job running on Perun.
#
# Usage:
#   qwen_remote.sh [workspace] [--continue|--new|--resume|--list]
#
# The script finds the running vLLM job, reads the node, port and served
# model from the job's metadata file, opens an SSH tunnel to it and starts
# Qwen Code inside the local Singularity container.
#
# The tunnel is two hops in one SSH process:
#
#   workstation:LOCAL_PORT -> perun:MID_PORT -> node:SERVER_PORT
#
# The second hop is made by Perun, so it uses the key stored on Perun.
# A direct 'ssh -J perun node' would instead make the node authenticate
# this workstation, which the compute nodes do not allow.

set -Eeuo pipefail

SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
CONFIG="${QWEN_REMOTE_CONFIG:-$SCRIPT_DIR/config.env}"

if [[ ! -r "$CONFIG" ]]; then
    echo "ERROR: Cannot read configuration: $CONFIG" >&2
    exit 1
fi

# Load workstation configuration.
# shellcheck disable=SC1090
source "$CONFIG"

: "${LOGIN_ALIAS:?LOGIN_ALIAS not set in $CONFIG}"
: "${QWEN_SIF_LOCAL:?QWEN_SIF_LOCAL not set in $CONFIG}"

VLLM_JOB_NAME="${VLLM_JOB_NAME:-vllm}"
REMOTE_RUNTIME="${REMOTE_RUNTIME:-\$HOME/llm/run}"
REMOTE_KEY_FILE="${REMOTE_KEY_FILE:-\$HOME/.llm_api_key}"
LOCAL_PORT="${LOCAL_PORT:-0}"
WAIT_TIMEOUT="${WAIT_TIMEOUT:-60}"
QWEN_BINDS="${QWEN_BINDS:-}"

# Compute nodes regenerate their host keys, so the login node must not
# check them against its known_hosts.
NODE_SSH_OPTS="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR"

usage() {
    cat <<USAGE
Usage:
  $0 [workspace]                 interactive session menu
  $0 [workspace] --continue      resume the most recent session
  $0 [workspace] --new           start a new session
  $0 [workspace] --resume        open the Qwen session selector
  $0 [workspace] --list          list sessions
USAGE
}

# ---------------------------------------------------------------------------
# Arguments: a workspace path and/or a mode flag, in any order.
# ---------------------------------------------------------------------------

WORKSPACE=""
MODE=""

for arg in "$@"; do
    case "$arg" in
        --continue|--new|--resume|--list)
            MODE="$arg"
            ;;
        -h|--help)
            usage
            exit 0
            ;;
        -*)
            echo "ERROR: Unknown option: $arg" >&2
            usage >&2
            exit 1
            ;;
        *)
            WORKSPACE="$arg"
            ;;
    esac
done

WORKSPACE=${WORKSPACE:-$PWD}

if [[ ! -d "$WORKSPACE" ]]; then
    echo "ERROR: Workspace does not exist: $WORKSPACE" >&2
    exit 1
fi

WORKSPACE=$(cd "$WORKSPACE" && pwd)

case "$MODE" in
    --continue) QWEN_ARGS=(--continue) ;;
    --new)      QWEN_ARGS=() ;;
    --resume)   QWEN_ARGS=(--resume) ;;
    --list)     QWEN_ARGS=(sessions list) ;;
    "")
        echo
        echo "Qwen session:"
        echo
        echo "  1) Continue last session"
        echo "  2) Start new session"
        echo "  3) Select previous session"
        echo "  4) List sessions"
        echo "  5) Exit"
        echo
        read -r -p "Select [1]: " CHOICE
        CHOICE=${CHOICE:-1}
        case "$CHOICE" in
            1) QWEN_ARGS=(--continue) ;;
            2) QWEN_ARGS=() ;;
            3) QWEN_ARGS=(--resume) ;;
            4) QWEN_ARGS=(sessions list) ;;
            5) exit 0 ;;
            *) echo "ERROR: Invalid selection." >&2; exit 1 ;;
        esac
        ;;
esac

# ---------------------------------------------------------------------------
# Ask Perun about the running vLLM job.
#
# run_vllm.sh writes $LLM_RUNTIME/<jobid>.job with NODE, SERVER_PORT and
# SERVED_MODEL, and removes it when the job ends. That file is the source
# of truth; squeue is only used to find the job id.
# ---------------------------------------------------------------------------

echo "Looking for a running vLLM job on $LOGIN_ALIAS ..."

REMOTE_SCRIPT=$(cat <<REMOTE
set -eu
RUNTIME="$REMOTE_RUNTIME"
KEYFILE="$REMOTE_KEY_FILE"
JOB=\$(squeue -u "\$USER" -h -n "$VLLM_JOB_NAME" -t R -o '%i' | head -1)
if [ -z "\$JOB" ]; then
    echo "ERR=nojob"
    exit 0
fi
META="\$RUNTIME/\$JOB.job"
if [ ! -r "\$META" ]; then
    echo "ERR=nometa"
    echo "JOB_ID=\$JOB"
    echo "NODE=\$(squeue -j "\$JOB" -h -o '%N')"
    exit 0
fi
cat "\$META"
# A free port on the login node, for the middle of the tunnel.
echo "MID_PORT=\$(python3 -c "import socket; s=socket.socket(); s.bind(('127.0.0.1',0)); print(s.getsockname()[1]); s.close()")"
if [ -r "\$KEYFILE" ]; then
    echo "API_KEY=\$(cat "\$KEYFILE")"
else
    echo "ERR=nokey"
fi
REMOTE
)

REMOTE_INFO=$(ssh -o BatchMode=yes "$LOGIN_ALIAS" "bash -s" <<<"$REMOTE_SCRIPT") || {
    echo "ERROR: Cannot reach $LOGIN_ALIAS over SSH (passwordless login not working?)." >&2
    echo "       Try:  ssh -v $LOGIN_ALIAS true" >&2
    exit 1
}

JOB_ID=""; NODE=""; SERVER_PORT=""; MID_PORT=""; API_KEY=""; ERR=""
SERVED_MODEL=""; MODEL_NAME=""; STARTED=""

while IFS='=' read -r key value; do
    case "$key" in
        JOB_ID)       JOB_ID="$value" ;;
        NODE)         NODE="$value" ;;
        SERVER_PORT)  SERVER_PORT="$value" ;;
        MID_PORT)     MID_PORT="$value" ;;
        SERVED_MODEL) SERVED_MODEL="$value" ;;
        MODEL_NAME)   MODEL_NAME="$value" ;;
        STARTED)      STARTED="$value" ;;
        API_KEY)      API_KEY="$value" ;;
        ERR)          ERR="$value" ;;
    esac
done <<<"$REMOTE_INFO"

case "$ERR" in
    nojob)
        echo "ERROR: No running '$VLLM_JOB_NAME' job found on $LOGIN_ALIAS." >&2
        echo "       Start one with:  ssh $LOGIN_ALIAS 'cd ~/llm && sbatch submit_llm.sh'" >&2
        exit 1
        ;;
    nometa)
        echo "ERROR: Job $JOB_ID runs on ${NODE:-?} but its metadata file is missing." >&2
        echo "       vLLM may still be starting, or run_vllm.sh failed. Check:" >&2
        echo "       ssh $LOGIN_ALIAS 'tail -n 40 ~/llm/jobs/vllm-$JOB_ID.log'" >&2
        exit 1
        ;;
    nokey)
        echo "ERROR: The API key file does not exist on the cluster." >&2
        echo "       Create it with:" >&2
        echo "       ssh $LOGIN_ALIAS 'openssl rand -hex 24 > ~/.llm_api_key && chmod 600 ~/.llm_api_key'" >&2
        echo "       then restart the vLLM job so it picks the key up." >&2
        exit 1
        ;;
esac

: "${NODE:?Could not determine the compute node}"
: "${SERVER_PORT:?Could not determine the vLLM port}"
: "${MID_PORT:?Could not get a free port on $LOGIN_ALIAS}"
: "${API_KEY:?Could not determine the API key}"
: "${SERVED_MODEL:?Could not determine the served model name}"

echo "Job $JOB_ID on $NODE, port $SERVER_PORT, model $SERVED_MODEL (${MODEL_NAME:-?})"

# ---------------------------------------------------------------------------
# Pick a free local port.
# ---------------------------------------------------------------------------

port_free() {
    ! (exec 3<>"/dev/tcp/127.0.0.1/$1") 2>/dev/null
}

if [[ "$LOCAL_PORT" == "0" ]] || ! port_free "$LOCAL_PORT"; then
    if [[ "$LOCAL_PORT" != "0" ]]; then
        echo "Local port $LOCAL_PORT is busy, picking another one."
    fi
    for candidate in $(seq 44044 44144); do
        if port_free "$candidate"; then
            LOCAL_PORT="$candidate"
            break
        fi
    done
fi

echo "Local endpoint: http://127.0.0.1:${LOCAL_PORT}/v1"

# ---------------------------------------------------------------------------
# Open the tunnel.
#
# One local SSH process does both hops: it forwards
# 127.0.0.1:LOCAL_PORT -> perun:MID_PORT, and the command it runs on Perun
# forwards perun:MID_PORT -> node:SERVER_PORT. -tt gives that inner ssh a
# pty, so it dies with this process instead of lingering on the login node.
# ---------------------------------------------------------------------------

TUNNEL_PID=""
TUNNEL_LOG=$(mktemp "${TMPDIR:-/tmp}/qwen-tunnel.XXXXXX")

cleanup() {
    if [[ -n "$TUNNEL_PID" ]]; then
        kill "$TUNNEL_PID" 2>/dev/null || true
    fi
    # Make sure no forwarding process is left holding MID_PORT.
    ssh -o BatchMode=yes -o ConnectTimeout=5 "$LOGIN_ALIAS" \
        "pkill -f 'ssh -N .*-L 127.0.0.1:${MID_PORT}:' || true" \
        >/dev/null 2>&1 || true
    rm -f "$TUNNEL_LOG"
}
trap cleanup EXIT

echo "Opening ${LOGIN_ALIAS} -> ${NODE} tunnel ..."

ssh -tt \
    -o ExitOnForwardFailure=yes \
    -o ServerAliveInterval=30 \
    -L "127.0.0.1:${LOCAL_PORT}:127.0.0.1:${MID_PORT}" \
    "$LOGIN_ALIAS" \
    "ssh -N $NODE_SSH_OPTS -o ExitOnForwardFailure=yes -L 127.0.0.1:${MID_PORT}:127.0.0.1:${SERVER_PORT} ${NODE}" \
    >>"$TUNNEL_LOG" 2>&1 </dev/null &
TUNNEL_PID=$!

# ---------------------------------------------------------------------------
# Wait until vLLM answers through the tunnel.
# ---------------------------------------------------------------------------

printf 'Waiting for vLLM'
ready=0
for (( i = 0; i < WAIT_TIMEOUT; i++ )); do
    if ! kill -0 "$TUNNEL_PID" 2>/dev/null; then
        echo
        echo "ERROR: The SSH tunnel died:" >&2
        sed 's/^/       | /' "$TUNNEL_LOG" >&2
        echo "       Check that Perun can reach the node without a password:" >&2
        echo "       ssh $LOGIN_ALIAS \"ssh -o BatchMode=yes $NODE hostname\"" >&2
        exit 1
    fi
    if curl -fsS -m 3 -o /dev/null \
            -H "Authorization: Bearer ${API_KEY}" \
            "http://127.0.0.1:${LOCAL_PORT}/v1/models" 2>/dev/null; then
        ready=1
        break
    fi
    printf '.'
    sleep 1
done
echo

if (( ready == 0 )); then
    echo "ERROR: vLLM did not answer within ${WAIT_TIMEOUT}s." >&2
    echo "       The model may still be loading. Watch the job log:" >&2
    echo "       ssh $LOGIN_ALIAS 'tail -f ~/llm/jobs/vllm-${JOB_ID}.log'" >&2
    if [[ -s "$TUNNEL_LOG" ]]; then
        echo "       Tunnel output:" >&2
        sed 's/^/       | /' "$TUNNEL_LOG" >&2
    fi
    exit 1
fi

echo "vLLM is reachable."

# ---------------------------------------------------------------------------
# Launch Qwen Code in the local Singularity container.
# ---------------------------------------------------------------------------

BIND_ARGS=()
for b in $QWEN_BINDS; do
    BIND_ARGS+=(--bind "$b")
done

echo "Launching Qwen in $WORKSPACE ..."

singularity exec \
    "${BIND_ARGS[@]}" \
    --env OPENAI_BASE_URL="http://127.0.0.1:${LOCAL_PORT}/v1" \
    --env OPENAI_API_KEY="${API_KEY}" \
    --env OPENAI_MODEL="${SERVED_MODEL}" \
    --env QWEN_CODE_FORCE_SYNCHRONIZED_OUTPUT="${QWEN_CODE_FORCE_SYNCHRONIZED_OUTPUT:-1}" \
    --env TERM="${TERM:-xterm-256color}" \
    "${QWEN_SIF_LOCAL}" \
    bash -lc "cd '$WORKSPACE' && qwen ${QWEN_ARGS[*]}"

7. Starting Qwen Code¶

After configuring SSH and preparing the vLLM server, make the launcher executable:

chmod +x ~/llm/qwen_remote.sh

Start an interactive session:

~/llm/qwen_remote.sh

The launcher displays a session menu:

Qwen session:

  1) Continue last session
  2) Start new session
  3) Select previous session
  4) List sessions
  5) Exit

Alternatively, specify the session mode directly.

Command	Description
`qwen_remote.sh`	Interactive session menu.
`qwen_remote.sh . --continue`	Continue the latest session in the current directory.
`qwen_remote.sh . --new`	Start a new session.
`qwen_remote.sh ~/projects/foo --new`	Start a new session in a specified workspace.
`qwen_remote.sh . --resume`	Open the session selector.
`qwen_remote.sh . --list`	List available sessions.

The workspace defaults to the current working directory.

8. Runtime connection management¶

Several mechanisms are implemented to make the connection independent of individual SLURM allocations.

Dynamic remote port¶

The vLLM server selects an available port when starting.

The port is stored in the job metadata file and retrieved by the workstation launcher.

No fixed remote port needs to be configured.

Dynamic intermediate port¶

The launcher asks the Perun login node to select an available port for the intermediate SSH forwarding stage.

The port is selected dynamically using Python and is not stored in the workstation configuration.

Dynamic local port¶

The workstation uses LOCAL_PORT from config.env.

If the configured port is unavailable, the launcher searches for another available port in the range:

44044-44144

Setting LOCAL_PORT="0" enables automatic port selection.

API readiness check¶

The launcher does not assume that vLLM is ready immediately after establishing the SSH tunnel.

Instead, it periodically queries:

http://127.0.0.1:<local_port>/v1/models

The request includes the API key.

Qwen Code starts only after the endpoint responds successfully.

The maximum waiting time is controlled by WAIT_TIMEOUT.

Tunnel cleanup¶

The SSH tunnel is terminated when Qwen Code exits.

The launcher also attempts to terminate the intermediate SSH forwarding process on Perun.

This prevents abandoned tunnels from occupying ports.

9. Troubleshooting¶

Cannot reach Perun over SSH

The launcher uses BatchMode=yes, which prevents interactive password prompts.

Check SSH connectivity:

ssh -v perun true

Verify the SSH key, permissions and configuration described in Section 2.

No running vLLM job found

Check the SLURM queue:

ssh perun 'squeue -u username'

The job may still be pending (PD), may have terminated, or may have reached its walltime limit.

Job metadata file is missing¶

Check whether the metadata file exists:

ssh perun 'ls -l ~/llm/run/'

If the job is running but the metadata file is missing, vLLM may still be starting or run_vllm.sh may have failed.

Check the job log:

ssh perun 'tail -n 50 ~/llm/jobs/vllm-<jobid>.log'

Password prompt from the compute node

The current implementation requires passwordless SSH connectivity from the Perun login node to the allocated compute node.

Test the connection:

ssh perun "ssh -o BatchMode=yes <compute-node> hostname"

If the command fails, configure the internal SSH key as described in Section 2.4.

SSH tunnel terminated unexpectedly

The launcher prints the tunnel's SSH output when the connection terminates.

Check the reported SSH error.

Common causes include:

Passwordless SSH authentication failure.
Compute node unavailable.
Incorrect intermediate port forwarding.
vLLM server terminated.
SSH connection timeout.

vLLM did not answer within the configured timeout

The model may still be loading into GPU memory.

Check the job log:

ssh perun 'tail -f ~/llm/jobs/vllm-<jobid>.log'

Look for:

Application startup complete

If the model requires more time to initialize, increase:

WAIT_TIMEOUT="120"

in config.env.

An HTTP 401 response indicates an API key mismatch. Verify that ~/.llm_api_key on Perun contains the key used by the running vLLM server.

After changing the key, restart the vLLM job.

Manually test the API endpoint

With the SSH tunnel established, test the local endpoint:

curl -s \
    -H "Authorization: Bearer $(ssh perun cat ~/.llm_api_key)" \
    http://127.0.0.1:44044/v1/models

Replace 44044 with the actual local port selected by the launcher.

10. Summary¶

The complete workflow is:

Configure passwordless SSH access from the workstation to Perun.
Configure passwordless SSH access from Perun to compute nodes.
Create the API key on Perun.
Configure the workstation-side config.env.
Submit a vLLM SLURM job on Perun.
Wait until the job starts and vLLM publishes its metadata.
Run qwen_remote.sh on the workstation.
The launcher discovers the running job and retrieves its metadata.
The launcher retrieves the API key and establishes the SSH tunnel.
The local API endpoint is checked until vLLM responds.
Qwen Code starts inside the local Singularity container.
When Qwen Code exits, the SSH tunnel is terminated.

The workstation configuration remains unchanged between jobs, different compute node allocations and model changes.

Only the cluster-side vLLM configuration and submission parameters need to be modified when changing the model or server configuration.

Created by: Marek Štekláč

Connecting Qwen Code to vLLM on SAS HPC clusters¶

Architecture¶

1. Prerequisites¶

Workstation¶

Perun¶

File layout¶

Singularity image and SLURM integration¶

2. SSH configuration¶

2.1. Generate an SSH key¶

2.2. Install the public key¶

2.3. Configure SSH¶

2.4. Configure login-to-compute-node SSH¶

3. API key configuration¶

4. Workstation configuration¶

config.env¶

5. Starting the vLLM server¶

Runtime metadata¶

6. Workstation launcher¶

qwen_remote.sh¶

7. Starting Qwen Code¶

8. Runtime connection management¶

Dynamic remote port¶

Dynamic intermediate port¶

Dynamic local port¶

API readiness check¶

Tunnel cleanup¶

9. Troubleshooting¶

Cannot reach Perun over SSH

No running vLLM job found

Job metadata file is missing¶

Password prompt from the compute node

SSH tunnel terminated unexpectedly

vLLM did not answer within the configured timeout

Manually test the API endpoint

10. Summary¶