Qwen Code

Qwen Code is an AI coding assistant that runs in the terminal. It reads and edits files, runs commands, and answers questions about a codebase.

On SAV infrastructure it runs in a Singularity container on a login node and connects to a model you serve yourself on a GPU node. Set the server up first following Serving LLMs with vLLM.

Quickstart

cd ~/llm 
sbatch submit_llm.sh                    # start the model server
bash agent.sh <workspace>               # pick a session, then work

agent.sh asks which session you want, finds the running server job, reads which node and port it landed on, opens an SSH tunnel, waits for the model to finish loading, and launches the agent. The tunnel closes when you exit.

Qwen Code session:

  1) Continue last session
  2) Start new session
  3) Select previous session
  4) List sessions
  5) Exit

Select [1]:

Release the GPU when you finish:

scancel JOB_ID

---

Setup

You need the server setup done first: ~/llm/config.env, an API key in ~/.llm_api_key, and a server you can start. Then add two files:

chmod +x ~/llm/agent.sh

Check that QWEN_SIF_X86 in config.env points at the either global image or personal image installed in your project directory.

Then create the two account files the container needs. The base image has no entry for your UID, and SLURM commands inside the container will not authenticate until it can resolve your user and group:

getent passwd root "$USER"      > ~/llm/.container_passwd
getent group  root "$(id -gn)"  > ~/llm/.container_group

Regenerate them if your account details ever change. They are bound over /etc/passwd and /etc/group in the container — see Bind mounts.

---

Usage

agent.sh [WORKSPACE] [MODE] [-j JOBID] [-- ARGS...]

  WORKSPACE   directory to work in (default: current directory)
  MODE        --continue, --new, --resume or --list
              With no MODE, an interactive menu is shown.
  -j JOBID    vLLM job to connect to (default: the only running one)
  -- ARGS     everything after -- is passed to qwen verbatim

~/llm/agent.sh                      # menu, current directory
~/llm/agent.sh --continue           # straight back into the last session
~/llm/agent.sh ~/my-project --new   # new session, in that workspace
~/llm/agent.sh --list               # list sessions; no GPU needed
~/llm/agent.sh -j 123456 --resume   # pick which server to use
~/llm/agent.sh -- --help            # arguments passed through to qwen

Session modes

Mode	Effect
--continue	Resume the most recent session for this workspace
--new	Start a completely new session
--resume	Open Qwen Code's own session selector
--list	List saved sessions and exit
(none)	Show the interactive menu; Enter selects Continue

Session state lives in your workspace directory, which agent.sh binds into the container, so sessions survive across runs and across different server jobs — if a server job hits its walltime, start a new one and --continue picks up where you left off.

--list does not need a GPU

Listing sessions touches no model, so agent.sh skips job selection, the tunnel and the readiness wait entirely. The list appears immediately, even with no vLLM job running at all.

---

The launcher

In order, agent.sh:

1. Parses the command line — workspace, session mode, job ID, and anything after -- to pass through verbatim.
2. Resolves the session mode, showing the menu if none was given. This happens before anything connects, so the menu appears at once rather than after the model has loaded.
3. Picks the server job — unless the mode needs no server. With no job ID it looks for exactly one running job named vllm; if there are several it prints them and stops rather than guessing, because connecting to the wrong model is confusing and silent.
4. Reads the metadata for that job to learn the node and port, and verifies the job is still running before trusting it, so a stale file produces a clear error instead of a hanging tunnel.
5. Allocates a local port the same way the server does, so several agent sessions can run side by side on one login node.
6. Opens the tunnel in the background and registers a trap to close it.
7. Waits for readiness by polling /v1/models until the model has finished loading — up to five minutes — while checking the tunnel is still alive.
8. Launches Qwen Code in the chosen workspace, with the cluster bind mounts that give it access to SLURM.

Metadata is parsed, never sourced

The field() helper extracts values with sed, and the node name and port are validated before use. A file written by a job into a runtime directory should not be able to execute code as you.

~/llm/agent.sh

#!/bin/bash
# Open a tunnel to a running vLLM job and start Qwen Code against it.
#
#   ./agent.sh                      # menu, current directory
#   ./agent.sh --continue           # resume the most recent session
#   ./agent.sh ~/proj --new         # new session in ~/proj
#   ./agent.sh --list               # list sessions; no server needed
#   ./agent.sh -j 123456 --resume   # choose the server job
#   ./agent.sh -- --help            # pass arguments to qwen
set -euo pipefail

source "${LLM_HOME:-$HOME/llm}/config.env"
: "${KEY:?No API key. Create ~/.llm_api_key -- see config.env.}"

usage() {
    cat <<EOF
Usage: ${0##*/} [WORKSPACE] [MODE] [-j JOBID] [-- ARGS...]

  WORKSPACE   directory to work in (default: current directory)
  MODE        --continue, --new, --resume or --list
              With no MODE, an interactive menu is shown.
  -j JOBID    vLLM job to connect to (default: the only running one)
  -- ARGS     everything after -- is passed to qwen verbatim
EOF
}

# ---- Command line ------------------------------------------------------
JOB_ID=""
WORKSPACE=""
MODE=""
PASSTHRU=()

while [[ $# -gt 0 ]]; do
    case "$1" in
        -j) JOB_ID="${2:?-j needs a job ID}"; shift 2 ;;
        -h|--help) usage; exit 0 ;;
        --) shift; PASSTHRU=("$@"); break ;;
        --*) MODE="$1"; shift ;;
        *)  if [[ -n "$WORKSPACE" ]]; then
                echo "Unexpected argument: $1" >&2; usage >&2; exit 1
            fi
            WORKSPACE="$1"; shift ;;
    esac
done

WS="${WORKSPACE:-$PWD}"
if [[ ! -d "$WS" ]]; then
    echo "No such directory: $WS" >&2
    exit 1
fi
WORKSPACE="$(cd "$WS" && pwd)"

# -----------------------------------------------------------------------
# Select Qwen session mode
#
#   --continue   Resume the most recent session for this workspace.
#   --new        Start a completely new session.
#   --resume     Open Qwen's session selector.
#   --list       List available sessions.
#
# With no option, present an interactive menu.
#
# Resolved before anything connects, so the menu appears immediately
# rather than after the model has finished loading.
# -----------------------------------------------------------------------
NEEDS_SERVER=1

case "$MODE" in

    --continue)
        QWEN_ARGS=(--continue)
        ;;

    --new)
        QWEN_ARGS=()
        ;;

    --resume)
        QWEN_ARGS=(--resume)
        ;;

    --list)
        # Listing sessions needs no model, so skip the tunnel.
        QWEN_ARGS=(sessions list)
        NEEDS_SERVER=0
        ;;

    "")
        if [[ ! -t 0 ]]; then
            echo "Not a terminal. Pass --continue, --new," >&2
            echo "--resume or --list explicitly." >&2
            exit 1
        fi

        echo
        echo "Qwen session:"
        echo
        echo "  1) Continue last session"
        echo "  2) Start new session"
        echo "  3) Select previous session"
        echo "  4) List sessions"
        echo "  5) Exit"
        echo
        read -r -p "Select [1]: " CHOICE || CHOICE=1

        # Pressing Enter selects the default.
        CHOICE=${CHOICE:-1}

        case "$CHOICE" in
            1)
                QWEN_ARGS=(--continue)
                ;;
            2)
                QWEN_ARGS=()
                ;;
            3)
                QWEN_ARGS=(--resume)
                ;;
            4)
                QWEN_ARGS=(sessions list)
                NEEDS_SERVER=0
                ;;
            5)
                exit 0
                ;;
            *)
                echo "Invalid selection." >&2
                exit 1
                ;;
        esac
        ;;

    *)
        echo "Unknown option: $MODE" >&2
        echo
        usage >&2
        exit 1
        ;;
esac

# ---- Image -------------------------------------------------------------
# The Qwen Code image is built for x86_64 only, which is what the login
# and CPU nodes are. Fail here rather than inside singularity.
if [[ "$(uname -m)" != "x86_64" ]]; then
    echo "This node is $(uname -m); the Qwen Code image is x86_64 only." >&2
    echo "Run agent.sh from a login node." >&2
    exit 1
fi
QWEN_SIF="${QWEN_SIF_X86:?QWEN_SIF_X86 is not set}"

if [[ "$NEEDS_SERVER" == "1" ]]; then

    # ---- Pick the job --------------------------------------------------
    # $USER is not exported in every environment, and set -u would abort.
    ME="${USER:-$(id -un)}"

    if [[ -z "$JOB_ID" ]]; then
        mapfile -t jobs < <(squeue -u "$ME" -n vllm -h -t RUNNING -o '%i')

        case ${#jobs[@]} in

            0)  echo "No running vLLM job. Start one: sbatch submit_llm.sh" >&2
                exit 1 ;;
            1)  JOB_ID="${jobs[0]}" ;;
            *)  echo "Several vLLM jobs are running. Pass -j JOBID:" >&2
                squeue -u "$ME" -n vllm -t RUNNING -o '%.10i %.9T %.14R %.11L' >&2
                exit 1 ;;
        esac
    fi

    # ---- Read its metadata ---------------------------------------------
    META="$LLM_RUNTIME/$JOB_ID.job"
    if [[ ! -r "$META" ]]; then
        echo "No metadata for job $JOB_ID. It may still be starting up." >&2
        echo "Watch it with: tail -f $LLM_LOGS/vllm-$JOB_ID.log" >&2
        exit 1
    fi

    # Parsed, never sourced: this file should not be able to run code.
    field() { sed -n "s/^$1=//p" "$META" | head -n1; }
    NODE=$(field NODE)
    SERVER_PORT=$(field SERVER_PORT)
    SERVED_MODEL=$(field SERVED_MODEL)

    if ! squeue -h -j "$JOB_ID" -t RUNNING -o '%i' | grep -qx "$JOB_ID"; then
        echo "Job $JOB_ID is no longer running; metadata is stale." >&2
        exit 1
    fi
    if [[ ! "$NODE" =~ ^[A-Za-z0-9._-]+$ || ! "$SERVER_PORT" =~ ^[0-9]+$ ]]; then
        echo "Malformed metadata in $META" >&2
        exit 1
    fi

    # ---- Tunnel --------------------------------------------------------
    LOCAL_PORT=$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')

    ssh -o ExitOnForwardFailure=yes -o BatchMode=yes -N \
        -L "127.0.0.1:$LOCAL_PORT:127.0.0.1:$SERVER_PORT" "$NODE" &
    SSH_PID=$!
    cleanup() {
        kill "$SSH_PID" 2>/dev/null || true
        wait "$SSH_PID" 2>/dev/null || true
    }
    trap cleanup EXIT INT TERM

    # ---- Wait for the model to finish loading --------------------------
    BASE="http://127.0.0.1:$LOCAL_PORT"
    ready=0
    for _ in $(seq 150); do
        if ! kill -0 "$SSH_PID" 2>/dev/null; then
            echo "SSH tunnel to $NODE died." >&2
            exit 1
        fi
        if curl -fsS -m 2 -H "Authorization: Bearer $KEY" \
                "$BASE/v1/models" >/dev/null 2>&1; then
            ready=1; break
        fi
        sleep 2
    done
    if [[ $ready -ne 1 ]]; then
        echo "Server did not answer within 5 minutes." >&2
        echo "Check the log: tail -f $LLM_LOGS/vllm-$JOB_ID.log" >&2
        exit 1
    fi
    echo "Connected to '$SERVED_MODEL' on $NODE:$SERVER_PORT"

    export OPENAI_BASE_URL="$BASE/v1"
    export OPENAI_API_KEY="$KEY"
    export OPENAI_MODEL="$SERVED_MODEL"
fi

# ---- Launch ------------------------------------------------------------
LAUNCH=()

if [[ ${#QWEN_ARGS[@]} -gt 0 ]]; then LAUNCH+=("${QWEN_ARGS[@]}"); fi
if [[ ${#PASSTHRU[@]}  -gt 0 ]]; then LAUNCH+=("${PASSTHRU[@]}");  fi


# The last four binds and the PATH are what let the agent use SLURM from
# inside the container -- see "Bind mounts". Without them sbatch, srun
# and squeue fail to authenticate.
#
# No 'exec': the EXIT trap must still run to close the tunnel.
# ${LAUNCH[@]+...} guards against older bash treating an empty array as
# unset under set -u.

ml singularity

singularity exec \
    --bind "$WORKSPACE:$WORKSPACE" \
    --bind "$HOME:$HOME" \
    --bind "$LLM_PROJECT:$LLM_PROJECT" \
    --bind /run/munge:/run/munge \
    --bind /var/spool/slurmd/conf-cache:/etc/slurm \
    --bind "$LLM_HOME/.container_passwd:/etc/passwd" \
    --bind "$LLM_HOME/.container_group:/etc/group" \
    --pwd "$WORKSPACE" \
    --env "PATH=/usr/local/bin:/opt/agent_env/bin:/opt/slurm/bin:/opt/slurm/sbin:/usr/bin:/bin" \
    "$QWEN_SIF" qwen ${LAUNCH[@]+"${LAUNCH[@]}"}

Devana slurm bind

On devana slurm needs to be bind followingly -- bind /etc/slurm:/etc/slurm & --env "PATH=/usr/local/bin:/opt/agent_env/bin:$HOME/.local/bin:/usr/bin:/bin"

---

Running agents responsibly

A coding agent executes commands. Two cluster-specific consequences:

It runs on the login node, but it has SLURM Because agent.sh binds the munge socket and the SLURM configuration into the container, the agent can run sbatch, srun, squeue and scancel itself. Tell it to: ask it to submit a job rather than to run the build, and heavy work lands on compute nodes where it belongs instead of on a shared, unqueued login node.

> compile this and run the test suite -- submit it with sbatch to the
> cpu partition under account number <project_ID>, then check the job's output

Anything the agent runs directly still executes on the login node, so for a long interactive session you can also put the whole thing in a job:

srun --account=<project_ID> --partition=cpu --time=1:00:00 --pty bash
~/llm/agent.sh

The agent can submit jobs against your allocation

It spends your account's compute budget without a second confirmation step. Be specific about partitions and walltimes in your prompts, and check squeue -u "$USER" if you are not sure what it started.

It edits files for real

Pass a workspace or run from a project directory, not $HOME, and keep that project in version control so you can see and undo what it did. Be deliberate about any mode that auto-approves actions — the model is working from your prompt, but it is still a model.

---

Bind mounts

Singularity does not expose every host path. agent.sh binds seven, and the last four are what let the agent submit and monitor its own SLURM jobs:

Bind	Why
$WORKSPACE	The directory the agent is working in
$HOME	Agent configuration and saved session state
$LLM_PROJECT	Shared containers and the model cache
/run/munge	Munge authentication socket. Without it every SLURM command inside the container fails with a credential error
/var/spool/slurmd/conf-cache → /etc/slurm	Perun cluster configuration, so SLURM clients know which controller to talk to
/etc/slurm → /etc/slurm	Devana cluster configuration, so SLURM clients know which controller to talk to
.container_passwd → /etc/passwd	The base image has no entry for your UID; munge and SLURM need your account to resolve to a name
.container_group → /etc/group	The same for your primary group

--env PATH then puts /opt/slurm/bin and /opt/slurm/sbin on the path so sbatch, srun, squeue and scancel are actually found, alongside /opt/agent_env/bin where Qwen Code itself is installed.

Create the passwd and group files first

These are two small files you generate once; the binds fail if they do not exist. See Setup.

To run the image by hand with the same access:

ml singularity

singularity exec \
    --bind "$PWD:$PWD" \
    --bind "$HOME:$HOME" \
    --bind /project/p2075-26-2:/project/p2075-26-2 \
    --bind /run/munge:/run/munge \
    --bind /var/spool/slurmd/conf-cache:/etc/slurm \
    --bind "$HOME/llm/.container_passwd:/etc/passwd" \
    --bind "$HOME/llm/.container_group:/etc/group" \
    --pwd "$PWD" \
    --env "PATH=/usr/local/bin:/opt/agent_env/bin:/opt/slurm/bin:/opt/slurm/sbin:/usr/bin:/bin" \
    qwen-code.sif qwen

Devana slurm bind

On devana slurm needs to be bind followingly -- bind /etc/slurm:/etc/slurm & --env "PATH=/usr/local/bin:/opt/agent_env/bin:$HOME/.local/bin:/usr/bin:/bin"

Never bake API keys, SSH keys or tokens into an image — anyone who can read the .sif can read them back out.

Building an agent image

The stock images cover Qwen Code. Build your own for a different version, extra tools, or a different agent entirely. The image holds the agent and its runtime only; model weights stay in the server's cache.

qwen-code.def

Bootstrap: docker
From: ubuntu:24.04

%labels
    Description Qwen Code client

%environment
    export LANG=C.UTF-8
    export LC_ALL=C.UTF-8

%post
    set -e
    apt-get update
    DEBIAN_FRONTEND=noninteractive apt-get install -y \
        ca-certificates curl git nodejs npm
    npm install --global @qwen-code/qwen-code@0.2.1
    apt-get clean
    rm -rf /var/lib/apt/lists/*

%runscript
    exec qwen "$@"

Pin the version rather than using latest, so the image can be rebuilt identically later. If the distribution's Node.js is too old for the release you want, install a specific Node.js version in %post instead.

You can see recipe for building the qwen-code.sif at /apps/images/vllm/x86_64 on Perun cluster and /storage-apps on Devana cluster.

Build, test, install

singularity build qwen-code.sif qwen-code.def
singularity exec qwen-code.sif qwen --version

Building needs either elevated privileges and network access for the base image and packages. This means you need to build on your own machine of the same architecture and copy the .sif across.

Perun's login/CPU nodes are x86_64; GPU nodes are aarch64 (all Devana nodes are x86_64). Since the agent runs on a login node, an x86_64 image is normally all you need — agent.sh picks the matching one from uname -m.

Never overwrite an image in use

A running job may have it open, and you cannot tell whose session you break. Put a version and date in the filename, then update the profile.

Using a different coding agent

Nothing here is specific to Qwen Code except the image, the executable name, the session flags and the environment variables. To run a different agent:

1. Build a SIF for it, following that agent's own install instructions. An agent's image contains only that agent — one image cannot start another.
2. Copy agent.sh to, say, ~/llm/aider.sh and change four things: the image variables, the qwen command at the bottom, the session-mode case block, and the OPENAI_* exports if the agent reads different variables.
3. Add its image paths to config.env next to the Qwen Code ones.

Two things to check before committing to an agent:

• Which variables it reads. Not every agent uses OPENAI_BASE_URL. Some want ANTHROPIC_*, some want a config file written to $HOME.
• Which API it needs. vLLM serves the OpenAI API, so an Anthropic-only client needs a translating proxy, not just different variable names.

Tool calling, streaming and structured output vary between agents and between models, so test any new combination end to end before advertising it.

When a second agent actually arrives

If a third one follows, that is the point to factor the shared parts of agent.sh into one script with per-agent config files. Doing it with a single agent buys nothing but indirection.

---

Troubleshooting

Server-side problems — the job will not start, the model will not load, out of GPU memory — are on the vLLM page. Start by checking the server directly; if it answers curl, the problem is here.

`Unknown option: --foo`

The session mode is not one agent.sh understands. The modes are --continue, --new, --resume and --list. Anything Qwen Code itself should receive goes after -- instead:

~/llm/agent.sh -- --some-qwen-flag

The menu does not appear

agent.sh only prompts on a terminal. Run from a script or a pipe and it asks for an explicit mode instead, so an automated run cannot hang waiting for input that will never arrive.

`agent.sh` says there is no metadata

The server job has not got far enough to publish its port, or it died during startup:

squeue -u "$USER"
tail -50 ~/llm/jobs/vllm-JOBID.log

A job in PD is queued, not broken.

The server never becomes ready

agent.sh waits five minutes. A cold cache on a large model can exceed that; pre-download the weights, then retry.

The agent connects but cannot edit files

Tool calling is not working. The agent will describe edits in prose instead of making them — a clear symptom. Confirm the model supports tool calling, then check that TOOL_PARSER in config.env matches what the model emits; the server log shows the raw tool call when parsing fails.

SLURM commands fail inside the container

Check which part is missing, from inside the same container:

singularity exec \
    --bind /run/munge:/run/munge \
    --bind /var/spool/slurmd/conf-cache:/etc/slurm \
    --bind "$HOME/llm/.container_passwd:/etc/passwd" \
    --bind "$HOME/llm/.container_group:/etc/group" \
    --env "PATH=/opt/slurm/bin:/usr/bin:/bin" \
    "$QWEN_SIF_X86" squeue -u "$USER"

Devana slurm bind

On devana slurm needs to be bind followingly -- bind /etc/slurm:/etc/slurm & --env "PATH=/usr/local/bin:/opt/agent_env/bin:$HOME/.local/bin:/usr/bin:/bin"

Symptom	Cause
squeue: command not found	PATH is missing /opt/slurm/bin
Invalid user for ... or whoami fails	The passwd and group files are missing or stale — regenerate them, see Setup
Credential or munge socket errors	/run/munge is not bound, or munge is not running on this node
Unable to contact slurm controller	/var/spool/slurmd/conf-cache or /etc/slurm (depending on cluster) is not bound over /etc/slurm

All five paths must exist on the host; Singularity fails the whole launch if a bind source is missing, so a cryptic startup error usually means one of them.

A session will not resume

Sessions are stored per workspace, so --continue from a different directory finds nothing. Use --list to see what exists, or --resume to pick from the selector. Check also that your home directory is bound — session state lives there, not in the image.

The agent stops mid-session

Usually the server job hit its walltime:

sacct -j JOBID --format=JobID,State,Elapsed,Timelimit

Submit a new server job and reconnect with --continue; saved sessions survive, so you resume where you left off.

`This node is aarch64; the Qwen Code image is x86_64 only`

You are on a GPU node. The agent is meant to run on a login node and reach the model through the tunnel; no aarch64 client image exists. Log in to a login node and run it there.

If you need the agent inside a job for some other reason, request a CPU partition rather than a GPU one — those nodes are x86_64:

srun --account=p2075-26-2 --partition=cpu --time=1:00:00 --pty bash

Qwen Code is missing from the image

singularity exec qwen-code.sif which qwen

If this prints nothing, the install step failed silently during the build, or it is installed outside the PATH that agent.sh sets.

---

Reference

Commands

Task	Command
Menu	~/llm/agent.sh
Continue last session	~/llm/agent.sh --continue
New session in a workspace	~/llm/agent.sh ~/proj --new
List sessions (no GPU)	~/llm/agent.sh --list
Choose the server job	~/llm/agent.sh -j JOBID
Pass arguments through	~/llm/agent.sh -- --help
Start a server	sbatch ~/llm/submit_llm.sh
Stop a server	scancel JOBID

Configuration

Set in config.env:

Variable	Purpose
QWEN_SIF_X86	Qwen Code image (x86_64; login and CPU nodes)
LLM_PROJECT	Project directory, bound into the container
KEY	API key, read from ~/.llm_api_key

Links

• Qwen Code
• documentation
• Apptainer documentation
• Serving LLMs with vLLM

Created by: Marek Štekláč