galaxy/specs/APP-4386/TECH.md

APP-4386 — SSH Remote Server Install Fallback (wget + SCP)

Linear: APP-4386

Context

When a user SSHes into a remote host, the client installs the remote server binary by piping install_remote_server.sh through bash -s on the remote. The script unconditionally uses curl (line 43) to download the tarball. On minimal hosts (Alpine, BusyBox, stripped Docker images), curl is absent → bash: line 43: curl: command not found (exit 127), and the install fails with no recovery path.

Other remote-development editors solve this with a multi-tier fallback strategy: try curl on the remote → fall back to wget → fall back to downloading locally and uploading via SCP. Warp currently has a single tier: curl only, no fallback.

Relevant code

• crates/remote_server/src/install_remote_server.sh — the install script; line 43 is the sole curl invocation
• crates/remote_server/src/setup.rs:385-414 — INSTALL_SCRIPT_TEMPLATE loaded via include_str!; install_script() substitutes placeholders ({download_base_url}, {channel}, {install_dir}, {binary_name}, {version_query}, {version_suffix})
• crates/remote_server/src/setup.rs:416-441 — download_url() and download_channel() construct the full CDN URL
• app/src/remote_server/ssh_transport.rs:194-217 — SshTransport::install_binary() runs the script via run_ssh_script and surfaces success/failure
• crates/remote_server/src/transport.rs:117-127 — RemoteTransport::install_binary trait method; returns Result<(), String>
• crates/remote_server/src/ssh.rs:95-155 — run_ssh_command and run_ssh_script utilities
• crates/remote_server/src/manager.rs:596-646 — RemoteServerManager::install_binary orchestrates the install, emits SetupStateChanged and BinaryInstallComplete
• crates/remote_server/src/setup.rs:202-237 — RemotePlatform, RemoteOs, RemoteArch — already detected before install via detect_platform

Proposed changes

Two phases, both in this PR.

Phase 1: wget fallback in the shell script

Modify install_remote_server.sh to detect which HTTP client is available and use whichever is present. The download URL construction stays identical — only the download command changes.

Replace the current curl invocation (lines 43-44):

curl -fSL "{download_base_url}?package=tar&os=$os_name&arch=$arch_name&channel={channel}{version_query}" \
  -o "$tmpdir/oz.tar.gz"

With a detection block:

url="{download_base_url}?package=tar&os=$os_name&arch=$arch_name&channel={channel}{version_query}"

if command -v curl >/dev/null 2>&1; then
  curl -fSL "$url" -o "$tmpdir/oz.tar.gz"
elif command -v wget >/dev/null 2>&1; then
  wget -q -O "$tmpdir/oz.tar.gz" "$url"
else
  echo "error: neither curl nor wget is available" >&2
  exit 3
fi

The exit code for "no HTTP client" is shared as a constant in setup.rs and injected into the script via placeholder substitution:

/// Exit code the install script uses when neither curl nor wget is
/// available on the remote host. The Rust side matches on this to
/// trigger the SCP upload fallback.
pub const NO_HTTP_CLIENT_EXIT_CODE: i32 = 3;

The script template uses exit {no_http_client_exit_code} instead of a hardcoded exit 3, and install_script() substitutes it alongside the existing placeholders.

Key details:

• command -v is POSIX-compliant and works on BusyBox sh, dash, bash, and zsh. Preferred over which (non-POSIX, absent on some minimal systems) and type (output format varies across shells).
• Exit code 3 is the next unused code after exit 1 (no binary in tarball) and exit 2 (unsupported arch/OS).
• wget -q -O matches the semantics of curl -fSL -o: quiet output, write to a specific file, follow redirects (wget follows by default, up to 20 hops). The -f (fail on HTTP errors) has no direct wget equivalent, but wget exits non-zero on 4xx/5xx by default.
• The url variable is extracted to avoid duplicating the long URL string between the curl and wget branches.
• The {placeholder} substitution from setup.rs is unchanged — no Rust changes needed for Phase 1 beyond the new constant and placeholder.

Phase 2: SCP upload fallback in Rust

When the install script exits with code 3 (no HTTP client), the client downloads the tarball locally and uploads it to the remote via scp through the existing ControlMaster socket.

This requires changes across three layers:

2a. New download_url() and install_tarball_path() public helpers in setup.rs

Expose the download URL construction so the Rust-side SCP fallback can download the same tarball the shell script would have fetched.

/// Returns the full download URL for the remote server tarball,
/// parameterized by the remote platform.
pub fn download_tarball_url(platform: &RemotePlatform) -> String {
    format!(
        "{}?package=tar&os={}&arch={}&channel={}{}",
        download_url(),
        platform.os.as_str(),
        platform.arch.as_str(),
        download_channel(),
        version_query(),
    )
}

/// Returns the remote path where the tarball should be uploaded
/// before the extraction script runs.
pub fn remote_tarball_staging_path() -> String {
    format!("{}/oz-upload.tar.gz", remote_server_dir())
}

Also extract a version_query() helper (currently inlined in install_script()) so both the shell script and the Rust download path use the same query string.

2b. New scp_upload utility in ssh.rs

Add an scp helper that uploads a local file to the remote through the ControlMaster socket:

/// Upload a local file to the remote host via `scp`, reusing the
/// ControlMaster socket for authentication. Returns `Ok(())` on
/// success or an error describing the failure.
pub async fn scp_upload(
    socket_path: &Path,
    local_path: &Path,
    remote_path: &str,
    timeout: Duration,
) -> Result<()> {
    async {
        Command::new("scp")
            .arg("-o").arg(format!("ControlPath={}", socket_path.display()))
            .arg("-o").arg("ControlMaster=no")
            .arg("-o").arg("ConnectTimeout=15")
            .arg(local_path.as_os_str())
            .arg(format!("placeholder@placeholder:{remote_path}"))
            .kill_on_drop(true)
            .output()
            .await
    }
    .with_timeout(timeout)
    .await
    .map_err(|_| anyhow!("scp timed out after {timeout:?}"))?
    .map_err(|e| anyhow!("scp failed to execute: {e}"))
    .and_then(|output| {
        if output.status.success() {
            Ok(())
        } else {
            let stderr = String::from_utf8_lossy(&output.stderr);
            Err(anyhow!("scp failed (exit {:?}): {stderr}", output.status.code()))
        }
    })
}

Key details:

• Reuses the same ControlMaster socket (-o ControlPath=...) that all other SSH operations use — no re-authentication needed.
• placeholder@placeholder matches the convention in ssh_args() (ssh.rs:26): the ControlMaster socket already has the real user/host baked in, so the CLI arguments are placeholders.
• ControlMaster=no ensures scp joins the existing master session rather than trying to become one.
• Timeout uses INSTALL_TIMEOUT (60s) from the caller since the upload replaces the download step.

2c. Shared extraction logic — single script with a tarball-path argument

Rather than maintaining two scripts with duplicated extraction code, refactor install_remote_server.sh so the download and extraction phases are cleanly separated within the same file. The script accepts an optional $1 argument: a path to an already-uploaded tarball. When provided, the script skips the download phase entirely and extracts from that path. When omitted, it runs the curl/wget download as before.

if [ -n "$1" ]; then
  # SCP fallback: tarball already uploaded by the client.
  tarball_src="$1"
  mv "$tarball_src" "$tmpdir/oz.tar.gz"
else
  # Normal path: download via curl or wget.
  url="{download_base_url}?package=tar&os=$os_name&arch=$arch_name&channel={channel}{version_query}"
  if command -v curl >/dev/null 2>&1; then
    curl -fSL "$url" -o "$tmpdir/oz.tar.gz"
  elif command -v wget >/dev/null 2>&1; then
    wget -q -O "$tmpdir/oz.tar.gz" "$url"
  else
    echo "error: neither curl nor wget is available" >&2
    exit {no_http_client_exit_code}
  fi
fi

# Shared extraction tail (unchanged from today's lines 45-50).
tar -xzf "$tmpdir/oz.tar.gz" -C "$tmpdir"
bin=$(find "$tmpdir" -type f -name 'oz*' ! -name '*.tar.gz' | head -n1)
if [ -z "$bin" ]; then echo "no binary found in tarball" >&2; exit 1; fi
chmod +x "$bin"
mv "$bin" "$install_dir/{binary_name}{version_suffix}"

The SCP fallback in Rust invokes the same script with the staging path as $1 via run_ssh_script by passing bash -s -- <staging_path> (or equivalently prepending the argument to the script). This eliminates code duplication and ensures any future extraction changes (e.g. checksum verification) apply to both paths.

The {staging_path} used by the SCP fallback is the expanded form of remote_tarball_staging_path().

2d. Modify SshTransport::install_binary to orchestrate the fallback

Change install_binary in ssh_transport.rs from a single run_ssh_script call to a two-step flow:

1. Run the existing install script (now with wget fallback from Phase 1).
2. If the script exits with code 3, fall back to SCP: a. Build the download URL using download_tarball_url with the RemotePlatform already detected by detect_platform (called earlier in the setup flow). b. Download the tarball locally using the system's HTTP client (reqwest or a curl subprocess on the local machine — the local machine is guaranteed to have internet access). c. Upload via scp_upload to the staging path. d. Run the extraction-only script via run_ssh_script.

To support this, SshTransport needs access to the RemotePlatform detected earlier. Add it as a field set during construction or via a setter called by the controller after detect_platform succeeds (it's already called before install_binary in the setup flow at manager.rs:500-530).

fn install_binary(&self) -> Pin<Box<dyn Future<Output = Result<(), String>> + Send>> {
    let socket_path = self.socket_path.clone();
    let platform = self.platform.clone();
    Box::pin(async move {
        let script = remote_server::setup::install_script();
        match remote_server::ssh::run_ssh_script(
            &socket_path, &script, remote_server::setup::INSTALL_TIMEOUT,
        ).await {
            Ok(output) if output.status.success() => Ok(()),
            Ok(output) if output.status.code() == Some(remote_server::setup::NO_HTTP_CLIENT_EXIT_CODE) => {
                // No HTTP client on remote — fall back to local download + SCP.
                log::info!("Remote has no curl/wget, falling back to SCP upload");
                let Some(platform) = platform else {
                    return Err("SCP fallback requires platform detection".into());
                };
                scp_install_fallback(&socket_path, &platform).await
            }
            Ok(output) => {
                let code = output.status.code().unwrap_or(-1);
                let stderr = String::from_utf8_lossy(&output.stderr);
                Err(format!("install script failed (exit {code}): {stderr}"))
            }
            Err(e) => Err(format!("{e:#}")),
        }
    })
}

The scp_install_fallback helper:

1. Constructs the URL via download_tarball_url(&platform).
2. Creates a local temp directory (tempfile::tempdir()) and downloads the tarball into it via a local curl subprocess. The temp directory is cleaned up automatically when the TempDir guard drops (including on early-return errors).
3. Calls scp_upload to transfer the local tarball to the remote staging path.
4. Re-invokes the install script with the staging path as $1 so the shared extraction tail runs.
5. TempDir drop handles cleanup.

Error handling and timeout budget

The SCP fallback uses a separate, longer timeout:

/// Timeout for the SCP upload fallback path (local download + SCP + extraction).
/// Longer than `INSTALL_TIMEOUT` because SCP transfers the ~30-50 MB tarball
/// over the user's SSH link, which is typically slower than the remote host's
/// direct internet connection. On a 1 MB/s link, upload alone takes 30-50s.
pub const SCP_INSTALL_TIMEOUT: Duration = Duration::from_secs(120);

The standard INSTALL_TIMEOUT (60s) is sufficient for the curl/wget path because the remote host downloads directly from the CDN. The SCP path adds a local download step (~5s) plus an SCP upload that depends entirely on the SSH link bandwidth — embedded devices, VPNs, and high-latency connections can easily exceed 60s for a ~30-50 MB transfer. Each sub-step (scp_upload, run_ssh_script for extraction) uses the full SCP_INSTALL_TIMEOUT individually to avoid splitting a single budget across steps, which would require coordination logic for diminishing remaining time.

• If the SCP upload or extraction fails, the error surfaces the same way as a normal install failure — BinaryInstallComplete { result: Err(_) } — and the session falls back to ControlMaster wormholing.

Testing and validation

Unit tests (setup_tests.rs)

• install_script_contains_wget_fallback — Assert that install_script() output contains command -v curl, command -v wget, and the exit {no_http_client_exit_code} sentinel. This is the main guardrail against accidentally regressing the fallback logic during future script edits.
• download_tarball_url_formats_correctly — Test download_tarball_url for each (RemoteOs, RemoteArch) combination. Catches URL construction drift between the shell script placeholders and the Rust-side URL builder, which would cause the SCP fallback to download the wrong artifact.

Integration / manual testing

• No curl, has wget: SSH into a Docker container with apt-get remove curl / Alpine with only wget. Verify install succeeds via wget path.
• No curl, no wget: SSH into a minimal BusyBox container with neither. Verify the script exits with NO_HTTP_CLIENT_EXIT_CODE, the client downloads locally, SCPs the tarball, and the extraction script installs successfully.
• Happy path unchanged: SSH into a standard Ubuntu/Debian host. Verify curl is still used (check logs for absence of "falling back" message).

Risks and mitigations

• BusyBox wget differences: BusyBox's wget is a stripped-down implementation that lacks some GNU wget flags. The flags used (-q -O) are supported by BusyBox wget. Notably, BusyBox wget does NOT support --connect-timeout — we omit it and rely on the outer SSH timeout (60s) instead.
• SCP deprecation: OpenSSH has been moving toward SFTP as the default transfer protocol. Modern scp (OpenSSH 9.0+) uses the SFTP protocol under the hood by default, so this isn't a practical concern. If a host has a very old SSH that lacks SCP, it almost certainly has curl or wget.
• Local download assumes curl on client: The local machine (macOS, Linux desktop, or Windows with WSL) is expected to have curl. macOS ships curl, and it's near-universal on Linux desktops.

Parallelization

This is a small, focused change. Sub-agents are not beneficial — the shell script change (Phase 1) and the Rust SCP fallback (Phase 2) are tightly coupled and should be in the same PR.