142 lines
10 KiB
Markdown
142 lines
10 KiB
Markdown
# Remote Server Integration Tests
|
||
|
||
## Context
|
||
|
||
The `SshRemoteServer` feature flag gates a new SSH session flow where a persistent binary (`remote-server-proxy`) runs on the remote host, replacing the legacy ControlMaster-based command execution. The feature has unit test coverage at the protocol layer (`crates/remote_server/src/client_tests.rs`) but no integration test coverage exercising the full client ↔ server lifecycle over a real SSH connection.
|
||
|
||
### Current integration test infra
|
||
|
||
The existing SSH integration tests (`crates/integration/src/test/ssh.rs`) cover the legacy warpification flow:
|
||
- Connect to a GCP-hosted Ubuntu VM (`ubuntu-14-04`) via IAP tunnel with password auth
|
||
- Helper steps in `app/src/integration_testing/subshell/` — `setup_gcloud_sdk()`, `enter_ssh_command()`, `enter_ssh_password()`, `wait_for_password_prompt()`
|
||
- Builder pattern: `new_builder().with_step(TestStep)` with assertion callbacks
|
||
- Feature flag gating via `set_should_run_test(|| FeatureFlag::X.is_enabled())`
|
||
|
||
### Remote server flow
|
||
|
||
When `SshRemoteServer` is enabled for a legacy SSH session (`app/src/terminal/writeable_pty/remote_server_controller.rs`):
|
||
|
||
1. `RemoteServerController` intercepts `SshInitShell`, stashes the bootstrap script
|
||
2. Runs `check_binary` via `RemoteServerManager` → if missing, `install_binary` → then `connect_session`
|
||
3. `connect_session` (`crates/remote_server/src/manager.rs:368`) spawns the proxy over SSH, performs the proto `Initialize` handshake → emits `SessionConnected { host_id }`
|
||
4. Bootstrap is flushed; `RemoteServerCommandExecutor` (`app/src/terminal/model/session/command_executor/remote_server_executor.rs`) is wired as the session's `CommandExecutor`
|
||
5. On CWD change, `navigate_to_directory` fires → returns `is_git` flag + triggers `RepoMetadataSnapshot` push
|
||
|
||
Key config: `SshExtensionInstallMode::AlwaysInstall` (setting in `app/src/terminal/warpify/settings.rs:85`) bypasses the choice block UI, needed for deterministic test flow.
|
||
|
||
### Binary deployment problem
|
||
|
||
The production install script (`crates/remote_server/src/install_remote_server.sh`) downloads from the CDN (`app.warp.dev/download/cli`). This fetches a published binary, not one built from the developer's branch. For integration tests, the binary must come from the current codebase so that changes to the remote-server protocol or logic are tested. The existing `script/deploy_remote_server` already solves this for local development — it cross-compiles for `x86_64-unknown-linux-musl` and uploads via rsync.
|
||
|
||
## Proposed changes
|
||
|
||
### 1. CI step: cross-compile and deploy binary to test VM
|
||
|
||
Add `script/deploy_remote_server_to_test_vm` that:
|
||
|
||
1. Cross-compiles the Oz CLI for the test VM target:
|
||
```
|
||
cargo build -p warp --bin warp --target x86_64-unknown-linux-musl \
|
||
--profile dev-remote \
|
||
--features release_bundle,crash_reporting,standalone,agent_mode_debug
|
||
```
|
||
Same build command as `script/deploy_remote_server`.
|
||
|
||
2. Uploads the binary to `ubuntu-14-04` at `~/.warp-dev/remote-server/oz-dev` via `sshpass` + `scp` through the GCP IAP tunnel (the test VM uses password auth; `sshpass` provides it non-interactively). Uses the same proxy command as the SSH integration tests (`app/src/integration_testing/subshell/util.rs:2`).
|
||
|
||
CI calls this script once before launching the integration test suite. Since `check_binary` will find the binary already present, the `RemoteServerController` flow becomes `check_binary → Ok(true) → connect_session`, skipping the CDN-based install.
|
||
|
||
### 2. Assertion helpers — `app/src/integration_testing/remote_server.rs`
|
||
|
||
New module with reusable test steps and action helpers:
|
||
|
||
- **`wait_for_remote_server_ready(tab_idx)`** — `TestStep` that polls until `Sessions::remote_server_setup_states` for the active session reaches `RemoteServerSetupState::Ready`.
|
||
- **`assert_remote_server_connected(tab_idx)`** — reads `RemoteServerManager` singleton, asserts the active session is in `RemoteSessionState::Connected`.
|
||
- **`assert_command_executor_is_remote_server(tab_idx)`** — downcasts the session's `CommandExecutor` via `as_any().downcast_ref::<RemoteServerCommandExecutor>()`.
|
||
- **`assert_remote_server_has_navigated(tab_idx)`** — asserts that `host_id_for_session` is populated for the active session.
|
||
- **`write_file_via_remote_server(tab_idx, path, content)`** — action callback that calls `RemoteServerClient::write_file` on a background thread (async → sync bridge via `tokio::runtime::Runtime::block_on`).
|
||
- **`load_repo_metadata_directory_via_remote_server(tab_idx, repo_path, dir_path)`** — action callback that calls `RemoteServerManager::load_remote_repo_metadata_directory` through the model handle.
|
||
|
||
Also adds `Session::command_executor()` accessor gated on `#[cfg(any(test, feature = "integration_tests"))]` (`app/src/terminal/model/session.rs`).
|
||
|
||
Registered in `app/src/integration_testing/mod.rs`.
|
||
|
||
### 3. Integration tests — `crates/integration/src/test/remote_server.rs`
|
||
|
||
All tests gated on `FeatureFlag::SshRemoteServer.is_enabled()` and configured with `with_user_defaults` setting `SshExtensionInstallMode` to `AlwaysInstall`.
|
||
|
||
#### Test A — Connection and handshake (`test_remote_server_connect_bash` / `_zsh`)
|
||
|
||
Validates the core flow: SSH → binary check → proto handshake → executor wiring.
|
||
|
||
Steps:
|
||
1. `wait_until_bootstrapped_single_pane_for_tab(0)` — local shell ready
|
||
2. `setup_gcloud_sdk()`
|
||
3. `enter_ssh_command(shell)` + `wait_for_password_prompt` + `enter_ssh_password`
|
||
4. `wait_for_remote_server_ready(0)` — covers check → connect → handshake
|
||
5. `wait_until_bootstrapped_single_pane_for_tab(0)` — remote shell bootstrapped
|
||
6. `assert_remote_server_connected(0)` — manager has `Connected` state with a `HostId`
|
||
7. `assert_command_executor_is_remote_server(0)` — session uses `RemoteServerCommandExecutor`
|
||
|
||
#### Test B — Repo metadata (`test_remote_server_navigate_to_repo`)
|
||
|
||
Validates the full navigate-to-directory flow: create git repo on remote → `cd` into it → `NavigatedToDirectory` response received → session has `host_id` tracked.
|
||
|
||
After Test A setup, plus:
|
||
8. Create a git repo on the remote via `execute_command("mkdir -p /tmp/warp-test-repo && cd /tmp/warp-test-repo && git init -b main ...")`
|
||
9. `execute_command("cd /tmp/warp-test-repo")` — triggers CWD change → `navigate_to_directory`
|
||
10. `assert_remote_server_has_navigated(0)` — host_id present for active session
|
||
11. `assert_remote_server_connected(0)` — session still healthy after navigation
|
||
|
||
#### Test C — Completions routing (`test_remote_server_completions`)
|
||
|
||
Validates that completions run through `RemoteServerCommandExecutor::execute_command` (the `RunCommand` proto path) rather than falling back to the legacy `RemoteCommandExecutor`.
|
||
|
||
After Test A setup, plus:
|
||
8. Run a command to trigger completions loading
|
||
9. `assert_command_executor_is_remote_server(0)` — confirm executor type
|
||
|
||
#### Test D — File write via proto client API (`test_remote_server_file_operations`)
|
||
|
||
Validates `WriteFile` through the `RemoteServerClient` proto API. Uses `write_file_via_remote_server` helper to dispatch the async write from an action callback, then reads the file back via a shell command (which goes through `RemoteServerCommandExecutor::run_command`) to confirm content integrity.
|
||
|
||
After Test A setup, plus:
|
||
8. `write_file_via_remote_server(0, "/tmp/warp-rs-test-file.txt", "hello from proto")` — writes via proto
|
||
9. `execute_command("cat /tmp/warp-rs-test-file.txt")` — reads back via `RunCommand`, asserts `"hello from proto"`
|
||
10. Clean up and verify executor type
|
||
|
||
#### Test E — Lazy loading repo metadata (`test_remote_server_lazy_load_directory`)
|
||
|
||
Validates the `LoadRepoMetadataDirectory` proto round-trip: navigate to a git repo → create a subdirectory → call `load_remote_repo_metadata_directory` for that subdirectory → verify the response flows through without error.
|
||
|
||
After Test A setup, plus:
|
||
8. Create a git repo with a `subdir/nested` file on the remote
|
||
9. `cd /tmp/warp-lazy-repo` → triggers `NavigatedToDirectory` and full indexing
|
||
10. `load_repo_metadata_directory_via_remote_server(0, repo_path, "subdir")` — triggers lazy-load proto request
|
||
11. `assert_remote_server_connected(0)` — connection still healthy
|
||
12. `execute_command("cat subdir/nested")` — verifies subdirectory content accessible
|
||
|
||
### 4. Wire into the test runner
|
||
|
||
- Add `mod remote_server;` + `pub use remote_server::*;` in `crates/integration/src/test.rs`
|
||
- Register test functions in `crates/integration/tests/integration/shell_integration_tests.rs`
|
||
- Register in `crates/integration/src/bin/integration.rs` for the manual runner
|
||
|
||
## Testing and validation
|
||
|
||
The tests in this spec *are* the validation — they verify the remote-server feature works end-to-end over a real SSH connection. Specifically:
|
||
|
||
- **Test A** proves the install-check → handshake → executor-wiring pipeline works against a real remote host, catching protocol mismatches or connection failures that unit tests with mock streams cannot.
|
||
- **Test B** proves the `NavigatedToDirectory` → repo metadata pipeline works through the proto, catching serialization issues or git-detection regressions on the remote.
|
||
- **Test C** proves completions are routed through the remote-server binary rather than silently falling back to the legacy ControlMaster executor, which would mask remote-server regressions.
|
||
- **Test D** proves the `WriteFile` proto message works end-to-end by writing via the client API and reading back via `RunCommand`, catching serialization or `FileModel` regressions invisible to shell-only tests.
|
||
- **Test E** proves the `LoadRepoMetadataDirectory` lazy-loading proto round-trip works, catching regressions in the subdirectory expansion path that is distinct from the initial `NavigatedToDirectory` indexing.
|
||
|
||
All tests run against a binary built from the current branch (via the CI deploy step), ensuring protocol changes are tested before merge.
|
||
|
||
## Parallelization
|
||
|
||
- **CI script** (step 1) and **assertion helpers** (step 2) can be built in parallel — they touch disjoint files.
|
||
- **Test module** (step 3) depends on the assertion helpers but the test groups (A–E) are independent functions that can be written in parallel once the shared helpers exist.
|
||
- **Test runner wiring** (step 4) is trivial and can be done alongside any other step.
|