Replay agent events on restore (#9251)

## Description

Restore orchestration event delivery on the client after a Warp restart
so that a parent conversation continues to receive lifecycle events and
inbox messages from its children — including terminal events that
arrived while Warp was not running.

See the full design in `specs/replay-agent-events-on-restore/PRODUCT.md`
and `specs/replay-agent-events-on-restore/TECH.md`.

This is a re-land of warpdotdev/warp-internal#24999, which was reverted
in warpdotdev/warp-internal#25055 due to a CI race. This version fixes
the test compilation issue (missing fields in `AmbientAgentTask` struct
literal in `conversation_ended_tombstone_view_tests.rs`).

### What

- Persists the per-conversation event cursor across restarts.
- Adds `last_event_sequence: Option<i64>` to `AgentConversationData`
(SQLite) and `AIConversation`.
- New `BlocklistAIHistoryModel::update_event_sequence` helper writes the
cursor through `write_updated_conversation_state` after each event
batch.
- Also persists the cursor to the server (fire-and-forget) so driver /
cloud restarts can resume without local SQLite state.
- Restores `OrchestrationEventPoller.watched_run_ids` and re-establishes
event delivery on `BlocklistAIHistoryEvent::RestoredConversations`.
- New `on_restored_conversations` handler issues `GET
/agent/runs/{run_id}` for each restored parent and uses the response
inline `children` and `last_event_sequence` to populate watched run ids
and merge the cursor (`max(SQLite, server)`).
- Fetch failures retry with exponential backoff (1s, 2s, 5s, 10s capped)
keyed off a per-conversation `restore_fetch_failures` counter; reset on
success and on conversation removal.
- Gated on `OrchestrationV2`. Shared-session viewers and conversations
without children are skipped.
- `Success` parents resume delivery immediately; `InProgress` parents
defer to the existing `on_conversation_status_updated` path.
- Restores V1 lifecycle subscriptions on restart by extending the
existing `RestoredConversations` handler in `OrchestrationEventService`
to re-register `lifecycle_subscription_routes` for restored child
conversations whose parents are present locally.

### Why

After a Warp restart, parent conversations were silently receiving no
further events from children. The `event_cursor` in
`OrchestrationEventPoller` was always initialized to 0, so even if
delivery had resumed, every event since the start of the conversation
would have replayed and produced duplicate messages. V1 lifecycle
subscription routes were also not restored, so V1 parents missed child
status transitions.

## Testing

- Added unit tests in
`app/src/ai/blocklist/orchestration_event_poller_tests.rs` covering:
cursor merge from server vs SQLite, retry on `get_ambient_agent_task`
failure, V2 gating, shared-session-viewer exclusion, cleanup on delete,
and `last_event_sequence` round-trip through
`AIConversation::new_restored`.
- Added unit test coverage in
`app/src/ai/blocklist/orchestration_events_tests.rs` for V1 lifecycle
re-registration on restore.
- Manual verification per
`specs/replay-agent-events-on-restore/TECH.md`.

## Server API dependencies

- [x] Does this change rely on a [new server
API](https://www.notion.so/warpdev/How-to-add-a-new-full-stack-feature-8412cede405a4ec194b32bdd4b951035?pvs=4#04da1e6a493542d68b3e998c7d339640)?
- [x] If so, is the use of this API restricted to client channels that
rely on the staging server (e.g. WarpDev)?

The companion server change adds:
- `last_event_sequence` on `Task` (`ai_tasks` column), surfaced inline
on `GET /agent/runs/:run_id`.
- `PATCH /agent/runs/:run_id/event-sequence` for client cursor writes.
- `children` inline on the `GET /agent/runs/:run_id` response.

## Agent Mode
- [x] Warp Agent Mode - This PR was created via Warp's AI Agent Mode

Co-Authored-By: Oz <oz-agent@warp.dev>

---------

Co-authored-by: Oz <oz-agent@warp.dev>
This commit is contained in:
Matthew Albright
2026-04-28 14:24:54 -04:00
committed by GitHub
co-authored by Oz
parent 4e807624c6
commit 57e8e3e9be
22 changed files with 1334 additions and 1 deletions
+62
View File
@@ -1036,6 +1036,11 @@ pub struct AgentConversationData {
pub run_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub autoexecute_override: Option<PersistedAutoexecuteMode>,
/// The last event sequence number from the v2 orchestration event log
/// that this conversation has observed. Used on restore to resume event
/// delivery without re-delivering already-processed events.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub last_event_sequence: Option<i64>,
}
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
@@ -1327,6 +1332,63 @@ pub struct NewMCPServerInstallation {
pub last_modified_at: NaiveDateTime,
}
#[cfg(test)]
mod tests {
use super::AgentConversationData;
#[test]
fn agent_conversation_data_roundtrips_last_event_sequence() {
let data = AgentConversationData {
server_conversation_token: None,
conversation_usage_metadata: None,
reverted_action_ids: None,
forked_from_server_conversation_token: None,
artifacts_json: None,
parent_agent_id: None,
agent_name: None,
parent_conversation_id: None,
run_id: None,
autoexecute_override: None,
last_event_sequence: Some(42),
};
let json = serde_json::to_string(&data).expect("serialize");
let roundtripped: AgentConversationData = serde_json::from_str(&json).expect("deserialize");
assert_eq!(roundtripped.last_event_sequence, Some(42));
}
#[test]
fn agent_conversation_data_deserializes_legacy_payload_without_last_event_sequence() {
// Legacy rows persisted before this feature landed omit the field
// entirely. `#[serde(default)]` must accept them as `None`.
let legacy_json = r#"{"server_conversation_token":null}"#;
let data: AgentConversationData =
serde_json::from_str(legacy_json).expect("legacy rows must deserialize");
assert_eq!(data.last_event_sequence, None);
}
#[test]
fn agent_conversation_data_skips_serializing_none_last_event_sequence() {
let data = AgentConversationData {
server_conversation_token: None,
conversation_usage_metadata: None,
reverted_action_ids: None,
forked_from_server_conversation_token: None,
artifacts_json: None,
parent_agent_id: None,
agent_name: None,
parent_conversation_id: None,
run_id: None,
autoexecute_override: None,
last_event_sequence: None,
};
let json = serde_json::to_string(&data).expect("serialize");
assert!(
!json.contains("last_event_sequence"),
"None should be skipped in serialized output: {json}"
);
}
}
#[derive(Insertable)]
#[diesel(table_name = panels)]
pub struct NewPanel {