## Description Restore orchestration event delivery on the client after a Warp restart so that a parent conversation continues to receive lifecycle events and inbox messages from its children — including terminal events that arrived while Warp was not running. See the full design in `specs/replay-agent-events-on-restore/PRODUCT.md` and `specs/replay-agent-events-on-restore/TECH.md`. This is a re-land of warpdotdev/warp-internal#24999, which was reverted in warpdotdev/warp-internal#25055 due to a CI race. This version fixes the test compilation issue (missing fields in `AmbientAgentTask` struct literal in `conversation_ended_tombstone_view_tests.rs`). ### What - Persists the per-conversation event cursor across restarts. - Adds `last_event_sequence: Option<i64>` to `AgentConversationData` (SQLite) and `AIConversation`. - New `BlocklistAIHistoryModel::update_event_sequence` helper writes the cursor through `write_updated_conversation_state` after each event batch. - Also persists the cursor to the server (fire-and-forget) so driver / cloud restarts can resume without local SQLite state. - Restores `OrchestrationEventPoller.watched_run_ids` and re-establishes event delivery on `BlocklistAIHistoryEvent::RestoredConversations`. - New `on_restored_conversations` handler issues `GET /agent/runs/{run_id}` for each restored parent and uses the response inline `children` and `last_event_sequence` to populate watched run ids and merge the cursor (`max(SQLite, server)`). - Fetch failures retry with exponential backoff (1s, 2s, 5s, 10s capped) keyed off a per-conversation `restore_fetch_failures` counter; reset on success and on conversation removal. - Gated on `OrchestrationV2`. Shared-session viewers and conversations without children are skipped. - `Success` parents resume delivery immediately; `InProgress` parents defer to the existing `on_conversation_status_updated` path. - Restores V1 lifecycle subscriptions on restart by extending the existing `RestoredConversations` handler in `OrchestrationEventService` to re-register `lifecycle_subscription_routes` for restored child conversations whose parents are present locally. ### Why After a Warp restart, parent conversations were silently receiving no further events from children. The `event_cursor` in `OrchestrationEventPoller` was always initialized to 0, so even if delivery had resumed, every event since the start of the conversation would have replayed and produced duplicate messages. V1 lifecycle subscription routes were also not restored, so V1 parents missed child status transitions. ## Testing - Added unit tests in `app/src/ai/blocklist/orchestration_event_poller_tests.rs` covering: cursor merge from server vs SQLite, retry on `get_ambient_agent_task` failure, V2 gating, shared-session-viewer exclusion, cleanup on delete, and `last_event_sequence` round-trip through `AIConversation::new_restored`. - Added unit test coverage in `app/src/ai/blocklist/orchestration_events_tests.rs` for V1 lifecycle re-registration on restore. - Manual verification per `specs/replay-agent-events-on-restore/TECH.md`. ## Server API dependencies - [x] Does this change rely on a [new server API](https://www.notion.so/warpdev/How-to-add-a-new-full-stack-feature-8412cede405a4ec194b32bdd4b951035?pvs=4#04da1e6a493542d68b3e998c7d339640)? - [x] If so, is the use of this API restricted to client channels that rely on the staging server (e.g. WarpDev)? The companion server change adds: - `last_event_sequence` on `Task` (`ai_tasks` column), surfaced inline on `GET /agent/runs/:run_id`. - `PATCH /agent/runs/:run_id/event-sequence` for client cursor writes. - `children` inline on the `GET /agent/runs/:run_id` response. ## Agent Mode - [x] Warp Agent Mode - This PR was created via Warp's AI Agent Mode Co-Authored-By: Oz <oz-agent@warp.dev> --------- Co-authored-by: Oz <oz-agent@warp.dev>
273 lines
9.2 KiB
Rust
273 lines
9.2 KiB
Rust
use std::collections::HashMap;
|
|
|
|
use chrono::{Local, Utc};
|
|
use persistence::model::AgentConversationData;
|
|
use warp_cli::agent::Harness;
|
|
use warp_core::features::FeatureFlag;
|
|
use warp_multi_agent_api as api;
|
|
use warpui::{App, EntityId};
|
|
|
|
use crate::ai::agent::conversation::{AIConversation, AIConversationId};
|
|
use crate::ai::ambient_agents::task::{AgentConfigSnapshot, HarnessConfig, TaskCreatorInfo};
|
|
use crate::ai::ambient_agents::{AmbientAgentTask, AmbientAgentTaskState};
|
|
use crate::ai::blocklist::history_model::BlocklistAIHistoryModel;
|
|
|
|
use super::{ConversationDetailsData, PanelMode};
|
|
|
|
fn create_test_task(task_id: &str) -> AmbientAgentTask {
|
|
let now = Utc::now();
|
|
AmbientAgentTask {
|
|
task_id: task_id.parse().unwrap(),
|
|
parent_run_id: None,
|
|
title: "Task".to_string(),
|
|
state: AmbientAgentTaskState::Succeeded,
|
|
prompt: "test".to_string(),
|
|
created_at: now,
|
|
started_at: None,
|
|
updated_at: now,
|
|
status_message: None,
|
|
source: None,
|
|
session_id: None,
|
|
session_link: None,
|
|
creator: Some(TaskCreatorInfo {
|
|
creator_type: "USER".to_string(),
|
|
uid: "user-1".to_string(),
|
|
display_name: Some("User 1".to_string()),
|
|
}),
|
|
conversation_id: None,
|
|
request_usage: None,
|
|
agent_config_snapshot: None,
|
|
artifacts: vec![],
|
|
is_sandbox_running: false,
|
|
last_event_sequence: None,
|
|
children: vec![],
|
|
}
|
|
}
|
|
|
|
fn create_message_with_directory(id: &str, task_id: &str, directory: &str) -> api::Message {
|
|
api::Message {
|
|
id: id.to_string(),
|
|
task_id: task_id.to_string(),
|
|
server_message_data: String::new(),
|
|
citations: vec![],
|
|
message: Some(api::message::Message::UserQuery(api::message::UserQuery {
|
|
query: "test query".to_string(),
|
|
context: Some(api::InputContext {
|
|
directory: Some(api::input_context::Directory {
|
|
pwd: directory.to_string(),
|
|
home: String::new(),
|
|
pwd_file_symbols_indexed: false,
|
|
}),
|
|
..Default::default()
|
|
}),
|
|
referenced_attachments: HashMap::new(),
|
|
mode: None,
|
|
intended_agent: Default::default(),
|
|
})),
|
|
request_id: "request-1".to_string(),
|
|
timestamp: None,
|
|
}
|
|
}
|
|
|
|
fn create_agent_output_message(id: &str, task_id: &str) -> api::Message {
|
|
api::Message {
|
|
id: id.to_string(),
|
|
task_id: task_id.to_string(),
|
|
server_message_data: String::new(),
|
|
citations: vec![],
|
|
message: Some(api::message::Message::AgentOutput(
|
|
api::message::AgentOutput {
|
|
text: "done".to_string(),
|
|
},
|
|
)),
|
|
request_id: "request-1".to_string(),
|
|
timestamp: None,
|
|
}
|
|
}
|
|
|
|
fn create_restored_conversation(
|
|
conversation_id: AIConversationId,
|
|
root_task_id: &str,
|
|
directory: &str,
|
|
conversation_data: AgentConversationData,
|
|
) -> AIConversation {
|
|
let task = api::Task {
|
|
id: root_task_id.to_string(),
|
|
messages: vec![
|
|
create_message_with_directory("message-1", root_task_id, directory),
|
|
create_agent_output_message("message-2", root_task_id),
|
|
],
|
|
dependencies: None,
|
|
description: String::new(),
|
|
summary: String::new(),
|
|
server_data: String::new(),
|
|
};
|
|
|
|
AIConversation::new_restored(conversation_id, vec![task], Some(conversation_data))
|
|
.expect("restored conversation should build")
|
|
}
|
|
|
|
#[test]
|
|
fn test_from_task_includes_linked_directory_when_run_id_matches() {
|
|
App::test((), |mut app| async move {
|
|
let _orchestration_v2_guard = FeatureFlag::OrchestrationV2.override_enabled(true);
|
|
let history_model = app.add_singleton_model(|_| BlocklistAIHistoryModel::new(vec![], &[]));
|
|
|
|
let conversation_id = AIConversationId::new();
|
|
let task_id = "550e8400-e29b-41d4-a716-000000004000";
|
|
let directory = "/tmp/run-id-directory";
|
|
|
|
let conversation = create_restored_conversation(
|
|
conversation_id,
|
|
"root-task",
|
|
directory,
|
|
AgentConversationData {
|
|
server_conversation_token: None,
|
|
conversation_usage_metadata: None,
|
|
reverted_action_ids: None,
|
|
forked_from_server_conversation_token: None,
|
|
artifacts_json: None,
|
|
parent_agent_id: None,
|
|
agent_name: None,
|
|
parent_conversation_id: None,
|
|
run_id: Some(task_id.to_string()),
|
|
autoexecute_override: None,
|
|
last_event_sequence: None,
|
|
},
|
|
);
|
|
|
|
history_model.update(&mut app, |model, ctx| {
|
|
model.restore_conversations(EntityId::new(), vec![conversation], ctx);
|
|
});
|
|
|
|
let task = create_test_task(task_id);
|
|
app.update(|ctx| {
|
|
let data = ConversationDetailsData::from_task(&task, None, None, ctx);
|
|
assert!(matches!(
|
|
data.mode,
|
|
PanelMode::Task {
|
|
directory: Some(ref task_directory),
|
|
..
|
|
} if task_directory == directory
|
|
));
|
|
});
|
|
});
|
|
}
|
|
|
|
#[test]
|
|
fn test_from_conversation_metadata_passes_harness_through() {
|
|
for harness in [
|
|
None,
|
|
Some(Harness::Oz),
|
|
Some(Harness::Claude),
|
|
Some(Harness::Gemini),
|
|
Some(Harness::Unknown),
|
|
] {
|
|
let data = ConversationDetailsData::from_conversation_metadata(
|
|
AIConversationId::new(),
|
|
"Title".to_string(),
|
|
None,
|
|
Utc::now().with_timezone(&Local),
|
|
None,
|
|
None,
|
|
None,
|
|
vec![],
|
|
None,
|
|
None,
|
|
None,
|
|
None,
|
|
harness,
|
|
);
|
|
assert_eq!(
|
|
data.harness, harness,
|
|
"harness {harness:?} should pass through"
|
|
);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn test_from_task_resolves_harness() {
|
|
App::test((), |mut app| async move {
|
|
let _history_model = app.add_singleton_model(|_| BlocklistAIHistoryModel::new(vec![], &[]));
|
|
|
|
// Base task has `agent_config_snapshot: None`; cloning lets us mutate per case.
|
|
let base_task = create_test_task("550e8400-e29b-41d4-a716-000000004020");
|
|
|
|
app.update(|ctx| {
|
|
// No snapshot → harness unknown.
|
|
let data = ConversationDetailsData::from_task(&base_task, None, None, ctx);
|
|
assert_eq!(data.harness, None);
|
|
|
|
// Snapshot without an explicit harness → default to Warp Agent.
|
|
let mut task = base_task.clone();
|
|
task.agent_config_snapshot = Some(AgentConfigSnapshot::default());
|
|
let data = ConversationDetailsData::from_task(&task, None, None, ctx);
|
|
assert_eq!(data.harness, Some(Harness::Oz));
|
|
|
|
// Snapshot with explicit harness_type.
|
|
for harness in [
|
|
Harness::Oz,
|
|
Harness::Claude,
|
|
Harness::Gemini,
|
|
Harness::Unknown,
|
|
] {
|
|
let mut task = base_task.clone();
|
|
task.agent_config_snapshot = Some(AgentConfigSnapshot {
|
|
harness: Some(HarnessConfig::from_harness_type(harness)),
|
|
..Default::default()
|
|
});
|
|
let data = ConversationDetailsData::from_task(&task, None, None, ctx);
|
|
assert_eq!(data.harness, Some(harness), "harness {harness:?}");
|
|
}
|
|
});
|
|
});
|
|
}
|
|
|
|
#[test]
|
|
fn test_from_task_includes_linked_directory_when_server_token_matches() {
|
|
App::test((), |mut app| async move {
|
|
let history_model = app.add_singleton_model(|_| BlocklistAIHistoryModel::new(vec![], &[]));
|
|
|
|
let conversation_id = AIConversationId::new();
|
|
let server_token = "server-token-123";
|
|
let directory = "/tmp/server-token-directory";
|
|
|
|
let conversation = create_restored_conversation(
|
|
conversation_id,
|
|
"root-task",
|
|
directory,
|
|
AgentConversationData {
|
|
server_conversation_token: Some(server_token.to_string()),
|
|
conversation_usage_metadata: None,
|
|
reverted_action_ids: None,
|
|
forked_from_server_conversation_token: None,
|
|
artifacts_json: None,
|
|
parent_agent_id: None,
|
|
agent_name: None,
|
|
parent_conversation_id: None,
|
|
run_id: None,
|
|
autoexecute_override: None,
|
|
last_event_sequence: None,
|
|
},
|
|
);
|
|
|
|
history_model.update(&mut app, |model, ctx| {
|
|
model.restore_conversations(EntityId::new(), vec![conversation], ctx);
|
|
});
|
|
|
|
let mut task = create_test_task("550e8400-e29b-41d4-a716-000000004001");
|
|
task.conversation_id = Some(server_token.to_string());
|
|
|
|
app.update(|ctx| {
|
|
let data = ConversationDetailsData::from_task(&task, None, None, ctx);
|
|
assert!(matches!(
|
|
data.mode,
|
|
PanelMode::Task {
|
|
directory: Some(ref task_directory),
|
|
..
|
|
} if task_directory == directory
|
|
));
|
|
});
|
|
});
|
|
}
|