Initial public release of Warp.
Repo-Sync-Origin: warpdotdev/warp-internal@12af1d983b
This commit is contained in:
David Stern
@@ -0,0 +1,60 @@
|
||||
# How to perform migrations
|
||||
|
||||
## How do migrations work?
|
||||
A sqlite database is one single file on the user's computer, and instead of being a separate process, it's a set of C functions
|
||||
that are bundled in our app. When the warp app starts, we upgrade the schema to the latest version in a transaction.
|
||||
Since we don't control the machine, we need to be super careful about the migrations that we ship.
|
||||
|
||||
TODO: C.I. and remediation of failed migrations
|
||||
|
||||
## Step 1: One-time setup
|
||||
Make sure you have run `/script/bootstrap` at least once before. That installs our fork of
|
||||
`diesel_cli`. Our fork is an old version that [bundles](https://github.com/warpdotdev/diesel/blob/b2c58897c39c519a946314bd5b63765d3af56204/diesel_cli/Cargo.toml#L54)
|
||||
SQLite in with the `diesel_cli`. We use these version of Diesel and SQLite instead of relying on
|
||||
the versions on our machines. So do not follow the official Diesel CLI installation instructions.
|
||||
|
||||
Additionally, the `sqlite3` binary is useful in development. The one on your Macbook is ok, but
|
||||
likely will be missing some basic language features. You can grab the latest binary from the
|
||||
official website https://www.sqlite.org/download.html. Note that we use SQL language features that
|
||||
are not available on old versions of sqlite. Note that you'll have to approve it to be run in your mac system preferences.
|
||||
|
||||
## Step 2: Write the migration
|
||||
```
|
||||
diesel migration generate <descriptive name of your migration>
|
||||
```
|
||||
This will create a new folder with an up.sql and down.sql.
|
||||
|
||||
## Step 3: Run the migration + generate the schema
|
||||
```
|
||||
cd <warp-internal repo>
|
||||
diesel migration run --database-url="/Users/$USER/Library/Application Support/dev.warp.Warp-Local/warp.sqlite"
|
||||
```
|
||||
This will run the migration on the same warp that runs when you run the app locally. This automatically generates or updates the `app/src/persistence/schema.rs`. We do not make manual edits to `schema.rs`.
|
||||
|
||||
You can also print the schema from a database that already has the migration with:
|
||||
```
|
||||
diesel print-schema --database-url="/Users/$USER/Library/Application Support/dev.warp.Warp-Local/warp.sqlite"
|
||||
```
|
||||
|
||||
## Reverting/redo-ing migrations
|
||||
As you are writing features and changing branches, you'll want to undo migrations to fix your database and make it compatible with older code. Redo-ing can also be helpful as you are iterating on your schema.
|
||||
```
|
||||
diesel migration revert --database-url="/Users/$USER/Library/Application Support/dev.warp.Warp-Local/warp.sqlite"
|
||||
diesel migration redo --database-url="/Users/$USER/Library/Application Support/dev.warp.Warp-Local/warp.sqlite"
|
||||
```
|
||||
|
||||
# Schema style
|
||||
- We use `id` for integer primary keys. If there's something more special about the primary key, consider a more descriptive name.
|
||||
- Plural table names, singular for structs in the rust model code.
|
||||
- If there's a table `foos` and a table `bars`, and `bars` has a column with a foreign key that references `foos.id`, name it `foo_id`.
|
||||
|
||||
# The `schema.patch` file
|
||||
The `app/src/persistence/schema.patch` is manually updated by us. This file lets us make manual
|
||||
changes on top of the auto-generated `schema.rs` file produced by `diesel_cli`.
|
||||
|
||||
To create the `schema.patch` file, we:
|
||||
1. Run the diesel migrations
|
||||
1. Manually edit `schema.rs`
|
||||
1. Run `git diff -U6 > app/src/persistence/schema.patch`.
|
||||
|
||||
You can read more about this patch file in the official [Diesel documentation](https://diesel.rs/guides/configuring-diesel-cli.html#the-patch_file-field).
|
||||
@@ -0,0 +1,214 @@
|
||||
use diesel::associations::HasTable;
|
||||
use diesel::{prelude::*, result::Error, SqliteConnection};
|
||||
use prost::Message;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use warp_multi_agent_api as api;
|
||||
|
||||
use super::model::{AgentConversation, AgentConversationData};
|
||||
use crate::persistence::model::{AgentConversationRecord, AgentTaskRecord};
|
||||
use crate::persistence::schema::{self, agent_conversations, agent_tasks};
|
||||
|
||||
#[derive(Debug, Insertable, AsChangeset)]
|
||||
#[diesel(table_name = agent_conversations)]
|
||||
struct NewAgentConversation {
|
||||
conversation_id: String,
|
||||
conversation_data: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Insertable, AsChangeset)]
|
||||
#[diesel(table_name = agent_tasks)]
|
||||
struct NewAgentTask {
|
||||
conversation_id: String,
|
||||
task_id: String,
|
||||
task: Vec<u8>,
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub(super) enum UpsertConversationError {
|
||||
#[error("Failed to serialize conversation data: {0:?}")]
|
||||
Serialization(#[from] serde_json::Error),
|
||||
#[error("Failed to upsert conversation to sqlite: {0:?}")]
|
||||
DB(#[from] diesel::result::Error),
|
||||
}
|
||||
|
||||
pub(super) fn upsert_agent_conversation<'a>(
|
||||
conn: &mut SqliteConnection,
|
||||
conversation_id_param: &str,
|
||||
tasks: impl IntoIterator<Item = &'a api::Task>,
|
||||
conversation_data_param: AgentConversationData,
|
||||
) -> Result<(), UpsertConversationError> {
|
||||
use diesel::ExpressionMethods;
|
||||
use diesel::QueryDsl;
|
||||
use schema::agent_conversations::dsl::*;
|
||||
use schema::agent_tasks::dsl as tasks_dsl;
|
||||
const MAX_PERSISTED_CONVERSATION_COUNT: i64 = 100;
|
||||
|
||||
let serialized_conversation_data = serde_json::to_string(&conversation_data_param)?;
|
||||
|
||||
conn.transaction::<_, Error, _>(|conn| {
|
||||
// Upsert the conversation level metadata
|
||||
let new_conversation = NewAgentConversation {
|
||||
conversation_id: conversation_id_param.to_owned(),
|
||||
conversation_data: serialized_conversation_data,
|
||||
};
|
||||
|
||||
diesel::insert_into(agent_conversations::table())
|
||||
.values(&new_conversation)
|
||||
.on_conflict(conversation_id)
|
||||
.do_update()
|
||||
.set(&new_conversation)
|
||||
.execute(conn)?;
|
||||
|
||||
// Upsert each task
|
||||
for task in tasks {
|
||||
let task_binary = task.encode_to_vec();
|
||||
let new_task = NewAgentTask {
|
||||
conversation_id: conversation_id_param.to_owned(),
|
||||
task_id: task.id.clone(),
|
||||
task: task_binary,
|
||||
};
|
||||
|
||||
if let Err(e) = diesel::insert_into(agent_tasks::table)
|
||||
.values(&new_task)
|
||||
.on_conflict(tasks_dsl::task_id)
|
||||
.do_update()
|
||||
.set(&new_task)
|
||||
.execute(conn)
|
||||
{
|
||||
log::warn!("Failed to upsert task {e:?}");
|
||||
return Err(e);
|
||||
}
|
||||
}
|
||||
|
||||
// Prune old conversations if we exceed MAX_PERSISTED_CONVERSATION_COUNT conversations
|
||||
let conversation_count: i64 = agent_conversations::table().count().get_result(conn)?;
|
||||
if conversation_count > MAX_PERSISTED_CONVERSATION_COUNT {
|
||||
// Remove the oldest conversations, keeping only the most recent MAX_PERSISTED_CONVERSATION_COUNT
|
||||
let conversations_to_remove: Vec<String> = agent_conversations::table()
|
||||
.order(last_modified_at.asc())
|
||||
.limit(conversation_count - MAX_PERSISTED_CONVERSATION_COUNT)
|
||||
.select(conversation_id)
|
||||
.load(conn)?;
|
||||
|
||||
delete_agent_conversations(conn, conversations_to_remove)?;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(super) fn read_agent_conversations(
|
||||
conn: &mut SqliteConnection,
|
||||
) -> Result<Vec<AgentConversation>, diesel::result::Error> {
|
||||
use schema::agent_conversations::dsl::*;
|
||||
|
||||
let mut conversations_by_id = HashMap::<String, AgentConversation>::from_iter(
|
||||
agent_conversations
|
||||
.select(AgentConversationRecord::as_select())
|
||||
.load(conn)?
|
||||
.into_iter()
|
||||
.map(|conversation| {
|
||||
(
|
||||
conversation.conversation_id.clone(),
|
||||
AgentConversation {
|
||||
conversation,
|
||||
tasks: vec![],
|
||||
},
|
||||
)
|
||||
}),
|
||||
);
|
||||
|
||||
let task_records: Vec<AgentTaskRecord> = agent_tasks::table
|
||||
.select(AgentTaskRecord::as_select())
|
||||
.load(conn)?;
|
||||
|
||||
let mut invalid_conversation_ids = HashSet::new();
|
||||
for task_record in task_records {
|
||||
if let Some(conversation) = conversations_by_id.get_mut(&task_record.conversation_id) {
|
||||
match api::Task::decode(&task_record.task[..]) {
|
||||
Ok(api_task) => {
|
||||
conversation.tasks.push(api_task);
|
||||
}
|
||||
Err(e) => {
|
||||
log::error!("Failed to decode task protobuf: {e}");
|
||||
|
||||
invalid_conversation_ids
|
||||
.insert(conversation.conversation.conversation_id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
conversations_by_id.retain(|c_id, _| !invalid_conversation_ids.contains(c_id));
|
||||
|
||||
Ok(conversations_by_id.into_values().collect())
|
||||
}
|
||||
|
||||
/// Read a single agent conversation by its ID, including decoded tasks.
|
||||
pub(crate) fn read_agent_conversation_by_id(
|
||||
conn: &mut SqliteConnection,
|
||||
conversation_id_str: &str,
|
||||
) -> Result<Option<AgentConversation>, diesel::result::Error> {
|
||||
use schema::agent_conversations::dsl as convo_dsl;
|
||||
use schema::agent_tasks::dsl as tasks_dsl;
|
||||
|
||||
let maybe_record: Option<AgentConversationRecord> = convo_dsl::agent_conversations
|
||||
.filter(convo_dsl::conversation_id.eq(conversation_id_str.to_owned()))
|
||||
.select(AgentConversationRecord::as_select())
|
||||
.first(conn)
|
||||
.optional()?;
|
||||
|
||||
let Some(conversation_record) = maybe_record else {
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
let task_records: Vec<AgentTaskRecord> = schema::agent_tasks::table
|
||||
.filter(tasks_dsl::conversation_id.eq(conversation_id_str))
|
||||
.select(AgentTaskRecord::as_select())
|
||||
.load(conn)?;
|
||||
|
||||
let mut decoded_tasks = Vec::new();
|
||||
for task_record in task_records.into_iter() {
|
||||
match api::Task::decode(&task_record.task[..]) {
|
||||
Ok(task) => decoded_tasks.push(task),
|
||||
Err(e) => {
|
||||
log::error!("Failed to decode task protobuf: {e}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(Some(AgentConversation {
|
||||
conversation: conversation_record,
|
||||
tasks: decoded_tasks,
|
||||
}))
|
||||
}
|
||||
|
||||
pub(super) fn delete_agent_conversations(
|
||||
conn: &mut SqliteConnection,
|
||||
conversation_ids: Vec<String>,
|
||||
) -> Result<(), diesel::result::Error> {
|
||||
use diesel::ExpressionMethods;
|
||||
use diesel::QueryDsl;
|
||||
use schema::agent_conversations::dsl::*;
|
||||
use schema::agent_tasks::dsl as tasks_dsl;
|
||||
|
||||
conn.transaction::<_, Error, _>(|conn| {
|
||||
// Delete tasks for these conversations first (due to foreign key constraint)
|
||||
diesel::delete(
|
||||
agent_tasks::table.filter(tasks_dsl::conversation_id.eq_any(&conversation_ids)),
|
||||
)
|
||||
.execute(conn)?;
|
||||
|
||||
// Delete the conversations themselves
|
||||
diesel::delete(
|
||||
agent_conversations::table().filter(conversation_id.eq_any(&conversation_ids)),
|
||||
)
|
||||
.execute(conn)?;
|
||||
|
||||
Ok(())
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,292 @@
|
||||
//! Manages how we write to and read from our SQLite database for our AI features.
|
||||
|
||||
use std::{collections::HashMap, sync::Arc};
|
||||
|
||||
use chrono::{Local, NaiveDateTime, TimeZone};
|
||||
use diesel::{prelude::*, result::Error, sqlite::SqliteConnection};
|
||||
|
||||
use itertools::Itertools;
|
||||
|
||||
use crate::ai::blocklist::{PersistedAIInput, SerializedBlockListItem};
|
||||
use crate::terminal::model::block::{SerializedAgentViewVisibility, SerializedBlock};
|
||||
use crate::{app_state::PaneUuid, persistence::schema::ai_queries};
|
||||
|
||||
use super::model::Block;
|
||||
use super::{model, schema};
|
||||
|
||||
const MAX_TERMINAL_BLOCKS_TO_PERSIST_PER_SESSION: i64 = 100;
|
||||
|
||||
type PersistedBlocks = HashMap<PaneUuid, Vec<SerializedBlockListItem>>;
|
||||
|
||||
/// An AI query read from the SQLite DB.
|
||||
#[derive(Identifiable, Insertable, Queryable, Selectable)]
|
||||
#[diesel(table_name = ai_queries)]
|
||||
#[diesel(primary_key(id))]
|
||||
pub(super) struct AIQuery {
|
||||
pub(super) id: i32,
|
||||
pub(super) exchange_id: String,
|
||||
pub(super) conversation_id: String,
|
||||
pub(super) start_ts: NaiveDateTime,
|
||||
pub(super) output_status: String,
|
||||
pub(super) input: String,
|
||||
pub(super) working_directory: Option<String>,
|
||||
pub(super) model_id: String,
|
||||
pub(super) coding_model_id: String,
|
||||
|
||||
// Planning model selection is deprecated and unused.
|
||||
#[allow(unused)]
|
||||
pub(super) planning_model_id: String,
|
||||
}
|
||||
|
||||
impl TryFrom<AIQuery> for PersistedAIInput {
|
||||
type Error = anyhow::Error;
|
||||
|
||||
fn try_from(value: AIQuery) -> Result<Self, Self::Error> {
|
||||
Ok(Self {
|
||||
start_ts: Local.from_utc_datetime(&value.start_ts),
|
||||
inputs: serde_json::from_str(&value.input)?,
|
||||
exchange_id: value.exchange_id.try_into()?,
|
||||
conversation_id: value.conversation_id.try_into()?,
|
||||
output_status: serde_json::from_str(&value.output_status)?,
|
||||
working_directory: value.working_directory,
|
||||
model_id: value.model_id.into(),
|
||||
coding_model_id: value.coding_model_id.into(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// A new AI query to be inserted into the SQLite DB.
|
||||
#[derive(Insertable, AsChangeset)]
|
||||
#[diesel(table_name = ai_queries)]
|
||||
#[diesel(treat_none_as_null = true)]
|
||||
pub(super) struct NewAIQuery {
|
||||
pub(super) exchange_id: String,
|
||||
pub(super) conversation_id: String,
|
||||
pub(super) start_ts: NaiveDateTime,
|
||||
pub(super) output_status: String,
|
||||
pub(super) input: String,
|
||||
pub(super) working_directory: Option<String>,
|
||||
pub(super) model_id: String,
|
||||
}
|
||||
|
||||
impl TryFrom<&PersistedAIInput> for NewAIQuery {
|
||||
type Error = anyhow::Error;
|
||||
|
||||
fn try_from(value: &PersistedAIInput) -> Result<Self, Self::Error> {
|
||||
Ok(Self {
|
||||
start_ts: value.start_ts.naive_utc(),
|
||||
input: serde_json::to_string(&value.inputs)?,
|
||||
working_directory: value.working_directory.clone(),
|
||||
exchange_id: value.exchange_id.to_string(),
|
||||
conversation_id: value.conversation_id.to_string(),
|
||||
output_status: serde_json::to_string(&value.output_status)?,
|
||||
model_id: value.model_id.clone().into(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn read_ai_queries(
|
||||
conn: &mut SqliteConnection,
|
||||
) -> Result<Vec<PersistedAIInput>, diesel::result::Error> {
|
||||
// Only load at most 100 AI queries; there's a very low chance that the user
|
||||
// will ever try rerunning AI queries older than this duration and loading
|
||||
// all AI queries in perpetuity has performance implications on app startup.
|
||||
// TOOD(alokedesai): Consider loading all AI queries by paginating the SQL query.
|
||||
const MAX_AI_QUERIES_TO_READ: i64 = 100;
|
||||
Ok(schema::ai_queries::table
|
||||
.select(AIQuery::as_select())
|
||||
.order_by(schema::ai_queries::columns::start_ts.desc())
|
||||
.limit(MAX_AI_QUERIES_TO_READ)
|
||||
.load::<AIQuery>(conn)?
|
||||
.into_iter()
|
||||
.filter_map(|ai_query| PersistedAIInput::try_from(ai_query).ok())
|
||||
.rev()
|
||||
.collect_vec())
|
||||
}
|
||||
|
||||
pub(super) fn upsert_ai_query(
|
||||
conn: &mut SqliteConnection,
|
||||
query: Arc<PersistedAIInput>,
|
||||
) -> anyhow::Result<()> {
|
||||
use schema::ai_queries::dsl::*;
|
||||
|
||||
let new_ai_query = NewAIQuery::try_from(query.as_ref())?;
|
||||
|
||||
Ok(conn.transaction::<_, Error, _>(|conn| {
|
||||
diesel::insert_into(ai_queries)
|
||||
.values(&new_ai_query)
|
||||
.on_conflict(exchange_id)
|
||||
.do_update()
|
||||
.set(&new_ai_query)
|
||||
.execute(conn)?;
|
||||
|
||||
Ok(())
|
||||
})?)
|
||||
}
|
||||
|
||||
/// Returns the most recent [`MAX_BLOCK_COUNT_PER_SESSION`] block list items for each session. The
|
||||
/// items are in chronological order.
|
||||
pub(super) fn get_all_restored_blocks(
|
||||
conn: &mut SqliteConnection,
|
||||
) -> Result<PersistedBlocks, diesel::result::Error> {
|
||||
let terminal_sessions = schema::terminal_panes::table
|
||||
.select(model::TerminalSession::as_select())
|
||||
.load::<model::TerminalSession>(conn)?;
|
||||
|
||||
let block_lists = Block::belonging_to(&terminal_sessions)
|
||||
.select(Block::as_select())
|
||||
.order_by(schema::blocks::columns::id.asc())
|
||||
.load::<Block>(conn)?
|
||||
.grouped_by(&terminal_sessions);
|
||||
|
||||
let mut all_block_items_by_pane = block_lists
|
||||
.into_iter()
|
||||
.zip(terminal_sessions)
|
||||
.map(|(blocks, terminal_pane)| {
|
||||
(
|
||||
PaneUuid(terminal_pane.uuid),
|
||||
blocks.into_iter().map(Into::into).collect(),
|
||||
)
|
||||
})
|
||||
.collect::<HashMap<_, Vec<SerializedBlockListItem>>>();
|
||||
|
||||
for (_, blocks) in all_block_items_by_pane.iter_mut() {
|
||||
blocks.sort_by_key(|item| item.start_ts());
|
||||
// Only keep most recent command blocks
|
||||
blocks.drain(
|
||||
0..blocks
|
||||
.len()
|
||||
.saturating_sub(MAX_TERMINAL_BLOCKS_TO_PERSIST_PER_SESSION as usize),
|
||||
);
|
||||
}
|
||||
|
||||
Ok(all_block_items_by_pane)
|
||||
}
|
||||
|
||||
pub(super) fn save_block(
|
||||
conn: &mut SqliteConnection,
|
||||
pane_id: Vec<u8>,
|
||||
block: &SerializedBlock,
|
||||
is_local_block: bool,
|
||||
) -> Result<(), Error> {
|
||||
use schema::blocks::dsl::*;
|
||||
conn.transaction::<_, Error, _>(|conn| {
|
||||
let saved_blocks_count: i64 = schema::blocks::dsl::blocks
|
||||
.filter(pane_leaf_uuid.eq(pane_id.clone()))
|
||||
.filter(id.is_not_null())
|
||||
.filter(is_background.ne(true))
|
||||
.count()
|
||||
.first(conn)?;
|
||||
|
||||
// add 1 because we are about to save a new block
|
||||
let diff = saved_blocks_count - MAX_TERMINAL_BLOCKS_TO_PERSIST_PER_SESSION + 1;
|
||||
if diff > 0 {
|
||||
// Find the oldest block to keep.
|
||||
let last_kept_id: Option<i32> = schema::blocks::dsl::blocks
|
||||
.filter(pane_leaf_uuid.eq(pane_id.clone()))
|
||||
.filter(id.is_not_null())
|
||||
.filter(is_background.ne(true))
|
||||
.select(id)
|
||||
.order(id.asc())
|
||||
.offset(diff)
|
||||
.limit(1)
|
||||
.first(conn)?;
|
||||
|
||||
if let Some(last_kept_id) = last_kept_id {
|
||||
diesel::delete(
|
||||
schema::blocks::dsl::blocks
|
||||
.filter(id.lt(last_kept_id))
|
||||
.filter(pane_leaf_uuid.eq(pane_id.clone())),
|
||||
)
|
||||
.execute(conn)?;
|
||||
}
|
||||
}
|
||||
|
||||
let block = create_block(pane_id, block, is_local_block);
|
||||
diesel::insert_into(schema::blocks::dsl::blocks)
|
||||
.values(block)
|
||||
.execute(conn)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
// TODO(vorporeal): can move this to a `to_persisted_block()` function on `SerializedBlock`
|
||||
// to get it out of the persistence layer.
|
||||
fn create_block<'a>(
|
||||
pane_leaf_uuid: Vec<u8>,
|
||||
block: &'a SerializedBlock,
|
||||
is_local: bool,
|
||||
) -> model::NewBlock<'a> {
|
||||
model::NewBlock {
|
||||
block_id: block.id.as_str(),
|
||||
pane_leaf_uuid,
|
||||
stylized_command: &block.stylized_command,
|
||||
stylized_output: &block.stylized_output,
|
||||
pwd: block.pwd.as_ref(),
|
||||
// This sqlite column still uses the legacy `git_branch` name, but it now stores the
|
||||
// block's git head for backwards compatibility with existing persisted data.
|
||||
git_branch: block.git_head.as_ref(),
|
||||
git_branch_name: block.git_branch_name.as_ref(),
|
||||
virtual_env: block.virtual_env.as_ref(),
|
||||
conda_env: block.conda_env.as_ref(),
|
||||
exit_code: block.exit_code.value(),
|
||||
did_execute: block.did_execute,
|
||||
completed_ts: block.completed_ts.map(|ts| ts.naive_utc()),
|
||||
start_ts: block.start_ts.map(|ts| ts.naive_utc()),
|
||||
ps1: block.ps1.as_ref(),
|
||||
rprompt: block.rprompt.as_ref(),
|
||||
honor_ps1: block.honor_ps1,
|
||||
is_background: block.is_background,
|
||||
shell: block.shell_host.as_ref().map(|host| host.shell_type.name()),
|
||||
user: block.shell_host.as_ref().map(|host| host.user.as_str()),
|
||||
host: block.shell_host.as_ref().map(|host| host.hostname.as_str()),
|
||||
prompt_snapshot: block.prompt_snapshot.as_ref(),
|
||||
ai_metadata: block.ai_metadata.as_ref(),
|
||||
is_local: Some(is_local),
|
||||
agent_view_visibility: block
|
||||
.agent_view_visibility
|
||||
.as_ref()
|
||||
.and_then(|v| serde_json::to_string(v).ok()),
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn delete_blocks(conn: &mut SqliteConnection, pane_id: Vec<u8>) -> Result<(), Error> {
|
||||
use schema::blocks::dsl::*;
|
||||
conn.transaction::<_, Error, _>(|conn| {
|
||||
diesel::delete(schema::blocks::dsl::blocks.filter(pane_leaf_uuid.eq(pane_id.clone())))
|
||||
.execute(conn)?;
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
pub(super) fn update_block_agent_view_visibility(
|
||||
conn: &mut SqliteConnection,
|
||||
target_block_id: &str,
|
||||
visibility: &SerializedAgentViewVisibility,
|
||||
) -> anyhow::Result<()> {
|
||||
use schema::blocks::dsl::*;
|
||||
let visibility_json = serde_json::to_string(visibility)?;
|
||||
diesel::update(blocks.filter(block_id.eq(target_block_id)))
|
||||
.set(agent_view_visibility.eq(visibility_json))
|
||||
.execute(conn)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(super) fn delete_ai_conversation(
|
||||
conn: &mut SqliteConnection,
|
||||
conversation_id_str: &str,
|
||||
) -> anyhow::Result<()> {
|
||||
use schema::ai_queries::dsl as queries_dsl;
|
||||
|
||||
conn.transaction::<_, Error, _>(|conn| {
|
||||
// Delete the AI query
|
||||
diesel::delete(
|
||||
queries_dsl::ai_queries.filter(queries_dsl::conversation_id.eq(conversation_id_str)),
|
||||
)
|
||||
.execute(conn)?;
|
||||
|
||||
Ok(())
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
use lazy_static::lazy_static;
|
||||
use session_sharing_protocol::common::{InputReplicaId, ProfileData};
|
||||
|
||||
use crate::{
|
||||
auth::UserUid,
|
||||
cloud_object::{CloudObjectGuest, ServerObjectContainer},
|
||||
drive::sharing::{LinkSharingSubjectType, SharingAccessLevel, Subject, TeamKind, UserKind},
|
||||
server::ids::ServerId,
|
||||
};
|
||||
|
||||
#[test]
|
||||
fn test_roundtrip_guests() {
|
||||
let guests = vec![
|
||||
CloudObjectGuest {
|
||||
subject: Subject::User(UserKind::Account(UserUid::new("firebase_uid"))),
|
||||
access_level: SharingAccessLevel::Edit,
|
||||
source: None,
|
||||
},
|
||||
CloudObjectGuest {
|
||||
subject: Subject::PendingUser {
|
||||
email: Some("pending@warp.dev".to_string()),
|
||||
},
|
||||
access_level: SharingAccessLevel::View,
|
||||
source: Some(ServerObjectContainer::Folder {
|
||||
folder_uid: 123.into(),
|
||||
}),
|
||||
},
|
||||
CloudObjectGuest {
|
||||
subject: Subject::Team(TeamKind::Team {
|
||||
team_uid: ServerId::from(99),
|
||||
}),
|
||||
access_level: SharingAccessLevel::Edit,
|
||||
source: None,
|
||||
},
|
||||
];
|
||||
|
||||
let encoded = super::encode_guests(&guests).expect("encode should succeed");
|
||||
let decoded = super::decode_guests(&encoded).expect("decode should succeed");
|
||||
|
||||
assert_eq!(guests, decoded);
|
||||
}
|
||||
|
||||
lazy_static! {
|
||||
/// By construction, [`CloudObjectGuest`] only accepts `'static`-lifetime [`Subject`]s.
|
||||
///
|
||||
/// In most cases, this would prevent persisting a shared session subject, but we work around
|
||||
/// it here for completeness;
|
||||
static ref PROFILE_DATA: ProfileData = ProfileData {
|
||||
firebase_uid: "2YP93GScglXJMdEr2Id12dI7HCG3".to_string(),
|
||||
display_name: "Some User".to_string(),
|
||||
photo_url: Some("http://example.com/some-image".to_string()),
|
||||
email: Some("user@warp.dev".to_string()),
|
||||
input_replica_id: InputReplicaId::from("some-id".to_string()),
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_fail_unsupported_subjects() {
|
||||
let result = super::encode_guests(&[CloudObjectGuest {
|
||||
subject: Subject::AnyoneWithLink(LinkSharingSubjectType::Anyone),
|
||||
access_level: SharingAccessLevel::View,
|
||||
source: None,
|
||||
}]);
|
||||
assert!(result.is_err());
|
||||
|
||||
let result = super::encode_guests(&[CloudObjectGuest {
|
||||
subject: Subject::User(UserKind::SharedSessionParticipant(PROFILE_DATA.clone())),
|
||||
access_level: SharingAccessLevel::View,
|
||||
source: None,
|
||||
}]);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
//! Supporting types for persisting cloud objects to SQLite.
|
||||
|
||||
pub use warp_server_client::persistence::{decode_guests, decode_link_sharing};
|
||||
|
||||
#[cfg(test)]
|
||||
pub use warp_server_client::persistence::encode_guests;
|
||||
|
||||
#[cfg(test)]
|
||||
#[path = "cloud_object_tests.rs"]
|
||||
mod tests;
|
||||
@@ -0,0 +1,69 @@
|
||||
use anyhow::Result;
|
||||
use diesel::{sqlite::SqliteConnection, ExpressionMethods, QueryDsl, RunQueryDsl};
|
||||
|
||||
use crate::terminal::event::UserBlockCompleted;
|
||||
|
||||
/// Returns the command that was run right after `command`
|
||||
/// in the same session, if any.
|
||||
pub fn get_next_command(
|
||||
conn: &mut SqliteConnection,
|
||||
command: &super::model::Command,
|
||||
) -> Result<super::model::Command> {
|
||||
let next_command = super::schema::commands::dsl::commands
|
||||
.filter(super::schema::commands::columns::id.gt(command.id))
|
||||
.filter(super::schema::commands::columns::session_id.eq(&command.session_id))
|
||||
// Skip any empty blocks
|
||||
.filter(super::schema::commands::columns::command.ne(""))
|
||||
.order(super::schema::commands::columns::id.asc())
|
||||
.limit(1)
|
||||
.first::<super::model::Command>(conn)?;
|
||||
Ok(next_command)
|
||||
}
|
||||
|
||||
/// Returns the commands that were run right before `command`
|
||||
/// in the same session, if any. They are ordered from oldest to newest.
|
||||
pub fn get_previous_commands(
|
||||
conn: &mut SqliteConnection,
|
||||
command: &super::model::Command,
|
||||
num_commands: usize,
|
||||
) -> Result<Vec<super::model::Command>> {
|
||||
let previous_commands = super::schema::commands::dsl::commands
|
||||
.filter(super::schema::commands::columns::id.lt(command.id))
|
||||
.filter(super::schema::commands::columns::session_id.eq(&command.session_id))
|
||||
// Skip any empty blocks
|
||||
.filter(super::schema::commands::columns::command.ne(""))
|
||||
.order(super::schema::commands::columns::id.desc())
|
||||
.limit(num_commands as i64)
|
||||
.load::<super::model::Command>(conn)?;
|
||||
Ok(previous_commands.into_iter().rev().collect())
|
||||
}
|
||||
|
||||
/// Gets the last num_commands times the same command was run in a similar context
|
||||
/// (same pwd, exit code, shell, hostname), from newest to oldest.
|
||||
pub fn get_same_commands_from_history(
|
||||
conn: &mut SqliteConnection,
|
||||
completed_block: &UserBlockCompleted,
|
||||
num_commands: usize,
|
||||
) -> Result<Vec<super::model::Command>> {
|
||||
let shell_host = completed_block.serialized_block.shell_host.as_ref();
|
||||
let commands = super::schema::commands::dsl::commands
|
||||
.filter(super::schema::commands::columns::command.eq(&completed_block.command))
|
||||
.filter(super::schema::commands::columns::pwd.eq(&completed_block.serialized_block.pwd))
|
||||
.filter(
|
||||
super::schema::commands::columns::exit_code
|
||||
.eq(completed_block.serialized_block.exit_code.value()),
|
||||
)
|
||||
.filter(
|
||||
super::schema::commands::columns::shell
|
||||
.eq(shell_host.map(|host| host.shell_type.name())),
|
||||
)
|
||||
.filter(
|
||||
super::schema::commands::columns::hostname.eq(shell_host.map(|host| &host.hostname)),
|
||||
)
|
||||
// Get newest to oldest commands.
|
||||
.order(super::schema::commands::columns::id.desc())
|
||||
.limit(num_commands as i64)
|
||||
.load::<super::model::Command>(conn)?;
|
||||
|
||||
Ok(commands)
|
||||
}
|
||||
@@ -0,0 +1,388 @@
|
||||
#![cfg_attr(not(feature = "local_fs"), allow(dead_code))]
|
||||
|
||||
cfg_if::cfg_if! {
|
||||
if #[cfg(feature = "local_fs")] {
|
||||
pub mod agent;
|
||||
mod block_list;
|
||||
mod cloud_objects;
|
||||
mod sqlite;
|
||||
pub mod commands;
|
||||
}
|
||||
}
|
||||
|
||||
pub use persistence::model;
|
||||
#[cfg_attr(not(feature = "local_fs"), expect(unused_imports))]
|
||||
pub use persistence::schema;
|
||||
|
||||
#[cfg(feature = "integration_tests")]
|
||||
pub mod testing;
|
||||
|
||||
use instant::Instant;
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::mpsc::SyncSender;
|
||||
use std::sync::Arc;
|
||||
use std::thread::JoinHandle;
|
||||
|
||||
use crate::ai::persisted_workspace::EnablementState;
|
||||
use ai::project_context::model::ProjectRulePath;
|
||||
use chrono::{DateTime, Local, Utc};
|
||||
use lsp::supported_servers::LSPServerType;
|
||||
use uuid::Uuid;
|
||||
use warp_core::command::ExitCode;
|
||||
use warp_graphql::scalars::time::ServerTimestamp;
|
||||
use warp_multi_agent_api as api;
|
||||
use warpui::{AppContext, Entity, SingletonEntity};
|
||||
|
||||
use crate::ai::blocklist::PersistedAIInput;
|
||||
use crate::ai::mcp::TemplatableMCPServerInstallation;
|
||||
use crate::app_state::AppState;
|
||||
use crate::auth::auth_manager::PersistedCurrentUserInformation;
|
||||
use crate::cloud_object::model::actions::ObjectAction;
|
||||
use crate::cloud_object::model::generic_string_model::CloudStringObject;
|
||||
|
||||
use crate::cloud_object::{
|
||||
CloudObject, CloudObjectMetadata, ObjectIdType, RevisionAndLastEditor, ServerCreationInfo,
|
||||
};
|
||||
use crate::drive::folders::CloudFolder;
|
||||
use crate::notebooks::CloudNotebook;
|
||||
use crate::server::experiments::ServerExperiment;
|
||||
use crate::server::ids::SyncId;
|
||||
use crate::suggestions::ignored_suggestions_model::SuggestionType;
|
||||
use crate::terminal::history::PersistedCommand;
|
||||
use crate::terminal::model::block::{SerializedAgentViewVisibility, SerializedBlock};
|
||||
use crate::terminal::model::session::SessionId;
|
||||
use crate::workflows::CloudWorkflow;
|
||||
use crate::workspaces::user_profiles::UserProfileWithUID;
|
||||
use crate::workspaces::workspace::{Workspace as WorkspaceMetadata, WorkspaceUid};
|
||||
use ai::workspace::WorkspaceMetadata as CodeWorkspaceMetadata;
|
||||
|
||||
use self::model::{AgentConversation, AgentConversationData, Project};
|
||||
|
||||
#[cfg(any(feature = "local_fs", feature = "integration_tests"))]
|
||||
pub use sqlite::database_file_path;
|
||||
#[cfg(any(feature = "local_fs", feature = "integration_tests"))]
|
||||
pub use sqlite::establish_ro_connection;
|
||||
|
||||
/// Initializes the persistence "subsystem".
|
||||
///
|
||||
/// Returns the previously-persisted data, if any, and handles for
|
||||
/// writing updated data to persist, if the persistence subsystem is
|
||||
/// available.
|
||||
#[cfg_attr(not(feature = "local_fs"), allow(unused_variables))]
|
||||
pub fn initialize(ctx: &mut AppContext) -> (Option<PersistedData>, Option<WriterHandles>) {
|
||||
cfg_if::cfg_if! {
|
||||
if #[cfg(feature = "local_fs")] {
|
||||
sqlite::initialize(ctx)
|
||||
} else {
|
||||
(None, None)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Remove sqlite database as part of Logout v0.
|
||||
// TODO: Implement per user scoping of sqlite.
|
||||
#[cfg_attr(not(feature = "local_fs"), allow(unused_variables))]
|
||||
pub fn remove(sender: &Option<SyncSender<ModelEvent>>) {
|
||||
cfg_if::cfg_if! {
|
||||
if #[cfg(feature = "local_fs")] {
|
||||
if let Some(sender) = sender.clone() {
|
||||
sqlite::remove(sender);
|
||||
}
|
||||
} else {
|
||||
log::info!("Local filesystem persistence is not enabled.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Reconstruct sqlite database as part of Logout v0.
|
||||
#[cfg_attr(not(feature = "local_fs"), allow(unused_variables))]
|
||||
pub fn reconstruct(sender: &Option<SyncSender<ModelEvent>>) {
|
||||
cfg_if::cfg_if! {
|
||||
if #[cfg(feature = "local_fs")] {
|
||||
if let Some(sender) = sender.clone() {
|
||||
sqlite::reconstruct(sender);
|
||||
}
|
||||
} else {
|
||||
log::info!("Local filesystem persistence is not enabled.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Holds interfaces to the writer thread.
|
||||
pub struct WriterHandles {
|
||||
pub handle: JoinHandle<()>,
|
||||
pub sender: SyncSender<ModelEvent>,
|
||||
}
|
||||
|
||||
/// Model for interacting with the writer thread.
|
||||
pub struct PersistenceWriter {
|
||||
thread_handle: Option<JoinHandle<()>>,
|
||||
model_event_sender: Option<SyncSender<ModelEvent>>,
|
||||
}
|
||||
|
||||
impl PersistenceWriter {
|
||||
pub fn new(handle: Option<WriterHandles>) -> Self {
|
||||
let (thread_handle, model_event_sender) = match handle {
|
||||
Some(handle) => (Some(handle.handle), Some(handle.sender)),
|
||||
None => (None, None),
|
||||
};
|
||||
Self {
|
||||
thread_handle,
|
||||
model_event_sender,
|
||||
}
|
||||
}
|
||||
|
||||
/// Sending half for sending model updates to the persistence writer thread.
|
||||
pub fn sender(&self) -> Option<SyncSender<ModelEvent>> {
|
||||
self.model_event_sender.clone()
|
||||
}
|
||||
|
||||
/// Synchronously terminate the SQLite writer thread.
|
||||
pub fn terminate(&mut self) {
|
||||
if let Some(handle) = self.thread_handle.take() {
|
||||
let start = Instant::now();
|
||||
let Some(sender) = self.sender() else {
|
||||
log::error!("Model event sender should exist if thread handle is set");
|
||||
return;
|
||||
};
|
||||
if let Err(err) = sender.send(ModelEvent::Terminate) {
|
||||
log::error!("Could not terminate SQLite writer thread: {err}");
|
||||
}
|
||||
if handle.join().is_err() {
|
||||
// If crash reporting is enabled, Sentry will have already handled the panic.
|
||||
log::error!("SQLite writer thread panicked");
|
||||
}
|
||||
log::info!("Shut down SQLite writer in {:?}", start.elapsed());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for PersistenceWriter {
|
||||
fn drop(&mut self) {
|
||||
self.terminate();
|
||||
}
|
||||
}
|
||||
|
||||
impl Entity for PersistenceWriter {
|
||||
type Event = ();
|
||||
}
|
||||
|
||||
impl SingletonEntity for PersistenceWriter {}
|
||||
|
||||
/// TODO: all of this data should eventually be indexed by user_id so that
|
||||
/// the logged in user sees the data for their user (and if another user logs in,
|
||||
/// they see their respective data). To do this, we can simply return a mapping
|
||||
/// of user ID->SqliteData and get the respective AppState after the user logs in.
|
||||
///
|
||||
/// For now, to address the global scoping here, we clear all persisted data on logout.
|
||||
pub struct PersistedData {
|
||||
/// Session restoration data
|
||||
pub app_state: AppState,
|
||||
|
||||
/// Shareable objects.
|
||||
pub cloud_objects: Vec<Box<dyn CloudObject>>,
|
||||
pub workspaces: Vec<WorkspaceMetadata>,
|
||||
pub current_workspace_uid: Option<WorkspaceUid>,
|
||||
pub command_history: Vec<PersistedCommand>,
|
||||
pub user_profiles: Vec<UserProfileWithUID>,
|
||||
pub time_of_next_force_object_refresh: Option<DateTime<Utc>>,
|
||||
pub object_actions: Vec<ObjectAction>,
|
||||
pub experiments: Vec<ServerExperiment>,
|
||||
pub ai_queries: Vec<PersistedAIInput>,
|
||||
pub codebase_indices: Vec<CodeWorkspaceMetadata>,
|
||||
pub workspace_language_servers: HashMap<PathBuf, HashMap<LSPServerType, EnablementState>>,
|
||||
pub multi_agent_conversations: Vec<AgentConversation>,
|
||||
pub projects: Vec<Project>,
|
||||
pub project_rules: Vec<ProjectRulePath>,
|
||||
pub ignored_suggestions: Vec<(String, SuggestionType)>,
|
||||
pub mcp_server_installations: HashMap<Uuid, TemplatableMCPServerInstallation>,
|
||||
pub mcp_servers_to_restore: Vec<Uuid>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct BlockCompleted {
|
||||
pub pane_id: Vec<u8>,
|
||||
/// Indicates if the block was created locally (e.g. not in a remote session)
|
||||
pub is_local: bool,
|
||||
pub block: Arc<SerializedBlock>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct StartedCommandMetadata {
|
||||
pub command: String,
|
||||
pub start_ts: Option<DateTime<Local>>,
|
||||
pub pwd: Option<String>,
|
||||
pub shell: Option<String>,
|
||||
pub username: Option<String>,
|
||||
pub hostname: Option<String>,
|
||||
pub session_id: Option<SessionId>,
|
||||
pub git_branch: Option<String>,
|
||||
pub cloud_workflow_id: Option<SyncId>,
|
||||
pub workflow_command: Option<String>,
|
||||
pub is_agent_executed: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct FinishedCommandMetadata {
|
||||
pub exit_code: ExitCode,
|
||||
pub start_ts: DateTime<Local>,
|
||||
pub completed_ts: DateTime<Local>,
|
||||
pub session_id: SessionId,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum ModelEvent {
|
||||
SaveBlock(BlockCompleted),
|
||||
DeleteBlocks(Vec<u8>),
|
||||
Snapshot(AppState),
|
||||
UpsertWorkflows(Vec<CloudWorkflow>),
|
||||
UpsertNotebooks(Vec<CloudNotebook>),
|
||||
UpsertFolders(Vec<CloudFolder>),
|
||||
MarkObjectAsSynced {
|
||||
hashed_sqlite_id: String,
|
||||
revision_and_editor: RevisionAndLastEditor,
|
||||
metadata_ts: Option<ServerTimestamp>,
|
||||
},
|
||||
IncrementRetryCount(String),
|
||||
UpsertGenericStringObject {
|
||||
object: Box<dyn CloudStringObject>,
|
||||
},
|
||||
UpsertGenericStringObjects(Vec<Box<dyn CloudStringObject>>),
|
||||
UpsertNotebook {
|
||||
notebook: CloudNotebook,
|
||||
},
|
||||
UpsertWorkflow {
|
||||
workflow: CloudWorkflow,
|
||||
},
|
||||
UpsertFolder {
|
||||
folder: CloudFolder,
|
||||
},
|
||||
UpdateObjectAfterServerCreation {
|
||||
client_id: String,
|
||||
server_creation_info: ServerCreationInfo,
|
||||
},
|
||||
DeleteObjects {
|
||||
ids: Vec<(SyncId, ObjectIdType)>,
|
||||
},
|
||||
UpsertWorkspace {
|
||||
workspace: Box<WorkspaceMetadata>,
|
||||
},
|
||||
UpsertWorkspaces {
|
||||
workspaces: Vec<WorkspaceMetadata>,
|
||||
},
|
||||
SetCurrentWorkspace {
|
||||
workspace_uid: WorkspaceUid,
|
||||
},
|
||||
UpdateObjectMetadata {
|
||||
id: String,
|
||||
metadata: CloudObjectMetadata,
|
||||
},
|
||||
InsertCommand {
|
||||
metadata: StartedCommandMetadata,
|
||||
},
|
||||
UpdateFinishedCommand {
|
||||
metadata: FinishedCommandMetadata,
|
||||
},
|
||||
UpsertUserProfiles {
|
||||
profiles: Vec<UserProfileWithUID>,
|
||||
},
|
||||
ClearUserProfiles,
|
||||
RecordTimeOfNextRefresh {
|
||||
timestamp: DateTime<Utc>,
|
||||
},
|
||||
SaveExperiments {
|
||||
experiments: Vec<ServerExperiment>,
|
||||
},
|
||||
// `PauseAndRemoveDatabase` and `ReconstructAndResume` are used to pause and resume the writer thread.
|
||||
// These are employed as part of Logout v0 to ensure that the writer thread
|
||||
// does not continue writing to the DB after the user has logged out and the DB is deleted.
|
||||
PauseAndRemoveDatabase,
|
||||
#[cfg(feature = "local_fs")]
|
||||
ReconstructAndResume,
|
||||
InsertObjectAction {
|
||||
object_action: ObjectAction,
|
||||
},
|
||||
SyncObjectActions {
|
||||
actions_to_sync: Vec<ObjectAction>,
|
||||
},
|
||||
/// Close the SQLite writer thread when the app is about to quit.
|
||||
Terminate,
|
||||
UpsertAIQuery {
|
||||
query: Arc<PersistedAIInput>,
|
||||
},
|
||||
/// Delete the AI query and related data for a given conversation.
|
||||
DeleteAIConversation {
|
||||
conversation_id: String,
|
||||
},
|
||||
UpdateMultiAgentConversation {
|
||||
conversation_id: String,
|
||||
updated_tasks: Vec<api::Task>,
|
||||
conversation_data: AgentConversationData,
|
||||
},
|
||||
DeleteMultiAgentConversations {
|
||||
conversation_ids: Vec<String>,
|
||||
},
|
||||
|
||||
UpsertCurrentUserInformation {
|
||||
user_information: PersistedCurrentUserInformation,
|
||||
},
|
||||
UpsertCodebaseIndexMetadata {
|
||||
index_metadata: Box<CodeWorkspaceMetadata>,
|
||||
},
|
||||
DeleteCodebaseIndexMetadata {
|
||||
repo_path: PathBuf,
|
||||
},
|
||||
UpsertProject {
|
||||
project: Project,
|
||||
},
|
||||
DeleteProject {
|
||||
path: String,
|
||||
},
|
||||
UpsertMCPServerEnvironmentVariables {
|
||||
mcp_server_uuid: Vec<u8>,
|
||||
environment_variables: String,
|
||||
},
|
||||
UpsertProjectRules {
|
||||
project_rule_paths: Vec<ProjectRulePath>,
|
||||
},
|
||||
DeleteProjectRules {
|
||||
path: Vec<PathBuf>,
|
||||
},
|
||||
AddIgnoredSuggestion {
|
||||
suggestion: String,
|
||||
suggestion_type: SuggestionType,
|
||||
},
|
||||
RemoveIgnoredSuggestion {
|
||||
suggestion: String,
|
||||
suggestion_type: SuggestionType,
|
||||
},
|
||||
UpsertMCPServerInstallation {
|
||||
mcp_server_installation: TemplatableMCPServerInstallation,
|
||||
},
|
||||
DeleteMCPServerInstallations {
|
||||
installation_uuids: Vec<Uuid>,
|
||||
},
|
||||
DeleteMCPServerInstallationsByTemplateUuid {
|
||||
template_uuid: Uuid,
|
||||
},
|
||||
UpdateMCPInstallationRunning {
|
||||
installation_uuid: Uuid,
|
||||
running: bool,
|
||||
},
|
||||
UpsertWorkspaceLanguageServer {
|
||||
workspace_path: PathBuf,
|
||||
lsp_type: LSPServerType,
|
||||
enabled: EnablementState,
|
||||
},
|
||||
UpdateBlockAgentViewVisibility {
|
||||
block_id: String,
|
||||
agent_view_visibility: SerializedAgentViewVisibility,
|
||||
},
|
||||
SaveAIDocumentContent {
|
||||
document_id: String,
|
||||
content: String,
|
||||
version: i32,
|
||||
title: String,
|
||||
},
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,417 @@
|
||||
use std::{path::PathBuf, sync::Arc};
|
||||
|
||||
use warp_core::features::FeatureFlag;
|
||||
use warp_graphql::scalars::time::ServerTimestamp;
|
||||
|
||||
use crate::{
|
||||
app_state::{
|
||||
AppState, CodePaneSnapShot, CodePaneTabSnapshot, LeafContents, LeafSnapshot,
|
||||
PaneNodeSnapshot, TabSnapshot, TerminalPaneSnapshot, WindowSnapshot,
|
||||
},
|
||||
cloud_object::{CloudObjectPermissions, Owner},
|
||||
code::editor_management::CodeSource,
|
||||
notebooks::{CloudNotebook, CloudNotebookModel},
|
||||
persistence::{model::ObjectPermissions, BlockCompleted, ModelEvent},
|
||||
server::ids::ClientId,
|
||||
tab::SelectedTabColor,
|
||||
terminal::model::block::SerializedBlock,
|
||||
terminal::ShellLaunchData,
|
||||
};
|
||||
|
||||
use super::{
|
||||
decode_path, deduplicate_events, encode_path, read_sqlite_data, save_app_state, setup_database,
|
||||
};
|
||||
|
||||
#[test]
|
||||
fn test_deduplicate_snapshots() {
|
||||
let local_notebook = CloudNotebook::new_local(
|
||||
CloudNotebookModel {
|
||||
title: "Hello".to_string(),
|
||||
data: "World".to_string(),
|
||||
ai_document_id: None,
|
||||
conversation_id: None,
|
||||
},
|
||||
Owner::mock_current_user(),
|
||||
None,
|
||||
ClientId::new(),
|
||||
);
|
||||
let completed_block_1 = BlockCompleted {
|
||||
pane_id: vec![1, 2, 3],
|
||||
block: Arc::new(SerializedBlock::default()),
|
||||
is_local: true,
|
||||
};
|
||||
let completed_block_2 = BlockCompleted {
|
||||
pane_id: vec![4, 5, 6],
|
||||
block: Arc::new(SerializedBlock::default()),
|
||||
is_local: true,
|
||||
};
|
||||
let snapshot_1 = AppState {
|
||||
active_window_index: Some(1),
|
||||
block_lists: Default::default(),
|
||||
windows: Default::default(),
|
||||
running_mcp_servers: Default::default(),
|
||||
};
|
||||
let snapshot_2 = AppState {
|
||||
active_window_index: Some(2),
|
||||
block_lists: Default::default(),
|
||||
windows: Default::default(),
|
||||
running_mcp_servers: Default::default(),
|
||||
};
|
||||
let snapshot_3 = AppState {
|
||||
active_window_index: Some(3),
|
||||
block_lists: Default::default(),
|
||||
windows: Default::default(),
|
||||
running_mcp_servers: Default::default(),
|
||||
};
|
||||
|
||||
let original_events = vec![
|
||||
ModelEvent::UpsertNotebook {
|
||||
notebook: local_notebook.clone(),
|
||||
},
|
||||
ModelEvent::Snapshot(snapshot_1.clone()),
|
||||
ModelEvent::SaveBlock(completed_block_1.clone()),
|
||||
ModelEvent::Snapshot(snapshot_2.clone()),
|
||||
ModelEvent::SaveBlock(completed_block_2.clone()),
|
||||
ModelEvent::Snapshot(snapshot_3.clone()),
|
||||
ModelEvent::UpsertNotebook {
|
||||
notebook: local_notebook.clone(),
|
||||
},
|
||||
];
|
||||
|
||||
let filtered_events = deduplicate_events(original_events);
|
||||
assert_eq!(filtered_events.len(), 5);
|
||||
|
||||
assert!(matches!(
|
||||
&filtered_events[0],
|
||||
&ModelEvent::UpsertNotebook { .. }
|
||||
));
|
||||
// The first snapshot should have been filtered out.
|
||||
assert!(matches!(&filtered_events[1], &ModelEvent::SaveBlock(_)));
|
||||
// The second snapshot should have been filtered out.
|
||||
assert!(matches!(&filtered_events[2], &ModelEvent::SaveBlock(_)));
|
||||
// The third snapshot should be preserved.
|
||||
match &filtered_events[3] {
|
||||
ModelEvent::Snapshot(snapshot) => assert_eq!(snapshot, &snapshot_3),
|
||||
other => panic!("Expected ModelEvent::Snapshot, got {other:?}"),
|
||||
}
|
||||
assert!(matches!(
|
||||
&filtered_events[4],
|
||||
&ModelEvent::UpsertNotebook { .. }
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deduplicate_no_snapshots() {
|
||||
let original_events = vec![ModelEvent::SaveBlock(BlockCompleted {
|
||||
pane_id: vec![1, 2, 3],
|
||||
block: Default::default(),
|
||||
is_local: true,
|
||||
})];
|
||||
let filtered_events = deduplicate_events(original_events);
|
||||
assert_eq!(filtered_events.len(), 1);
|
||||
assert!(matches!(&filtered_events[0], &ModelEvent::SaveBlock(_)));
|
||||
}
|
||||
|
||||
fn test_terminal_window_snapshot(vertical_tabs_panel_open: bool) -> WindowSnapshot {
|
||||
WindowSnapshot {
|
||||
tabs: vec![TabSnapshot {
|
||||
custom_title: None,
|
||||
root: PaneNodeSnapshot::Leaf(LeafSnapshot {
|
||||
is_focused: true,
|
||||
custom_vertical_tabs_title: None,
|
||||
contents: LeafContents::Terminal(TerminalPaneSnapshot {
|
||||
uuid: vec![u8::from(vertical_tabs_panel_open) + 1],
|
||||
cwd: Some("/tmp".to_string()),
|
||||
shell_launch_data: Some(ShellLaunchData::Executable {
|
||||
executable_path: PathBuf::from("/bin/zsh"),
|
||||
shell_type: crate::terminal::shell::ShellType::Zsh,
|
||||
}),
|
||||
is_active: true,
|
||||
is_read_only: false,
|
||||
input_config: None,
|
||||
llm_model_override: None,
|
||||
active_profile_id: None,
|
||||
conversation_ids_to_restore: vec![],
|
||||
active_conversation_id: None,
|
||||
}),
|
||||
}),
|
||||
default_directory_color: None,
|
||||
selected_color: SelectedTabColor::default(),
|
||||
left_panel: None,
|
||||
right_panel: None,
|
||||
}],
|
||||
active_tab_index: 0,
|
||||
bounds: None,
|
||||
fullscreen_state: Default::default(),
|
||||
quake_mode: false,
|
||||
universal_search_width: None,
|
||||
warp_ai_width: None,
|
||||
voltron_width: None,
|
||||
warp_drive_index_width: None,
|
||||
left_panel_open: false,
|
||||
vertical_tabs_panel_open,
|
||||
left_panel_width: None,
|
||||
right_panel_width: None,
|
||||
agent_management_filters: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_sqlite_round_trips_vertical_tabs_panel_open() {
|
||||
let tempdir = tempfile::tempdir().expect("tempdir should be created");
|
||||
let database_path = tempdir.path().join("warp.sqlite");
|
||||
let mut conn = setup_database(&database_path).expect("database should initialize");
|
||||
|
||||
let app_state = AppState {
|
||||
windows: vec![
|
||||
test_terminal_window_snapshot(false),
|
||||
test_terminal_window_snapshot(true),
|
||||
],
|
||||
active_window_index: Some(1),
|
||||
block_lists: Default::default(),
|
||||
running_mcp_servers: Default::default(),
|
||||
};
|
||||
|
||||
save_app_state(&mut conn, &app_state).expect("app state should save");
|
||||
|
||||
let restored = read_sqlite_data(&mut conn, None)
|
||||
.expect("app state should load")
|
||||
.app_state;
|
||||
|
||||
assert_eq!(restored.active_window_index, Some(1));
|
||||
assert_eq!(
|
||||
restored
|
||||
.windows
|
||||
.iter()
|
||||
.map(|window| window.vertical_tabs_panel_open)
|
||||
.collect::<Vec<_>>(),
|
||||
vec![false, true]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_sqlite_round_trips_custom_vertical_tabs_title() {
|
||||
let tempdir = tempfile::tempdir().expect("tempdir should be created");
|
||||
let database_path = tempdir.path().join("warp.sqlite");
|
||||
let mut conn = setup_database(&database_path).expect("database should initialize");
|
||||
|
||||
let app_state = AppState {
|
||||
windows: vec![WindowSnapshot {
|
||||
tabs: vec![TabSnapshot {
|
||||
custom_title: None,
|
||||
root: PaneNodeSnapshot::Leaf(LeafSnapshot {
|
||||
is_focused: true,
|
||||
custom_vertical_tabs_title: Some("Production API".to_string()),
|
||||
contents: LeafContents::Terminal(TerminalPaneSnapshot {
|
||||
uuid: vec![42],
|
||||
cwd: Some("/tmp".to_string()),
|
||||
shell_launch_data: Some(ShellLaunchData::Executable {
|
||||
executable_path: PathBuf::from("/bin/zsh"),
|
||||
shell_type: crate::terminal::shell::ShellType::Zsh,
|
||||
}),
|
||||
is_active: true,
|
||||
is_read_only: false,
|
||||
input_config: None,
|
||||
llm_model_override: None,
|
||||
active_profile_id: None,
|
||||
conversation_ids_to_restore: vec![],
|
||||
active_conversation_id: None,
|
||||
}),
|
||||
}),
|
||||
default_directory_color: None,
|
||||
selected_color: SelectedTabColor::default(),
|
||||
left_panel: None,
|
||||
right_panel: None,
|
||||
}],
|
||||
active_tab_index: 0,
|
||||
bounds: None,
|
||||
fullscreen_state: Default::default(),
|
||||
quake_mode: false,
|
||||
universal_search_width: None,
|
||||
warp_ai_width: None,
|
||||
voltron_width: None,
|
||||
warp_drive_index_width: None,
|
||||
left_panel_open: false,
|
||||
vertical_tabs_panel_open: false,
|
||||
left_panel_width: None,
|
||||
right_panel_width: None,
|
||||
agent_management_filters: None,
|
||||
}],
|
||||
active_window_index: Some(0),
|
||||
block_lists: Default::default(),
|
||||
running_mcp_servers: Default::default(),
|
||||
};
|
||||
|
||||
save_app_state(&mut conn, &app_state).expect("app state should save");
|
||||
|
||||
let restored = read_sqlite_data(&mut conn, None)
|
||||
.expect("app state should load")
|
||||
.app_state;
|
||||
|
||||
let PaneNodeSnapshot::Leaf(LeafSnapshot {
|
||||
custom_vertical_tabs_title,
|
||||
..
|
||||
}) = &restored.windows[0].tabs[0].root
|
||||
else {
|
||||
panic!("Expected terminal pane leaf");
|
||||
};
|
||||
assert_eq!(
|
||||
custom_vertical_tabs_title.as_deref(),
|
||||
Some("Production API")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_sqlite_round_trips_code_pane_with_multiple_tabs() {
|
||||
let tempdir = tempfile::tempdir().expect("tempdir should be created");
|
||||
let database_path = tempdir.path().join("warp.sqlite");
|
||||
let mut conn = setup_database(&database_path).expect("database should initialize");
|
||||
|
||||
let app_state = AppState {
|
||||
windows: vec![WindowSnapshot {
|
||||
tabs: vec![TabSnapshot {
|
||||
custom_title: None,
|
||||
root: PaneNodeSnapshot::Leaf(LeafSnapshot {
|
||||
is_focused: true,
|
||||
custom_vertical_tabs_title: None,
|
||||
contents: LeafContents::Code(CodePaneSnapShot::Local {
|
||||
tabs: vec![
|
||||
CodePaneTabSnapshot {
|
||||
path: Some(PathBuf::from("/tmp/main.rs")),
|
||||
},
|
||||
CodePaneTabSnapshot {
|
||||
path: Some(PathBuf::from("/tmp/lib.rs")),
|
||||
},
|
||||
CodePaneTabSnapshot { path: None },
|
||||
],
|
||||
active_tab_index: 1,
|
||||
source: Some(CodeSource::FileTree {
|
||||
path: PathBuf::from("/tmp/main.rs"),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
default_directory_color: None,
|
||||
selected_color: SelectedTabColor::default(),
|
||||
left_panel: None,
|
||||
right_panel: None,
|
||||
}],
|
||||
active_tab_index: 0,
|
||||
bounds: None,
|
||||
fullscreen_state: Default::default(),
|
||||
quake_mode: false,
|
||||
universal_search_width: None,
|
||||
warp_ai_width: None,
|
||||
voltron_width: None,
|
||||
warp_drive_index_width: None,
|
||||
left_panel_open: false,
|
||||
vertical_tabs_panel_open: false,
|
||||
left_panel_width: None,
|
||||
right_panel_width: None,
|
||||
agent_management_filters: None,
|
||||
}],
|
||||
active_window_index: Some(0),
|
||||
block_lists: Default::default(),
|
||||
running_mcp_servers: Default::default(),
|
||||
};
|
||||
|
||||
save_app_state(&mut conn, &app_state).expect("app state should save");
|
||||
|
||||
let restored = read_sqlite_data(&mut conn, None)
|
||||
.expect("app state should load")
|
||||
.app_state;
|
||||
|
||||
assert_eq!(restored.windows.len(), 1);
|
||||
let restored_tab = &restored.windows[0].tabs[0];
|
||||
let PaneNodeSnapshot::Leaf(LeafSnapshot {
|
||||
contents:
|
||||
LeafContents::Code(CodePaneSnapShot::Local {
|
||||
tabs,
|
||||
active_tab_index,
|
||||
source,
|
||||
}),
|
||||
..
|
||||
}) = &restored_tab.root
|
||||
else {
|
||||
panic!("Expected code pane leaf");
|
||||
};
|
||||
|
||||
assert_eq!(tabs.len(), 3);
|
||||
assert_eq!(*active_tab_index, 1);
|
||||
assert_eq!(tabs[0].path, Some(PathBuf::from("/tmp/main.rs")));
|
||||
assert_eq!(tabs[1].path, Some(PathBuf::from("/tmp/lib.rs")));
|
||||
assert_eq!(tabs[2].path, None);
|
||||
assert!(matches!(source, Some(CodeSource::FileTree { .. })));
|
||||
}
|
||||
|
||||
fn assert_encode_then_decode_preserves_original_path(original_path: PathBuf) {
|
||||
let bytes = encode_path(original_path.clone());
|
||||
let decoded_path = decode_path(bytes);
|
||||
assert_eq!(original_path, decoded_path);
|
||||
}
|
||||
|
||||
/// Test that a local path can be encoded and decoded. We use this when persisting a local
|
||||
/// file path for notebooks in sqlite. We need this test because Windows `OsString`s are
|
||||
/// often arbitrary sequences of 16-bit values, unlike Unix which uses sequences of 8-bit
|
||||
/// values (bytes). Since `diesel::sql_types::Binary` deals with sequences of bytes (`u8`)
|
||||
/// we need to perform special casting on `OsString`s on Windows.
|
||||
#[test]
|
||||
fn test_path_encode_decode() {
|
||||
// Empty path
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::new());
|
||||
|
||||
// Windows-style paths
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(r"C:\windows\system32.dll"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from("c:temp"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(r"\temp"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(r"\temp\emoji\🙈.txt"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(r"\temp\ñoñàscii\temp.txt"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(r"\temp\hindi\हिन्दी"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(r"\temp\cjk\狗没有耐心"));
|
||||
|
||||
// Unix-style paths
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from(
|
||||
"/home/persistence/example.sql",
|
||||
));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from("./database/log.txt"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from("/temp/emoji/🙈.txt"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from("/temp/ñoñàscii/temp.txt"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from("/temp/hindi/हिन्दी"));
|
||||
assert_encode_then_decode_preserves_original_path(PathBuf::from("/temp/cjk/狗没有耐心"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deserialize_corrupted_guests() {
|
||||
let _ = FeatureFlag::SharedWithMe.override_enabled(true);
|
||||
// Use a hardcoded timestamp to ensure this test works on systems with more-than-microsecond
|
||||
// precision.
|
||||
let permissions_ts_micros = 123456;
|
||||
let permissions_ts =
|
||||
ServerTimestamp::from_unix_timestamp_micros(permissions_ts_micros).unwrap();
|
||||
|
||||
let db_permissions = ObjectPermissions {
|
||||
id: 42,
|
||||
object_metadata_id: 10,
|
||||
subject_type: "TEAM".to_string(),
|
||||
subject_id: Some("7".to_string()),
|
||||
subject_uid: "team_uid12345678912345".to_string(),
|
||||
permissions_last_updated_at: Some(permissions_ts_micros),
|
||||
// This is not a valid set of encoded object guests.
|
||||
object_guests: Some(vec![1, 2, 3]),
|
||||
anyone_with_link_access_level: None,
|
||||
anyone_with_link_source: None,
|
||||
};
|
||||
|
||||
// The overall permissions should successfully convert, minus the object guests.
|
||||
let cloud_permissions = super::to_cloud_object_permissions(&db_permissions, None);
|
||||
assert_eq!(
|
||||
cloud_permissions,
|
||||
Some(CloudObjectPermissions {
|
||||
owner: Owner::Team {
|
||||
team_uid: crate::server::ids::ServerId::from_string_lossy("team_uid12345678912345"),
|
||||
},
|
||||
permissions_last_updated_ts: Some(permissions_ts),
|
||||
anyone_with_link: None,
|
||||
guests: vec![],
|
||||
})
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
//! Module with integration test-only util methods setting up sqlite.
|
||||
|
||||
use diesel::{ExpressionMethods, QueryDsl, RunQueryDsl};
|
||||
|
||||
use super::{schema, sqlite::init_db};
|
||||
|
||||
/// Updates the 'user' and 'host' columns for stored blocks to the given values.
|
||||
///
|
||||
/// This is used at runtime to update the user and host values to real values based on the running
|
||||
/// machine in integration tests that rely on accuracy of these values.
|
||||
pub fn set_user_and_hostname_for_blocks(user: String, hostname: String) {
|
||||
let mut conn = init_db().expect("Should be able to establish sqlite connection.");
|
||||
|
||||
// Update the 'user' and 'host' columns to their real values (based on the machine on which this test is running)
|
||||
// for blocks that were stored with the placeholder 'local:user' and 'local:host' values.
|
||||
//
|
||||
// This allows us to use real (rather than mocked out) logic for matching restored
|
||||
// blocks to the appropriate session based on session hostnamebased on system hostname.
|
||||
diesel::update(schema::blocks::dsl::blocks.filter(schema::blocks::user.eq("local:user")))
|
||||
.set((
|
||||
schema::blocks::user.eq(user),
|
||||
schema::blocks::host.eq(hostname),
|
||||
))
|
||||
.execute(&mut conn)
|
||||
.expect("Failed to update user and hostname for restored blocks.");
|
||||
}
|
||||
|
||||
pub fn set_user_and_hostname_for_commands(user: String, hostname: String) {
|
||||
let mut conn = init_db().expect("Should be able to establish sqlite connection.");
|
||||
|
||||
// Update the 'user' and 'host' columns to their real values (based on the machine on which
|
||||
// this test is running) for commands that were stored with the placeholder 'local:user' and
|
||||
// 'local:host' values.
|
||||
//
|
||||
// This allows us to use real (rather than mocked out) logic for matching history commands to
|
||||
// the appropriate session based on session hostnamebased on system hostname.
|
||||
diesel::update(
|
||||
schema::commands::dsl::commands.filter(schema::commands::username.eq("local:user")),
|
||||
)
|
||||
.set((
|
||||
schema::commands::username.eq(user),
|
||||
schema::commands::hostname.eq(hostname),
|
||||
))
|
||||
.execute(&mut conn)
|
||||
.expect("Failed to update user and hostname for persisted commands.");
|
||||
}
|
||||
Reference in New Issue
Block a user