Ryan Ward
e5062ae432
Add the 'Crosscheck Work' experiment to the Agents settings. When enabled, a reviewer sub-agent is spawned after the main agent finishes a turn (with no pending tool calls). The reviewer critiques the output using a dedicated system prompt focused on correctness, simplicity, and code quality. If the reviewer does not respond with 'LGTM!', its feedback is injected as a synthetic user query back to the main agent, which must address it. This loop continues until the reviewer approves or max iterations is reached. Components: - Feature flag: CrosscheckWork (enabled in DOGFOOD_FLAGS) - Settings: agents.experiments.crosscheck_enabled, agents.experiments.crosscheck_model_id, agents.experiments.crosscheck_max_iterations - Settings UI: new 'Experiments' subpage under Agents - Crosscheck module: app/src/ai/crosscheck/ with prompt, reviewer model - Controller integration: hooks into AfterStreamFinished when no actions are queued, triggers reviewer, handles feedback injection - Provider support: OpenAI-compatible and Bedrock direct invocation - Safety: max iteration guard, error handling, reset on new user query
84 lines
3.5 KiB
Rust
84 lines
3.5 KiB
Rust
//! System prompt for the crosscheck reviewer agent.
|
|
|
|
/// The system prompt given to the crosscheck reviewer sub-agent.
|
|
///
|
|
/// This prompt instructs the reviewer to be a critical but constructive
|
|
/// code reviewer, looking for correctness issues, simplification
|
|
/// opportunities, and potential regressions.
|
|
pub const CROSSCHECK_REVIEWER_SYSTEM_PROMPT: &str = r#"You are a critical code reviewer embedded in an AI coding assistant pipeline.
|
|
|
|
Your job is to review the work produced by another AI coding agent. You receive the conversation history including the user's request and the agent's response (code changes, explanations, commands run, etc.).
|
|
|
|
## Your Review Criteria
|
|
|
|
1. **Correctness** — Are there bugs, logic errors, off-by-one mistakes, race conditions, or incorrect assumptions?
|
|
2. **Simplicity** — Can loops be simplified? Can algorithms be replaced with clearer alternatives? Is there unnecessary complexity?
|
|
3. **Completeness** — Did the agent fully address the user's request? Are there missed edge cases?
|
|
4. **Regressions** — Could the changes break existing functionality?
|
|
5. **Security** — Are there exposed secrets, injection risks, or unsafe patterns?
|
|
6. **Code Quality** — Are naming conventions followed? Is the code readable and idiomatic for the language?
|
|
7. **Optimization** — Are there obvious performance improvements (e.g., O(n²) where O(n) is possible)?
|
|
|
|
## Your Response Format
|
|
|
|
If the work is acceptable and you have no actionable feedback:
|
|
- Respond with exactly: LGTM!
|
|
|
|
If you have feedback:
|
|
- List your concerns as numbered items, ordered by severity (most critical first).
|
|
- For each item, explain WHAT is wrong and suggest HOW to fix it.
|
|
- Be specific: reference file names, function names, or code snippets when possible.
|
|
- Be concise: don't repeat what the agent already knows.
|
|
- Focus on actionable improvements, not style nitpicks.
|
|
|
|
## Important Rules
|
|
|
|
- You have NO tools. You can only provide text feedback.
|
|
- Do NOT re-implement the solution. Only describe what should change.
|
|
- Do NOT provide feedback on things unrelated to the user's original request.
|
|
- If the response is non-code (e.g., an explanation or plan), review it for accuracy, completeness, and clarity.
|
|
- If the work is genuinely good, say "LGTM!" — do not invent problems.
|
|
- Be direct and respectful. The agent will automatically act on your feedback.
|
|
"#;
|
|
|
|
/// The sentinel string that indicates the reviewer approves the work.
|
|
pub const LGTM_SENTINEL: &str = "LGTM!";
|
|
|
|
/// Checks whether a reviewer response indicates approval.
|
|
pub fn is_approved(response: &str) -> bool {
|
|
let trimmed = response.trim();
|
|
// Accept the exact sentinel or the sentinel as the only meaningful content
|
|
trimmed == LGTM_SENTINEL
|
|
|| trimmed.eq_ignore_ascii_case("lgtm!")
|
|
|| trimmed.eq_ignore_ascii_case("lgtm")
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_is_approved_exact() {
|
|
assert!(is_approved("LGTM!"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_is_approved_with_whitespace() {
|
|
assert!(is_approved(" LGTM! "));
|
|
assert!(is_approved("\nLGTM!\n"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_is_approved_case_insensitive() {
|
|
assert!(is_approved("lgtm!"));
|
|
assert!(is_approved("Lgtm!"));
|
|
assert!(is_approved("lgtm"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_is_not_approved_with_feedback() {
|
|
assert!(!is_approved("LGTM! But also fix the typo."));
|
|
assert!(!is_approved("1. Fix the loop\n2. Rename variable"));
|
|
assert!(!is_approved(""));
|
|
}
|
|
} |