Fix: Ensure OpenAI requests end with user message for Bedrock/LiteLLM compatibility

When LiteLLM routes OpenAI-format requests to AWS Bedrock, the provider
rejects conversations ending with an assistant message with:
'This model does not support assistant message prefill.'

Add ensure_ends_with_user_message() as the final sanitization step in
sanitize_messages_for_openai(). If the conversation ends with an assistant
message, a minimal 'Continue.' user message is appended.

Updated tests to reflect the new behavior and added dedicated tests for
the new function.
This commit is contained in:
Ryan Ward
2026-07-16 09:55:38 -05:00
parent 5a2e3acb31
commit 345057a147
2 changed files with 81 additions and 7 deletions
+23
View File
@@ -6,9 +6,11 @@ use crate::ai::provider::types::{ContentPart, ConversationMessage, MessageConten
/// alternation and allows system messages anywhere. The main constraints are:
/// - Tool results must reference a valid tool_call_id from a preceding assistant message
/// - Tool calls in assistant messages must eventually have matching tool results
/// - When routed to Bedrock via LiteLLM, the conversation must end with a user message
pub fn sanitize_messages_for_openai(messages: &mut Vec<ConversationMessage>) {
remove_orphaned_tool_results(messages);
synthesize_missing_tool_results(messages);
ensure_ends_with_user_message(messages);
}
/// Removes tool_result messages that reference tool_use_ids not found in any
@@ -218,3 +220,24 @@ fn collect_tool_result_ids(content: &MessageContent, ids: &mut std::collections:
_ => {}
}
}
/// Ensures the message array ends with a user message. Some providers (e.g. Bedrock
/// via LiteLLM) reject requests where the conversation ends with an assistant message
/// ("assistant message prefill"). If the last message is from the assistant, append a
/// minimal user message to satisfy this constraint.
fn ensure_ends_with_user_message(messages: &mut Vec<ConversationMessage>) {
if messages.is_empty() {
return;
}
if let Some(last) = messages.last() {
if last.role == MessageRole::Assistant {
log::info!(
"[openai] Conversation ends with assistant message — appending user continuation message"
);
messages.push(ConversationMessage {
role: MessageRole::User,
content: MessageContent::Text("Continue.".to_string()),
});
}
}
}