public/galaxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial public release of Warp.

Browse Source 
Repo-Sync-Origin: warpdotdev/warp-internal@12af1d983b
This commit is contained in:
David Stern
2026-04-28 08:43:33 -05:00
commit 0dbd3d567a
4982 changed files with 1431549 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
resources/bundled/skills/add-mcp-server/SKILL.md
+85
View File
@@ -0,0 +1,85 @@
---
name: add-mcp-server
description: Use this skill when helping users add MCP servers to their Warp configuration.
---
# Adding MCP Servers to Warp
Warp supports MCP servers via native config files. Follow these steps when helping a user add an MCP server.
## Step 1: Determine Scope
If the user hasn't specified, ask whether they want to configure the server **globally** (for all projects) or **project-scoped** (for a specific repository only).
Config file paths:
- **Global (user-scoped):** `~/.warp/.mcp.json`
- **Project-scoped:** `{repo_root}/.warp/.mcp.json`
## Step 2: Gather Server Details
If the user hasn't provided the server's connection details, use WebSearch to find the correct configuration for the named server.
If it's unclear whether the server should be run as a local CLI process (stdio transport) or connected to via URL (HTTP/SSE streaming transport), ask the user which they prefer.
## Step 3: Check and Prepare the Config File
Check whether the target config file exists.
- **If it does not exist**, create it with `mkdir -p` for the directory and initialize it with an empty `mcpServers` wrapper key:
```json
{
"mcpServers": {}
}
```
- **If it exists**, read it to determine which top-level wrapper key is already in use. Recognized wrapper keys (in order of preference):
- `mcpServers` *(preferred)*
- `mcp_servers`
- `servers`
- `mcp.servers` (nested under a `mcp` key)
- Flat map (each top-level key is a server name)
Preserve the existing wrapper key when writing. If the existing key is unrecognized or incompatible, switch to `mcpServers`.
**Never remove existing server entries** — only add or update the new server.
## Step 4: Write the Server Configuration
### Command-based server (stdio transport)
```json
{
"mcpServers": {
"server-name": {
"command": "npx",
"args": ["-y", "@scope/package-name"],
"env": {
"API_KEY": "${API_KEY}"
}
}
}
}
```
### URL-based server (HTTP/SSE streaming transport)
```json
{
"mcpServers": {
"server-name": {
"url": "https://example.com/mcp",
"env": {
"API_KEY": "${API_KEY}"
}
}
}
}
```
For environment variables containing secrets, use `${VAR_NAME}` syntax — Warp will substitute the value from the user's environment at runtime.
## Notes
- Warp auto-detects changes to `.mcp.json` files on save — no restart required.
- Configured servers appear in Warp's Settings under MCP, labeled **"Detected from Warp"**.
- Global config applies across all sessions; project config only applies when working inside that repository.
resources/bundled/skills/claude-api/LICENSE.txt
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Anthropic, PBC.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
resources/bundled/skills/claude-api/SKILL.md
+324
View File
@@ -0,0 +1,324 @@
---
name: claude-api
description: "Build, debug, and optimize Claude API / Anthropic SDK apps. Apps built with this skill should include prompt caching. Also handles migrating existing Claude API code between Claude model versions (4.5 → 4.6, 4.6 → 4.7, retired-model replacements). TRIGGER when: code imports `anthropic`/`@anthropic-ai/sdk`; user asks for the Claude API, Anthropic SDK, or Managed Agents; user adds/modifies/tunes a Claude feature (caching, thinking, compaction, tool use, batch, files, citations, memory) or model (Opus/Sonnet/Haiku) in a file; questions about prompt caching / cache hit rate in an Anthropic SDK project. SKIP: file imports `openai`/other-provider SDK, filename like `*-openai.py`/`*-generic.py`, provider-neutral code, general programming/ML."
license: Complete terms in LICENSE.txt
---
# Building LLM-Powered Applications with Claude
This skill helps you build LLM-powered applications with Claude. Choose the right surface based on your needs, detect the project language, then read the relevant language-specific documentation.
## Before You Start
Scan the target file (or, if no target file, the prompt and project) for non-Anthropic provider markers — `import openai`, `from openai`, `langchain_openai`, `OpenAI(`, `gpt-4`, `gpt-5`, file names like `agent-openai.py` or `*-generic.py`, or any explicit instruction to keep the code provider-neutral. If you find any, stop and tell the user that this skill produces Claude/Anthropic SDK code; ask whether they want to switch the file to Claude or want a non-Claude implementation. Do not edit a non-Anthropic file with Anthropic SDK calls.
## Output Requirement
When the user asks you to add, modify, or implement a Claude feature, your code must call Claude through one of:
1. **The official Anthropic SDK** for the project's language (`anthropic`, `@anthropic-ai/sdk`, `com.anthropic.*`, etc.). This is the default whenever a supported SDK exists for the project.
2. **Raw HTTP** (`curl`, `requests`, `fetch`, `httpx`, etc.) — only when the user explicitly asks for cURL/REST/raw HTTP, the project is a shell/cURL project, or the language has no official SDK.
Never mix the two — don't reach for `requests`/`fetch` in a Python or TypeScript project just because it feels lighter. Never fall back to OpenAI-compatible shims.
**Never guess SDK usage.** Function names, class names, namespaces, method signatures, and import paths must come from explicit documentation — either the `{lang}/` files in this skill or the official SDK repositories or documentation links listed in `shared/live-sources.md`. If the binding you need is not explicitly documented in the skill files, WebFetch the relevant SDK repo from `shared/live-sources.md` before writing code. Do not infer Ruby/Java/Go/PHP/C# APIs from cURL shapes or from another language's SDK.
## Defaults
Unless the user requests otherwise:
For the Claude model version, please use Claude Opus 4.7, which you can access via the exact model string `claude-opus-4-7`. Please default to using adaptive thinking (`thinking: {type: "adaptive"}`) for anything remotely complicated. And finally, please default to streaming for any request that may involve long input, long output, or high `max_tokens` — it prevents hitting request timeouts. Use the SDK's `.get_final_message()` / `.finalMessage()` helper to get the complete response if you don't need to handle individual stream events
---
## Subcommands
If the User Request at the bottom of this prompt is a bare subcommand string (no prose), search every **Subcommands** table in this document — including any in sections appended below — and follow the matching Action column directly. This lets users invoke specific flows via `/claude-api <subcommand>`. If no table in the document matches, treat the request as normal prose.
---
## Language Detection
Before reading code examples, determine which language the user is working in:
1. **Look at project files** to infer the language:
- `*.py`, `requirements.txt`, `pyproject.toml`, `setup.py`, `Pipfile` → **Python** — read from `python/`
- `*.ts`, `*.tsx`, `package.json`, `tsconfig.json` → **TypeScript** — read from `typescript/`
- `*.js`, `*.jsx` (no `.ts` files present) → **TypeScript** — JS uses the same SDK, read from `typescript/`
- `*.java`, `pom.xml`, `build.gradle` → **Java** — read from `java/`
- `*.kt`, `*.kts`, `build.gradle.kts` → **Java** — Kotlin uses the Java SDK, read from `java/`
- `*.scala`, `build.sbt` → **Java** — Scala uses the Java SDK, read from `java/`
- `*.go`, `go.mod` → **Go** — read from `go/`
- `*.rb`, `Gemfile` → **Ruby** — read from `ruby/`
- `*.cs`, `*.csproj` → **C#** — read from `csharp/`
- `*.php`, `composer.json` → **PHP** — read from `php/`
2. **If multiple languages detected** (e.g., both Python and TypeScript files):
- Check which language the user's current file or question relates to
- If still ambiguous, ask: "I detected both Python and TypeScript files. Which language are you using for the Claude API integration?"
3. **If language can't be inferred** (empty project, no source files, or unsupported language):
- Use AskUserQuestion with options: Python, TypeScript, Java, Go, Ruby, cURL/raw HTTP, C#, PHP
- If AskUserQuestion is unavailable, default to Python examples and note: "Showing Python examples. Let me know if you need a different language."
4. **If unsupported language detected** (Rust, Swift, C++, Elixir, etc.):
- Suggest cURL/raw HTTP examples from `curl/` and note that community SDKs may exist
- Offer to show Python or TypeScript examples as reference implementations
5. **If user needs cURL/raw HTTP examples**, read from `curl/`.
### Language-Specific Feature Support
| Language | Tool Runner | Managed Agents | Notes |
| ---------- | ----------- | -------------- | ------------------------------------- |
| Python | Yes (beta) | Yes (beta) | Full support — `@beta_tool` decorator |
| TypeScript | Yes (beta) | Yes (beta) | Full support — `betaZodTool` + Zod |
| Java | Yes (beta) | Yes (beta) | Beta tool use with annotated classes |
| Go | Yes (beta) | Yes (beta) | `BetaToolRunner` in `toolrunner` pkg |
| Ruby | Yes (beta) | Yes (beta) | `BaseTool` + `tool_runner` in beta |
| C# | No | No | Official SDK |
| PHP | Yes (beta) | Yes (beta) | `BetaRunnableTool` + `toolRunner()` |
| cURL | N/A | Yes (beta) | Raw HTTP, no SDK features |
> **Managed Agents code examples**: dedicated language-specific READMEs are provided for Python, TypeScript, Go, Ruby, PHP, Java, and cURL (`{lang}/managed-agents/README.md`, `curl/managed-agents.md`). Read your language's README plus the language-agnostic `shared/managed-agents-*.md` concept files. **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `agents.create` and pass it to every subsequent `sessions.create`; do not call `agents.create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. If a binding you need isn't shown in the README, WebFetch the relevant entry from `shared/live-sources.md` rather than guess. C# does not currently have Managed Agents support; use cURL-style raw HTTP requests against the API.
---
## Which Surface Should I Use?
> **Start simple.** Default to the simplest tier that meets your needs. Single API calls and workflows handle most use cases — only reach for agents when the task genuinely requires open-ended, model-driven exploration.
| Use Case | Tier | Recommended Surface | Why |
| ----------------------------------------------- | --------------- | ------------------------- | ------------------------------------------------------------ |
| Classification, summarization, extraction, Q&A | Single LLM call | **Claude API** | One request, one response |
| Batch processing or embeddings | Single LLM call | **Claude API** | Specialized endpoints |
| Multi-step pipelines with code-controlled logic | Workflow | **Claude API + tool use** | You orchestrate the loop |
| Custom agent with your own tools | Agent | **Claude API + tool use** | Maximum flexibility |
| Server-managed stateful agent with workspace | Agent | **Managed Agents** | Anthropic runs the loop and hosts the tool-execution sandbox |
| Persisted, versioned agent configs | Agent | **Managed Agents** | Agents are stored objects; sessions pin to a version |
| Long-running multi-turn agent with file mounts | Agent | **Managed Agents** | Per-session containers, SSE event stream, Skills + MCP |
> **Note:** Managed Agents is the right choice when you want Anthropic to run the agent loop *and* host the container where tools execute — file ops, bash, code execution all run in the per-session workspace. If you want to host the compute yourself or run your own custom tool runtime, Claude API + tool use is the right choice — use the tool runner for automatic loop handling, or the manual loop for fine-grained control (approval gates, custom logging, conditional execution).
> **Third-party providers (Amazon Bedrock, Google Vertex AI, Microsoft Foundry):** Managed Agents is **not available** on Bedrock, Vertex, or Foundry. If you are deploying through any third-party provider, use **Claude API + tool use** for all use cases — including ones where Managed Agents would otherwise be the recommended surface.
### Decision Tree
```
What does your application need?
0. Are you deploying through Amazon Bedrock, Google Vertex AI, or Microsoft Foundry?
└── Yes → Claude API (+ tool use for agents) — Managed Agents is 1P only.
No → continue.
1. Single LLM call (classification, summarization, extraction, Q&A)
└── Claude API — one request, one response
2. Do you want Anthropic to run the agent loop and host a per-session
container where Claude executes tools (bash, file ops, code)?
└── Yes → Managed Agents — server-managed sessions, persisted agent configs,
SSE event stream, Skills + MCP, file mounts.
Examples: "stateful coding agent with a workspace per task",
"long-running research agent that streams events to a UI",
"agent with persisted, versioned config used across many sessions"
3. Workflow (multi-step, code-orchestrated, with your own tools)
└── Claude API with tool use — you control the loop
4. Open-ended agent (model decides its own trajectory, your own tools, you host the compute)
└── Claude API agentic loop (maximum flexibility)
```
### Should I Build an Agent?
Before choosing the agent tier, check all four criteria:
- **Complexity** — Is the task multi-step and hard to fully specify in advance? (e.g., "turn this design doc into a PR" vs. "extract the title from this PDF")
- **Value** — Does the outcome justify higher cost and latency?
- **Viability** — Is Claude capable at this task type?
- **Cost of error** — Can errors be caught and recovered from? (tests, review, rollback)
If the answer is "no" to any of these, stay at a simpler tier (single call or workflow).
---
## Architecture
Everything goes through `POST /v1/messages`. Tools and output constraints are features of this single endpoint — not separate APIs.
**User-defined tools** — You define tools (via decorators, Zod schemas, or raw JSON), and the SDK's tool runner handles calling the API, executing your functions, and looping until Claude is done. For full control, you can write the loop manually.
**Server-side tools** — Anthropic-hosted tools that run on Anthropic's infrastructure. Code execution is fully server-side (declare it in `tools`, Claude runs code automatically). Computer use can be server-hosted or self-hosted.
**Structured outputs** — Constrains the Messages API response format (`output_config.format`) and/or tool parameter validation (`strict: true`). The recommended approach is `client.messages.parse()` which validates responses against your schema automatically. Note: the old `output_format` parameter is deprecated; use `output_config: {format: {...}}` on `messages.create()`.
**Supporting endpoints** — Batches (`POST /v1/messages/batches`), Files (`POST /v1/files`), Token Counting, and Models (`GET /v1/models`, `GET /v1/models/{id}` — live capability/context-window discovery) feed into or support Messages API requests.
---
## Current Models (cached: 2026-04-15)
| Model | Model ID | Context | Input $/1M | Output $/1M |
| ----------------- | ------------------- | -------------- | ---------- | ----------- |
| Claude Opus 4.7 | `claude-opus-4-7` | 1M | $5.00 | $25.00 |
| Claude Opus 4.6 | `claude-opus-4-6` | 1M | $5.00 | $25.00 |
| Claude Sonnet 4.6 | `claude-sonnet-4-6` | 1M | $3.00 | $15.00 |
| Claude Haiku 4.5 | `claude-haiku-4-5` | 200K | $1.00 | $5.00 |
**ALWAYS use `claude-opus-4-7` unless the user explicitly names a different model.** This is non-negotiable. Do not use `claude-sonnet-4-6`, `claude-sonnet-4-5`, or any other model unless the user literally says "use sonnet" or "use haiku". Never downgrade for cost — that's the user's decision, not yours.
**CRITICAL: Use only the exact model ID strings from the table above — they are complete as-is. Do not append date suffixes.** For example, use `claude-sonnet-4-5`, never `claude-sonnet-4-5-20250514` or any other date-suffixed variant you might recall from training data. If the user requests an older model not in the table (e.g., "opus 4.5", "sonnet 3.7"), read `shared/models.md` for the exact ID — do not construct one yourself.
A note: if any of the model strings above look unfamiliar to you, that's to be expected — that just means they were released after your training data cutoff. Rest assured they are real models; we wouldn't mess with you like that.
**Live capability lookup:** The table above is cached. When the user asks "what's the context window for X", "does X support vision/thinking/effort", or "which models support Y", query the Models API (`client.models.retrieve(id)` / `client.models.list()`) — see `shared/models.md` for the field reference and capability-filter examples.
---
## Thinking & Effort (Quick Reference)
**Opus 4.7 — Adaptive thinking only:** Use `thinking: {type: "adaptive"}`. `thinking: {type: "enabled", budget_tokens: N}` returns a 400 on Opus 4.7 — adaptive is the only on-mode. `{type: "disabled"}` and omitting `thinking` both work. Sampling parameters (`temperature`, `top_p`, `top_k`) are also removed and will 400. See `shared/model-migration.md` → Migrating to Opus 4.7 for the full breaking-change list.
**Opus 4.6 — Adaptive thinking (recommended):** Use `thinking: {type: "adaptive"}`. Claude dynamically decides when and how much to think. No `budget_tokens` needed — `budget_tokens` is deprecated on Opus 4.6 and Sonnet 4.6 and should not be used for new code. Adaptive thinking also automatically enables interleaved thinking (no beta header needed). **When the user asks for "extended thinking", a "thinking budget", or `budget_tokens`: always use Opus 4.7 or 4.6 with `thinking: {type: "adaptive"}`. The concept of a fixed token budget for thinking is deprecated — adaptive thinking replaces it. Do NOT use `budget_tokens` for new 4.6/4.7 code and do NOT switch to an older model.** *Gradual-migration carve-out:* `budget_tokens` is still functional on Opus 4.6 and Sonnet 4.6 as a transitional escape hatch — if you're migrating existing code and need a hard token ceiling before you've tuned `effort`, see `shared/model-migration.md` → Transitional escape hatch. Note: this carve-out does **not** apply to Opus 4.7 — `budget_tokens` is fully removed there.
**Effort parameter (GA, no beta header):** Controls thinking depth and overall token spend via `output_config: {effort: "low"|"medium"|"high"|"max"}` (inside `output_config`, not top-level). Default is `high` (equivalent to omitting it). `max` is Opus-tier only (Opus 4.6 and later — not Sonnet or Haiku). Opus 4.7 adds `"xhigh"` (between `high` and `max`) — the best setting for most coding and agentic use cases on 4.7, and the default in Claude Code; use a minimum of `high` for most intelligence-sensitive work. Works on Opus 4.5, Opus 4.6, Opus 4.7, and Sonnet 4.6. Will error on Sonnet 4.5 / Haiku 4.5. On Opus 4.7, effort matters more than on any prior Opus — re-tune it when migrating. Combine with adaptive thinking for the best cost-quality tradeoffs. Lower effort means fewer and more-consolidated tool calls, less preamble, and terser confirmations — `high` is often the sweet spot balancing quality and token efficiency; use `max` when correctness matters more than cost; use `low` for subagents or simple tasks.
**Opus 4.7 — thinking content omitted by default:** `thinking` blocks still stream but their text is empty unless you opt in with `thinking: {type: "adaptive", display: "summarized"}` (default is `"omitted"`). Silent change — no error. If you stream reasoning to users, the default looks like a long pause before output; set `"summarized"` to restore visible progress.
**Task Budgets (beta, Opus 4.7):** `output_config: {task_budget: {type: "tokens", total: N}}` tells the model how many tokens it has for a full agentic loop — it sees a running countdown and self-moderates (minimum 20,000; beta header `task-budgets-2026-03-13`). Distinct from `max_tokens`, which is an enforced per-response ceiling the model is not aware of. See `shared/model-migration.md` → Task Budgets.
**Sonnet 4.6:** Supports adaptive thinking (`thinking: {type: "adaptive"}`). `budget_tokens` is deprecated on Sonnet 4.6 — use adaptive thinking instead.
**Older models (only if explicitly requested):** If the user specifically asks for Sonnet 4.5 or another older model, use `thinking: {type: "enabled", budget_tokens: N}`. `budget_tokens` must be less than `max_tokens` (minimum 1024). Never choose an older model just because the user mentions `budget_tokens` — use Opus 4.7 with adaptive thinking instead.
---
## Compaction (Quick Reference)
**Beta, Opus 4.7, Opus 4.6, and Sonnet 4.6.** For long-running conversations that may exceed the 1M context window, enable server-side compaction. The API automatically summarizes earlier context when it approaches the trigger threshold (default: 150K tokens). Requires beta header `compact-2026-01-12`.
**Critical:** Append `response.content` (not just the text) back to your messages on every turn. Compaction blocks in the response must be preserved — the API uses them to replace the compacted history on the next request. Extracting only the text string and appending that will silently lose the compaction state.
See `{lang}/claude-api/README.md` (Compaction section) for code examples. Full docs via WebFetch in `shared/live-sources.md`.
---
## Prompt Caching (Quick Reference)
**Prefix match.** Any byte change anywhere in the prefix invalidates everything after it. Render order is `tools` → `system` → `messages`. Keep stable content first (frozen system prompt, deterministic tool list), put volatile content (timestamps, per-request IDs, varying questions) after the last `cache_control` breakpoint.
**Top-level auto-caching** (`cache_control: {type: "ephemeral"}` on `messages.create()`) is the simplest option when you don't need fine-grained placement. Max 4 breakpoints per request. Minimum cacheable prefix is ~1024 tokens — shorter prefixes silently won't cache.
**Verify with `usage.cache_read_input_tokens`** — if it's zero across repeated requests, a silent invalidator is at work (`datetime.now()` in system prompt, unsorted JSON, varying tool set).
For placement patterns, architectural guidance, and the silent-invalidator audit checklist: read `shared/prompt-caching.md`. Language-specific syntax: `{lang}/claude-api/README.md` (Prompt Caching section).
---
## Managed Agents (Beta)
**Managed Agents** is a third surface: server-managed stateful agents with Anthropic-hosted tool execution. You create a persisted, versioned Agent config (`POST /v1/agents`), then start Sessions that reference it. Each session provisions a container as the agent's workspace — bash, file ops, and code execution run there; the agent loop itself runs on Anthropic's orchestration layer and acts on the container via tools. The session streams events; you send messages and tool results back.
**Managed Agents is first-party only.** It is not available on Amazon Bedrock, Google Vertex AI, or Microsoft Foundry. For agents on third-party providers, use Claude API + tool use.
**Mandatory flow:** Agent (once) → Session (every run). `model`/`system`/`tools` live on the agent, never the session. See `shared/managed-agents-overview.md` for the full reading guide, beta headers, and pitfalls.
**Beta headers:** `managed-agents-2026-04-01` — the SDK sets this automatically for all `client.beta.{agents,environments,sessions,vaults}.*` calls. Skills API uses `skills-2025-10-02` and Files API uses `files-api-2025-04-14`, but you don't need to explicitly pass those in for endpoints other than `/v1/skills` and `/v1/files`.
**Subcommands** — invoke directly with `/claude-api <subcommand>`:
| Subcommand | Action |
|---|---|
| `managed-agents-onboard` | Walk the user through setting up a Managed Agent from scratch. **Read `shared/managed-agents-onboarding.md` immediately** and follow its interview script: mental model → know-or-explore branch → template config → session setup → emit code. Do not summarize — run the interview. |
**Reading guide:** Start with `shared/managed-agents-overview.md`, then the topical `shared/managed-agents-*.md` files (core, environments, tools, events, client-patterns, onboarding, api-reference). For Python, TypeScript, Go, Ruby, PHP, and Java, read `{lang}/managed-agents/README.md` for code examples. For cURL, read `curl/managed-agents.md`. **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `agents.create` and pass it to every subsequent `sessions.create`; do not call `agents.create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML (URL in `shared/live-sources.md`). If a binding you need isn't shown in the language README, WebFetch the relevant entry from `shared/live-sources.md` rather than guess. C# does not currently have Managed Agents support; use raw HTTP from `curl/managed-agents.md` as a reference.
**When the user wants to set up a Managed Agent from scratch** (e.g. "how do I get started", "walk me through creating one", "set up a new agent"): read `shared/managed-agents-onboarding.md` and run its interview — same flow as the `managed-agents-onboard` subcommand.
**When the user asks "how do I write the client code for X":** reach for `shared/managed-agents-client-patterns.md` — covers lossless stream reconnect, `processed_at` queued/processed gate, interrupt, `tool_confirmation` round-trip, the correct idle/terminated break gate, post-idle status race, stream-first ordering, file-mount gotchas, keeping credentials host-side via custom tools, etc.
---
## Reading Guide
After detecting the language, read the relevant files based on what the user needs:
### Quick Task Reference
**Single text classification/summarization/extraction/Q&A:**
→ Read only `{lang}/claude-api/README.md`
**Chat UI or real-time response display:**
→ Read `{lang}/claude-api/README.md` + `{lang}/claude-api/streaming.md`
**Long-running conversations (may exceed context window):**
→ Read `{lang}/claude-api/README.md` — see Compaction section
**Migrating to a newer model (Opus 4.7 / Opus 4.6 / Sonnet 4.6) or replacing a retired model:**
→ Read `shared/model-migration.md`
**Prompt caching / optimize caching / "why is my cache hit rate low":**
→ Read `shared/prompt-caching.md` + `{lang}/claude-api/README.md` (Prompt Caching section)
**Function calling / tool use / agents:**
→ Read `{lang}/claude-api/README.md` + `shared/tool-use-concepts.md` + `{lang}/claude-api/tool-use.md`
**Agent design (tool surface, context management, caching strategy):**
→ Read `shared/agent-design.md`
**Batch processing (non-latency-sensitive):**
→ Read `{lang}/claude-api/README.md` + `{lang}/claude-api/batches.md`
**File uploads across multiple requests:**
→ Read `{lang}/claude-api/README.md` + `{lang}/claude-api/files-api.md`
**Managed Agents (server-managed stateful agents with workspace):**
→ Read `shared/managed-agents-overview.md` + the rest of the `shared/managed-agents-*.md` files. For Python, TypeScript, Go, Ruby, PHP, and Java, read `{lang}/managed-agents/README.md` for code examples. For cURL, read `curl/managed-agents.md`. **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `agents.create` and pass it to every subsequent `sessions.create`; do not call `agents.create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML (URL in `shared/live-sources.md`). If a binding you need isn't shown in the language README, WebFetch the relevant entry from `shared/live-sources.md` rather than guess. C# does not currently support Managed Agents — use raw HTTP from `curl/managed-agents.md` as a reference.
### Claude API (Full File Reference)
Read the **language-specific Claude API folder** (`{language}/claude-api/`):
1. **`{language}/claude-api/README.md`** — **Read this first.** Installation, quick start, common patterns, error handling.
2. **`shared/tool-use-concepts.md`** — Read when the user needs function calling, code execution, memory, or structured outputs. Covers conceptual foundations.
3. **`shared/agent-design.md`** — Read when designing an agent: bash vs. dedicated tools, programmatic tool calling, tool search/skills, context editing vs. compaction vs. memory, caching principles.
4. **`{language}/claude-api/tool-use.md`** — Read for language-specific tool use code examples (tool runner, manual loop, code execution, memory, structured outputs).
5. **`{language}/claude-api/streaming.md`** — Read when building chat UIs or interfaces that display responses incrementally.
6. **`{language}/claude-api/batches.md`** — Read when processing many requests offline (not latency-sensitive). Runs asynchronously at 50% cost.
7. **`{language}/claude-api/files-api.md`** — Read when sending the same file across multiple requests without re-uploading.
8. **`shared/prompt-caching.md`** — Read when adding or optimizing prompt caching. Covers prefix-stability design, breakpoint placement, and anti-patterns that silently invalidate cache.
9. **`shared/error-codes.md`** — Read when debugging HTTP errors or implementing error handling.
10. **`shared/model-migration.md`** — Read when upgrading to newer models, replacing retired models, or translating `budget_tokens` / prefill patterns to the current API.
11. **`shared/live-sources.md`** — WebFetch URLs for fetching the latest official documentation.
> **Note:** For Java, Go, Ruby, C#, PHP, and cURL — these have a single file each covering all basics. Read that file plus `shared/tool-use-concepts.md` and `shared/error-codes.md` as needed.
> **Note:** For the Managed Agents file reference, see the `## Managed Agents (Beta)` section above — it lists every `shared/managed-agents-*.md` file and the language-specific READMEs.
---
## When to Use WebFetch
Use WebFetch to get the latest documentation when:
- User asks for "latest" or "current" information
- Cached data seems incorrect
- User asks about features not covered here
Live documentation URLs are in `shared/live-sources.md`.
## Common Pitfalls
- Don't truncate inputs when passing files or content to the API. If the content is too long to fit in the context window, notify the user and discuss options (chunking, summarization, etc.) rather than silently truncating.
- **Opus 4.7 thinking:** Adaptive only. `thinking: {type: "enabled", budget_tokens: N}` returns 400 on Opus 4.7 — `budget_tokens` is fully removed there (along with `temperature`, `top_p`, `top_k`). Use `thinking: {type: "adaptive"}`.
- **Opus 4.6 / Sonnet 4.6 thinking:** Use `thinking: {type: "adaptive"}` — do NOT use `budget_tokens` for new 4.6 code (deprecated on both Opus 4.6 and Sonnet 4.6; for gradual migration of existing code, see the transitional escape hatch in `shared/model-migration.md` — note this carve-out does not apply to Opus 4.7). For older models, `budget_tokens` must be less than `max_tokens` (minimum 1024). This will throw an error if you get it wrong.
- **4.6/4.7 family prefill removed:** Assistant message prefills (last-assistant-turn prefills) return a 400 error on Opus 4.6, Opus 4.7, and Sonnet 4.6. Use structured outputs (`output_config.format`) or system prompt instructions to control response format instead.
- **Confirm migration scope before editing:** When a user asks to migrate code to a newer Claude model without naming a specific file, directory, or file list, **ask which scope to apply first** — the entire working directory, a specific subdirectory, or a specific set of files. Do not start editing until the user confirms. Imperative phrasings like "migrate my codebase", "move my project to X", "upgrade to Sonnet 4.6", or bare "migrate to Opus 4.7" are **still ambiguous** — they tell you what to do but not where, so ask. Proceed without asking only when the prompt names an exact file, a specific directory, or an explicit file list ("migrate `app.py`", "migrate everything under `services/`", "update `a.py` and `b.py`"). See `shared/model-migration.md` Step 0.
- **`max_tokens` defaults:** Don't lowball `max_tokens` — hitting the cap truncates output mid-thought and requires a retry. For non-streaming requests, default to `~16000` (keeps responses under SDK HTTP timeouts). For streaming requests, default to `~64000` (timeouts aren't a concern, so give the model room). Only go lower when you have a hard reason: classification (`~256`), cost caps, or deliberately short outputs.
- **128K output tokens:** Opus 4.6 and Opus 4.7 support up to 128K `max_tokens`, but the SDKs require streaming for values that large to avoid HTTP timeouts. Use `.stream()` with `.get_final_message()` / `.finalMessage()`.
- **Tool call JSON parsing (4.6/4.7 family):** Opus 4.6, Opus 4.7, and Sonnet 4.6 may produce different JSON string escaping in tool call `input` fields (e.g., Unicode or forward-slash escaping). Always parse tool inputs with `json.loads()` / `JSON.parse()` — never do raw string matching on the serialized input.
- **Structured outputs (all models):** Use `output_config: {format: {...}}` instead of the deprecated `output_format` parameter on `messages.create()`. This is a general API change, not 4.6-specific.
- **Don't reimplement SDK functionality:** The SDK provides high-level helpers — use them instead of building from scratch. Specifically: use `stream.finalMessage()` instead of wrapping `.on()` events in `new Promise()`; use typed exception classes (`Anthropic.RateLimitError`, etc.) instead of string-matching error messages; use SDK types (`Anthropic.MessageParam`, `Anthropic.Tool`, `Anthropic.Message`, etc.) instead of redefining equivalent interfaces.
- **Don't define custom types for SDK data structures:** The SDK exports types for all API objects. Use `Anthropic.MessageParam` for messages, `Anthropic.Tool` for tool definitions, `Anthropic.ToolUseBlock` / `Anthropic.ToolResultBlockParam` for tool results, `Anthropic.Message` for responses. Defining your own `interface ChatMessage { role: string; content: unknown }` duplicates what the SDK already provides and loses type safety.
- **Report and document output:** For tasks that produce reports, documents, or visualizations, the code execution sandbox has `python-docx`, `python-pptx`, `matplotlib`, `pillow`, and `pypdf` pre-installed. Claude can generate formatted files (DOCX, PDF, charts) and return them via the Files API — consider this for "report" or "document" type requests instead of plain stdout text.
resources/bundled/skills/claude-api/csharp/claude-api.md
+402
View File
@@ -0,0 +1,402 @@
# Claude API — C#
> **Note:** The C# SDK is the official Anthropic SDK for C#. Tool use is supported via the Messages API. A class-annotation-based tool runner is not available; use raw tool definitions with JSON schema. The SDK also supports Microsoft.Extensions.AI IChatClient integration with function invocation.
## Installation
```bash
dotnet add package Anthropic
```
## Client Initialization
```csharp
using Anthropic;
// Default (uses ANTHROPIC_API_KEY env var)
AnthropicClient client = new();
// Explicit API key (use environment variables — never hardcode keys)
AnthropicClient client = new() {
ApiKey = Environment.GetEnvironmentVariable("ANTHROPIC_API_KEY")
};
```
---
## Basic Message Request
```csharp
using Anthropic.Models.Messages;
var parameters = new MessageCreateParams
{
Model = Model.ClaudeOpus4_6,
MaxTokens = 16000,
Messages = [new() { Role = Role.User, Content = "What is the capital of France?" }]
};
var response = await client.Messages.Create(parameters);
// ContentBlock is a union wrapper. .Value unwraps to the variant object,
// then OfType<T> filters to the type you want. Or use the TryPick* idiom
// shown in the Thinking section below.
foreach (var text in response.Content.Select(b => b.Value).OfType<TextBlock>())
{
Console.WriteLine(text.Text);
}
```
---
## Streaming
```csharp
using Anthropic.Models.Messages;
var parameters = new MessageCreateParams
{
Model = Model.ClaudeOpus4_6,
MaxTokens = 64000,
Messages = [new() { Role = Role.User, Content = "Write a haiku" }]
};
await foreach (RawMessageStreamEvent streamEvent in client.Messages.CreateStreaming(parameters))
{
if (streamEvent.TryPickContentBlockDelta(out var delta) &&
delta.Delta.TryPickText(out var text))
{
Console.Write(text.Text);
}
}
```
**`RawMessageStreamEvent` TryPick methods** (naming drops the `Message`/`Raw` prefix): `TryPickStart`, `TryPickDelta`, `TryPickStop`, `TryPickContentBlockStart`, `TryPickContentBlockDelta`, `TryPickContentBlockStop`. There is no `TryPickMessageStop` — use `TryPickStop`.
---
## Thinking
**Adaptive thinking is the recommended mode for Claude 4.6+ models.** Claude decides dynamically when and how much to think.
```csharp
using Anthropic.Models.Messages;
var response = await client.Messages.Create(new MessageCreateParams
{
Model = Model.ClaudeOpus4_6,
MaxTokens = 16000,
// ThinkingConfigParam? implicitly converts from the concrete variant classes —
// no wrapper needed.
Thinking = new ThinkingConfigAdaptive(),
Messages =
[
new() { Role = Role.User, Content = "Solve: 27 * 453" },
],
});
// ThinkingBlock(s) precede TextBlock in Content. TryPick* narrows the union.
foreach (var block in response.Content)
{
if (block.TryPickThinking(out ThinkingBlock? t))
{
Console.WriteLine($"[thinking] {t.Thinking}");
}
else if (block.TryPickText(out TextBlock? text))
{
Console.WriteLine(text.Text);
}
}
```
> **Deprecated:** `new ThinkingConfigEnabled { BudgetTokens = N }` (fixed-budget extended thinking) still works on Claude 4.6 but is deprecated. Use adaptive thinking above.
Alternative to `TryPick*`: `.Select(b => b.Value).OfType<ThinkingBlock>()` (same LINQ pattern as the Basic Message example).
---
## Tool Use
### Defining a tool
`Tool` (NOT `ToolParam`) with an `InputSchema` record. `InputSchema.Type` is auto-set to `"object"` by the constructor — don't set it. `ToolUnion` has an implicit conversion from `Tool`, triggered by the collection expression `[...]`.
```csharp
using System.Text.Json;
using Anthropic.Models.Messages;
var parameters = new MessageCreateParams
{
Model = Model.ClaudeSonnet4_6,
MaxTokens = 16000,
Tools = [
new Tool {
Name = "get_weather",
Description = "Get the current weather in a given location",
InputSchema = new() {
Properties = new Dictionary<string, JsonElement> {
["location"] = JsonSerializer.SerializeToElement(
new { type = "string", description = "City name" }),
},
Required = ["location"],
},
},
],
Messages = [new() { Role = Role.User, Content = "Weather in Paris?" }],
};
```
Derived from `anthropic-sdk-csharp/src/Anthropic/Models/Messages/Tool.cs` and `ToolUnion.cs:799` (implicit conversion).
See [shared tool use concepts](../shared/tool-use-concepts.md) for the loop pattern.
### Converting response content to the follow-up assistant message
When echoing Claude's response back in the assistant turn, **there is no `.ToParam()` helper** — manually reconstruct each `ContentBlock` variant as its `*Param` counterpart. Do NOT use `new ContentBlockParam(block.Json)`: it compiles and serializes, but `.Value` stays `null` so `TryPick*`/`Validate()` fail (degraded JSON pass-through, not the typed path).
```csharp
using Anthropic.Models.Messages;
Message response = await client.Messages.Create(parameters);
// No .ToParam() — reconstruct per variant. Implicit conversions from each
// *Param type to ContentBlockParam mean no explicit wrapper.
List<ContentBlockParam> assistantContent = [];
List<ContentBlockParam> toolResults = [];
foreach (ContentBlock block in response.Content)
{
if (block.TryPickText(out TextBlock? text))
{
assistantContent.Add(new TextBlockParam { Text = text.Text });
}
else if (block.TryPickThinking(out ThinkingBlock? thinking))
{
// Signature MUST be preserved — the API rejects tampering
assistantContent.Add(new ThinkingBlockParam
{
Thinking = thinking.Thinking,
Signature = thinking.Signature,
});
}
else if (block.TryPickRedactedThinking(out RedactedThinkingBlock? redacted))
{
assistantContent.Add(new RedactedThinkingBlockParam { Data = redacted.Data });
}
else if (block.TryPickToolUse(out ToolUseBlock? toolUse))
{
// ToolUseBlock has required Caller; ToolUseBlockParam.Caller is optional — don't copy it
assistantContent.Add(new ToolUseBlockParam
{
ID = toolUse.ID,
Name = toolUse.Name,
Input = toolUse.Input,
});
// Execute the tool; collect ONE result per tool_use block — the API
// rejects the follow-up if any tool_use ID lacks a matching tool_result.
string result = ExecuteYourTool(toolUse.Name, toolUse.Input);
toolResults.Add(new ToolResultBlockParam
{
ToolUseID = toolUse.ID,
Content = result,
});
}
}
// Follow-up: prior messages + assistant echo + user tool_result(s)
List<MessageParam> followUpMessages =
[
.. parameters.Messages,
new() { Role = Role.Assistant, Content = assistantContent },
new() { Role = Role.User, Content = toolResults },
];
```
`ToolResultBlockParam` has no tuple constructor — use the object initializer. `Content` is a string-or-list union; a plain `string` implicitly converts.
---
## Context Editing / Compaction (Beta)
**Beta-namespace prefix is inconsistent** (source-verified against `src/Anthropic/Models/Beta/Messages/*.cs` @ 12.9.0). No prefix: `MessageCreateParams`, `MessageCountTokensParams`, `Role`. **Everything else has the `Beta` prefix**: `BetaMessageParam`, `BetaMessage`, `BetaContentBlock`, `BetaToolUseBlock`, all block param types. The unprefixed `Role` WILL collide with `Anthropic.Models.Messages.Role` if you import both namespaces (CS0104). Safest: import only Beta; if mixing, alias the beta `Role`:
```csharp
using Anthropic.Models.Beta.Messages;
using NonBeta = Anthropic.Models.Messages; // only if you also need non-beta types
// Now: MessageCreateParams, BetaMessageParam, Role (beta's), NonBeta.Role (if needed)
```
`BetaMessage.Content` is `IReadOnlyList<BetaContentBlock>` — a 15-variant discriminated union. Narrow with `TryPick*`. **Response `BetaContentBlock` is NOT assignable to param `BetaContentBlockParam`** — there's no `.ToParam()` in C#. Round-trip by converting each block:
```csharp
using Anthropic.Models.Beta.Messages;
var betaParams = new MessageCreateParams // no Beta prefix — one of only 2 unprefixed
{
Model = Model.ClaudeOpus4_6,
MaxTokens = 16000,
Betas = ["compact-2026-01-12"],
ContextManagement = new BetaContextManagementConfig
{
Edits = [new BetaCompact20260112Edit()],
},
Messages = messages,
};
BetaMessage resp = await client.Beta.Messages.Create(betaParams);
foreach (BetaContentBlock block in resp.Content)
{
if (block.TryPickCompaction(out BetaCompactionBlock? compaction))
{
// Content is nullable — compaction can fail server-side
Console.WriteLine($"compaction summary: {compaction.Content}");
}
}
// Context-edit metadata lives on a separate nullable field
if (resp.ContextManagement is { } ctx)
{
foreach (var edit in ctx.AppliedEdits)
Console.WriteLine($"cleared {edit.ClearedInputTokens} tokens");
}
// ROUND-TRIP: BetaMessageParam.Content is BetaMessageParamContent (a string|list
// union). It implicit-converts from List<BetaContentBlockParam>, NOT from the
// response's IReadOnlyList<BetaContentBlock>. Convert each block:
List<BetaContentBlockParam> paramBlocks = [];
foreach (var b in resp.Content)
{
if (b.TryPickText(out var t)) paramBlocks.Add(new BetaTextBlockParam { Text = t.Text });
else if (b.TryPickCompaction(out var c)) paramBlocks.Add(new BetaCompactionBlockParam { Content = c.Content });
// ... other variants as needed
}
messages.Add(new BetaMessageParam { Role = Role.Assistant, Content = paramBlocks });
```
All 15 `BetaContentBlock.TryPick*` variants: `Text`, `Thinking`, `RedactedThinking`, `ToolUse`, `ServerToolUse`, `WebSearchToolResult`, `WebFetchToolResult`, `CodeExecutionToolResult`, `BashCodeExecutionToolResult`, `TextEditorCodeExecutionToolResult`, `ToolSearchToolResult`, `McpToolUse`, `McpToolResult`, `ContainerUpload`, `Compaction`.
**`BetaToolUseBlock.Input` is `IReadOnlyDictionary<string, JsonElement>`** — index by key then call the `JsonElement` extractor:
```csharp
if (block.TryPickToolUse(out BetaToolUseBlock? tu))
{
int a = tu.Input["a"].GetInt32();
string s = tu.Input["name"].GetString()!;
}
```
---
## Effort Parameter
Effort is nested under `OutputConfig`, NOT a top-level property. `ApiEnum<string, Effort>` has an implicit conversion from the enum, so assign `Effort.High` directly.
```csharp
OutputConfig = new OutputConfig { Effort = Effort.High },
```
Values: `Effort.Low`, `Effort.Medium`, `Effort.High`, `Effort.Max`. Combine with `Thinking = new ThinkingConfigAdaptive()` for cost-quality control.
---
## Prompt Caching
`System` takes `MessageCreateParamsSystem?` — a union of `string` or `List<TextBlockParam>`. There is no `SystemTextBlockParam`; use plain `TextBlockParam`. The implicit conversion needs the concrete `List<TextBlockParam>` type (array literals won't convert). For placement patterns and the silent-invalidator audit checklist, see `shared/prompt-caching.md`.
```csharp
System = new List<TextBlockParam> {
new() {
Text = longSystemPrompt,
CacheControl = new CacheControlEphemeral(), // auto-sets Type = "ephemeral"
},
},
```
Optional `Ttl` on `CacheControlEphemeral`: `new() { Ttl = Ttl.Ttl1h }` or `Ttl.Ttl5m`. `CacheControl` also exists on `Tool.CacheControl` and top-level `MessageCreateParams.CacheControl`.
Verify hits via `response.Usage.CacheCreationInputTokens` / `response.Usage.CacheReadInputTokens`.
---
## Token Counting
```csharp
MessageTokensCount result = await client.Messages.CountTokens(new MessageCountTokensParams {
Model = Model.ClaudeOpus4_6,
Messages = [new() { Role = Role.User, Content = "Hello" }],
});
long tokens = result.InputTokens;
```
`MessageCountTokensParams.Tools` uses a different union type (`MessageCountTokensTool`) than `MessageCreateParams.Tools` (`ToolUnion`) — if you're passing tools, the compiler will tell you when it matters.
---
## Structured Output
```csharp
OutputConfig = new OutputConfig {
Format = new JsonOutputFormat {
Schema = new Dictionary<string, JsonElement> {
["type"] = JsonSerializer.SerializeToElement("object"),
["properties"] = JsonSerializer.SerializeToElement(
new { name = new { type = "string" } }),
["required"] = JsonSerializer.SerializeToElement(new[] { "name" }),
},
},
},
```
`JsonOutputFormat.Type` is auto-set to `"json_schema"` by the constructor. `Schema` is `required`.
---
## PDF / Document Input
`DocumentBlockParam` takes a `DocumentBlockParamSource` union: `Base64PdfSource` / `UrlPdfSource` / `PlainTextSource` / `ContentBlockSource`. `Base64PdfSource` auto-sets `MediaType = "application/pdf"` and `Type = "base64"`.
```csharp
new MessageParam {
Role = Role.User,
Content = new List<ContentBlockParam> {
new DocumentBlockParam { Source = new Base64PdfSource { Data = base64String } },
new TextBlockParam { Text = "Summarize this PDF" },
},
}
```
---
## Server-Side Tools
Web search, bash, text editor, and code execution are built-in server tools. Type names are version-suffixed; constructors auto-set `name`/`type`. All implicit-convert to `ToolUnion`.
```csharp
Tools = [
new WebSearchTool20260209(),
new ToolBash20250124(),
new ToolTextEditor20250728(),
new CodeExecutionTool20260120(),
],
```
Also available: `WebFetchTool20260209`, `MemoryTool20250818`. `WebSearchTool20260209` optionals: `AllowedDomains`, `BlockedDomains`, `MaxUses`, `UserLocation`.
---
## Files API (Beta)
Files live under `client.Beta.Files` (namespace `Anthropic.Models.Beta.Files`). `BinaryContent` implicit-converts from `Stream` and `byte[]`.
```csharp
using Anthropic.Models.Beta.Files;
using Anthropic.Models.Beta.Messages;
FileMetadata meta = await client.Beta.Files.Upload(
new FileUploadParams { File = File.OpenRead("doc.pdf") });
// Referencing the uploaded file requires Beta message types:
new BetaRequestDocumentBlock {
Source = new BetaFileDocumentSource { FileID = meta.ID },
}
```
The non-beta `DocumentBlockParamSource` union has no file-ID variant — file references need `client.Beta.Messages.Create()`.
resources/bundled/skills/claude-api/curl/examples.md
+216
View File
@@ -0,0 +1,216 @@
# Claude API — cURL / Raw HTTP
Use these examples when the user needs raw HTTP requests or is working in a language without an official SDK.
## Setup
```bash
export ANTHROPIC_API_KEY="your-api-key"
```
---
## Basic Message Request
```bash
curl https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}'
```
### Parsing the response
Use `jq` to extract fields from the JSON response. Do not use `grep`/`sed` —
JSON strings can contain any character and regex parsing will break on quotes,
escapes, or multi-line content.
```bash
# Capture the response, then extract fields
response=$(curl -s https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{"model":"claude-opus-4-7","max_tokens":16000,"messages":[{"role":"user","content":"Hello"}]}')
# Print the first text block (-r strips the JSON quotes)
echo "$response" | jq -r '.content[0].text'
# Read usage fields
input_tokens=$(echo "$response" | jq -r '.usage.input_tokens')
output_tokens=$(echo "$response" | jq -r '.usage.output_tokens')
# Read stop reason (for tool-use loops)
stop_reason=$(echo "$response" | jq -r '.stop_reason')
# Extract all text blocks (content is an array; filter to type=="text")
echo "$response" | jq -r '.content[] | select(.type == "text") | .text'
```
---
## Streaming (SSE)
```bash
curl https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-7",
"max_tokens": 64000,
"stream": true,
"messages": [{"role": "user", "content": "Write a haiku"}]
}'
```
The response is a stream of Server-Sent Events:
```
event: message_start
data: {"type":"message_start","message":{"id":"msg_...","type":"message",...}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":12}}
event: message_stop
data: {"type":"message_stop"}
```
---
## Tool Use
```bash
curl https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"tools": [{
"name": "get_weather",
"description": "Get current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"}
},
"required": ["location"]
}
}],
"messages": [{"role": "user", "content": "What is the weather in Paris?"}]
}'
```
When Claude responds with a `tool_use` block, send the result back:
```bash
curl https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"tools": [{
"name": "get_weather",
"description": "Get current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"}
},
"required": ["location"]
}
}],
"messages": [
{"role": "user", "content": "What is the weather in Paris?"},
{"role": "assistant", "content": [
{"type": "text", "text": "Let me check the weather."},
{"type": "tool_use", "id": "toolu_abc123", "name": "get_weather", "input": {"location": "Paris"}}
]},
{"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "toolu_abc123", "content": "72°F and sunny"}
]}
]
}'
```
---
## Prompt Caching
Put `cache_control` on the last block of the stable prefix. See `shared/prompt-caching.md` for placement patterns and the silent-invalidator audit checklist.
```bash
curl https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"system": [
{"type": "text", "text": "<large shared prompt...>", "cache_control": {"type": "ephemeral"}}
],
"messages": [{"role": "user", "content": "Summarize the key points"}]
}'
```
For 1-hour TTL: `"cache_control": {"type": "ephemeral", "ttl": "1h"}`. Top-level `"cache_control"` on the request body auto-places on the last cacheable block. Verify hits via the response `usage.cache_creation_input_tokens` / `usage.cache_read_input_tokens` fields.
---
## Extended Thinking
> **Opus 4.7, Opus 4.6, and Sonnet 4.6:** Use adaptive thinking. `budget_tokens` is removed on Opus 4.7 (400 if sent); deprecated on Opus 4.6 and Sonnet 4.6.
> **Older models:** Use `"type": "enabled"` with `"budget_tokens": N` (must be < `max_tokens`, min 1024).
```bash
# Opus 4.7 / 4.6: adaptive thinking (recommended)
curl https://api.anthropic.com/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"thinking": {
"type": "adaptive"
},
"output_config": {
"effort": "high"
},
"messages": [{"role": "user", "content": "Solve this step by step..."}]
}'
```
---
## Required Headers
| Header | Value | Description |
| ------------------- | ------------------ | -------------------------- |
| `Content-Type` | `application/json` | Required |
| `x-api-key` | Your API key | Authentication |
| `anthropic-version` | `2023-06-01` | API version |
| `anthropic-beta` | Beta feature IDs | Required for beta features |
resources/bundled/skills/claude-api/curl/managed-agents.md
+337
View File
@@ -0,0 +1,337 @@
# Managed Agents — cURL / Raw HTTP
Use these examples when the user needs raw HTTP requests or is working without an SDK.
## Setup
```bash
export ANTHROPIC_API_KEY="your-api-key"
# Common headers
HEADERS=(
-H "Content-Type: application/json"
-H "x-api-key: $ANTHROPIC_API_KEY"
-H "anthropic-version: 2023-06-01"
-H "anthropic-beta: managed-agents-2026-04-01"
)
```
---
## Create an Environment
```bash
curl -X POST https://api.anthropic.com/v1/environments \
"${HEADERS[@]}" \
-d '{
"name": "my-dev-env",
"config": {
"type": "cloud",
"networking": { "type": "unrestricted" }
}
}'
```
### With restricted networking
```bash
curl -X POST https://api.anthropic.com/v1/environments \
"${HEADERS[@]}" \
-d '{
"name": "restricted-env",
"config": {
"type": "cloud",
"networking": {
"type": "package_managers_and_custom",
"allowed_hosts": ["api.example.com"]
}
}
}'
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** Under `managed-agents-2026-04-01`, `model`/`system`/`tools` are top-level fields on `POST /v1/agents`, not on the session. Always create the agent first — the session only takes `"agent": {"type": "agent", "id": "..."}`.
### Minimal
```bash
# 1. Create the agent
curl -X POST https://api.anthropic.com/v1/agents \
"${HEADERS[@]}" \
-d '{
"name": "Coding Assistant",
"model": "claude-opus-4-7",
"tools": [{ "type": "agent_toolset_20260401" }]
}'
# → { "id": "agent_abc123", ... }
# 2. Start a session
curl -X POST https://api.anthropic.com/v1/sessions \
"${HEADERS[@]}" \
-d '{
"agent": { "type": "agent", "id": "agent_abc123", "version": "1772585501101368014" },
"environment_id": "env_abc123"
}'
```
### With system prompt, custom tools, and GitHub repo
```bash
# 1. Create the agent
curl -X POST https://api.anthropic.com/v1/agents \
"${HEADERS[@]}" \
-d '{
"name": "Code Reviewer",
"model": "claude-opus-4-7",
"system": "You are a senior code reviewer. Be thorough and constructive.",
"tools": [
{ "type": "agent_toolset_20260401" },
{
"type": "custom",
"name": "run_linter",
"description": "Run the project linter on a file",
"input_schema": {
"type": "object",
"properties": {
"file_path": { "type": "string", "description": "Path to lint" }
},
"required": ["file_path"]
}
}
]
}'
# 2. Start a session with the repo mounted
curl -X POST https://api.anthropic.com/v1/sessions \
"${HEADERS[@]}" \
-d '{
"agent": { "type": "agent", "id": "agent_abc123", "version": "1772585501101368014" },
"environment_id": "env_abc123",
"title": "Code review session",
"resources": [
{
"type": "github_repository",
"url": "https://github.com/owner/repo",
"mount_path": "/workspace/repo",
"authorization_token": "ghp_...",
"branch": "feature-branch"
}
]
}'
```
---
## Send a User Message
```bash
curl -X POST https://api.anthropic.com/v1/sessions/$SESSION_ID/events \
"${HEADERS[@]}" \
-d '{
"events": [
{
"type": "user.message",
"content": [{ "type": "text", "text": "Review the auth module for security issues" }]
}
]
}'
```
---
## Stream Events (SSE)
```bash
curl -N https://api.anthropic.com/v1/sessions/$SESSION_ID/events/stream \
"${HEADERS[@]}"
```
Response format:
```
event: session.status_running
data: {"type":"session.status_running","id":"sevt_...","processed_at":"..."}
event: agent.message
data: {"type":"agent.message","id":"sevt_...","content":[{"type":"text","text":"I'll review..."}],"processed_at":"..."}
event: session.status_idle
data: {"type":"session.status_idle","id":"sevt_...","processed_at":"..."}
```
---
## Poll Events
```bash
# Get all events
curl https://api.anthropic.com/v1/sessions/$SESSION_ID/events \
"${HEADERS[@]}"
# Paginated — get next page of events
curl "https://api.anthropic.com/v1/sessions/$SESSION_ID/events?page=page_abc123" \
"${HEADERS[@]}"
```
---
## Provide Custom Tool Result
When the agent calls a custom tool, send the result back:
```bash
curl -X POST https://api.anthropic.com/v1/sessions/$SESSION_ID/events \
"${HEADERS[@]}" \
-d '{
"events": [
{
"type": "user.custom_tool_result",
"custom_tool_use_id": "sevt_abc123",
"content": [{ "type": "text", "text": "No linting errors found." }]
}
]
}'
```
---
## Interrupt a Running Session
```bash
curl -X POST https://api.anthropic.com/v1/sessions/$SESSION_ID/events \
"${HEADERS[@]}" \
-d '{
"events": [
{
"type": "interrupt"
}
]
}'
```
---
## Get Session Details
```bash
curl https://api.anthropic.com/v1/sessions/$SESSION_ID \
"${HEADERS[@]}"
```
---
## List Sessions
```bash
curl https://api.anthropic.com/v1/sessions \
"${HEADERS[@]}"
```
---
## Delete a Session
```bash
curl -X DELETE https://api.anthropic.com/v1/sessions/$SESSION_ID \
"${HEADERS[@]}"
```
---
## Upload a File
```bash
curl -X POST https://api.anthropic.com/v1/files \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "anthropic-beta: files-api-2025-04-14" \
-F "file=@path/to/file.txt" \
-F "purpose=agent"
```
---
## List and Download Session Files
List files the agent wrote to `/mnt/session/outputs/` during a session, then download them.
```bash
# List files associated with a session
curl "https://api.anthropic.com/v1/files?scope_id=$SESSION_ID" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "anthropic-beta: files-api-2025-04-14,managed-agents-2026-04-01"
# Download a specific file
curl "https://api.anthropic.com/v1/files/$FILE_ID/content" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "anthropic-beta: files-api-2025-04-14,managed-agents-2026-04-01" \
-o downloaded_file.txt
```
---
## List Agents
```bash
curl https://api.anthropic.com/v1/agents \
"${HEADERS[@]}"
```
---
## MCP Server Integration
```bash
# 1. Agent declares MCP server (no auth here — auth goes in a vault)
curl -X POST https://api.anthropic.com/v1/agents \
"${HEADERS[@]}" \
-d '{
"name": "MCP Agent",
"model": "claude-opus-4-7",
"mcp_servers": [
{ "type": "url", "name": "my-tools", "url": "https://my-mcp-server.example.com/sse" }
],
"tools": [
{ "type": "agent_toolset_20260401" },
{ "type": "mcp_toolset", "mcp_server_name": "my-tools" }
]
}'
# 2. Session attaches vault containing credentials for that MCP server URL
curl -X POST https://api.anthropic.com/v1/sessions \
"${HEADERS[@]}" \
-d '{
"agent": "agent_abc123",
"environment_id": "env_abc123",
"vault_ids": ["vlt_abc123"]
}'
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
---
## Tool Configuration
```bash
curl -X POST https://api.anthropic.com/v1/agents \
"${HEADERS[@]}" \
-d '{
"name": "Restricted Agent",
"model": "claude-opus-4-7",
"tools": [
{
"type": "agent_toolset_20260401",
"default_config": { "enabled": true },
"configs": [
{ "name": "bash", "enabled": false }
]
}
]
}'
```
resources/bundled/skills/claude-api/go/claude-api.md
+421
View File
@@ -0,0 +1,421 @@
# Claude API — Go
> **Note:** The Go SDK supports the Claude API and beta tool use with `BetaToolRunner`. Agent SDK is not yet available for Go.
## Installation
```bash
go get github.com/anthropics/anthropic-sdk-go
```
## Client Initialization
```go
import (
"github.com/anthropics/anthropic-sdk-go"
"github.com/anthropics/anthropic-sdk-go/option"
)
// Default (uses ANTHROPIC_API_KEY env var)
client := anthropic.NewClient()
// Explicit API key
client := anthropic.NewClient(
option.WithAPIKey("your-api-key"),
)
```
---
## Basic Message Request
```go
response, err := client.Messages.New(context.Background(), anthropic.MessageNewParams{
Model: anthropic.ModelClaudeOpus4_6,
MaxTokens: 16000,
Messages: []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("What is the capital of France?")),
},
})
if err != nil {
log.Fatal(err)
}
for _, block := range response.Content {
switch variant := block.AsAny().(type) {
case anthropic.TextBlock:
fmt.Println(variant.Text)
}
}
```
---
## Streaming
```go
stream := client.Messages.NewStreaming(context.Background(), anthropic.MessageNewParams{
Model: anthropic.ModelClaudeOpus4_6,
MaxTokens: 64000,
Messages: []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("Write a haiku")),
},
})
for stream.Next() {
event := stream.Current()
switch eventVariant := event.AsAny().(type) {
case anthropic.ContentBlockDeltaEvent:
switch deltaVariant := eventVariant.Delta.AsAny().(type) {
case anthropic.TextDelta:
fmt.Print(deltaVariant.Text)
}
}
}
if err := stream.Err(); err != nil {
log.Fatal(err)
}
```
**Accumulating the final message** (there is no `GetFinalMessage()` on the stream):
```go
stream := client.Messages.NewStreaming(ctx, params)
message := anthropic.Message{}
for stream.Next() {
message.Accumulate(stream.Current())
}
if err := stream.Err(); err != nil { log.Fatal(err) }
// message.Content now has the complete response
```
---
## Tool Use
### Tool Runner (Beta — Recommended)
**Beta:** The Go SDK provides `BetaToolRunner` for automatic tool use loops via the `toolrunner` package.
```go
import (
"context"
"fmt"
"log"
"github.com/anthropics/anthropic-sdk-go"
"github.com/anthropics/anthropic-sdk-go/toolrunner"
)
// Define tool input with jsonschema tags for automatic schema generation
type GetWeatherInput struct {
City string `json:"city" jsonschema:"required,description=The city name"`
}
// Create a tool with automatic schema generation from struct tags
weatherTool, err := toolrunner.NewBetaToolFromJSONSchema(
"get_weather",
"Get current weather for a city",
func(ctx context.Context, input GetWeatherInput) (anthropic.BetaToolResultBlockParamContentUnion, error) {
return anthropic.BetaToolResultBlockParamContentUnion{
OfText: &anthropic.BetaTextBlockParam{
Text: fmt.Sprintf("The weather in %s is sunny, 72°F", input.City),
},
}, nil
},
)
if err != nil {
log.Fatal(err)
}
// Create a tool runner that handles the conversation loop automatically
runner := client.Beta.Messages.NewToolRunner(
[]anthropic.BetaTool{weatherTool},
anthropic.BetaToolRunnerParams{
BetaMessageNewParams: anthropic.BetaMessageNewParams{
Model: anthropic.ModelClaudeOpus4_6,
MaxTokens: 16000,
Messages: []anthropic.BetaMessageParam{
anthropic.NewBetaUserMessage(anthropic.NewBetaTextBlock("What's the weather in Paris?")),
},
},
MaxIterations: 5,
},
)
// Run until Claude produces a final response
message, err := runner.RunToCompletion(context.Background())
if err != nil {
log.Fatal(err)
}
// RunToCompletion returns *BetaMessage; content is []BetaContentBlockUnion.
// Narrow via AsAny() switch — note the Beta-namespace types (BetaTextBlock,
// not TextBlock):
for _, block := range message.Content {
switch block := block.AsAny().(type) {
case anthropic.BetaTextBlock:
fmt.Println(block.Text)
}
}
```
**Key features of the Go tool runner:**
- Automatic schema generation from Go structs via `jsonschema` tags
- `RunToCompletion()` for simple one-shot usage
- `All()` iterator for processing each message in the conversation
- `NextMessage()` for step-by-step iteration
- Streaming variant via `NewToolRunnerStreaming()` with `AllStreaming()`
### Manual Loop
For fine-grained control over the agentic loop, define tools with `ToolParam`, check `StopReason`, execute tools yourself, and feed `tool_result` blocks back. This is the pattern when you need to intercept, validate, or log tool calls.
Derived from `anthropic-sdk-go/examples/tools/main.go`.
```go
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"github.com/anthropics/anthropic-sdk-go"
)
func main() {
client := anthropic.NewClient()
// 1. Define tools. ToolParam.InputSchema uses a map, no struct tags needed.
addTool := anthropic.ToolParam{
Name: "add",
Description: anthropic.String("Add two integers"),
InputSchema: anthropic.ToolInputSchemaParam{
Properties: map[string]any{
"a": map[string]any{"type": "integer"},
"b": map[string]any{"type": "integer"},
},
},
}
// ToolParam must be wrapped in ToolUnionParam for the Tools slice
tools := []anthropic.ToolUnionParam{{OfTool: &addTool}}
messages := []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("What is 2 + 3?")),
}
for {
resp, err := client.Messages.New(context.Background(), anthropic.MessageNewParams{
Model: anthropic.ModelClaudeSonnet4_6,
MaxTokens: 16000,
Messages: messages,
Tools: tools,
})
if err != nil {
log.Fatal(err)
}
// 2. Append the assistant response to history BEFORE processing tool calls.
// resp.ToParam() converts Message → MessageParam in one call.
messages = append(messages, resp.ToParam())
// 3. Walk content blocks. ContentBlockUnion is a flattened struct;
// use block.AsAny().(type) to switch on the actual variant.
toolResults := []anthropic.ContentBlockParamUnion{}
for _, block := range resp.Content {
switch variant := block.AsAny().(type) {
case anthropic.TextBlock:
fmt.Println(variant.Text)
case anthropic.ToolUseBlock:
// 4. Parse the tool input. Use variant.JSON.Input.Raw() to get the
// raw JSON — block.Input is json.RawMessage, not the parsed value.
var in struct {
A int `json:"a"`
B int `json:"b"`
}
if err := json.Unmarshal([]byte(variant.JSON.Input.Raw()), &in); err != nil {
log.Fatal(err)
}
result := fmt.Sprintf("%d", in.A+in.B)
// 5. NewToolResultBlock(toolUseID, content, isError) builds the
// ContentBlockParamUnion for you. block.ID is the tool_use_id.
toolResults = append(toolResults,
anthropic.NewToolResultBlock(block.ID, result, false))
}
}
// 6. Exit when Claude stops asking for tools
if resp.StopReason != anthropic.StopReasonToolUse {
break
}
// 7. Tool results go in a user message (variadic: all results in one turn)
messages = append(messages, anthropic.NewUserMessage(toolResults...))
}
}
```
**Key API surface:**
| Symbol | Purpose |
|---|---|
| `resp.ToParam()` | Convert `Message` response → `MessageParam` for history |
| `block.AsAny().(type)` | Type-switch on `ContentBlockUnion` variants |
| `variant.JSON.Input.Raw()` | Raw JSON string of tool input (for `json.Unmarshal`) |
| `anthropic.NewToolResultBlock(id, content, isError)` | Build `tool_result` block |
| `anthropic.NewUserMessage(blocks...)` | Wrap tool results as a user turn |
| `anthropic.StopReasonToolUse` | `StopReason` constant to check loop termination |
| `anthropic.ToolUnionParam{OfTool: &t}` | Wrap `ToolParam` in the union for `Tools:` |
---
## Thinking
Enable Claude's internal reasoning by setting `Thinking` in `MessageNewParams`. The response will contain `ThinkingBlock` content before the final `TextBlock`.
**Adaptive thinking is the recommended mode for Claude 4.6+ models.** Claude decides dynamically when and how much to think. Combine with the `effort` parameter for cost-quality control.
Derived from `anthropic-sdk-go/message.go` (`ThinkingConfigParamUnion`, `NewThinkingConfigAdaptiveParam`).
```go
// There is no ThinkingConfigParamOfAdaptive helper — construct the union
// struct-literal directly and take the address of the variant.
adaptive := anthropic.NewThinkingConfigAdaptiveParam()
params := anthropic.MessageNewParams{
Model: anthropic.ModelClaudeSonnet4_6,
MaxTokens: 16000,
Thinking: anthropic.ThinkingConfigParamUnion{OfAdaptive: &adaptive},
Messages: []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("How many r's in strawberry?")),
},
}
resp, err := client.Messages.New(context.Background(), params)
if err != nil {
log.Fatal(err)
}
// ThinkingBlock(s) precede TextBlock in content
for _, block := range resp.Content {
switch b := block.AsAny().(type) {
case anthropic.ThinkingBlock:
fmt.Println("[thinking]", b.Thinking)
case anthropic.TextBlock:
fmt.Println(b.Text)
}
}
```
> **Deprecated:** `ThinkingConfigParamOfEnabled(budgetTokens)` (fixed-budget extended thinking) still works on Claude 4.6 but is deprecated. Use adaptive thinking above.
To disable: `anthropic.ThinkingConfigParamUnion{OfDisabled: &anthropic.ThinkingConfigDisabledParam{}}`.
---
## Prompt Caching
`System` is `[]TextBlockParam`; set `CacheControl` on the last block to cache tools + system together. For placement patterns and the silent-invalidator audit checklist, see `shared/prompt-caching.md`.
```go
System: []anthropic.TextBlockParam{{
Text: longSystemPrompt,
CacheControl: anthropic.NewCacheControlEphemeralParam(), // default 5m TTL
}},
```
For 1-hour TTL: `anthropic.CacheControlEphemeralParam{TTL: anthropic.CacheControlEphemeralTTLTTL1h}`. There's also a top-level `CacheControl` on `MessageNewParams` that auto-places on the last cacheable block.
Verify hits via `resp.Usage.CacheCreationInputTokens` / `resp.Usage.CacheReadInputTokens`.
---
## Server-Side Tools
Version-suffixed struct names with `Param` suffix. `Name`/`Type` are `constant.*` types — zero value marshals correctly, so `{}` works. Wrap in `ToolUnionParam` with the matching `Of*` field.
```go
Tools: []anthropic.ToolUnionParam{
{OfWebSearchTool20260209: &anthropic.WebSearchTool20260209Param{}},
{OfBashTool20250124: &anthropic.ToolBash20250124Param{}},
{OfTextEditor20250728: &anthropic.ToolTextEditor20250728Param{}},
{OfCodeExecutionTool20260120: &anthropic.CodeExecutionTool20260120Param{}},
},
```
Also available: `WebFetchTool20260209Param`, `MemoryTool20250818Param`, `ToolSearchToolBm25_20251119Param`, `ToolSearchToolRegex20251119Param`.
---
## PDF / Document Input
`NewDocumentBlock` generic helper accepts any source type. `MediaType`/`Type` are auto-set.
```go
b64 := base64.StdEncoding.EncodeToString(pdfBytes)
msg := anthropic.NewUserMessage(
anthropic.NewDocumentBlock(anthropic.Base64PDFSourceParam{Data: b64}),
anthropic.NewTextBlock("Summarize this document"),
)
```
Other sources: `URLPDFSourceParam{URL: "https://..."}`, `PlainTextSourceParam{Data: "..."}`.
---
## Files API (Beta)
Under `client.Beta.Files`. Method is **`Upload`** (NOT `New`/`Create`), params struct is `BetaFileUploadParams`. The `File` field takes an `io.Reader`; use `anthropic.File()` to attach a filename + content-type for the multipart encoding.
```go
f, _ := os.Open("./upload_me.txt")
defer f.Close()
meta, err := client.Beta.Files.Upload(ctx, anthropic.BetaFileUploadParams{
File: anthropic.File(f, "upload_me.txt", "text/plain"),
Betas: []anthropic.AnthropicBeta{anthropic.AnthropicBetaFilesAPI2025_04_14},
})
// meta.ID is the file_id to reference in subsequent message requests
```
Other `Beta.Files` methods: `List`, `Delete`, `Download`, `GetMetadata`.
---
## Context Editing / Compaction (Beta)
Use `Beta.Messages.New` with `ContextManagement` on `BetaMessageNewParams`. There is no `NewBetaAssistantMessage` — use `.ToParam()` for the round-trip.
```go
params := anthropic.BetaMessageNewParams{
Model: anthropic.ModelClaudeOpus4_6, // also supported: ModelClaudeSonnet4_6
MaxTokens: 16000,
Betas: []anthropic.AnthropicBeta{"compact-2026-01-12"},
ContextManagement: anthropic.BetaContextManagementConfigParam{
Edits: []anthropic.BetaContextManagementConfigEditUnionParam{
{OfCompact20260112: &anthropic.BetaCompact20260112EditParam{}},
},
},
Messages: []anthropic.BetaMessageParam{ /* ... */ },
}
resp, err := client.Beta.Messages.New(ctx, params)
if err != nil {
log.Fatal(err)
}
// Round-trip: append response to history via .ToParam()
params.Messages = append(params.Messages, resp.ToParam())
// Read compaction blocks from the response
for _, block := range resp.Content {
if c, ok := block.AsAny().(anthropic.BetaCompactionBlock); ok {
fmt.Println("compaction summary:", c.Content)
}
}
```
Other edit types: `BetaClearToolUses20250919EditParam`, `BetaClearThinking20251015EditParam`.
resources/bundled/skills/claude-api/go/managed-agents/README.md
+561
View File
@@ -0,0 +1,561 @@
# Managed Agents — Go
> **Bindings not shown here:** This README covers the most common managed-agents flows for Go. If you need a class, method, namespace, field, or behavior that isn't shown, WebFetch the Go SDK repo **or the relevant docs page** from `shared/live-sources.md` rather than guess. Do not extrapolate from cURL shapes or another language's SDK.
> **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `agents.New` and pass it to every subsequent `sessions.New`; do not call `agents.New` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. The examples below show in-code creation for completeness; in production the create call belongs in setup, not in the request path.
## Installation
```bash
go get github.com/anthropics/anthropic-sdk-go
```
## Client Initialization
```go
import (
"context"
"github.com/anthropics/anthropic-sdk-go"
"github.com/anthropics/anthropic-sdk-go/option"
)
// Default (uses ANTHROPIC_API_KEY env var)
client := anthropic.NewClient()
// Explicit API key
client := anthropic.NewClient(
option.WithAPIKey("your-api-key"),
)
ctx := context.Background()
```
---
## Create an Environment
```go
environment, err := client.Beta.Environments.New(ctx, anthropic.BetaEnvironmentNewParams{
Name: "my-dev-env",
Config: anthropic.BetaCloudConfigParams{
Networking: anthropic.BetaCloudConfigParamsNetworkingUnion{
OfUnrestricted: &anthropic.UnrestrictedNetworkParam{},
},
},
})
if err != nil {
panic(err)
}
fmt.Println(environment.ID) // env_...
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** `Model`/`System`/`Tools` live on the agent object, not the session. Always start with `Beta.Agents.New()` — the session only takes `Agent: anthropic.BetaSessionNewParamsAgentUnion{OfString: anthropic.String(agent.ID)}` (or the typed `OfBetaManagedAgentsAgents` variant when you need a specific version).
### Minimal
```go
// 1. Create the agent (reusable, versioned)
agent, err := client.Beta.Agents.New(ctx, anthropic.BetaAgentNewParams{
Name: "Coding Assistant",
Model: anthropic.BetaManagedAgentsModelConfigParams{
ID: "claude-opus-4-7",
Type: anthropic.BetaManagedAgentsModelConfigParamsTypeModelConfig,
},
System: anthropic.String("You are a helpful coding assistant."),
Tools: []anthropic.BetaAgentNewParamsToolUnion{{
OfAgentToolset20260401: &anthropic.BetaManagedAgentsAgentToolset20260401Params{
Type: anthropic.BetaManagedAgentsAgentToolset20260401ParamsTypeAgentToolset20260401,
},
}},
})
if err != nil {
panic(err)
}
// 2. Start a session
session, err := client.Beta.Sessions.New(ctx, anthropic.BetaSessionNewParams{
Agent: anthropic.BetaSessionNewParamsAgentUnion{
OfBetaManagedAgentsAgents: &anthropic.BetaManagedAgentsAgentParams{
Type: anthropic.BetaManagedAgentsAgentParamsTypeAgent,
ID: agent.ID,
Version: anthropic.Int(agent.Version),
},
},
EnvironmentID: environment.ID,
Title: anthropic.String("Quickstart session"),
})
if err != nil {
panic(err)
}
fmt.Printf("Session ID: %s, status: %s\n", session.ID, session.Status)
```
### Updating an Agent
Updates create new versions; the agent object is immutable per version.
```go
updatedAgent, err := client.Beta.Agents.Update(ctx, agent.ID, anthropic.BetaAgentUpdateParams{
Version: agent.Version,
System: anthropic.String("You are a helpful coding agent. Always write tests."),
})
if err != nil {
panic(err)
}
fmt.Printf("New version: %d\n", updatedAgent.Version)
// List all versions
iter := client.Beta.Agents.Versions.ListAutoPaging(ctx, agent.ID, anthropic.BetaAgentVersionListParams{})
for iter.Next() {
version := iter.Current()
fmt.Printf("Version %d: %s\n", version.Version, version.UpdatedAt.Format(time.RFC3339))
}
if err := iter.Err(); err != nil {
panic(err)
}
// Archive the agent
_, err = client.Beta.Agents.Archive(ctx, agent.ID, anthropic.BetaAgentArchiveParams{})
if err != nil {
panic(err)
}
```
---
## Send a User Message
```go
_, err = client.Beta.Sessions.Events.Send(ctx, session.ID, anthropic.BetaSessionEventSendParams{
Events: []anthropic.SendEventsParamsUnion{{
OfUserMessage: &anthropic.BetaManagedAgentsUserMessageEventParams{
Type: anthropic.BetaManagedAgentsUserMessageEventParamsTypeUserMessage,
Content: []anthropic.BetaManagedAgentsUserMessageEventParamsContentUnion{{
OfText: &anthropic.BetaManagedAgentsTextBlockParam{
Type: anthropic.BetaManagedAgentsTextBlockTypeText,
Text: "Review the auth module",
},
}},
},
}},
})
if err != nil {
panic(err)
}
```
> 💡 **Stream-first:** Open the stream *before* (or concurrently with) sending the message. The stream only delivers events that occur after it opens — stream-after-send means early events arrive buffered in one batch. See [Steering Patterns](../../shared/managed-agents-events.md#steering-patterns).
---
## Stream Events (SSE)
```go
// Open the stream first, then send the user message
stream := client.Beta.Sessions.Events.StreamEvents(ctx, session.ID, anthropic.BetaSessionEventStreamParams{})
defer stream.Close()
if _, err := client.Beta.Sessions.Events.Send(ctx, session.ID, anthropic.BetaSessionEventSendParams{
Events: []anthropic.SendEventsParamsUnion{{
OfUserMessage: &anthropic.BetaManagedAgentsUserMessageEventParams{
Type: anthropic.BetaManagedAgentsUserMessageEventParamsTypeUserMessage,
Content: []anthropic.BetaManagedAgentsUserMessageEventParamsContentUnion{{
OfText: &anthropic.BetaManagedAgentsTextBlockParam{
Type: anthropic.BetaManagedAgentsTextBlockTypeText,
Text: "Summarize the repo README",
},
}},
},
}},
}); err != nil {
panic(err)
}
events:
for stream.Next() {
switch event := stream.Current().AsAny().(type) {
case anthropic.BetaManagedAgentsAgentMessageEvent:
for _, block := range event.Content {
fmt.Print(block.Text)
}
case anthropic.BetaManagedAgentsAgentToolUseEvent:
fmt.Printf("\n[Using tool: %s]\n", event.Name)
case anthropic.BetaManagedAgentsSessionStatusIdleEvent:
break events
case anthropic.BetaManagedAgentsSessionErrorEvent:
fmt.Printf("\n[Error: %s]\n", event.Error.Message)
break events
}
}
if err := stream.Err(); err != nil {
panic(err)
}
```
### Reconnecting and Tailing
When reconnecting mid-session, list past events first to dedupe, then tail live events:
```go
stream := client.Beta.Sessions.Events.StreamEvents(ctx, session.ID, anthropic.BetaSessionEventStreamParams{})
defer stream.Close()
// Stream is open and buffering. List history before tailing live.
seenEventIDs := map[string]struct{}{}
history := client.Beta.Sessions.Events.ListAutoPaging(ctx, session.ID, anthropic.BetaSessionEventListParams{})
for history.Next() {
seenEventIDs[history.Current().ID] = struct{}{}
}
if err := history.Err(); err != nil {
panic(err)
}
// Tail live events, skipping anything already seen
tail:
for stream.Next() {
event := stream.Current()
if _, seen := seenEventIDs[event.ID]; seen {
continue
}
seenEventIDs[event.ID] = struct{}{}
switch event := event.AsAny().(type) {
case anthropic.BetaManagedAgentsAgentMessageEvent:
for _, block := range event.Content {
fmt.Print(block.Text)
}
case anthropic.BetaManagedAgentsSessionStatusIdleEvent:
break tail
}
}
if err := stream.Err(); err != nil {
panic(err)
}
```
---
## Provide Custom Tool Result
> ℹ️ The Go managed-agents bindings for `user.custom_tool_result` are not yet documented in this skill or in the apps source examples. Refer to `shared/managed-agents-events.md` for the wire format and the `github.com/anthropics/anthropic-sdk-go` repository for the corresponding Go params types.
---
## Poll Events
```go
// Auto-paginating iterator
iter := client.Beta.Sessions.Events.ListAutoPaging(ctx, session.ID, anthropic.BetaSessionEventListParams{})
for iter.Next() {
event := iter.Current()
fmt.Printf("%s: %s\n", event.Type, event.ID)
}
if err := iter.Err(); err != nil {
panic(err)
}
```
---
## Upload a File
```go
csvFile, err := os.Open("data.csv")
if err != nil {
panic(err)
}
defer csvFile.Close()
file, err := client.Beta.Files.Upload(ctx, anthropic.BetaFileUploadParams{
File: csvFile,
})
if err != nil {
panic(err)
}
fmt.Printf("File ID: %s\n", file.ID)
// Mount in a session
session, err := client.Beta.Sessions.New(ctx, anthropic.BetaSessionNewParams{
Agent: anthropic.BetaSessionNewParamsAgentUnion{
OfString: anthropic.String(agent.ID),
},
EnvironmentID: environment.ID,
Resources: []anthropic.BetaSessionNewParamsResourceUnion{{
OfFile: &anthropic.BetaManagedAgentsFileResourceParams{
Type: anthropic.BetaManagedAgentsFileResourceParamsTypeFile,
FileID: file.ID,
MountPath: anthropic.String("/workspace/data.csv"),
},
}},
})
if err != nil {
panic(err)
}
```
### Add and Manage Resources on an Existing Session
```go
// Attach an additional file to an open session
resource, err := client.Beta.Sessions.Resources.Add(ctx, session.ID, anthropic.BetaSessionResourceAddParams{
BetaManagedAgentsFileResourceParams: anthropic.BetaManagedAgentsFileResourceParams{
Type: anthropic.BetaManagedAgentsFileResourceParamsTypeFile,
FileID: file.ID,
},
})
if err != nil {
panic(err)
}
fmt.Println(resource.ID) // "sesrsc_01ABC..."
// List resources on the session
listed, err := client.Beta.Sessions.Resources.List(ctx, session.ID, anthropic.BetaSessionResourceListParams{})
if err != nil {
panic(err)
}
for _, entry := range listed.Data {
fmt.Println(entry.ID, entry.Type)
}
// Detach a resource
if _, err := client.Beta.Sessions.Resources.Delete(ctx, resource.ID, anthropic.BetaSessionResourceDeleteParams{
SessionID: session.ID,
}); err != nil {
panic(err)
}
```
---
## List and Download Session Files
> ℹ️ Listing and downloading files an agent wrote during a session is not yet documented for Go in this skill or in the apps source examples. See `shared/managed-agents-events.md` and the `github.com/anthropics/anthropic-sdk-go` repository for the `Beta.Files.List` and `Beta.Files.Download` Go params types.
---
## Session Management
```go
// List environments
environments, err := client.Beta.Environments.List(ctx, anthropic.BetaEnvironmentListParams{})
if err != nil {
panic(err)
}
// Retrieve a specific environment
env, err := client.Beta.Environments.Get(ctx, environment.ID, anthropic.BetaEnvironmentGetParams{})
if err != nil {
panic(err)
}
// Archive an environment (read-only, existing sessions continue)
_, err = client.Beta.Environments.Archive(ctx, environment.ID, anthropic.BetaEnvironmentArchiveParams{})
if err != nil {
panic(err)
}
// Delete an environment (only if no sessions reference it)
_, err = client.Beta.Environments.Delete(ctx, environment.ID, anthropic.BetaEnvironmentDeleteParams{})
if err != nil {
panic(err)
}
// Delete a session
_, err = client.Beta.Sessions.Delete(ctx, session.ID, anthropic.BetaSessionDeleteParams{})
if err != nil {
panic(err)
}
```
---
## MCP Server Integration
```go
// Agent declares MCP server (no auth here — auth goes in a vault)
agent, err := client.Beta.Agents.New(ctx, anthropic.BetaAgentNewParams{
Name: "GitHub Assistant",
Model: anthropic.BetaManagedAgentsModelConfigParams{
ID: "claude-opus-4-7",
Type: anthropic.BetaManagedAgentsModelConfigParamsTypeModelConfig,
},
MCPServers: []anthropic.BetaManagedAgentsUrlmcpServerParams{{
Type: anthropic.BetaManagedAgentsUrlmcpServerParamsTypeURL,
Name: "github",
URL: "https://api.githubcopilot.com/mcp/",
}},
Tools: []anthropic.BetaAgentNewParamsToolUnion{
{
OfAgentToolset20260401: &anthropic.BetaManagedAgentsAgentToolset20260401Params{
Type: anthropic.BetaManagedAgentsAgentToolset20260401ParamsTypeAgentToolset20260401,
},
},
{
OfMCPToolset: &anthropic.BetaManagedAgentsMCPToolsetParams{
Type: anthropic.BetaManagedAgentsMCPToolsetParamsTypeMCPToolset,
MCPServerName: "github",
},
},
},
})
if err != nil {
panic(err)
}
// Session attaches vault(s) containing credentials for those MCP server URLs
session, err := client.Beta.Sessions.New(ctx, anthropic.BetaSessionNewParams{
Agent: anthropic.BetaSessionNewParamsAgentUnion{
OfBetaManagedAgentsAgents: &anthropic.BetaManagedAgentsAgentParams{
Type: anthropic.BetaManagedAgentsAgentParamsTypeAgent,
ID: agent.ID,
Version: anthropic.Int(agent.Version),
},
},
EnvironmentID: environment.ID,
VaultIDs: []string{vault.ID},
})
if err != nil {
panic(err)
}
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
---
## Vaults
```go
// Create a vault
vault, err := client.Beta.Vaults.New(ctx, anthropic.BetaVaultNewParams{
DisplayName: "Alice",
Metadata: map[string]string{"external_user_id": "usr_abc123"},
})
if err != nil {
panic(err)
}
// Add an OAuth credential
credential, err := client.Beta.Vaults.Credentials.New(ctx, vault.ID, anthropic.BetaVaultCredentialNewParams{
DisplayName: anthropic.String("Alice's Slack"),
Auth: anthropic.BetaVaultCredentialNewParamsAuthUnion{
OfMCPOAuth: &anthropic.BetaManagedAgentsMCPOAuthCreateParams{
Type: anthropic.BetaManagedAgentsMCPOAuthCreateParamsTypeMCPOAuth,
MCPServerURL: "https://mcp.slack.com/mcp",
AccessToken: "xoxp-...",
ExpiresAt: anthropic.Time(time.Date(2026, time.April, 15, 0, 0, 0, 0, time.UTC)),
Refresh: anthropic.BetaManagedAgentsMCPOAuthRefreshParams{
TokenEndpoint: "https://slack.com/api/oauth.v2.access",
ClientID: "1234567890.0987654321",
Scope: anthropic.String("channels:read chat:write"),
RefreshToken: "xoxe-1-...",
TokenEndpointAuth: anthropic.BetaManagedAgentsMCPOAuthRefreshParamsTokenEndpointAuthUnion{
OfClientSecretPost: &anthropic.BetaManagedAgentsTokenEndpointAuthPostParam{
Type: anthropic.BetaManagedAgentsTokenEndpointAuthPostParamTypeClientSecretPost,
ClientSecret: "abc123...",
},
},
},
},
},
})
if err != nil {
panic(err)
}
// Rotate the credential (e.g., after a token refresh)
_, err = client.Beta.Vaults.Credentials.Update(ctx, credential.ID, anthropic.BetaVaultCredentialUpdateParams{
VaultID: vault.ID,
Auth: anthropic.BetaVaultCredentialUpdateParamsAuthUnion{
OfMCPOAuth: &anthropic.BetaManagedAgentsMCPOAuthUpdateParams{
Type: anthropic.BetaManagedAgentsMCPOAuthUpdateParamsTypeMCPOAuth,
AccessToken: anthropic.String("xoxp-new-..."),
ExpiresAt: anthropic.Time(time.Date(2026, time.May, 15, 0, 0, 0, 0, time.UTC)),
Refresh: anthropic.BetaManagedAgentsMCPOAuthRefreshUpdateParams{
RefreshToken: anthropic.String("xoxe-1-new-..."),
},
},
},
})
if err != nil {
panic(err)
}
// Archive a vault
_, err = client.Beta.Vaults.Archive(ctx, vault.ID, anthropic.BetaVaultArchiveParams{})
if err != nil {
panic(err)
}
```
---
## GitHub Repository Integration
Mount a GitHub repository as a session resource (a vault holds the GitHub MCP credential):
```go
session, err := client.Beta.Sessions.New(ctx, anthropic.BetaSessionNewParams{
Agent: anthropic.BetaSessionNewParamsAgentUnion{OfString: anthropic.String(agent.ID)},
EnvironmentID: environment.ID,
VaultIDs: []string{vault.ID},
Resources: []anthropic.BetaSessionNewParamsResourceUnion{
{
OfGitHubRepository: &anthropic.BetaManagedAgentsGitHubRepositoryResourceParams{
Type: anthropic.BetaManagedAgentsGitHubRepositoryResourceParamsTypeGitHubRepository,
URL: "https://github.com/org/repo",
MountPath: anthropic.String("/workspace/repo"),
AuthorizationToken: "ghp_your_github_token",
},
},
},
})
if err != nil {
panic(err)
}
```
Multiple repositories on the same session:
```go
resources := []anthropic.BetaSessionNewParamsResourceUnion{
{
OfGitHubRepository: &anthropic.BetaManagedAgentsGitHubRepositoryResourceParams{
Type: anthropic.BetaManagedAgentsGitHubRepositoryResourceParamsTypeGitHubRepository,
URL: "https://github.com/org/frontend",
MountPath: anthropic.String("/workspace/frontend"),
AuthorizationToken: "ghp_your_github_token",
},
},
{
OfGitHubRepository: &anthropic.BetaManagedAgentsGitHubRepositoryResourceParams{
Type: anthropic.BetaManagedAgentsGitHubRepositoryResourceParamsTypeGitHubRepository,
URL: "https://github.com/org/backend",
MountPath: anthropic.String("/workspace/backend"),
AuthorizationToken: "ghp_your_github_token",
},
},
}
```
Rotating a repository's authorization token:
```go
listed, err := client.Beta.Sessions.Resources.List(ctx, session.ID, anthropic.BetaSessionResourceListParams{})
if err != nil {
panic(err)
}
repoResourceID := listed.Data[0].ID
_, err = client.Beta.Sessions.Resources.Update(ctx, repoResourceID, anthropic.BetaSessionResourceUpdateParams{
SessionID: session.ID,
AuthorizationToken: "ghp_your_new_github_token",
})
if err != nil {
panic(err)
}
```
resources/bundled/skills/claude-api/java/claude-api.md
+432
View File
@@ -0,0 +1,432 @@
# Claude API — Java
> **Note:** The Java SDK supports the Claude API and beta tool use with annotated classes. Agent SDK is not yet available for Java.
## Installation
Maven:
```xml
<dependency>
<groupId>com.anthropic</groupId>
<artifactId>anthropic-java</artifactId>
<version>2.17.0</version>
</dependency>
```
Gradle:
```groovy
implementation("com.anthropic:anthropic-java:2.17.0")
```
## Client Initialization
```java
import com.anthropic.client.AnthropicClient;
import com.anthropic.client.okhttp.AnthropicOkHttpClient;
// Default (reads ANTHROPIC_API_KEY from environment)
AnthropicClient client = AnthropicOkHttpClient.fromEnv();
// Explicit API key
AnthropicClient client = AnthropicOkHttpClient.builder()
.apiKey("your-api-key")
.build();
```
---
## Basic Message Request
```java
import com.anthropic.models.messages.MessageCreateParams;
import com.anthropic.models.messages.Message;
import com.anthropic.models.messages.Model;
MessageCreateParams params = MessageCreateParams.builder()
.model(Model.CLAUDE_OPUS_4_6)
.maxTokens(16000L)
.addUserMessage("What is the capital of France?")
.build();
Message response = client.messages().create(params);
response.content().stream()
.flatMap(block -> block.text().stream())
.forEach(textBlock -> System.out.println(textBlock.text()));
```
---
## Streaming
```java
import com.anthropic.core.http.StreamResponse;
import com.anthropic.models.messages.RawMessageStreamEvent;
MessageCreateParams params = MessageCreateParams.builder()
.model(Model.CLAUDE_OPUS_4_6)
.maxTokens(64000L)
.addUserMessage("Write a haiku")
.build();
try (StreamResponse<RawMessageStreamEvent> streamResponse = client.messages().createStreaming(params)) {
streamResponse.stream()
.flatMap(event -> event.contentBlockDelta().stream())
.flatMap(deltaEvent -> deltaEvent.delta().text().stream())
.forEach(textDelta -> System.out.print(textDelta.text()));
}
```
---
## Thinking
**Adaptive thinking is the recommended mode for Claude 4.6+ models.** Claude decides dynamically when and how much to think. The builder has a direct `.thinking(ThinkingConfigAdaptive)` overload — no manual union wrapping.
```java
import com.anthropic.models.messages.ContentBlock;
import com.anthropic.models.messages.MessageCreateParams;
import com.anthropic.models.messages.Model;
import com.anthropic.models.messages.ThinkingConfigAdaptive;
MessageCreateParams params = MessageCreateParams.builder()
.model(Model.CLAUDE_SONNET_4_6)
.maxTokens(16000L)
.thinking(ThinkingConfigAdaptive.builder().build())
.addUserMessage("Solve this step by step: 27 * 453")
.build();
for (ContentBlock block : client.messages().create(params).content()) {
block.thinking().ifPresent(t -> System.out.println("[thinking] " + t.thinking()));
block.text().ifPresent(t -> System.out.println(t.text()));
}
```
> **Deprecated:** `ThinkingConfigEnabled.builder().budgetTokens(N)` (and the `.enabledThinking(N)` shortcut) still works on Claude 4.6 but is deprecated. Use adaptive thinking above.
`ContentBlock` narrowing: `.thinking()` / `.text()` return `Optional<T>` — use `.ifPresent(...)` or `.stream().flatMap(...)`. Alternative: `isThinking()` / `asThinking()` boolean+unwrap pairs (throws on wrong variant).
---
## Tool Use (Beta)
The Java SDK supports beta tool use with annotated classes. Tool classes implement `Supplier<String>` for automatic execution via `BetaToolRunner`.
### Tool Runner (automatic loop)
```java
import com.anthropic.models.beta.messages.MessageCreateParams;
import com.anthropic.models.beta.messages.BetaMessage;
import com.anthropic.helpers.BetaToolRunner;
import com.fasterxml.jackson.annotation.JsonClassDescription;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import java.util.function.Supplier;
@JsonClassDescription("Get the weather in a given location")
static class GetWeather implements Supplier<String> {
@JsonPropertyDescription("The city and state, e.g. San Francisco, CA")
public String location;
@Override
public String get() {
return "The weather in " + location + " is sunny and 72°F";
}
}
BetaToolRunner toolRunner = client.beta().messages().toolRunner(
MessageCreateParams.builder()
.model("claude-opus-4-7")
.maxTokens(16000L)
.putAdditionalHeader("anthropic-beta", "structured-outputs-2025-11-13")
.addTool(GetWeather.class)
.addUserMessage("What's the weather in San Francisco?")
.build());
for (BetaMessage message : toolRunner) {
System.out.println(message);
}
```
### Memory Tool
The Java SDK provides `BetaMemoryToolHandler` for implementing the memory tool backend. You supply a handler that manages file storage, and the `BetaToolRunner` handles memory tool calls automatically.
```java
import com.anthropic.helpers.BetaMemoryToolHandler;
import com.anthropic.helpers.BetaToolRunner;
import com.anthropic.models.beta.messages.BetaMemoryTool20250818;
import com.anthropic.models.beta.messages.BetaMessage;
import com.anthropic.models.beta.messages.MessageCreateParams;
import com.anthropic.models.beta.messages.ToolRunnerCreateParams;
// Implement BetaMemoryToolHandler with your storage backend (e.g., filesystem)
BetaMemoryToolHandler memoryHandler = new FileSystemMemoryToolHandler(sandboxRoot);
MessageCreateParams createParams = MessageCreateParams.builder()
.model("claude-opus-4-7")
.maxTokens(4096L)
.addTool(BetaMemoryTool20250818.builder().build())
.addUserMessage("Remember that my favorite color is blue")
.build();
BetaToolRunner toolRunner = client.beta().messages().toolRunner(
ToolRunnerCreateParams.builder()
.betaMemoryToolHandler(memoryHandler)
.initialMessageParams(createParams)
.build());
for (BetaMessage message : toolRunner) {
System.out.println(message);
}
```
See the [shared memory tool concepts](../shared/tool-use-concepts.md) for more details on the memory tool.
### Non-Beta Tool Declaration (manual JSON schema)
`Tool.InputSchema.Properties` is a freeform `Map<String, JsonValue>` wrapper — build property schemas via `putAdditionalProperty`. `type: "object"` is the default. The builder has a direct `.addTool(Tool)` overload that wraps in `ToolUnion` automatically.
```java
import com.anthropic.core.JsonValue;
import com.anthropic.models.messages.Tool;
Tool tool = Tool.builder()
.name("get_weather")
.description("Get the current weather in a given location")
.inputSchema(Tool.InputSchema.builder()
.properties(Tool.InputSchema.Properties.builder()
.putAdditionalProperty("location", JsonValue.from(Map.of("type", "string")))
.build())
.required(List.of("location"))
.build())
.build();
MessageCreateParams params = MessageCreateParams.builder()
.model(Model.CLAUDE_SONNET_4_6)
.maxTokens(16000L)
.addTool(tool)
.addUserMessage("Weather in Paris?")
.build();
```
For manual tool loops, handle `tool_use` blocks in the response, send `tool_result` back, loop until `stop_reason` is `"end_turn"`. See [shared tool use concepts](../shared/tool-use-concepts.md).
### Building `MessageParam` with Content Blocks (Tool Result Round-Trip)
`MessageParam.Content` is an inner union class (string | list). Use the builder's `.contentOfBlockParams(List<ContentBlockParam>)` alias — there is NO separate `MessageParamContent` class with a static `ofBlockParams`:
```java
import com.anthropic.models.messages.MessageParam;
import com.anthropic.models.messages.ContentBlockParam;
import com.anthropic.models.messages.ToolResultBlockParam;
List<ContentBlockParam> results = List.of(
ContentBlockParam.ofToolResult(ToolResultBlockParam.builder()
.toolUseId(toolUseBlock.id())
.content(yourResultString)
.build())
);
MessageParam toolResultMsg = MessageParam.builder()
.role(MessageParam.Role.USER)
.contentOfBlockParams(results) // builder alias for Content.ofBlockParams(...)
.build();
```
---
## Effort Parameter
Effort is nested inside `OutputConfig` — there is NO `.effort()` directly on `MessageCreateParams.Builder`.
```java
import com.anthropic.models.messages.OutputConfig;
.outputConfig(OutputConfig.builder()
.effort(OutputConfig.Effort.HIGH) // or LOW, MEDIUM, MAX
.build())
```
Combine with `Thinking = ThinkingConfigAdaptive` for cost-quality control.
---
## Prompt Caching
System message as a list of `TextBlockParam` with `CacheControlEphemeral`. Use `.systemOfTextBlockParams(...)` — the plain `.system(String)` overload can't carry cache control. For placement patterns and the silent-invalidator audit checklist, see `shared/prompt-caching.md`.
```java
import com.anthropic.models.messages.TextBlockParam;
import com.anthropic.models.messages.CacheControlEphemeral;
.systemOfTextBlockParams(List.of(
TextBlockParam.builder()
.text(longSystemPrompt)
.cacheControl(CacheControlEphemeral.builder()
.ttl(CacheControlEphemeral.Ttl.TTL_1H) // optional; also TTL_5M
.build())
.build()))
```
There's also a top-level `.cacheControl(CacheControlEphemeral)` on `MessageCreateParams.Builder` and on `Tool.builder()`.
Verify hits via `response.usage().cacheCreationInputTokens()` / `response.usage().cacheReadInputTokens()`.
---
## Token Counting
```java
import com.anthropic.models.messages.MessageCountTokensParams;
long tokens = client.messages().countTokens(
MessageCountTokensParams.builder()
.model(Model.CLAUDE_SONNET_4_6)
.addUserMessage("Hello")
.build()
).inputTokens();
```
---
## Structured Output
The class-based overload auto-derives the JSON schema from your POJO and gives you a typed `.text()` return — no manual schema, no manual parsing.
```java
import com.anthropic.models.messages.StructuredMessageCreateParams;
record Book(String title, String author) {}
record BookList(List<Book> books) {}
StructuredMessageCreateParams<BookList> params = MessageCreateParams.builder()
.model(Model.CLAUDE_SONNET_4_6)
.maxTokens(16000L)
.outputConfig(BookList.class) // returns a typed builder
.addUserMessage("List 3 classic novels")
.build();
client.messages().create(params).content().stream()
.flatMap(cb -> cb.text().stream())
.forEach(typed -> {
// typed.text() returns BookList, not String
for (Book b : typed.text().books()) System.out.println(b.title());
});
```
Supports Jackson annotations: `@JsonPropertyDescription`, `@JsonIgnore`, `@ArraySchema(minItems=...)`. Manual schema path: `OutputConfig.builder().format(JsonOutputFormat.builder().schema(...).build())`.
---
## PDF / Document Input
`DocumentBlockParam` builder has source shortcuts. Wrap in `ContentBlockParam.ofDocument()` and pass via `.addUserMessageOfBlockParams()`.
```java
import com.anthropic.models.messages.DocumentBlockParam;
import com.anthropic.models.messages.ContentBlockParam;
import com.anthropic.models.messages.TextBlockParam;
DocumentBlockParam doc = DocumentBlockParam.builder()
.base64Source(base64String) // or .urlSource("https://...") or .textSource("...")
.title("My Document") // optional
.build();
.addUserMessageOfBlockParams(List.of(
ContentBlockParam.ofDocument(doc),
ContentBlockParam.ofText(TextBlockParam.builder().text("Summarize this").build())))
```
---
## Server-Side Tools
Version-suffixed types; `name`/`type` auto-set by builder. Direct `.addTool()` overloads exist for every type — no manual `ToolUnion` wrapping.
```java
import com.anthropic.models.messages.WebSearchTool20260209;
import com.anthropic.models.messages.ToolBash20250124;
import com.anthropic.models.messages.ToolTextEditor20250728;
import com.anthropic.models.messages.CodeExecutionTool20260120;
.addTool(WebSearchTool20260209.builder()
.maxUses(5L) // optional
.allowedDomains(List.of("example.com")) // optional
.build())
.addTool(ToolBash20250124.builder().build())
.addTool(ToolTextEditor20250728.builder().build())
.addTool(CodeExecutionTool20260120.builder().build())
```
Also available: `WebFetchTool20260209`, `MemoryTool20250818`, `ToolSearchToolBm25_20251119`.
### Beta namespace (MCP, compaction)
For beta-only features use `com.anthropic.models.beta.messages.*` — class names have a `Beta` prefix AND live in the beta package. The beta `MessageCreateParams.Builder` has direct `.addTool(BetaToolBash20250124)` overloads AND `.addMcpServer()`:
```java
import com.anthropic.models.beta.messages.MessageCreateParams;
import com.anthropic.models.beta.messages.BetaToolBash20250124;
import com.anthropic.models.beta.messages.BetaCodeExecutionTool20260120;
import com.anthropic.models.beta.messages.BetaRequestMcpServerUrlDefinition;
MessageCreateParams params = MessageCreateParams.builder()
.model(Model.CLAUDE_OPUS_4_6)
.maxTokens(16000L)
.addBeta("mcp-client-2025-11-20")
.addTool(BetaToolBash20250124.builder().build())
.addTool(BetaCodeExecutionTool20260120.builder().build())
.addMcpServer(BetaRequestMcpServerUrlDefinition.builder()
.name("my-server")
.url("https://example.com/mcp")
.build())
.addUserMessage("...")
.build();
client.beta().messages().create(params);
```
`BetaTool*` types are NOT interchangeable with non-beta `Tool*` — pick one namespace per request.
**Reading server-tool blocks in the response:** `ServerToolUseBlock` has `.id()`, `.name()` (enum), and `._input()` returning raw `JsonValue` — there is NO typed `.input()`. For code execution results, unwrap two levels:
```java
for (ContentBlock block : response.content()) {
block.serverToolUse().ifPresent(stu -> {
System.out.println("tool: " + stu.name() + " input: " + stu._input());
});
block.codeExecutionToolResult().ifPresent(r -> {
r.content().resultBlock().ifPresent(result -> {
System.out.println("stdout: " + result.stdout());
System.out.println("stderr: " + result.stderr());
System.out.println("exit: " + result.returnCode());
});
});
}
```
---
## Files API (Beta)
Under `client.beta().files()`. File references in messages need the beta message types (non-beta `DocumentBlockParam.Source` has no file-ID variant).
```java
import com.anthropic.models.beta.files.FileUploadParams;
import com.anthropic.models.beta.files.FileMetadata;
import com.anthropic.models.beta.messages.BetaRequestDocumentBlock;
import java.nio.file.Paths;
FileMetadata meta = client.beta().files().upload(
FileUploadParams.builder()
.file(Paths.get("/path/to/doc.pdf")) // or .file(InputStream) or .file(byte[])
.build());
// Reference in a beta message:
BetaRequestDocumentBlock doc = BetaRequestDocumentBlock.builder()
.fileSource(meta.id())
.build();
```
Other methods: `.list()`, `.delete(String fileId)`, `.download(String fileId)`, `.retrieveMetadata(String fileId)`.
resources/bundled/skills/claude-api/java/managed-agents/README.md
+442
View File
@@ -0,0 +1,442 @@
# Managed Agents — Java
> **Bindings not shown here:** This README covers the most common managed-agents flows for Java. If you need a class, method, namespace, field, or behavior that isn't shown, WebFetch the Java SDK repo **or the relevant docs page** from `shared/live-sources.md` rather than guess. Do not extrapolate from cURL shapes or another language's SDK.
> **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `client.beta().agents().create` and pass it to every subsequent `client.beta().sessions().create`; do not call `agents().create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. The examples below show in-code creation for completeness; in production the create call belongs in setup, not in the request path.
## Installation
```xml
<dependency>
<groupId>com.anthropic</groupId>
<artifactId>anthropic-java</artifactId>
</dependency>
```
## Client Initialization
```java
import com.anthropic.client.okhttp.AnthropicOkHttpClient;
// Default (uses ANTHROPIC_API_KEY env var)
var client = AnthropicOkHttpClient.fromEnv();
```
---
## Create an Environment
```java
import com.anthropic.models.beta.environments.BetaCloudConfigParams;
import com.anthropic.models.beta.environments.EnvironmentCreateParams;
import com.anthropic.models.beta.environments.UnrestrictedNetwork;
var environment = client.beta().environments().create(EnvironmentCreateParams.builder()
.name("my-dev-env")
.config(BetaCloudConfigParams.builder()
.networking(UnrestrictedNetwork.builder().build())
.build())
.build());
System.out.println("Environment ID: " + environment.id()); // env_...
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** Model, system, and tools live on the agent object, not the session. Always start with `client.beta().agents().create()` — the session takes either `.agent(agent.id())` or the typed `BetaManagedAgentsAgentParams.builder()...build()`.
### Minimal
```java
import com.anthropic.models.beta.agents.AgentCreateParams;
import com.anthropic.models.beta.agents.BetaManagedAgentsAgentToolset20260401Params;
import com.anthropic.models.beta.sessions.BetaManagedAgentsAgentParams;
import com.anthropic.models.beta.sessions.SessionCreateParams;
// 1. Create the agent (reusable, versioned)
var agent = client.beta().agents().create(AgentCreateParams.builder()
.name("Coding Assistant")
.model("claude-opus-4-7")
.system("You are a helpful coding assistant.")
.addTool(BetaManagedAgentsAgentToolset20260401Params.builder()
.type(BetaManagedAgentsAgentToolset20260401Params.Type.AGENT_TOOLSET_20260401)
.build())
.build());
// 2. Start a session
var session = client.beta().sessions().create(SessionCreateParams.builder()
.agent(BetaManagedAgentsAgentParams.builder()
.type(BetaManagedAgentsAgentParams.Type.AGENT)
.id(agent.id())
.version(agent.version())
.build())
.environmentId(environment.id())
.title("Quickstart session")
.build());
System.out.println("Session ID: " + session.id());
```
### Updating an Agent
Updates create new versions; the agent object is immutable per version.
```java
import com.anthropic.models.beta.agents.AgentUpdateParams;
var updatedAgent = client.beta().agents().update(agent.id(), AgentUpdateParams.builder()
.version(agent.version())
.system("You are a helpful coding agent. Always write tests.")
.build());
System.out.println("New version: " + updatedAgent.version());
// List all versions
for (var version : client.beta().agents().versions().list(agent.id()).autoPager()) {
System.out.println("Version " + version.version() + ": " + version.updatedAt());
}
// Archive the agent
var archived = client.beta().agents().archive(agent.id());
System.out.println("Archived at: " + archived.archivedAt().orElseThrow());
```
---
## Send a User Message
```java
import com.anthropic.models.beta.sessions.events.BetaManagedAgentsUserMessageEventParams;
import com.anthropic.models.beta.sessions.events.EventSendParams;
client.beta().sessions().events().send(session.id(), EventSendParams.builder()
.addEvent(BetaManagedAgentsUserMessageEventParams.builder()
.type(BetaManagedAgentsUserMessageEventParams.Type.USER_MESSAGE)
.addTextContent("Review the auth module")
.build())
.build());
```
> 💡 **Stream-first:** Open the stream *before* (or concurrently with) sending the message. The stream only delivers events that occur after it opens — stream-after-send means early events arrive buffered in one batch. See [Steering Patterns](../../shared/managed-agents-events.md#steering-patterns).
---
## Stream Events (SSE)
```java
import com.anthropic.models.beta.sessions.events.StreamEvents;
// Open the stream first, then send the user message
try (var stream = client.beta().sessions().events().streamStreaming(session.id())) {
client.beta().sessions().events().send(session.id(), EventSendParams.builder()
.addEvent(BetaManagedAgentsUserMessageEventParams.builder()
.type(BetaManagedAgentsUserMessageEventParams.Type.USER_MESSAGE)
.addTextContent("Summarize the repo README")
.build())
.build());
for (var event : (Iterable<StreamEvents>) stream.stream()::iterator) {
if (event.isAgentMessage()) {
event.asAgentMessage().content().forEach(block -> System.out.print(block.text()));
} else if (event.isAgentToolUse()) {
System.out.println("\n[Using tool: " + event.asAgentToolUse().name() + "]");
} else if (event.isSessionStatusIdle()) {
break;
} else if (event.isSessionError()) {
System.out.println("\n[Error]");
break;
}
}
}
```
### Reconnecting and Tailing
When reconnecting mid-session, list past events first to dedupe, then tail live events. The cross-variant `id` field is read from the raw `_json()` value:
```java
import com.anthropic.core.JsonValue;
import java.util.HashSet;
import java.util.Map;
import java.util.Optional;
try (var stream = client.beta().sessions().events().streamStreaming(session.id())) {
// Stream is open and buffering. List history before tailing live.
var seenEventIds = new HashSet<String>();
for (var past : client.beta().sessions().events().list(session.id()).autoPager()) {
Optional<Map<String, JsonValue>> obj = past._json().orElseThrow().asObject();
seenEventIds.add(obj.orElseThrow().get("id").asStringOrThrow());
}
// Tail live events, skipping anything already seen
for (var event : (Iterable<StreamEvents>) stream.stream()::iterator) {
Optional<Map<String, JsonValue>> obj = event._json().orElseThrow().asObject();
if (!seenEventIds.add(obj.orElseThrow().get("id").asStringOrThrow())) continue;
if (event.isAgentMessage()) {
event.asAgentMessage().content().forEach(block -> System.out.print(block.text()));
} else if (event.isSessionStatusIdle()) {
break;
}
}
}
```
---
## Provide Custom Tool Result
> ℹ️ The Java managed-agents bindings for `user.custom_tool_result` are not yet documented in this skill or in the apps source examples. Refer to `shared/managed-agents-events.md` for the wire format and the `anthropic-java` repository for the corresponding params types.
---
## Poll Events
```java
for (var event : client.beta().sessions().events().list(session.id()).autoPager()) {
System.out.println(event.type() + ": " + event);
}
```
---
## Upload a File
```java
import com.anthropic.models.beta.files.FileUploadParams;
import com.anthropic.models.beta.sessions.BetaManagedAgentsFileResourceParams;
import java.nio.file.Path;
var dataCsv = Path.of("data.csv");
var file = client.beta().files().upload(FileUploadParams.builder()
.file(dataCsv)
.build());
System.out.println("File ID: " + file.id());
// Mount in a session
var session = client.beta().sessions().create(SessionCreateParams.builder()
.agent(agent.id())
.environmentId(environment.id())
.addResource(BetaManagedAgentsFileResourceParams.builder()
.type(BetaManagedAgentsFileResourceParams.Type.FILE)
.fileId(file.id())
.mountPath("/workspace/data.csv")
.build())
.build());
```
### Add and Manage Resources on an Existing Session
```java
import com.anthropic.models.beta.sessions.resources.ResourceAddParams;
import com.anthropic.models.beta.sessions.resources.ResourceDeleteParams;
// Attach an additional file to an open session
var resource = client.beta().sessions().resources().add(session.id(), ResourceAddParams.builder()
.betaManagedAgentsFileResourceParams(BetaManagedAgentsFileResourceParams.builder()
.type(BetaManagedAgentsFileResourceParams.Type.FILE)
.fileId(file.id())
.build())
.build());
System.out.println(resource.id()); // "sesrsc_01ABC..."
// List resources on the session — entries are a discriminated union
var listed = client.beta().sessions().resources().list(session.id());
for (var entry : listed.data()) {
if (entry.isFile()) {
var fileResource = entry.asFile();
System.out.println(fileResource.id() + " " + fileResource.type());
} else if (entry.isGitHubRepository()) {
var repoResource = entry.asGitHubRepository();
System.out.println(repoResource.id() + " " + repoResource.type());
}
}
// Detach a resource
client.beta().sessions().resources().delete(resource.id(), ResourceDeleteParams.builder()
.sessionId(session.id())
.build());
```
---
## List and Download Session Files
> ℹ️ Listing and downloading files an agent wrote during a session is not yet documented for Java in this skill or in the apps source examples. See `shared/managed-agents-events.md` and the `anthropic-java` repository for the file list/download bindings.
---
## Session Management
```java
// List environments
var environments = client.beta().environments().list();
// Retrieve a specific environment
var env = client.beta().environments().retrieve(environment.id());
// Archive an environment (read-only, existing sessions continue)
client.beta().environments().archive(environment.id());
// Delete an environment (only if no sessions reference it)
client.beta().environments().delete(environment.id());
// Delete a session
client.beta().sessions().delete(session.id());
```
---
## MCP Server Integration
```java
import com.anthropic.models.beta.agents.BetaManagedAgentsMcpToolsetParams;
import com.anthropic.models.beta.agents.BetaManagedAgentsUrlmcpServerParams;
// Agent declares MCP server (no auth here — auth goes in a vault)
var agent = client.beta().agents().create(AgentCreateParams.builder()
.name("GitHub Assistant")
.model("claude-opus-4-7")
.addMcpServer(BetaManagedAgentsUrlmcpServerParams.builder()
.type(BetaManagedAgentsUrlmcpServerParams.Type.URL)
.name("github")
.url("https://api.githubcopilot.com/mcp/")
.build())
.addTool(BetaManagedAgentsAgentToolset20260401Params.builder()
.type(BetaManagedAgentsAgentToolset20260401Params.Type.AGENT_TOOLSET_20260401)
.build())
.addTool(BetaManagedAgentsMcpToolsetParams.builder()
.type(BetaManagedAgentsMcpToolsetParams.Type.MCP_TOOLSET)
.mcpServerName("github")
.build())
.build());
// Session attaches vault(s) containing credentials for those MCP server URLs
var session = client.beta().sessions().create(SessionCreateParams.builder()
.agent(BetaManagedAgentsAgentParams.builder()
.type(BetaManagedAgentsAgentParams.Type.AGENT)
.id(agent.id())
.version(agent.version())
.build())
.environmentId(environment.id())
.addVaultId(vault.id())
.build());
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
---
## Vaults
```java
import com.anthropic.core.JsonValue;
import com.anthropic.models.beta.vaults.VaultCreateParams;
import com.anthropic.models.beta.vaults.credentials.BetaManagedAgentsMcpOAuthCreateParams;
import com.anthropic.models.beta.vaults.credentials.BetaManagedAgentsMcpOAuthRefreshParams;
import com.anthropic.models.beta.vaults.credentials.BetaManagedAgentsMcpOAuthRefreshUpdateParams;
import com.anthropic.models.beta.vaults.credentials.BetaManagedAgentsMcpOAuthUpdateParams;
import com.anthropic.models.beta.vaults.credentials.CredentialCreateParams;
import com.anthropic.models.beta.vaults.credentials.CredentialUpdateParams;
import java.time.OffsetDateTime;
// Create a vault
var vault = client.beta().vaults().create(VaultCreateParams.builder()
.displayName("Alice")
.metadata(VaultCreateParams.Metadata.builder()
.putAdditionalProperty("external_user_id", JsonValue.from("usr_abc123"))
.build())
.build());
System.out.println(vault.id()); // "vlt_01ABC..."
// Add an OAuth credential
var credential = client.beta().vaults().credentials().create(vault.id(),
CredentialCreateParams.builder()
.displayName("Alice's Slack")
.auth(BetaManagedAgentsMcpOAuthCreateParams.builder()
.type(BetaManagedAgentsMcpOAuthCreateParams.Type.MCP_OAUTH)
.mcpServerUrl("https://mcp.slack.com/mcp")
.accessToken("xoxp-...")
.expiresAt(OffsetDateTime.parse("2026-04-15T00:00:00Z"))
.refresh(BetaManagedAgentsMcpOAuthRefreshParams.builder()
.tokenEndpoint("https://slack.com/api/oauth.v2.access")
.clientId("1234567890.0987654321")
.scope("channels:read chat:write")
.refreshToken("xoxe-1-...")
.clientSecretPostTokenEndpointAuth("abc123...")
.build())
.build())
.build());
// Rotate the credential (e.g., after a token refresh)
client.beta().vaults().credentials().update(credential.id(),
CredentialUpdateParams.builder()
.vaultId(vault.id())
.auth(BetaManagedAgentsMcpOAuthUpdateParams.builder()
.type(BetaManagedAgentsMcpOAuthUpdateParams.Type.MCP_OAUTH)
.accessToken("xoxp-new-...")
.expiresAt(OffsetDateTime.parse("2026-05-15T00:00:00Z"))
.refresh(BetaManagedAgentsMcpOAuthRefreshUpdateParams.builder()
.refreshToken("xoxe-1-new-...")
.build())
.build())
.build());
// Archive a vault
client.beta().vaults().archive(vault.id());
```
---
## GitHub Repository Integration
Mount a GitHub repository as a session resource (a vault holds the GitHub MCP credential):
```java
import com.anthropic.models.beta.sessions.BetaManagedAgentsGitHubRepositoryResourceParams;
var session = client.beta().sessions().create(SessionCreateParams.builder()
.agent(agent.id())
.environmentId(environment.id())
.addVaultId(vault.id())
.addResource(BetaManagedAgentsGitHubRepositoryResourceParams.builder()
.type(BetaManagedAgentsGitHubRepositoryResourceParams.Type.GITHUB_REPOSITORY)
.url("https://github.com/org/repo")
.mountPath("/workspace/repo")
.authorizationToken("ghp_your_github_token")
.build())
.build());
```
Multiple repositories on the same session:
```java
import java.util.List;
var resources = List.of(
BetaManagedAgentsGitHubRepositoryResourceParams.builder()
.type(BetaManagedAgentsGitHubRepositoryResourceParams.Type.GITHUB_REPOSITORY)
.url("https://github.com/org/frontend")
.mountPath("/workspace/frontend")
.authorizationToken("ghp_your_github_token")
.build(),
BetaManagedAgentsGitHubRepositoryResourceParams.builder()
.type(BetaManagedAgentsGitHubRepositoryResourceParams.Type.GITHUB_REPOSITORY)
.url("https://github.com/org/backend")
.mountPath("/workspace/backend")
.authorizationToken("ghp_your_github_token")
.build());
```
Rotating a repository's authorization token:
```java
import com.anthropic.models.beta.sessions.resources.ResourceUpdateParams;
var listed = client.beta().sessions().resources().list(session.id());
var repoResourceId = listed.data().get(0).asGitHubRepository().id();
client.beta().sessions().resources().update(repoResourceId, ResourceUpdateParams.builder()
.sessionId(session.id())
.authorizationToken("ghp_your_new_github_token")
.build());
```
resources/bundled/skills/claude-api/php/claude-api.md
+375
View File
@@ -0,0 +1,375 @@
# Claude API — PHP
> **Note:** The PHP SDK is the official Anthropic SDK for PHP. A beta tool runner is available via `$client->beta->messages->toolRunner()`. Structured output helpers are supported via `StructuredOutputModel` classes. Agent SDK is not available. Bedrock, Vertex AI, and Foundry clients are supported.
## Installation
```bash
composer require "anthropic-ai/sdk"
```
## Client Initialization
```php
use Anthropic\Client;
// Using API key from environment variable
$client = new Client(apiKey: getenv("ANTHROPIC_API_KEY"));
```
### Amazon Bedrock
```php
use Anthropic\Bedrock;
// Constructor is private — use the static factory. Reads AWS credentials from env.
$client = Bedrock\Client::fromEnvironment(region: 'us-east-1');
```
### Google Vertex AI
```php
use Anthropic\Vertex;
// Constructor is private. Parameter is `location`, not `region`.
$client = Vertex\Client::fromEnvironment(
location: 'us-east5',
projectId: 'my-project-id',
);
```
### Anthropic Foundry
```php
use Anthropic\Foundry;
// Constructor is private. baseUrl or resource is required.
$client = Foundry\Client::withCredentials(
authToken: getenv('ANTHROPIC_FOUNDRY_AUTH_TOKEN'),
baseUrl: 'https://<resource>.services.ai.azure.com/anthropic',
);
```
---
## Basic Message Request
```php
$message = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
messages: [
['role' => 'user', 'content' => 'What is the capital of France?'],
],
);
// content is an array of polymorphic blocks (TextBlock, ToolUseBlock,
// ThinkingBlock). Accessing ->text on content[0] without checking the block
// type will throw if the first block is not a TextBlock (e.g., when extended
// thinking is enabled and a ThinkingBlock comes first). Always guard:
foreach ($message->content as $block) {
if ($block->type === 'text') {
echo $block->text;
}
}
```
If you only want the first text block:
```php
foreach ($message->content as $block) {
if ($block->type === 'text') {
echo $block->text;
break;
}
}
```
---
## Streaming
> **Requires SDK v0.5.0+.** v0.4.0 and earlier used a single `$params` array; calling with named parameters throws `Unknown named parameter $model`. Upgrade: `composer require "anthropic-ai/sdk:^0.7"`
```php
use Anthropic\Messages\RawContentBlockDeltaEvent;
use Anthropic\Messages\TextDelta;
$stream = $client->messages->createStream(
model: 'claude-opus-4-7',
maxTokens: 64000,
messages: [
['role' => 'user', 'content' => 'Write a haiku'],
],
);
foreach ($stream as $event) {
if ($event instanceof RawContentBlockDeltaEvent && $event->delta instanceof TextDelta) {
echo $event->delta->text;
}
}
```
---
## Tool Use
### Tool Runner (Beta)
**Beta:** The PHP SDK provides a tool runner via `$client->beta->messages->toolRunner()`. Define tools with `BetaRunnableTool` — a definition array plus a `run` closure:
```php
use Anthropic\Lib\Tools\BetaRunnableTool;
$weatherTool = new BetaRunnableTool(
definition: [
'name' => 'get_weather',
'description' => 'Get the current weather for a location.',
'input_schema' => [
'type' => 'object',
'properties' => [
'location' => ['type' => 'string', 'description' => 'City and state'],
],
'required' => ['location'],
],
],
run: function (array $input): string {
return "The weather in {$input['location']} is sunny and 72°F.";
},
);
$runner = $client->beta->messages->toolRunner(
maxTokens: 16000,
messages: [['role' => 'user', 'content' => 'What is the weather in Paris?']],
model: 'claude-opus-4-7',
tools: [$weatherTool],
);
foreach ($runner as $message) {
foreach ($message->content as $block) {
if ($block->type === 'text') {
echo $block->text;
}
}
}
```
### Manual Loop
Tools are passed as arrays. **The SDK uses camelCase keys** (`inputSchema`, `toolUseID`, `stopReason`) and auto-maps to the API's snake_case on the wire — since v0.5.0. See [shared tool use concepts](../shared/tool-use-concepts.md) for the loop pattern.
```php
use Anthropic\Messages\ToolUseBlock;
$tools = [
[
'name' => 'get_weather',
'description' => 'Get the current weather in a given location',
'inputSchema' => [ // camelCase, not input_schema
'type' => 'object',
'properties' => [
'location' => ['type' => 'string', 'description' => 'City and state'],
],
'required' => ['location'],
],
],
];
$messages = [['role' => 'user', 'content' => 'What is the weather in SF?']];
$response = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
tools: $tools,
messages: $messages,
);
while ($response->stopReason === 'tool_use') { // camelCase property
$toolResults = [];
foreach ($response->content as $block) {
if ($block instanceof ToolUseBlock) {
// $block->name : string — tool name to dispatch on
// $block->input : array<string,mixed> — parsed JSON input
// $block->id : string — pass back as toolUseID
$result = executeYourTool($block->name, $block->input);
$toolResults[] = [
'type' => 'tool_result',
'toolUseID' => $block->id, // camelCase, not tool_use_id
'content' => $result,
];
}
}
// Append assistant turn + user turn with tool results
$messages[] = ['role' => 'assistant', 'content' => $response->content];
$messages[] = ['role' => 'user', 'content' => $toolResults];
$response = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
tools: $tools,
messages: $messages,
);
}
// Final text response
foreach ($response->content as $block) {
if ($block->type === 'text') {
echo $block->text;
}
}
```
`$block->type === 'tool_use'` also works; `instanceof ToolUseBlock` narrows for PHPStan.
---
## Extended Thinking
**Adaptive thinking is the recommended mode for Claude 4.6+ models.** Claude decides dynamically when and how much to think.
```php
use Anthropic\Messages\ThinkingBlock;
$message = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
thinking: ['type' => 'adaptive'],
messages: [
['role' => 'user', 'content' => 'Solve: 27 * 453'],
],
);
// ThinkingBlock(s) precede TextBlock in content
foreach ($message->content as $block) {
if ($block instanceof ThinkingBlock) {
echo "Thinking:\n{$block->thinking}\n\n";
// $block->signature is an opaque string — preserve verbatim if
// passing thinking blocks back in multi-turn conversations
} elseif ($block->type === 'text') {
echo "Answer: {$block->text}\n";
}
}
```
> **Deprecated:** `['type' => 'enabled', 'budgetTokens' => N]` (fixed-budget extended thinking) still works on Claude 4.6 but is deprecated. Use adaptive thinking above.
`$block->type === 'thinking'` also works for the check; `instanceof` narrows for PHPStan.
---
## Prompt Caching
`system:` takes an array of text blocks; set `cacheControl` on the last block. Array-shape syntax (camelCase keys) is idiomatic. For placement patterns and the silent-invalidator audit checklist, see `shared/prompt-caching.md`.
```php
$message = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
system: [
['type' => 'text', 'text' => $longSystemPrompt, 'cacheControl' => ['type' => 'ephemeral']],
],
messages: [['role' => 'user', 'content' => 'Summarize the key points']],
);
```
For 1-hour TTL: `'cacheControl' => ['type' => 'ephemeral', 'ttl' => '1h']`. There's also a top-level `cacheControl:` on `messages->create(...)` that auto-places on the last cacheable block.
Verify hits via `$message->usage->cacheCreationInputTokens` / `$message->usage->cacheReadInputTokens`.
---
## Structured Outputs
### Using StructuredOutputModel (Recommended)
Define a PHP class implementing `StructuredOutputModel` and pass it as `outputConfig`:
```php
use Anthropic\Lib\Contracts\StructuredOutputModel;
use Anthropic\Lib\Concerns\StructuredOutputModelTrait;
use Anthropic\Lib\Attributes\Constrained;
class Person implements StructuredOutputModel
{
use StructuredOutputModelTrait;
#[Constrained(description: 'Full name')]
public string $name;
public int $age;
public ?string $email = null; // nullable = optional field
}
$message = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
messages: [['role' => 'user', 'content' => 'Generate a profile for Alice, age 30']],
outputConfig: ['format' => Person::class],
);
$person = $message->parsedOutput(); // Person instance
echo $person->name;
```
Types are inferred from PHP type hints. Use `#[Constrained(description: '...')]` to add descriptions. Nullable properties (`?string`) become optional fields.
### Raw Schema
```php
$message = $client->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
messages: [['role' => 'user', 'content' => 'Extract: John (john@co.com), Enterprise plan']],
outputConfig: [
'format' => [
'type' => 'json_schema',
'schema' => [
'type' => 'object',
'properties' => [
'name' => ['type' => 'string'],
'email' => ['type' => 'string'],
'plan' => ['type' => 'string'],
],
'required' => ['name', 'email', 'plan'],
'additionalProperties' => false,
],
],
],
);
// First text block contains valid JSON
foreach ($message->content as $block) {
if ($block->type === 'text') {
$data = json_decode($block->text, true);
break;
}
}
```
---
## Beta Features & Server-Side Tools
**`betas:` is NOT a param on `$client->messages->create()`** — it only exists on the beta namespace. Use it for features that need an explicit opt-in header:
```php
use Anthropic\Beta\Messages\BetaRequestMCPServerURLDefinition;
$response = $client->beta->messages->create(
model: 'claude-opus-4-7',
maxTokens: 16000,
mcpServers: [
BetaRequestMCPServerURLDefinition::with(
name: 'my-server',
url: 'https://example.com/mcp',
),
],
betas: ['mcp-client-2025-11-20'], // only valid on ->beta->messages
messages: [['role' => 'user', 'content' => 'Use the MCP tools']],
);
```
**Server-side tools** (bash, web_search, text_editor, code_execution) are GA and work on both paths — `Anthropic\Messages\ToolBash20250124` / `WebSearchTool20260209` / `ToolTextEditor20250728` / `CodeExecutionTool20260120` for non-beta, `Anthropic\Beta\Messages\BetaToolBash20250124` / `BetaWebSearchTool20260209` / `BetaToolTextEditor20250728` / `BetaCodeExecutionTool20260120` for beta. No `betas:` header needed for these.
resources/bundled/skills/claude-api/php/managed-agents/README.md
+435
View File
@@ -0,0 +1,435 @@
# Managed Agents — PHP
> **Bindings not shown here:** This README covers the most common managed-agents flows for PHP. If you need a class, method, namespace, field, or behavior that isn't shown, WebFetch the PHP SDK repo **or the relevant docs page** from `shared/live-sources.md` rather than guess. Do not extrapolate from cURL shapes or another language's SDK.
> **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `$client->beta->agents->create` and pass it to every subsequent `->sessions->create`; do not call `agents->create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. The examples below show in-code creation for completeness; in production the create call belongs in setup, not in the request path.
## Installation
```bash
composer require "anthropic-ai/sdk"
```
## Client Initialization
```php
use Anthropic\Client;
// Default (uses ANTHROPIC_API_KEY env var)
$client = new Client();
// Explicit API key
$client = new Client(apiKey: 'your-api-key');
```
---
## Create an Environment
```php
$environment = $client->beta->environments->create(
name: 'my-dev-env',
config: ['type' => 'cloud', 'networking' => ['type' => 'unrestricted']],
);
echo "Environment ID: {$environment->id}\n"; // env_...
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** `model`/`system`/`tools` live on the agent object, not the session. Always start with `$client->beta->agents->create()` — the session takes either `agent: $agent->id` or the typed `BetaManagedAgentsAgentParams::with(type: 'agent', id: $agent->id, version: $agent->version)`.
### Minimal
```php
use Anthropic\Beta\Agents\BetaManagedAgentsAgentToolset20260401Params;
// 1. Create the agent (reusable, versioned)
$agent = $client->beta->agents->create(
name: 'Coding Assistant',
model: 'claude-opus-4-7',
system: 'You are a helpful coding assistant.',
tools: [
BetaManagedAgentsAgentToolset20260401Params::with(
type: 'agent_toolset_20260401',
),
],
);
// 2. Start a session
$session = $client->beta->sessions->create(
agent: ['type' => 'agent', 'id' => $agent->id, 'version' => $agent->version],
environmentID: $environment->id,
title: 'Quickstart session',
);
echo "Session ID: {$session->id}\n";
```
### Updating an Agent
Updates create new versions; the agent object is immutable per version.
```php
$updatedAgent = $client->beta->agents->update(
$agent->id,
version: $agent->version,
system: 'You are a helpful coding agent. Always write tests.',
);
echo "New version: {$updatedAgent->version}\n";
// List all versions
foreach ($client->beta->agents->versions->list($agent->id)->pagingEachItem() as $version) {
echo "Version {$version->version}: {$version->updatedAt->format(DateTimeInterface::ATOM)}\n";
}
// Archive the agent
$archived = $client->beta->agents->archive($agent->id);
echo "Archived at: {$archived->archivedAt->format(DateTimeInterface::ATOM)}\n";
```
---
## Send a User Message
```php
$client->beta->sessions->events->send(
$session->id,
events: [
[
'type' => 'user.message',
'content' => [['type' => 'text', 'text' => 'Review the auth module']],
],
],
);
```
> 💡 **Stream-first:** Open the stream *before* (or concurrently with) sending the message. The stream only delivers events that occur after it opens — stream-after-send means early events arrive buffered in one batch. See [Steering Patterns](../../shared/managed-agents-events.md#steering-patterns).
---
## Stream Events (SSE)
> ℹ️ **Streaming transporter:** PHP's default buffered PSR-18 client never returns for the open-ended session event stream. Use a streaming Guzzle transporter for `streamStream()` calls — other calls keep the default client.
```php
$streamingClient = new GuzzleHttp\Client(['stream' => true]);
// Open the stream first, then send the user message
$stream = $client->beta->sessions->events->streamStream(
$session->id,
requestOptions: ['transporter' => $streamingClient],
);
$client->beta->sessions->events->send(
$session->id,
events: [
[
'type' => 'user.message',
'content' => [['type' => 'text', 'text' => 'Summarize the repo README']],
],
],
);
foreach ($stream as $event) {
match ($event->type) {
'agent.message' => array_walk(
$event->content,
static fn($block) => $block->type === 'text' ? print($block->text) : null,
),
'agent.tool_use' => print("\n[Using tool: {$event->name}]\n"),
'session.error' => printf("\n[Error: %s]", $event->error?->message ?? 'unknown'),
default => null,
};
if ($event->type === 'session.status_idle' || $event->type === 'session.error') {
break;
}
}
$stream->close();
```
### Reconnecting and Tailing
When reconnecting mid-session, list past events first to dedupe, then tail live events:
```php
$stream = $client->beta->sessions->events->streamStream(
$session->id,
requestOptions: ['transporter' => $streamingClient],
);
// Stream is open and buffering. List history before tailing live.
$seenEventIds = [];
foreach ($client->beta->sessions->events->list($session->id)->pagingEachItem() as $event) {
$seenEventIds[$event->id] = true;
}
// Tail live events, skipping anything already seen
foreach ($stream as $event) {
if (isset($seenEventIds[$event->id])) {
continue;
}
$seenEventIds[$event->id] = true;
match ($event->type) {
'agent.message' => array_walk(
$event->content,
static fn($block) => $block->type === 'text' ? print($block->text) : null,
),
default => null,
};
if ($event->type === 'session.status_idle') {
break;
}
}
$stream->close();
```
---
## Provide Custom Tool Result
> ℹ️ The PHP managed-agents bindings for `user.custom_tool_result` are not yet documented in this skill or in the apps source examples. Refer to `shared/managed-agents-events.md` for the wire format and the `anthropic-ai/sdk` PHP repository for the corresponding params.
---
## Poll Events
```php
foreach ($client->beta->sessions->events->list($session->id)->pagingEachItem() as $event) {
echo "{$event->type}: {$event->id}\n";
}
```
---
## Upload a File
> ℹ️ **PHP file upload:** The PHP SDK's beta managed-agents file upload binding is not shown in the apps source examples; the canonical PHP example uses raw cURL against `POST /v1/files`. If your codebase prefers the SDK, WebFetch the `anthropic-ai/sdk` PHP repository for the latest binding before writing code.
```php
use Anthropic\Beta\Sessions\BetaManagedAgentsFileResourceParams;
// Raw cURL upload (canonical example from the apps source)
$csvPath = 'data.csv';
$ch = curl_init('https://api.anthropic.com/v1/files');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => [
'x-api-key: ' . getenv('ANTHROPIC_API_KEY'),
'anthropic-version: 2023-06-01',
'anthropic-beta: files-api-2025-04-14',
],
CURLOPT_POSTFIELDS => ['file' => new CURLFile($csvPath, 'text/csv', 'data.csv')],
]);
$file = json_decode(curl_exec($ch));
echo "File ID: {$file->id}\n";
// Mount in a session
$session = $client->beta->sessions->create(
agent: $agent->id,
environmentID: $environment->id,
resources: [
BetaManagedAgentsFileResourceParams::with(
type: 'file',
fileID: $file->id,
mountPath: '/workspace/data.csv',
),
],
);
```
### Add and Manage Resources on an Existing Session
```php
// Attach an additional file to an open session
$resource = $client->beta->sessions->resources->add(
$session->id,
type: 'file',
fileID: $file->id,
);
echo "{$resource->id}\n"; // "sesrsc_01ABC..."
// List resources on the session
$listed = $client->beta->sessions->resources->list($session->id);
foreach ($listed->data as $entry) {
echo "{$entry->id} {$entry->type}\n";
}
// Detach a resource
$client->beta->sessions->resources->delete($resource->id, sessionID: $session->id);
```
---
## List and Download Session Files
> ℹ️ Listing and downloading files an agent wrote during a session is not yet documented for PHP in this skill or in the apps source examples. See `shared/managed-agents-events.md` and the `anthropic-ai/sdk` PHP repository for the file list/download bindings.
---
## Session Management
```php
// List environments
$environments = $client->beta->environments->list();
// Retrieve a specific environment
$env = $client->beta->environments->retrieve($environment->id);
// Archive an environment (read-only, existing sessions continue)
$client->beta->environments->archive($environment->id);
// Delete an environment (only if no sessions reference it)
$client->beta->environments->delete($environment->id);
// Delete a session
$client->beta->sessions->delete($session->id);
```
---
## MCP Server Integration
```php
use Anthropic\Beta\Agents\BetaManagedAgentsAgentToolset20260401Params;
use Anthropic\Beta\Agents\BetaManagedAgentsMCPToolsetParams;
use Anthropic\Beta\Agents\BetaManagedAgentsUrlmcpServerParams;
use Anthropic\Beta\Sessions\BetaManagedAgentsAgentParams;
// Agent declares MCP server (no auth here — auth goes in a vault)
$agent = $client->beta->agents->create(
name: 'GitHub Assistant',
model: 'claude-opus-4-7',
mcpServers: [
BetaManagedAgentsUrlmcpServerParams::with(
type: 'url',
name: 'github',
url: 'https://api.githubcopilot.com/mcp/',
),
],
tools: [
BetaManagedAgentsAgentToolset20260401Params::with(type: 'agent_toolset_20260401'),
BetaManagedAgentsMCPToolsetParams::with(
type: 'mcp_toolset',
mcpServerName: 'github',
),
],
);
// Session attaches vault(s) containing credentials for those MCP server URLs
$session = $client->beta->sessions->create(
agent: BetaManagedAgentsAgentParams::with(
type: 'agent',
id: $agent->id,
version: $agent->version,
),
environmentID: $environment->id,
vaultIDs: [$vault->id],
);
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
---
## Vaults
```php
// Create a vault
$vault = $client->beta->vaults->create(
displayName: 'Alice',
metadata: ['external_user_id' => 'usr_abc123'],
);
echo $vault->id . "\n"; // "vlt_01ABC..."
// Add an OAuth credential
$credential = $client->beta->vaults->credentials->create(
vaultID: $vault->id,
displayName: "Alice's Slack",
auth: [
'type' => 'mcp_oauth',
'mcp_server_url' => 'https://mcp.slack.com/mcp',
'access_token' => 'xoxp-...',
'expires_at' => '2026-04-15T00:00:00Z',
'refresh' => [
'token_endpoint' => 'https://slack.com/api/oauth.v2.access',
'client_id' => '1234567890.0987654321',
'scope' => 'channels:read chat:write',
'refresh_token' => 'xoxe-1-...',
'token_endpoint_auth' => [
'type' => 'client_secret_post',
'client_secret' => 'abc123...',
],
],
],
);
// Rotate the credential (e.g., after a token refresh)
$client->beta->vaults->credentials->update(
$credential->id,
vaultID: $vault->id,
auth: [
'type' => 'mcp_oauth',
'access_token' => 'xoxp-new-...',
'expires_at' => '2026-05-15T00:00:00Z',
'refresh' => ['refresh_token' => 'xoxe-1-new-...'],
],
);
// Archive a vault
$client->beta->vaults->archive($vault->id);
```
---
## GitHub Repository Integration
Mount a GitHub repository as a session resource (a vault holds the GitHub MCP credential):
```php
$session = $client->beta->sessions->create(
agent: $agent->id,
environmentID: $environment->id,
vaultIDs: [$vault->id],
resources: [
[
'type' => 'github_repository',
'url' => 'https://github.com/org/repo',
'mountPath' => '/workspace/repo',
'authorizationToken' => 'ghp_your_github_token',
],
],
);
```
Multiple repositories on the same session:
```php
$resources = [
[
'type' => 'github_repository',
'url' => 'https://github.com/org/frontend',
'mountPath' => '/workspace/frontend',
'authorizationToken' => 'ghp_your_github_token',
],
[
'type' => 'github_repository',
'url' => 'https://github.com/org/backend',
'mountPath' => '/workspace/backend',
'authorizationToken' => 'ghp_your_github_token',
],
];
```
Rotating a repository's authorization token:
```php
$listed = $client->beta->sessions->resources->list($session->id);
$repoResourceId = $listed->data[0]->id;
$client->beta->sessions->resources->update(
$repoResourceId,
sessionID: $session->id,
authorizationToken: 'ghp_your_new_github_token',
);
```
resources/bundled/skills/claude-api/python/claude-api/README.md
+420
View File
@@ -0,0 +1,420 @@
# Claude API — Python
## Installation
```bash
pip install anthropic
```
## Client Initialization
```python
import anthropic
# Default (uses ANTHROPIC_API_KEY env var)
client = anthropic.Anthropic()
# Explicit API key
client = anthropic.Anthropic(api_key="your-api-key")
# Async client
async_client = anthropic.AsyncAnthropic()
```
---
## Basic Message Request
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[
{"role": "user", "content": "What is the capital of France?"}
]
)
# response.content is a list of content block objects (TextBlock, ThinkingBlock,
# ToolUseBlock, ...). Check .type before accessing .text.
for block in response.content:
if block.type == "text":
print(block.text)
```
---
## System Prompts
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
system="You are a helpful coding assistant. Always provide examples in Python.",
messages=[{"role": "user", "content": "How do I read a JSON file?"}]
)
```
---
## Vision (Images)
### Base64
```python
import base64
with open("image.png", "rb") as f:
image_data = base64.standard_b64encode(f.read()).decode("utf-8")
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": image_data
}
},
{"type": "text", "text": "What's in this image?"}
]
}]
)
```
### URL
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "url",
"url": "https://example.com/image.png"
}
},
{"type": "text", "text": "Describe this image"}
]
}]
)
```
---
## Prompt Caching
Cache large context to reduce costs (up to 90% savings). **Caching is a prefix match** — any byte change anywhere in the prefix invalidates everything after it. For placement patterns, architectural guidance (frozen system prompt, deterministic tool order, where to put volatile content), and the silent-invalidator audit checklist, read `shared/prompt-caching.md`.
### Automatic Caching (Recommended)
Use top-level `cache_control` to automatically cache the last cacheable block in the request — no need to annotate individual content blocks:
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
cache_control={"type": "ephemeral"}, # auto-caches the last cacheable block
system="You are an expert on this large document...",
messages=[{"role": "user", "content": "Summarize the key points"}]
)
```
### Manual Cache Control
For fine-grained control, add `cache_control` to specific content blocks:
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
system=[{
"type": "text",
"text": "You are an expert on this large document...",
"cache_control": {"type": "ephemeral"} # default TTL is 5 minutes
}],
messages=[{"role": "user", "content": "Summarize the key points"}]
)
# With explicit TTL (time-to-live)
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
system=[{
"type": "text",
"text": "You are an expert on this large document...",
"cache_control": {"type": "ephemeral", "ttl": "1h"} # 1 hour TTL
}],
messages=[{"role": "user", "content": "Summarize the key points"}]
)
```
### Verifying Cache Hits
```python
print(response.usage.cache_creation_input_tokens) # tokens written to cache (~1.25x cost)
print(response.usage.cache_read_input_tokens) # tokens served from cache (~0.1x cost)
print(response.usage.input_tokens) # uncached tokens (full cost)
```
If `cache_read_input_tokens` is zero across repeated identical-prefix requests, a silent invalidator is at work — `datetime.now()` or a UUID in the system prompt, unsorted `json.dumps()`, or a varying tool set. See `shared/prompt-caching.md` for the full audit table.
---
## Extended Thinking
> **Opus 4.7, Opus 4.6, and Sonnet 4.6:** Use adaptive thinking. `budget_tokens` is removed on Opus 4.7 (400 if sent); deprecated on Opus 4.6 and Sonnet 4.6.
> **Older models:** Use `thinking: {type: "enabled", budget_tokens: N}` (must be < `max_tokens`, min 1024).
```python
# Opus 4.7 / 4.6: adaptive thinking (recommended)
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
thinking={"type": "adaptive"},
output_config={"effort": "high"}, # low | medium | high | max
messages=[{"role": "user", "content": "Solve this step by step..."}]
)
# Access thinking and response
for block in response.content:
if block.type == "thinking":
print(f"Thinking: {block.thinking}")
elif block.type == "text":
print(f"Response: {block.text}")
```
---
## Error Handling
```python
import anthropic
try:
response = client.messages.create(...)
except anthropic.BadRequestError as e:
print(f"Bad request: {e.message}")
except anthropic.AuthenticationError:
print("Invalid API key")
except anthropic.PermissionDeniedError:
print("API key lacks required permissions")
except anthropic.NotFoundError:
print("Invalid model or endpoint")
except anthropic.RateLimitError as e:
retry_after = int(e.response.headers.get("retry-after", "60"))
print(f"Rate limited. Retry after {retry_after}s.")
except anthropic.APIStatusError as e:
if e.status_code >= 500:
print(f"Server error ({e.status_code}). Retry later.")
else:
print(f"API error: {e.message}")
except anthropic.APIConnectionError:
print("Network error. Check internet connection.")
```
---
## Multi-Turn Conversations
The API is stateless — send the full conversation history each time.
```python
class ConversationManager:
"""Manage multi-turn conversations with the Claude API."""
def __init__(self, client: anthropic.Anthropic, model: str, system: str = None):
self.client = client
self.model = model
self.system = system
self.messages = []
def send(self, user_message: str, **kwargs) -> str:
"""Send a message and get a response."""
self.messages.append({"role": "user", "content": user_message})
response = self.client.messages.create(
model=self.model,
max_tokens=kwargs.get("max_tokens", 16000),
system=self.system,
messages=self.messages,
**kwargs
)
assistant_message = next(
(b.text for b in response.content if b.type == "text"), ""
)
self.messages.append({"role": "assistant", "content": assistant_message})
return assistant_message
# Usage
conversation = ConversationManager(
client=anthropic.Anthropic(),
model="claude-opus-4-7",
system="You are a helpful assistant."
)
response1 = conversation.send("My name is Alice.")
response2 = conversation.send("What's my name?") # Claude remembers "Alice"
```
**Rules:**
- Messages must alternate between `user` and `assistant`
- First message must be `user`
---
### Compaction (long conversations)
> **Beta, Opus 4.7, Opus 4.6, and Sonnet 4.6.** When conversations approach the 200K context window, compaction automatically summarizes earlier context server-side. The API returns a `compaction` block; you must pass it back on subsequent requests — append `response.content`, not just the text.
```python
import anthropic
client = anthropic.Anthropic()
messages = []
def chat(user_message: str) -> str:
messages.append({"role": "user", "content": user_message})
response = client.beta.messages.create(
betas=["compact-2026-01-12"],
model="claude-opus-4-7",
max_tokens=16000,
messages=messages,
context_management={
"edits": [{"type": "compact_20260112"}]
}
)
# Append full content — compaction blocks must be preserved
messages.append({"role": "assistant", "content": response.content})
return next(block.text for block in response.content if block.type == "text")
# Compaction triggers automatically when context grows large
print(chat("Help me build a Python web scraper"))
print(chat("Add support for JavaScript-rendered pages"))
print(chat("Now add rate limiting and error handling"))
```
---
## Stop Reasons
The `stop_reason` field in the response indicates why the model stopped generating:
| Value | Meaning |
|-------|---------|
| `end_turn` | Claude finished its response naturally |
| `max_tokens` | Hit the `max_tokens` limit — increase it or use streaming |
| `stop_sequence` | Hit a custom stop sequence |
| `tool_use` | Claude wants to call a tool — execute it and continue |
| `pause_turn` | Model paused and can be resumed (agentic flows) |
| `refusal` | Claude refused for safety reasons — output may not match your schema |
---
## Cost Optimization Strategies
### 1. Use Prompt Caching for Repeated Context
```python
# Automatic caching (simplest — caches the last cacheable block)
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
cache_control={"type": "ephemeral"},
system=large_document_text, # e.g., 50KB of context
messages=[{"role": "user", "content": "Summarize the key points"}]
)
# First request: full cost
# Subsequent requests: ~90% cheaper for cached portion
```
### 2. Choose the Right Model
```python
# Default to Opus for most tasks
response = client.messages.create(
model="claude-opus-4-7", # $5.00/$25.00 per 1M tokens
max_tokens=16000,
messages=[{"role": "user", "content": "Explain quantum computing"}]
)
# Use Sonnet for high-volume production workloads
standard_response = client.messages.create(
model="claude-sonnet-4-6", # $3.00/$15.00 per 1M tokens
max_tokens=16000,
messages=[{"role": "user", "content": "Summarize this document"}]
)
# Use Haiku only for simple, speed-critical tasks
simple_response = client.messages.create(
model="claude-haiku-4-5", # $1.00/$5.00 per 1M tokens
max_tokens=256,
messages=[{"role": "user", "content": "Classify this as positive or negative"}]
)
```
### 3. Use Token Counting Before Requests
```python
count_response = client.messages.count_tokens(
model="claude-opus-4-7",
messages=messages,
system=system
)
estimated_input_cost = count_response.input_tokens * 0.000005 # $5/1M tokens
print(f"Estimated input cost: ${estimated_input_cost:.4f}")
```
---
## Retry with Exponential Backoff
> **Note:** The Anthropic SDK automatically retries rate limit (429) and server errors (5xx) with exponential backoff. You can configure this with `max_retries` (default: 2). Only implement custom retry logic if you need behavior beyond what the SDK provides.
```python
import time
import random
import anthropic
def call_with_retry(
client: anthropic.Anthropic,
max_retries: int = 5,
base_delay: float = 1.0,
max_delay: float = 60.0,
**kwargs
):
"""Call the API with exponential backoff retry."""
last_exception = None
for attempt in range(max_retries):
try:
return client.messages.create(**kwargs)
except anthropic.RateLimitError as e:
last_exception = e
except anthropic.APIStatusError as e:
if e.status_code >= 500:
last_exception = e
else:
raise # Client errors (4xx except 429) should not be retried
delay = min(base_delay * (2 ** attempt) + random.uniform(0, 1), max_delay)
print(f"Retry {attempt + 1}/{max_retries} after {delay:.1f}s")
time.sleep(delay)
raise last_exception
```
resources/bundled/skills/claude-api/python/claude-api/batches.md
+185
View File
@@ -0,0 +1,185 @@
# Message Batches API — Python
The Batches API (`POST /v1/messages/batches`) processes Messages API requests asynchronously at 50% of standard prices.
## Key Facts
- Up to 100,000 requests or 256 MB per batch
- Most batches complete within 1 hour; maximum 24 hours
- Results available for 29 days after creation
- 50% cost reduction on all token usage
- All Messages API features supported (vision, tools, caching, etc.)
---
## Create a Batch
```python
import anthropic
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request
client = anthropic.Anthropic()
message_batch = client.messages.batches.create(
requests=[
Request(
custom_id="request-1",
params=MessageCreateParamsNonStreaming(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Summarize climate change impacts"}]
)
),
Request(
custom_id="request-2",
params=MessageCreateParamsNonStreaming(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Explain quantum computing basics"}]
)
),
]
)
print(f"Batch ID: {message_batch.id}")
print(f"Status: {message_batch.processing_status}")
```
---
## Poll for Completion
```python
import time
while True:
batch = client.messages.batches.retrieve(message_batch.id)
if batch.processing_status == "ended":
break
print(f"Status: {batch.processing_status}, processing: {batch.request_counts.processing}")
time.sleep(60)
print("Batch complete!")
print(f"Succeeded: {batch.request_counts.succeeded}")
print(f"Errored: {batch.request_counts.errored}")
```
---
## Retrieve Results
> **Note:** Examples below use `match/case` syntax, requiring Python 3.10+. For earlier versions, use `if/elif` chains instead.
```python
for result in client.messages.batches.results(message_batch.id):
match result.result.type:
case "succeeded":
msg = result.result.message
text = next((b.text for b in msg.content if b.type == "text"), "")
print(f"[{result.custom_id}] {text[:100]}")
case "errored":
if result.result.error.type == "invalid_request":
print(f"[{result.custom_id}] Validation error - fix request and retry")
else:
print(f"[{result.custom_id}] Server error - safe to retry")
case "canceled":
print(f"[{result.custom_id}] Canceled")
case "expired":
print(f"[{result.custom_id}] Expired - resubmit")
```
---
## Cancel a Batch
```python
cancelled = client.messages.batches.cancel(message_batch.id)
print(f"Status: {cancelled.processing_status}") # "canceling"
```
---
## Batch with Prompt Caching
```python
shared_system = [
{"type": "text", "text": "You are a literary analyst."},
{
"type": "text",
"text": large_document_text, # Shared across all requests
"cache_control": {"type": "ephemeral"}
}
]
message_batch = client.messages.batches.create(
requests=[
Request(
custom_id=f"analysis-{i}",
params=MessageCreateParamsNonStreaming(
model="claude-opus-4-7",
max_tokens=16000,
system=shared_system,
messages=[{"role": "user", "content": question}]
)
)
for i, question in enumerate(questions)
]
)
```
---
## Full End-to-End Example
```python
import anthropic
import time
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request
client = anthropic.Anthropic()
# 1. Prepare requests
items_to_classify = [
"The product quality is excellent!",
"Terrible customer service, never again.",
"It's okay, nothing special.",
]
requests = [
Request(
custom_id=f"classify-{i}",
params=MessageCreateParamsNonStreaming(
model="claude-haiku-4-5",
max_tokens=50,
messages=[{
"role": "user",
"content": f"Classify as positive/negative/neutral (one word): {text}"
}]
)
)
for i, text in enumerate(items_to_classify)
]
# 2. Create batch
batch = client.messages.batches.create(requests=requests)
print(f"Created batch: {batch.id}")
# 3. Wait for completion
while True:
batch = client.messages.batches.retrieve(batch.id)
if batch.processing_status == "ended":
break
time.sleep(10)
# 4. Collect results
results = {}
for result in client.messages.batches.results(batch.id):
if result.result.type == "succeeded":
msg = result.result.message
results[result.custom_id] = next((b.text for b in msg.content if b.type == "text"), "")
for custom_id, classification in sorted(results.items()):
print(f"{custom_id}: {classification}")
```
resources/bundled/skills/claude-api/python/claude-api/files-api.md
+165
View File
@@ -0,0 +1,165 @@
# Files API — Python
The Files API uploads files for use in Messages API requests. Reference files via `file_id` in content blocks, avoiding re-uploads across multiple API calls.
**Beta:** Pass `betas=["files-api-2025-04-14"]` in your API calls (the SDK sets the required header automatically).
## Key Facts
- Maximum file size: 500 MB
- Total storage: 100 GB per organization
- Files persist until deleted
- File operations (upload, list, delete) are free; content used in messages is billed as input tokens
- Not available on Amazon Bedrock or Google Vertex AI
---
## Upload a File
```python
import anthropic
client = anthropic.Anthropic()
uploaded = client.beta.files.upload(
file=("report.pdf", open("report.pdf", "rb"), "application/pdf"),
)
print(f"File ID: {uploaded.id}")
print(f"Size: {uploaded.size_bytes} bytes")
```
---
## Use a File in Messages
### PDF / Text Document
```python
response = client.beta.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Summarize the key findings in this report."},
{
"type": "document",
"source": {"type": "file", "file_id": uploaded.id},
"title": "Q4 Report", # optional
"citations": {"enabled": True} # optional, enables citations
}
]
}],
betas=["files-api-2025-04-14"],
)
for block in response.content:
if block.type == "text":
print(block.text)
```
### Image
```python
image_file = client.beta.files.upload(
file=("photo.png", open("photo.png", "rb"), "image/png"),
)
response = client.beta.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "What's in this image?"},
{
"type": "image",
"source": {"type": "file", "file_id": image_file.id}
}
]
}],
betas=["files-api-2025-04-14"],
)
```
---
## Manage Files
### List Files
```python
files = client.beta.files.list()
for f in files.data:
print(f"{f.id}: {f.filename} ({f.size_bytes} bytes)")
```
### Get File Metadata
```python
file_info = client.beta.files.retrieve_metadata("file_011CNha8iCJcU1wXNR6q4V8w")
print(f"Filename: {file_info.filename}")
print(f"MIME type: {file_info.mime_type}")
```
### Delete a File
```python
client.beta.files.delete("file_011CNha8iCJcU1wXNR6q4V8w")
```
### Download a File
Only files created by the code execution tool or skills can be downloaded (not user-uploaded files).
```python
file_content = client.beta.files.download("file_011CNha8iCJcU1wXNR6q4V8w")
file_content.write_to_file("output.txt")
```
---
## Full End-to-End Example
Upload a document once, ask multiple questions about it:
```python
import anthropic
client = anthropic.Anthropic()
# 1. Upload once
uploaded = client.beta.files.upload(
file=("contract.pdf", open("contract.pdf", "rb"), "application/pdf"),
)
print(f"Uploaded: {uploaded.id}")
# 2. Ask multiple questions using the same file_id
questions = [
"What are the key terms and conditions?",
"What is the termination clause?",
"Summarize the payment schedule.",
]
for question in questions:
response = client.beta.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": [
{"type": "text", "text": question},
{
"type": "document",
"source": {"type": "file", "file_id": uploaded.id}
}
]
}],
betas=["files-api-2025-04-14"],
)
print(f"\nQ: {question}")
text = next((b.text for b in response.content if b.type == "text"), "")
print(f"A: {text[:200]}")
# 3. Clean up when done
client.beta.files.delete(uploaded.id)
```
resources/bundled/skills/claude-api/python/claude-api/streaming.md
+162
View File
@@ -0,0 +1,162 @@
# Streaming — Python
## Quick Start
```python
with client.messages.stream(
model="claude-opus-4-7",
max_tokens=64000,
messages=[{"role": "user", "content": "Write a story"}]
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
```
### Async
```python
async with async_client.messages.stream(
model="claude-opus-4-7",
max_tokens=64000,
messages=[{"role": "user", "content": "Write a story"}]
) as stream:
async for text in stream.text_stream:
print(text, end="", flush=True)
```
---
## Handling Different Content Types
Claude may return text, thinking blocks, or tool use. Handle each appropriately:
> **Opus 4.7 / Opus 4.6:** Use `thinking: {type: "adaptive"}`. On older models, use `thinking: {type: "enabled", budget_tokens: N}` instead.
```python
with client.messages.stream(
model="claude-opus-4-7",
max_tokens=64000,
thinking={"type": "adaptive"},
messages=[{"role": "user", "content": "Analyze this problem"}]
) as stream:
for event in stream:
if event.type == "content_block_start":
if event.content_block.type == "thinking":
print("\n[Thinking...]")
elif event.content_block.type == "text":
print("\n[Response:]")
elif event.type == "content_block_delta":
if event.delta.type == "thinking_delta":
print(event.delta.thinking, end="", flush=True)
elif event.delta.type == "text_delta":
print(event.delta.text, end="", flush=True)
```
---
## Streaming with Tool Use
The Python tool runner currently returns complete messages. Use streaming for individual API calls within a manual loop if you need per-token streaming with tools:
```python
with client.messages.stream(
model="claude-opus-4-7",
max_tokens=64000,
tools=tools,
messages=messages
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
response = stream.get_final_message()
# Continue with tool execution if response.stop_reason == "tool_use"
```
---
## Getting the Final Message
```python
with client.messages.stream(
model="claude-opus-4-7",
max_tokens=64000,
messages=[{"role": "user", "content": "Hello"}]
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
# Get full message after streaming
final_message = stream.get_final_message()
print(f"\n\nTokens used: {final_message.usage.output_tokens}")
```
---
## Streaming with Progress Updates
```python
def stream_with_progress(client, **kwargs):
"""Stream a response with progress updates."""
total_tokens = 0
content_parts = []
with client.messages.stream(**kwargs) as stream:
for event in stream:
if event.type == "content_block_delta":
if event.delta.type == "text_delta":
text = event.delta.text
content_parts.append(text)
print(text, end="", flush=True)
elif event.type == "message_delta":
if event.usage and event.usage.output_tokens is not None:
total_tokens = event.usage.output_tokens
final_message = stream.get_final_message()
print(f"\n\n[Tokens used: {total_tokens}]")
return "".join(content_parts)
```
---
## Error Handling in Streams
```python
try:
with client.messages.stream(
model="claude-opus-4-7",
max_tokens=64000,
messages=[{"role": "user", "content": "Write a story"}]
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
except anthropic.APIConnectionError:
print("\nConnection lost. Please retry.")
except anthropic.RateLimitError:
print("\nRate limited. Please wait and retry.")
except anthropic.APIStatusError as e:
print(f"\nAPI error: {e.status_code}")
```
---
## Stream Event Types
| Event Type | Description | When it fires |
| --------------------- | --------------------------- | --------------------------------- |
| `message_start` | Contains message metadata | Once at the beginning |
| `content_block_start` | New content block beginning | When a text/tool_use block starts |
| `content_block_delta` | Incremental content update | For each token/chunk |
| `content_block_stop` | Content block complete | When a block finishes |
| `message_delta` | Message-level updates | Contains `stop_reason`, usage |
| `message_stop` | Message complete | Once at the end |
## Best Practices
1. **Always flush output** — Use `flush=True` to show tokens immediately
2. **Handle partial responses** — If the stream is interrupted, you may have incomplete content
3. **Track token usage** — The `message_delta` event contains usage information
4. **Use timeouts** — Set appropriate timeouts for your application
5. **Default to streaming** — Use `.get_final_message()` to get the complete response even when streaming, giving you timeout protection without needing to handle individual events
resources/bundled/skills/claude-api/python/claude-api/tool-use.md
+590
View File
@@ -0,0 +1,590 @@
# Tool Use — Python
For conceptual overview (tool definitions, tool choice, tips), see [shared/tool-use-concepts.md](../../shared/tool-use-concepts.md).
## Tool Runner (Recommended)
**Beta:** The tool runner is in beta in the Python SDK.
Use the `@beta_tool` decorator to define tools as typed functions, then pass them to `client.beta.messages.tool_runner()`:
```python
import anthropic
from anthropic import beta_tool
client = anthropic.Anthropic()
@beta_tool
def get_weather(location: str, unit: str = "celsius") -> str:
"""Get current weather for a location.
Args:
location: City and state, e.g., San Francisco, CA.
unit: Temperature unit, either "celsius" or "fahrenheit".
"""
# Your implementation here
return f"72°F and sunny in {location}"
# The tool runner handles the agentic loop automatically
runner = client.beta.messages.tool_runner(
model="claude-opus-4-7",
max_tokens=16000,
tools=[get_weather],
messages=[{"role": "user", "content": "What's the weather in Paris?"}],
)
# Each iteration yields a BetaMessage; iteration stops when Claude is done
for message in runner:
print(message)
```
For async usage, use `@beta_async_tool` with `async def` functions.
**Key benefits of the tool runner:**
- No manual loop — the SDK handles calling tools and feeding results back
- Type-safe tool inputs via decorators
- Tool schemas are generated automatically from function signatures
- Iteration stops automatically when Claude has no more tool calls
---
## MCP Tool Conversion Helpers
**Beta.** Convert [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) tools, prompts, and resources to Anthropic API types for use with the tool runner. Requires `pip install anthropic[mcp]` (Python 3.10+).
> **Note:** The Claude API also supports an `mcp_servers` parameter that lets Claude connect directly to remote MCP servers. Use these helpers instead when you need local MCP servers, prompts, resources, or more control over the MCP connection.
### MCP Tools with Tool Runner
```python
from anthropic import AsyncAnthropic
from anthropic.lib.tools.mcp import async_mcp_tool
from mcp import ClientSession
from mcp.client.stdio import stdio_client, StdioServerParameters
client = AsyncAnthropic()
async with stdio_client(StdioServerParameters(command="mcp-server")) as (read, write):
async with ClientSession(read, write) as mcp_client:
await mcp_client.initialize()
tools_result = await mcp_client.list_tools()
# tool_runner is sync — returns the runner, not a coroutine
runner = client.beta.messages.tool_runner(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Use the available tools"}],
tools=[async_mcp_tool(t, mcp_client) for t in tools_result.tools],
)
async for message in runner:
print(message)
```
For sync usage, use `mcp_tool` instead of `async_mcp_tool`.
### MCP Prompts
```python
from anthropic.lib.tools.mcp import mcp_message
prompt = await mcp_client.get_prompt(name="my-prompt")
response = await client.beta.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[mcp_message(m) for m in prompt.messages],
)
```
### MCP Resources as Content
```python
from anthropic.lib.tools.mcp import mcp_resource_to_content
resource = await mcp_client.read_resource(uri="file:///path/to/doc.txt")
response = await client.beta.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": [
mcp_resource_to_content(resource),
{"type": "text", "text": "Summarize this document"},
],
}],
)
```
### Upload MCP Resources as Files
```python
from anthropic.lib.tools.mcp import mcp_resource_to_file
resource = await mcp_client.read_resource(uri="file:///path/to/data.json")
uploaded = await client.beta.files.upload(file=mcp_resource_to_file(resource))
```
Conversion functions raise `UnsupportedMCPValueError` if an MCP value cannot be converted (e.g., unsupported content types like audio, unsupported MIME types).
---
## Manual Agentic Loop
Use this when you need fine-grained control over the loop (e.g., custom logging, conditional tool execution, human-in-the-loop approval):
```python
import anthropic
client = anthropic.Anthropic()
tools = [...] # Your tool definitions
messages = [{"role": "user", "content": user_input}]
# Agentic loop: keep going until Claude stops calling tools
while True:
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
tools=tools,
messages=messages
)
# If Claude is done (no more tool calls), break
if response.stop_reason == "end_turn":
break
# Server-side tool hit iteration limit; re-send to continue
if response.stop_reason == "pause_turn":
messages = [
{"role": "user", "content": user_input},
{"role": "assistant", "content": response.content},
]
continue
# Extract tool use blocks from the response
tool_use_blocks = [b for b in response.content if b.type == "tool_use"]
# Append assistant's response (including tool_use blocks)
messages.append({"role": "assistant", "content": response.content})
# Execute each tool and collect results
tool_results = []
for tool in tool_use_blocks:
result = execute_tool(tool.name, tool.input) # Your implementation
tool_results.append({
"type": "tool_result",
"tool_use_id": tool.id, # Must match the tool_use block's id
"content": result
})
# Append tool results as a user message
messages.append({"role": "user", "content": tool_results})
# Final response text
final_text = next(b.text for b in response.content if b.type == "text")
```
---
## Handling Tool Results
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
tools=tools,
messages=[{"role": "user", "content": "What's the weather in Paris?"}]
)
for block in response.content:
if block.type == "tool_use":
tool_name = block.name
tool_input = block.input
tool_use_id = block.id
result = execute_tool(tool_name, tool_input)
followup = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
tools=tools,
messages=[
{"role": "user", "content": "What's the weather in Paris?"},
{"role": "assistant", "content": response.content},
{
"role": "user",
"content": [{
"type": "tool_result",
"tool_use_id": tool_use_id,
"content": result
}]
}
]
)
```
---
## Multiple Tool Calls
```python
tool_results = []
for block in response.content:
if block.type == "tool_use":
result = execute_tool(block.name, block.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": result
})
# Send all results back at once
if tool_results:
followup = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
tools=tools,
messages=[
*previous_messages,
{"role": "assistant", "content": response.content},
{"role": "user", "content": tool_results}
]
)
```
---
## Error Handling in Tool Results
```python
tool_result = {
"type": "tool_result",
"tool_use_id": tool_use_id,
"content": "Error: Location 'xyz' not found. Please provide a valid city name.",
"is_error": True
}
```
---
## Tool Choice
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
tools=tools,
tool_choice={"type": "tool", "name": "get_weather"}, # Force specific tool
messages=[{"role": "user", "content": "What's the weather in Paris?"}]
)
```
---
## Code Execution
### Basic Usage
```python
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": "Calculate the mean and standard deviation of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"
}],
tools=[{
"type": "code_execution_20260120",
"name": "code_execution"
}]
)
for block in response.content:
if block.type == "text":
print(block.text)
elif block.type == "bash_code_execution_tool_result":
print(f"stdout: {block.content.stdout}")
```
### Upload Files for Analysis
```python
# 1. Upload a file
uploaded = client.beta.files.upload(file=open("sales_data.csv", "rb"))
# 2. Pass to code execution via container_upload block
# Code execution is GA; Files API is still beta (pass via extra_headers)
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
extra_headers={"anthropic-beta": "files-api-2025-04-14"},
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Analyze this sales data. Show trends and create a visualization."},
{"type": "container_upload", "file_id": uploaded.id}
]
}],
tools=[{"type": "code_execution_20260120", "name": "code_execution"}]
)
```
### Retrieve Generated Files
```python
import os
OUTPUT_DIR = "./claude_outputs"
os.makedirs(OUTPUT_DIR, exist_ok=True)
for block in response.content:
if block.type == "bash_code_execution_tool_result":
result = block.content
if result.type == "bash_code_execution_result" and result.content:
for file_ref in result.content:
if file_ref.type == "bash_code_execution_output":
metadata = client.beta.files.retrieve_metadata(file_ref.file_id)
file_content = client.beta.files.download(file_ref.file_id)
# Use basename to prevent path traversal; validate result
safe_name = os.path.basename(metadata.filename)
if not safe_name or safe_name in (".", ".."):
print(f"Skipping invalid filename: {metadata.filename}")
continue
output_path = os.path.join(OUTPUT_DIR, safe_name)
file_content.write_to_file(output_path)
print(f"Saved: {output_path}")
```
### Container Reuse
```python
# First request: set up environment
response1 = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Install tabulate and create data.json with sample data"}],
tools=[{"type": "code_execution_20260120", "name": "code_execution"}]
)
# Get container ID from response
container_id = response1.container.id
# Second request: reuse the same container
response2 = client.messages.create(
container=container_id,
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Read data.json and display as a formatted table"}],
tools=[{"type": "code_execution_20260120", "name": "code_execution"}]
)
```
### Response Structure
```python
for block in response.content:
if block.type == "text":
print(block.text) # Claude's explanation
elif block.type == "server_tool_use":
print(f"Running: {block.name} - {block.input}") # What Claude is doing
elif block.type == "bash_code_execution_tool_result":
result = block.content
if result.type == "bash_code_execution_result":
if result.return_code == 0:
print(f"Output: {result.stdout}")
else:
print(f"Error: {result.stderr}")
else:
print(f"Tool error: {result.error_code}")
elif block.type == "text_editor_code_execution_tool_result":
print(f"File operation: {block.content}")
```
---
## Memory Tool
### Basic Usage
```python
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Remember that my preferred language is Python."}],
tools=[{"type": "memory_20250818", "name": "memory"}],
)
```
### SDK Memory Helper
Subclass `BetaAbstractMemoryTool`:
```python
from anthropic.lib.tools import BetaAbstractMemoryTool
class MyMemoryTool(BetaAbstractMemoryTool):
def view(self, command): ...
def create(self, command): ...
def str_replace(self, command): ...
def insert(self, command): ...
def delete(self, command): ...
def rename(self, command): ...
memory = MyMemoryTool()
# Use with tool runner
runner = client.beta.messages.tool_runner(
model="claude-opus-4-7",
max_tokens=16000,
tools=[memory],
messages=[{"role": "user", "content": "Remember my preferences"}],
)
for message in runner:
print(message)
```
For full implementation examples, use WebFetch:
- `https://github.com/anthropics/anthropic-sdk-python/blob/main/examples/memory/basic.py`
---
## Structured Outputs
### JSON Outputs (Pydantic — Recommended)
```python
from pydantic import BaseModel
from typing import List
import anthropic
class ContactInfo(BaseModel):
name: str
email: str
plan: str
interests: List[str]
demo_requested: bool
client = anthropic.Anthropic()
response = client.messages.parse(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": "Extract: Jane Doe (jane@co.com) wants Enterprise, interested in API and SDKs, wants a demo."
}],
output_format=ContactInfo,
)
# response.parsed_output is a validated ContactInfo instance
contact = response.parsed_output
print(contact.name) # "Jane Doe"
print(contact.interests) # ["API", "SDKs"]
```
### Raw Schema
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{
"role": "user",
"content": "Extract info: John Smith (john@example.com) wants the Enterprise plan."
}],
output_config={
"format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"plan": {"type": "string"},
"demo_requested": {"type": "boolean"}
},
"required": ["name", "email", "plan", "demo_requested"],
"additionalProperties": False
}
}
}
)
import json
# output_config.format guarantees the first block is text with valid JSON
text = next(b.text for b in response.content if b.type == "text")
data = json.loads(text)
```
### Strict Tool Use
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Book a flight to Tokyo for 2 passengers on March 15"}],
tools=[{
"name": "book_flight",
"description": "Book a flight to a destination",
"strict": True,
"input_schema": {
"type": "object",
"properties": {
"destination": {"type": "string"},
"date": {"type": "string", "format": "date"},
"passengers": {"type": "integer", "enum": [1, 2, 3, 4, 5, 6, 7, 8]}
},
"required": ["destination", "date", "passengers"],
"additionalProperties": False
}
}]
)
```
### Using Both Together
```python
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=16000,
messages=[{"role": "user", "content": "Plan a trip to Paris next month"}],
output_config={
"format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"summary": {"type": "string"},
"next_steps": {"type": "array", "items": {"type": "string"}}
},
"required": ["summary", "next_steps"],
"additionalProperties": False
}
}
},
tools=[{
"name": "search_flights",
"description": "Search for available flights",
"strict": True,
"input_schema": {
"type": "object",
"properties": {
"destination": {"type": "string"},
"date": {"type": "string", "format": "date"}
},
"required": ["destination", "date"],
"additionalProperties": False
}
}]
)
```
resources/bundled/skills/claude-api/python/managed-agents/README.md
+332
View File
@@ -0,0 +1,332 @@
# Managed Agents — Python
> **Bindings not shown here:** This README covers the most common managed-agents flows for Python. If you need a class, method, namespace, field, or behavior that isn't shown, WebFetch the Python SDK repo **or the relevant docs page** from `shared/live-sources.md` rather than guess. Do not extrapolate from cURL shapes or another language's SDK.
> **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `agents.create` and pass it to every subsequent `sessions.create`; do not call `agents.create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. The examples below show in-code creation for completeness; in production the create call belongs in setup, not in the request path.
## Installation
```bash
pip install anthropic
```
## Client Initialization
```python
import anthropic
# Default (uses ANTHROPIC_API_KEY env var)
client = anthropic.Anthropic()
# Explicit API key
client = anthropic.Anthropic(api_key="your-api-key")
```
---
## Create an Environment
```python
environment = client.beta.environments.create(
name="my-dev-env",
config={
"type": "cloud",
"networking": {"type": "unrestricted"},
},
)
print(environment.id) # env_...
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** `model`/`system`/`tools` live on the agent object, not the session. Always start with `agents.create()` — the session only takes `agent={"type": "agent", "id": agent.id}`.
### Minimal
```python
# 1. Create the agent (reusable, versioned)
agent = client.beta.agents.create(
name="Coding Assistant",
model="claude-opus-4-7",
tools=[{"type": "agent_toolset_20260401", "default_config": {"enabled": True}}],
)
# 2. Start a session
session = client.beta.sessions.create(
agent={"type": "agent", "id": agent.id, "version": agent.version},
environment_id=environment.id,
)
print(session.id, session.status)
```
### With system prompt and custom tools
```python
import os
agent = client.beta.agents.create(
name="Code Reviewer",
model="claude-opus-4-7",
system="You are a senior code reviewer.",
tools=[
{"type": "agent_toolset_20260401"},
{
"type": "custom",
"name": "run_tests",
"description": "Run the test suite",
"input_schema": {
"type": "object",
"properties": {
"test_path": {"type": "string", "description": "Path to test file"}
},
"required": ["test_path"],
},
},
],
)
session = client.beta.sessions.create(
agent={"type": "agent", "id": agent.id, "version": agent.version},
environment_id=environment.id,
title="Code review session",
resources=[
{
"type": "github_repository",
"url": "https://github.com/owner/repo",
"mount_path": "/workspace/repo",
"authorization_token": os.environ["GITHUB_TOKEN"],
"branch": "main",
}
],
)
```
---
## Send a User Message
```python
client.beta.sessions.events.send(
session_id=session.id,
events=[
{
"type": "user.message",
"content": [{"type": "text", "text": "Review the auth module"}],
}
],
)
```
> 💡 **Stream-first:** Open the stream *before* (or concurrently with) sending the message. The stream only delivers events that occur after it opens — stream-after-send means early events arrive buffered in one batch. See [Steering Patterns](../../shared/managed-agents-events.md#steering-patterns).
---
## Stream Events (SSE)
```python
import json
# Stream-first: open stream, then send while stream is live
with client.beta.sessions.stream(
session_id=session.id,
) as stream:
client.beta.sessions.events.send(
session_id=session.id,
events=[{"type": "user.message", "content": [{"type": "text", "text": "..."}]}],
)
for event in stream:
... # process events
# Standalone stream iteration:
with client.beta.sessions.stream(
session_id=session.id,
) as stream:
for event in stream:
if event.type == "agent.message":
for block in event.content:
if block.type == "text":
print(block.text, end="", flush=True)
elif event.type == "agent.custom_tool_use":
# Custom tool invocation — session is now idle
print(f"\nCustom tool call: {event.tool_name}")
print(f"Input: {json.dumps(event.input)}")
# Send result back (see below)
elif event.type == "session.status_idle":
print("\n--- Agent idle ---")
elif event.type == "session.status_terminated":
print("\n--- Session terminated ---")
break
```
---
## Provide Custom Tool Result
```python
client.beta.sessions.events.send(
session_id=session.id,
events=[
{
"type": "user.custom_tool_result",
"custom_tool_use_id": "sevt_abc123",
"content": [{"type": "text", "text": "All 42 tests passed."}],
}
],
)
```
---
## Poll Events
```python
events = client.beta.sessions.events.list(
session_id=session.id,
)
for event in events.data:
print(f"{event.type}: {event.id}")
```
> ⚠️ **Prefer the SDK over raw `requests`/`httpx`.** If you hand-roll a poll loop, don't assume `timeout=(5, 60)` or `httpx.Timeout(120)` caps total call duration — both are **per-chunk** read timeouts (reset on every byte), so a trickling response can block forever. For a hard wall-clock deadline, track `time.monotonic()` at the loop level and bail explicitly, or wrap with `asyncio.wait_for()`. See [Receiving Events](../../shared/managed-agents-events.md#receiving-events).
---
## Full Streaming Loop with Custom Tools
```python
import json
def run_custom_tool(tool_name: str, tool_input: dict) -> str:
"""Execute a custom tool and return the result."""
if tool_name == "run_tests":
# Your tool implementation here
return "All tests passed."
return f"Unknown tool: {tool_name}"
def run_session(client, session_id: str):
"""Stream events and handle custom tool calls."""
while True:
with client.beta.sessions.stream(
session_id=session_id,
) as stream:
tool_calls = []
for event in stream:
if event.type == "agent.message":
for block in event.content:
if block.type == "text":
print(block.text, end="", flush=True)
elif event.type == "agent.custom_tool_use":
tool_calls.append(event)
elif event.type == "session.status_idle":
break
elif event.type == "session.status_terminated":
return
if not tool_calls:
break
# Process custom tool calls
results = []
for call in tool_calls:
result = run_custom_tool(call.tool_name, call.input)
results.append({
"type": "user.custom_tool_result",
"custom_tool_use_id": call.id,
"content": [{"type": "text", "text": result}],
})
client.beta.sessions.events.send(
session_id=session_id,
events=results,
)
```
---
## Upload a File
```python
with open("data.csv", "rb") as f:
file = client.beta.files.upload(
file=f,
)
# Use in a session
session = client.beta.sessions.create(
agent={"type": "agent", "id": agent.id, "version": agent.version},
environment_id=environment.id,
resources=[{"type": "file", "file_id": file.id, "mount_path": "/workspace/data.csv"}],
)
```
---
## List and Download Session Files
List files the agent wrote to `/mnt/session/outputs/` during a session, then download them.
```python
# List files associated with a session
files = client.beta.files.list(
scope_id=session.id,
betas=["managed-agents-2026-04-01"],
)
for f in files.data:
print(f.filename, f.size_bytes)
# Download each file and save to disk
file_content = client.beta.files.download(f.id)
file_content.write_to_file(f.filename)
```
> 💡 There's a brief indexing lag (~1–3s) between `session.status_idle` and output files appearing in `files.list`. Retry once or twice if the list is empty.
---
## Session Management
```python
# Get session details
session = client.beta.sessions.retrieve(session_id="sesn_011CZxAbc123Def456")
print(session.status, session.usage)
# List sessions
sessions = client.beta.sessions.list()
# Delete a session
client.beta.sessions.delete(session_id="sesn_011CZxAbc123Def456")
# Archive a session
client.beta.sessions.archive(session_id="sesn_011CZxAbc123Def456")
```
---
## MCP Server Integration
```python
# Agent declares MCP server (no auth here — auth goes in a vault)
agent = client.beta.agents.create(
name="MCP Agent",
model="claude-opus-4-7",
mcp_servers=[
{"type": "url", "name": "my-tools", "url": "https://my-mcp-server.example.com/sse"},
],
tools=[
{"type": "agent_toolset_20260401", "default_config": {"enabled": True}},
{"type": "mcp_toolset", "mcp_server_name": "my-tools"},
],
)
# Session attaches vault(s) containing credentials for those MCP server URLs
session = client.beta.sessions.create(
agent=agent.id,
environment_id=environment.id,
vault_ids=[vault.id],
)
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
resources/bundled/skills/claude-api/ruby/claude-api.md
+113
View File
@@ -0,0 +1,113 @@
# Claude API — Ruby
> **Note:** The Ruby SDK supports the Claude API. A tool runner is available in beta via `client.beta.messages.tool_runner()`. Agent SDK is not yet available for Ruby.
## Installation
```bash
gem install anthropic
```
## Client Initialization
```ruby
require "anthropic"
# Default (uses ANTHROPIC_API_KEY env var)
client = Anthropic::Client.new
# Explicit API key
client = Anthropic::Client.new(api_key: "your-api-key")
```
---
## Basic Message Request
```ruby
message = client.messages.create(
model: :"claude-opus-4-7",
max_tokens: 16000,
messages: [
{ role: "user", content: "What is the capital of France?" }
]
)
# content is an array of polymorphic block objects (TextBlock, ThinkingBlock,
# ToolUseBlock, ...). .type is a Symbol — compare with :text, not "text".
# .text raises NoMethodError on non-TextBlock entries.
message.content.each do |block|
puts block.text if block.type == :text
end
```
---
## Streaming
```ruby
stream = client.messages.stream(
model: :"claude-opus-4-7",
max_tokens: 64000,
messages: [{ role: "user", content: "Write a haiku" }]
)
stream.text.each { |text| print(text) }
```
---
## Tool Use
The Ruby SDK supports tool use via raw JSON schema definitions and also provides a beta tool runner for automatic tool execution.
### Tool Runner (Beta)
```ruby
class GetWeatherInput < Anthropic::BaseModel
required :location, String, doc: "City and state, e.g. San Francisco, CA"
end
class GetWeather < Anthropic::BaseTool
doc "Get the current weather for a location"
input_schema GetWeatherInput
def call(input)
"The weather in #{input.location} is sunny and 72°F."
end
end
client.beta.messages.tool_runner(
model: :"claude-opus-4-7",
max_tokens: 16000,
tools: [GetWeather.new],
messages: [{ role: "user", content: "What's the weather in San Francisco?" }]
).each_message do |message|
puts message.content
end
```
### Manual Loop
See the [shared tool use concepts](../shared/tool-use-concepts.md) for the tool definition format and agentic loop pattern.
---
## Prompt Caching
`system_:` (trailing underscore — avoids shadowing `Kernel#system`) takes an array of text blocks; set `cache_control` on the last block. Plain hashes work via the `OrHash` type alias. For placement patterns and the silent-invalidator audit checklist, see `shared/prompt-caching.md`.
```ruby
message = client.messages.create(
model: :"claude-opus-4-7",
max_tokens: 16000,
system_: [
{ type: "text", text: long_system_prompt, cache_control: { type: "ephemeral" } }
],
messages: [{ role: "user", content: "Summarize the key points" }]
)
```
For 1-hour TTL: `cache_control: { type: "ephemeral", ttl: "1h" }`. There's also a top-level `cache_control:` on `messages.create` that auto-places on the last cacheable block.
Verify hits via `message.usage.cache_creation_input_tokens` / `message.usage.cache_read_input_tokens`.
resources/bundled/skills/claude-api/ruby/managed-agents/README.md
+389
View File
@@ -0,0 +1,389 @@
# Managed Agents — Ruby
> **Bindings not shown here:** This README covers the most common managed-agents flows for Ruby. If you need a class, method, namespace, field, or behavior that isn't shown, WebFetch the Ruby SDK repo **or the relevant docs page** from `shared/live-sources.md` rather than guess. Do not extrapolate from cURL shapes or another language's SDK.
> **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `client.beta.agents.create` and pass it to every subsequent `client.beta.sessions.create`; do not call `agents.create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. The examples below show in-code creation for completeness; in production the create call belongs in setup, not in the request path.
## Installation
```bash
gem install anthropic
```
## Client Initialization
```ruby
require "anthropic"
# Default (uses ANTHROPIC_API_KEY env var)
client = Anthropic::Client.new
# Explicit API key
client = Anthropic::Client.new(api_key: "your-api-key")
```
> ⚠️ **Trailing underscores:** The Ruby SDK uses `system_:` and `send_(` (trailing underscore) to avoid shadowing `Kernel#system` and `Kernel#send`. Use these forms throughout managed-agents code.
---
## Create an Environment
```ruby
environment = client.beta.environments.create(
name: "my-dev-env",
config: {
type: "cloud",
networking: {type: "unrestricted"}
}
)
puts "Environment ID: #{environment.id}" # env_...
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** `model`/`system_`/`tools` live on the agent object, not the session. Always start with `client.beta.agents.create()` — the session takes either `agent: agent.id` or the typed hash form `agent: {type: "agent", id: agent.id, version: agent.version}`.
### Minimal
```ruby
# 1. Create the agent (reusable, versioned)
agent = client.beta.agents.create(
name: "Coding Assistant",
model: :"claude-opus-4-7",
system_: "You are a helpful coding assistant.",
tools: [{type: "agent_toolset_20260401"}]
)
# 2. Start a session
session = client.beta.sessions.create(
agent: {type: "agent", id: agent.id, version: agent.version},
environment_id: environment.id,
title: "Quickstart session"
)
puts "Session ID: #{session.id}"
```
### Updating an Agent
Updates create new versions; the agent object is immutable per version.
```ruby
updated_agent = client.beta.agents.update(
agent.id,
version: agent.version,
system_: "You are a helpful coding agent. Always write tests."
)
puts "New version: #{updated_agent.version}"
# List all versions
client.beta.agents.versions.list(agent.id).auto_paging_each do |version|
puts "Version #{version.version}: #{version.updated_at.iso8601}"
end
# Archive the agent
archived = client.beta.agents.archive(agent.id)
puts "Archived at: #{archived.archived_at.iso8601}"
```
---
## Send a User Message
```ruby
client.beta.sessions.events.send_(
session.id,
events: [{
type: "user.message",
content: [{type: "text", text: "Review the auth module"}]
}]
)
```
> 💡 **Stream-first:** Open the stream *before* (or concurrently with) sending the message. The stream only delivers events that occur after it opens — stream-after-send means early events arrive buffered in one batch. See [Steering Patterns](../../shared/managed-agents-events.md#steering-patterns).
---
## Stream Events (SSE)
```ruby
# Open the stream first, then send the user message
stream = client.beta.sessions.events.stream_events(session.id)
client.beta.sessions.events.send_(
session.id,
events: [{
type: "user.message",
content: [{type: "text", text: "Summarize the repo README"}]
}]
)
stream.each do |event|
case event.type
in :"agent.message"
event.content.each { |block| print block.text }
in :"agent.tool_use"
puts "\n[Using tool: #{event.name}]"
in :"session.status_idle"
break
in :"session.error"
puts "\n[Error: #{event.error&.message || "unknown"}]"
break
else
# ignore other event types
end
end
```
> ℹ️ Event `.type` is a Symbol (compare with `:"agent.message"`, not `"agent.message"`).
### Reconnecting and Tailing
When reconnecting mid-session, list past events first to dedupe, then tail live events:
```ruby
require "set"
stream = client.beta.sessions.events.stream_events(session.id)
# Stream is open and buffering. List history before tailing live.
seen_event_ids = Set.new
client.beta.sessions.events.list(session.id).auto_paging_each { |past| seen_event_ids << past.id }
# Tail live events, skipping anything already seen
stream.each do |event|
next if seen_event_ids.include?(event.id)
seen_event_ids << event.id
case event.type
in :"agent.message"
event.content.each { |block| print block.text }
in :"session.status_idle"
break
else
# ignore other event types
end
end
```
---
## Provide Custom Tool Result
> ℹ️ The Ruby managed-agents bindings for `user.custom_tool_result` are not yet documented in this skill or in the apps source examples. Refer to `shared/managed-agents-events.md` for the wire format and the `anthropic` Ruby gem repository for the corresponding params.
---
## Poll Events
```ruby
client.beta.sessions.events.list(session.id).auto_paging_each do |event|
puts "#{event.type}: #{event.id}"
end
```
---
## Upload a File
```ruby
require "pathname"
file = client.beta.files.upload(file: Pathname("data.csv"))
puts "File ID: #{file.id}"
# Mount in a session
session = client.beta.sessions.create(
agent: agent.id,
environment_id: environment.id,
resources: [
{
type: "file",
file_id: file.id,
mount_path: "/workspace/data.csv"
}
]
)
```
### Add and Manage Resources on an Existing Session
```ruby
# Attach an additional file to an open session
resource = client.beta.sessions.resources.add(
session.id,
type: "file",
file_id: file.id
)
puts resource.id # "sesrsc_01ABC..."
# List resources on the session
listed = client.beta.sessions.resources.list(session.id)
listed.data.each { |entry| puts "#{entry.id} #{entry.type}" }
# Detach a resource
client.beta.sessions.resources.delete(resource.id, session_id: session.id)
```
---
## List and Download Session Files
> ℹ️ Listing and downloading files an agent wrote during a session is not yet documented for Ruby in this skill or in the apps source examples. See `shared/managed-agents-events.md` and the `anthropic` Ruby gem repository for the file list/download bindings.
---
## Session Management
```ruby
# List environments
environments = client.beta.environments.list
# Retrieve a specific environment
env = client.beta.environments.retrieve(environment.id)
# Archive an environment (read-only, existing sessions continue)
client.beta.environments.archive(environment.id)
# Delete an environment (only if no sessions reference it)
client.beta.environments.delete(environment.id)
# Delete a session
client.beta.sessions.delete(session.id)
```
---
## MCP Server Integration
```ruby
# Agent declares MCP server (no auth here — auth goes in a vault)
agent = client.beta.agents.create(
name: "GitHub Assistant",
model: :"claude-opus-4-7",
mcp_servers: [
{
type: "url",
name: "github",
url: "https://api.githubcopilot.com/mcp/"
}
],
tools: [
{type: "agent_toolset_20260401"},
{type: "mcp_toolset", mcp_server_name: "github"}
]
)
# Session attaches vault(s) containing credentials for those MCP server URLs
session = client.beta.sessions.create(
agent: {type: "agent", id: agent.id, version: agent.version},
environment_id: environment.id,
vault_ids: [vault.id]
)
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
---
## Vaults
```ruby
# Create a vault
vault = client.beta.vaults.create(
display_name: "Alice",
metadata: {external_user_id: "usr_abc123"}
)
puts vault.id # "vlt_01ABC..."
# Add an OAuth credential
credential = client.beta.vaults.credentials.create(
vault.id,
display_name: "Alice's Slack",
auth: {
type: "mcp_oauth",
mcp_server_url: "https://mcp.slack.com/mcp",
access_token: "xoxp-...",
expires_at: "2026-04-15T00:00:00Z",
refresh: {
token_endpoint: "https://slack.com/api/oauth.v2.access",
client_id: "1234567890.0987654321",
scope: "channels:read chat:write",
refresh_token: "xoxe-1-...",
token_endpoint_auth: {
type: "client_secret_post",
client_secret: "abc123..."
}
}
}
)
# Rotate the credential (e.g., after a token refresh)
client.beta.vaults.credentials.update(
credential.id,
vault_id: vault.id,
auth: {
type: "mcp_oauth",
access_token: "xoxp-new-...",
expires_at: "2026-05-15T00:00:00Z",
refresh: {refresh_token: "xoxe-1-new-..."}
}
)
# Archive a vault
client.beta.vaults.archive(vault.id)
```
---
## GitHub Repository Integration
Mount a GitHub repository as a session resource (a vault holds the GitHub MCP credential):
```ruby
session = client.beta.sessions.create(
agent: agent.id,
environment_id: environment.id,
vault_ids: [vault.id],
resources: [
{
type: "github_repository",
url: "https://github.com/org/repo",
mount_path: "/workspace/repo",
authorization_token: "ghp_your_github_token"
}
]
)
```
Multiple repositories on the same session:
```ruby
resources = [
{
type: "github_repository",
url: "https://github.com/org/frontend",
mount_path: "/workspace/frontend",
authorization_token: "ghp_your_github_token"
},
{
type: "github_repository",
url: "https://github.com/org/backend",
mount_path: "/workspace/backend",
authorization_token: "ghp_your_github_token"
}
]
```
Rotating a repository's authorization token:
```ruby
listed = client.beta.sessions.resources.list(session.id)
repo_resource_id = listed.data.first.id
client.beta.sessions.resources.update(
repo_resource_id,
session_id: session.id,
authorization_token: "ghp_your_new_github_token"
)
```
resources/bundled/skills/claude-api/shared/agent-design.md
+101
View File
@@ -0,0 +1,101 @@
# Agent Design Patterns
This file covers decision heuristics for building agents on the Claude API: which primitives to reach for, how to design your tool surface, and how to manage context and cost over long runs. For per-tool mechanics and code examples, see `tool-use-concepts.md` and the language-specific folders.
---
## Model Parameters
| Parameter | When to use it | What to expect |
| --- | --- | --- |
| **Adaptive thinking** (`thinking: {type: "adaptive"}`) | When you want Claude to control when and how much to think. | Claude determines thinking depth per request and automatically interleaves thinking between tool calls. No token budget to tune. |
| **Effort** (`output_config: {effort: ...}`) | When adjusting the tradeoff between thoroughness and token efficiency. | Lower effort → fewer and more-consolidated tool calls, less preamble, terser confirmations. `medium` is often a favorable balance. Use `max` when correctness matters more than cost. |
See `SKILL.md` §Thinking & Effort for model support and parameter details.
---
## Designing Your Tool Surface
### Bash vs. dedicated tools
Claude doesn't know your application's security boundary, approval policy, or UX surface. Claude emits tool calls; your harness handles them. The shape of those tool calls determines what the harness can do.
A **bash tool** gives Claude broad programmatic leverage — it can perform almost any action. But it gives the harness only an opaque command string, the same shape for every action. Promoting an action to a **dedicated tool** gives the harness an action-specific hook with typed arguments it can intercept, gate, render, or audit.
**When to promote an action to a dedicated tool:**
- **Security boundary.** Actions that require gating are natural candidates. Reversibility is a useful criterion: hard-to-reverse actions (external API calls, sending messages, deleting data) can be gated behind user confirmation. A `send_email` tool is easy to gate; `bash -c "curl -X POST ..."` is not.
- **Staleness checks.** A dedicated `edit` tool can reject writes if the file changed since Claude last read it. Bash can't enforce that invariant.
- **Rendering.** Some actions benefit from custom UI. Claude Code promotes question-asking to a tool so it can render as a modal, present options, and block the agent loop until answered.
- **Scheduling.** Read-only tools like `glob` and `grep` can be marked parallel-safe. When the same actions run through bash, the harness can't tell a parallel-safe `grep` from a parallel-unsafe `git push`, so it must serialize.
**Rule of thumb:** Start with bash for breadth. Promote to dedicated tools when you need to gate, render, audit, or parallelize the action.
---
## Anthropic-Provided Tools
| Tool | Side | When to use it | What to expect |
| --- | --- | --- | --- |
| **Bash** | Client | Claude needs to execute shell commands. | Claude emits commands; your harness executes them. Reference implementation provided. |
| **Text editor** | Client | Claude needs to read or edit files. | Claude views, creates, and edits files via your implementation. Reference implementation provided. |
| **Computer use** | Client or Server | Claude needs to interact with GUIs, web apps, or visual interfaces. | Claude takes screenshots and issues mouse/keyboard commands. Can be self-hosted (you run the environment) or Anthropic-hosted. |
| **Code execution** | Server | Claude needs to run code in a sandbox you don't want to manage. | Anthropic-hosted container with built-in file and bash sub-tools. No client-side execution. |
| **Web search / fetch** | Server | Claude needs information past its training cutoff (news, current events, recent docs) or the content of a specific URL. | Claude issues a query or URL; Anthropic executes it and returns results with citations. |
| **Memory** | Client | Claude needs to save context across sessions. | Claude reads/writes a `/memories` directory. You implement the storage backend. |
**Client-side** tools are defined by Anthropic (name, schema, Claude's usage pattern) but executed by your harness. Anthropic provides reference implementations. **Server-side** tools run entirely on Anthropic infrastructure — declare them in `tools` and Claude handles the rest.
---
## Composing Tool Calls: Programmatic Tool Calling
With standard tool use, each tool call is a round trip: Claude calls the tool, the result lands in Claude's context, Claude reasons about it, then calls the next tool. Three sequential actions (read profile → look up orders → check inventory) means three round trips. Each adds latency and tokens, and most of the intermediate data is never needed again.
**Programmatic tool calling (PTC)** lets Claude compose those calls into a script instead. The script runs in the code execution container. When the script calls a tool, the container pauses, the call is executed (client-side or server-side), and the result returns to the running code — not to Claude's context. The script processes it with normal control flow (loops, filters, branches). Only the script's final output returns to Claude.
| When to use it | What to expect |
| --- | --- |
| Many sequential tool calls, or large intermediate results you want filtered before they hit the context window. | Claude writes code that invokes tools as functions. Runs in the code execution container. Token cost scales with final output, not intermediate results. |
---
## Scaling the Tool and Instruction Set
| Feature | When to use it | What to expect |
| --- | --- | --- |
| **Tool search** | Many tools available, but only a few relevant per request. Don't want all schemas in context upfront. | Claude searches the tool set and loads only relevant schemas. Tool definitions are appended, not swapped — preserves cache (see Caching below). |
| **Skills** | Task-specific instructions Claude should load only when relevant. | Each skill is a folder with a `SKILL.md`. The skill's description sits in context by default; Claude reads the full file when the task calls for it. |
Both patterns keep the fixed context small and load detail on demand.
---
## Long-Running Agents: Managing Context
| Pattern | When to use it | What to expect |
| --- | --- | --- |
| **Context editing** | Context grows stale over many turns (old tool results, completed thinking). | Tool results and thinking blocks are cleared based on configurable thresholds. Keeps the transcript lean without summarizing. |
| **Compaction** | Conversation likely to reach or exceed the context window limit. | Earlier context is summarized into a compaction block server-side. See `SKILL.md` §Compaction for the critical `response.content` handling. |
| **Memory** | State must persist across sessions (not just within one conversation). | Claude reads/writes files in a memory directory. Survives process restarts. |
**Choosing between them:** Context editing and compaction operate within a session — editing prunes stale turns, compaction summarizes when you're near the limit. Memory is for cross-session persistence. Many long-running agents use all three.
---
## Caching for Agents
**Read `prompt-caching.md` first.** It covers the prefix-match invariant, breakpoint placement, the silent-invalidator audit, and why changing tools or models mid-session breaks the cache. This section covers only the agent-specific workarounds for those constraints.
| Constraint (from `prompt-caching.md`) | Agent-specific workaround |
| --- | --- |
| Editing the system prompt mid-session invalidates the cache. | Append a `<system-reminder>` block in the `messages` array instead. The cached prefix stays intact. Claude Code uses this for time updates and mode transitions. |
| Switching models mid-session invalidates the cache. | Spawn a **subagent** with the cheaper model for the sub-task; keep the main loop on one model. Claude Code's Explore subagents use Haiku this way. |
| Adding/removing tools mid-session invalidates the cache. | Use **tool search** for dynamic discovery — it appends tool schemas rather than swapping them, so the existing prefix is preserved. |
For multi-turn breakpoint placement, use top-level auto-caching — see `prompt-caching.md` §Placement patterns.
---
For live documentation on any of these features, see `live-sources.md`.
resources/bundled/skills/claude-api/shared/error-codes.md
+213
View File
@@ -0,0 +1,213 @@
# HTTP Error Codes Reference
This file documents HTTP error codes returned by the Claude API, their common causes, and how to handle them. For language-specific error handling examples, see the `python/` or `typescript/` folders.
## Error Code Summary
| Code | Error Type | Retryable | Common Cause |
| ---- | ----------------------- | --------- | ------------------------------------ |
| 400 | `invalid_request_error` | No | Invalid request format or parameters |
| 401 | `authentication_error` | No | Invalid or missing API key |
| 403 | `permission_error` | No | API key lacks permission |
| 404 | `not_found_error` | No | Invalid endpoint or model ID |
| 413 | `request_too_large` | No | Request exceeds size limits |
| 429 | `rate_limit_error` | Yes | Too many requests |
| 500 | `api_error` | Yes | Anthropic service issue |
| 529 | `overloaded_error` | Yes | API is temporarily overloaded |
## Detailed Error Information
### 400 Bad Request
**Causes:**
- Malformed JSON in request body
- Missing required parameters (`model`, `max_tokens`, `messages`)
- Invalid parameter types (e.g., string where integer expected)
- Empty messages array
- Messages not alternating user/assistant
**Example error:**
```json
{
"type": "error",
"error": {
"type": "invalid_request_error",
"message": "messages: roles must alternate between \"user\" and \"assistant\""
},
"request_id": "req_011CSHoEeqs5C35K2UUqR7Fy"
}
```
**Fix:** Validate request structure before sending. Check that:
- `model` is a valid model ID
- `max_tokens` is a positive integer
- `messages` array is non-empty and alternates correctly
---
### 401 Unauthorized
**Causes:**
- Missing `x-api-key` header or `Authorization` header
- Invalid API key format
- Revoked or deleted API key
**Fix:** Ensure `ANTHROPIC_API_KEY` environment variable is set correctly.
---
### 403 Forbidden
**Causes:**
- API key doesn't have access to the requested model
- Organization-level restrictions
- Attempting to access beta features without beta access
**Fix:** Check your API key permissions in the Console. You may need a different API key or to request access to specific features.
---
### 404 Not Found
**Causes:**
- Typo in model ID (e.g., `claude-sonnet-4.6` instead of `claude-sonnet-4-6`)
- Using deprecated model ID
- Invalid API endpoint
**Fix:** Use exact model IDs from the models documentation. You can use aliases (e.g., `claude-opus-4-7`).
---
### 413 Request Too Large
**Causes:**
- Request body exceeds maximum size
- Too many tokens in input
- Image data too large
**Fix:** Reduce input size — truncate conversation history, compress/resize images, or split large documents into chunks.
---
### 400 Validation Errors
Some 400 errors are specifically related to parameter validation:
- `max_tokens` exceeds model's limit
- Invalid `temperature` value (must be 0.0-1.0)
- `budget_tokens` >= `max_tokens` in extended thinking
- Invalid tool definition schema
**Model-specific 400s on Opus 4.7:**
- `temperature`, `top_p`, `top_k` are removed — sending any of them returns 400. Delete the parameter; see `shared/model-migration.md` → Per-SDK Syntax Reference.
- `thinking: {type: "enabled", budget_tokens: N}` is removed — sending it returns 400. Use `thinking: {type: "adaptive"}` instead.
**Common mistake with extended thinking on older models (Opus 4.6 and earlier):**
```
# Wrong: budget_tokens must be < max_tokens
thinking: budget_tokens=10000, max_tokens=1000 → Error!
# Correct
thinking: budget_tokens=10000, max_tokens=16000
```
---
### 429 Rate Limited
**Causes:**
- Exceeded requests per minute (RPM)
- Exceeded tokens per minute (TPM)
- Exceeded tokens per day (TPD)
**Headers to check:**
- `retry-after`: Seconds to wait before retrying
- `x-ratelimit-limit-*`: Your limits
- `x-ratelimit-remaining-*`: Remaining quota
**Fix:** The Anthropic SDKs automatically retry 429 and 5xx errors with exponential backoff (default: `max_retries=2`). For custom retry behavior, see the language-specific error handling examples.
---
### 500 Internal Server Error
**Causes:**
- Temporary Anthropic service issue
- Bug in API processing
**Fix:** Retry with exponential backoff. If persistent, check [status.anthropic.com](https://status.anthropic.com).
---
### 529 Overloaded
**Causes:**
- High API demand
- Service capacity reached
**Fix:** Retry with exponential backoff. Consider using a different model (Haiku is often less loaded), spreading requests over time, or implementing request queuing.
---
## Common Mistakes and Fixes
| Mistake | Error | Fix |
| ------------------------------- | ---------------- | ------------------------------------------------------- |
| `temperature`/`top_p`/`top_k` on Opus 4.7 | 400 | Remove the parameter (see `shared/model-migration.md`) |
| `budget_tokens` on Opus 4.7 | 400 | Use `thinking: {type: "adaptive"}` |
| `budget_tokens` >= `max_tokens` (older models) | 400 | Ensure `budget_tokens` < `max_tokens` |
| Typo in model ID | 404 | Use valid model ID like `claude-opus-4-7` |
| First message is `assistant` | 400 | First message must be `user` |
| Consecutive same-role messages | 400 | Alternate `user` and `assistant` |
| API key in code | 401 (leaked key) | Use environment variable |
| Custom retry needs | 429/5xx | SDK retries automatically; customize with `max_retries` |
## Typed Exceptions in SDKs
**Always use the SDK's typed exception classes** instead of checking error messages with string matching. Each HTTP error code maps to a specific exception class:
| HTTP Code | TypeScript Class | Python Class |
| --------- | --------------------------------- | --------------------------------- |
| 400 | `Anthropic.BadRequestError` | `anthropic.BadRequestError` |
| 401 | `Anthropic.AuthenticationError` | `anthropic.AuthenticationError` |
| 403 | `Anthropic.PermissionDeniedError` | `anthropic.PermissionDeniedError` |
| 404 | `Anthropic.NotFoundError` | `anthropic.NotFoundError` |
| 429 | `Anthropic.RateLimitError` | `anthropic.RateLimitError` |
| 500+ | `Anthropic.InternalServerError` | `anthropic.InternalServerError` |
| Any | `Anthropic.APIError` | `anthropic.APIError` |
```typescript
// ✅ Correct: use typed exceptions
try {
const response = await client.messages.create({...});
} catch (error) {
if (error instanceof Anthropic.RateLimitError) {
// Handle rate limiting
} else if (error instanceof Anthropic.APIError) {
console.error(`API error ${error.status}:`, error.message);
}
}
// ❌ Wrong: don't check error messages with string matching
try {
const response = await client.messages.create({...});
} catch (error) {
const msg = error instanceof Error ? error.message : String(error);
if (msg.includes("429") || msg.includes("rate_limit")) { ... }
}
```
All exception classes extend `Anthropic.APIError`, which has a `status` property. Use `instanceof` checks from most specific to least specific (e.g., check `RateLimitError` before `APIError`).
resources/bundled/skills/claude-api/shared/live-sources.md
+132
View File
@@ -0,0 +1,132 @@
# Live Documentation Sources
This file contains WebFetch URLs for fetching current information from platform.claude.com and Agent SDK repositories. Use these when users need the latest data that may have changed since the cached content was last updated.
## When to Use WebFetch
- User explicitly asks for "latest" or "current" information
- Cached data seems incorrect
- User asks about features not covered in cached content
- User needs specific API details or examples
## Claude API Documentation URLs
### Models & Pricing
| Topic | URL | Extraction Prompt |
| --------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| Models Overview | `https://platform.claude.com/docs/en/about-claude/models/overview.md` | "Extract current model IDs, context windows, and pricing for all Claude models" |
| Migration Guide | `https://platform.claude.com/docs/en/about-claude/models/migration-guide.md` | "Extract breaking changes, deprecated parameters, and per-model migration steps when moving to a newer Claude model" |
| Pricing | `https://platform.claude.com/docs/en/pricing.md` | "Extract current pricing per million tokens for input and output" |
### Core Features
| Topic | URL | Extraction Prompt |
| ----------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Extended Thinking | `https://platform.claude.com/docs/en/build-with-claude/extended-thinking.md` | "Extract extended thinking parameters, budget_tokens requirements, and usage examples" |
| Adaptive Thinking | `https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking.md` | "Extract adaptive thinking setup, effort levels, and Claude Opus 4.7 usage examples" |
| Effort Parameter | `https://platform.claude.com/docs/en/build-with-claude/effort.md` | "Extract effort levels, cost-quality tradeoffs, and interaction with thinking" |
| Tool Use | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview.md` | "Extract tool definition schema, tool_choice options, and handling tool results" |
| Streaming | `https://platform.claude.com/docs/en/build-with-claude/streaming.md` | "Extract streaming event types, SDK examples, and best practices" |
| Prompt Caching | `https://platform.claude.com/docs/en/build-with-claude/prompt-caching.md` | "Extract cache_control usage, pricing benefits, and implementation examples" |
### Media & Files
| Topic | URL | Extraction Prompt |
| ----------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------- |
| Vision | `https://platform.claude.com/docs/en/build-with-claude/vision.md` | "Extract supported image formats, size limits, and code examples" |
| PDF Support | `https://platform.claude.com/docs/en/build-with-claude/pdf-support.md` | "Extract PDF handling capabilities, limits, and examples" |
### API Operations
| Topic | URL | Extraction Prompt |
| ---------------- | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| Batch Processing | `https://platform.claude.com/docs/en/build-with-claude/batch-processing.md` | "Extract batch API endpoints, request format, and polling for results" |
| Files API | `https://platform.claude.com/docs/en/build-with-claude/files.md` | "Extract file upload, download, and referencing in messages, including supported types and beta header" |
| Token Counting | `https://platform.claude.com/docs/en/build-with-claude/token-counting.md` | "Extract token counting API usage and examples" |
| Rate Limits | `https://platform.claude.com/docs/en/api/rate-limits.md` | "Extract current rate limits by tier and model" |
| Errors | `https://platform.claude.com/docs/en/api/errors.md` | "Extract HTTP error codes, meanings, and retry guidance" |
### Tools
| Topic | URL | Extraction Prompt |
| -------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Code Execution | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/code-execution-tool.md` | "Extract code execution tool setup, file upload, container reuse, and response handling" |
| Computer Use | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use.md` | "Extract computer use tool setup, capabilities, and implementation examples" |
| Bash Tool | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/bash-tool.md` | "Extract bash tool schema, reference implementation, and security considerations" |
| Text Editor | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool.md` | "Extract text editor tool commands, schema, and reference implementation" |
| Memory Tool | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool.md` | "Extract memory tool commands, directory structure, and implementation patterns" |
| Tool Search | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool.md` | "Extract tool search setup, when to use, and cache interaction" |
| Programmatic Tool Calling | `https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling.md` | "Extract PTC setup, script execution model, and tool invocation from code" |
| Skills | `https://platform.claude.com/docs/en/agents-and-tools/skills.md` | "Extract skill folder structure, SKILL.md format, and loading behavior" |
### Advanced Features
| Topic | URL | Extraction Prompt |
| ------------------ | ----------------------------------------------------------------------------- | --------------------------------------------------- |
| Structured Outputs | `https://platform.claude.com/docs/en/build-with-claude/structured-outputs.md` | "Extract output_config.format usage and schema enforcement" |
| Compaction | `https://platform.claude.com/docs/en/build-with-claude/compaction.md` | "Extract compaction setup, trigger config, and streaming with compaction" |
| Context Editing | `https://platform.claude.com/docs/en/build-with-claude/context-editing.md` | "Extract context editing thresholds, what gets cleared, and configuration" |
| Citations | `https://platform.claude.com/docs/en/build-with-claude/citations.md` | "Extract citation format and implementation" |
| Context Windows | `https://platform.claude.com/docs/en/build-with-claude/context-windows.md` | "Extract context window sizes and token management" |
### Managed Agents
Use these when a managed-agents binding, behavior, or wire-level detail isn't covered in the cached `shared/managed-agents-*.md` concept files or in `{lang}/managed-agents/README.md`.
| Topic | URL | Extraction Prompt |
| --------------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| Overview | `https://platform.claude.com/docs/en/managed-agents/overview.md` | "Extract the high-level architecture and how agents/sessions/environments/vaults fit together" |
| Quickstart | `https://platform.claude.com/docs/en/managed-agents/quickstart.md` | "Extract the minimal end-to-end agent → environment → session → stream code path" |
| Agent Setup | `https://platform.claude.com/docs/en/managed-agents/agent-setup.md` | "Extract agent create/update/list-versions/archive lifecycle and parameters" |
| Define Outcomes | `https://platform.claude.com/docs/en/managed-agents/define-outcomes.md` | "Extract outcome definitions, evaluation hooks, and success criteria configuration" |
| Sessions | `https://platform.claude.com/docs/en/managed-agents/sessions.md` | "Extract session lifecycle, status transitions, idle/terminated semantics, and resume rules" |
| Environments | `https://platform.claude.com/docs/en/managed-agents/environments.md` | "Extract environment config (cloud/networking), management endpoints, and reuse model" |
| Events and Streaming | `https://platform.claude.com/docs/en/managed-agents/events-and-streaming.md` | "Extract event stream types, stream-first ordering, reconnect/dedupe, and steering patterns" |
| Tools | `https://platform.claude.com/docs/en/managed-agents/tools.md` | "Extract built-in toolset, custom tool definitions, and tool result wire format" |
| Files | `https://platform.claude.com/docs/en/managed-agents/files.md` | "Extract file upload, mount paths, session resources, and listing/downloading session outputs" |
| Permission Policies | `https://platform.claude.com/docs/en/managed-agents/permission-policies.md` | "Extract permission policy types (allow/deny/confirm) and per-tool config" |
| Multi-Agent | `https://platform.claude.com/docs/en/managed-agents/multi-agent.md` | "Extract multi-agent composition patterns, sub-agent invocation, and result handoff" |
| Observability | `https://platform.claude.com/docs/en/managed-agents/observability.md` | "Extract logging, tracing, and usage telemetry exposed by managed agents" |
| GitHub | `https://platform.claude.com/docs/en/managed-agents/github.md` | "Extract github_repository resource shape, multi-repo mounting, and token rotation" |
| MCP Connector | `https://platform.claude.com/docs/en/managed-agents/mcp-connector.md` | "Extract MCP server declaration on agents and vault-based credential injection at session" |
| Vaults | `https://platform.claude.com/docs/en/managed-agents/vaults.md` | "Extract vault create, credential add/rotate, OAuth refresh shape, and archive" |
| Skills | `https://platform.claude.com/docs/en/managed-agents/skills.md` | "Extract skill packaging and loading model for managed agents" |
| Memory | `https://platform.claude.com/docs/en/managed-agents/memory.md` | "Extract memory resource shape, scoping, and lifecycle" |
| Onboarding | `https://platform.claude.com/docs/en/managed-agents/onboarding.md` | "Extract first-run setup, prerequisites, and account/region requirements" |
| Cloud Containers | `https://platform.claude.com/docs/en/managed-agents/cloud-containers.md` | "Extract cloud container runtime, image config, and network/storage knobs" |
| Migration | `https://platform.claude.com/docs/en/managed-agents/migration.md` | "Extract migration paths from earlier APIs/preview shapes to GA managed agents" |
### Anthropic CLI
The `ant` CLI provides terminal access to the Claude API. Every API resource is exposed as a subcommand. It is one convenient way to create agents, environments, sessions, and other resources from version-controlled YAML, and to inspect responses interactively.
| Topic | URL | Extraction Prompt |
| ------------- | ------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Anthropic CLI | `https://platform.claude.com/docs/en/api/sdks/cli.md` | "Extract CLI install, authentication, command structure, and the beta:agents/environments/sessions commands" |
---
## Claude API SDK Repositories
WebFetch these when a binding (class, method, namespace, field) isn't covered in the cached `{lang}/` skill files or in the managed-agents docs above. The SDKs include beta managed-agents support for `/v1/agents`, `/v1/sessions`, `/v1/environments`, and related resources — search the repo for `BetaManagedAgents`, `beta.agents`, `beta.sessions`, or the equivalent namespace for that language.
| SDK | URL | Extraction Prompt |
| ---------- | -------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| Python | `https://github.com/anthropics/anthropic-sdk-python` | "Extract beta managed-agents namespaces, classes, and method signatures (`client.beta.agents`, `client.beta.sessions`)" |
| TypeScript | `https://github.com/anthropics/anthropic-sdk-typescript` | "Extract beta managed-agents namespaces, classes, and method signatures (`client.beta.agents`, `client.beta.sessions`)" |
| Java | `https://github.com/anthropics/anthropic-sdk-java` | "Extract beta managed-agents classes, builders, and method signatures (`client.beta().agents()`, `BetaManagedAgents*`)" |
| Go | `https://github.com/anthropics/anthropic-sdk-go` | "Extract beta managed-agents types and method signatures (`client.Beta.Agents`, `BetaManagedAgents*` event types)" |
| Ruby | `https://github.com/anthropics/anthropic-sdk-ruby` | "Extract beta managed-agents methods and parameter shapes (`client.beta.agents`, `client.beta.sessions`)" |
| C# | `https://github.com/anthropics/anthropic-sdk-csharp` | "Extract beta managed-agents classes and method signatures (NuGet package, `BetaManagedAgents*` types)" |
| PHP | `https://github.com/anthropics/anthropic-sdk-php` | "Extract beta managed-agents classes and method signatures (`$client->beta->agents`, `BetaManagedAgents*` params)" |
---
## Fallback Strategy
If WebFetch fails (network issues, URL changed):
1. Use cached content from the language-specific files (note the cache date)
2. Inform user the data may be outdated
3. Suggest they check platform.claude.com or the GitHub repos directly
resources/bundled/skills/claude-api/shared/managed-agents-api-reference.md
+299
View File
@@ -0,0 +1,299 @@
# Managed Agents — Endpoint Reference
All endpoints require `x-api-key` and `anthropic-version: 2023-06-01` headers. Managed Agents endpoints additionally require the `anthropic-beta` header.
## Beta Headers
```
anthropic-beta: managed-agents-2026-04-01
```
The SDK adds this header automatically for all `client.beta.{agents,environments,sessions,vaults}.*` calls. Skills endpoints use `skills-2025-10-02`; Files endpoints use `files-api-2025-04-14`.
---
## SDK Method Reference
All resources are under the `beta` namespace. Python and TypeScript share identical method names.
| Resource | Python / TypeScript (`client.beta.*`) | Go (`client.Beta.*`) |
| --- | --- | --- |
| Agents | `agents.create` / `retrieve` / `update` / `list` / `archive` | `Agents.New` / `Get` / `Update` / `List` / `Archive` |
| Agent Versions | `agents.versions.list` | `Agents.Versions.List` |
| Environments | `environments.create` / `retrieve` / `update` / `list` / `delete` / `archive` | `Environments.New` / `Get` / `Update` / `List` / `Delete` / `Archive` |
| Sessions | `sessions.create` / `retrieve` / `update` / `list` / `delete` / `archive` | `Sessions.New` / `Get` / `Update` / `List` / `Delete` / `Archive` |
| Session Events | `sessions.events.list` / `send` / `stream` | `Sessions.Events.List` / `Send` / `StreamEvents` |
| Session Resources | `sessions.resources.add` / `retrieve` / `update` / `list` / `delete` | `Sessions.Resources.Add` / `Get` / `Update` / `List` / `Delete` |
| Vaults | `vaults.create` / `retrieve` / `update` / `list` / `delete` / `archive` | `Vaults.New` / `Get` / `Update` / `List` / `Delete` / `Archive` |
| Credentials | `vaults.credentials.create` / `retrieve` / `update` / `list` / `delete` / `archive` | `Vaults.Credentials.New` / `Get` / `Update` / `List` / `Delete` / `Archive` |
**Naming quirks to watch for:**
- Agents have **no delete** — only `archive`. Archive is **permanent**: the agent becomes read-only, new sessions cannot reference it, and there is no unarchive. Confirm with the user before archiving a production agent. Environments, Sessions, Vaults, and Credentials have both `delete` and `archive`; Session Resources, Files, and Skills are `delete`-only.
- Session resources use `add` (not `create`).
- Go's event stream is `StreamEvents` (not `Stream`).
**Agent shorthand:** `agent` on session create accepts either a bare string (`agent="agent_abc123"` — uses latest version) or the full reference object (`{type: "agent", id: "agent_abc123", version: 123}`).
**Model shorthand:** `model` on agent create accepts either a bare string (`model="claude-opus-4-7"` — uses `standard` speed) or the full config object (`{type: "model_config", id: "claude-opus-4-6", speed: "fast"}`). Note: `speed: "fast"` is only supported on Opus 4.6.
---
## Agents
**Step one of every flow.** Sessions require a pre-created agent — there is no inline agent config under `managed-agents-2026-04-01`.
| Method | Path | Operation | Description |
| -------- | ------------------------------------------------ | ---------------- | ---------------------------------------- |
| `GET` | `/v1/agents` | ListAgents | List agents |
| `POST` | `/v1/agents` | CreateAgent | Create a saved agent configuration |
| `GET` | `/v1/agents/{agent_id}` | GetAgent | Get agent details |
| `POST` | `/v1/agents/{agent_id}` | UpdateAgent | Update agent configuration |
| `POST` | `/v1/agents/{agent_id}/archive` | ArchiveAgent | Archive an agent. Makes it **read-only**; existing sessions continue, new sessions cannot reference it. No unarchive — this is the terminal state. |
| `GET` | `/v1/agents/{agent_id}/versions` | ListAgentVersions | List agent versions |
## Sessions
| Method | Path | Operation | Description |
| -------- | ------------------------------------------------ | ---------------- | ---------------------------------------- |
| `GET` | `/v1/sessions` | ListSessions | List sessions (paginated) |
| `POST` | `/v1/sessions` | CreateSession | Create a new session |
| `GET` | `/v1/sessions/{session_id}` | GetSession | Get session details |
| `POST` | `/v1/sessions/{session_id}` | UpdateSession | Update session metadata/title |
| `DELETE` | `/v1/sessions/{session_id}` | DeleteSession | Delete a session |
| `POST` | `/v1/sessions/{session_id}/archive` | ArchiveSession | Archive a session |
## Events
| Method | Path | Operation | Description |
| -------- | ------------------------------------------------ | ---------------- | ---------------------------------------- |
| `GET` | `/v1/sessions/{session_id}/events` | ListEvents | List events (polling, paginated) |
| `POST` | `/v1/sessions/{session_id}/events` | SendEvents | Send events (user message, tool result) |
| `GET` | `/v1/sessions/{session_id}/events/stream` | StreamEvents | Stream events via SSE |
## Session Resources
| Method | Path | Operation | Description |
| -------- | ------------------------------------------------------- | ---------------- | ---------------------------------------- |
| `GET` | `/v1/sessions/{session_id}/resources` | ListResources | List resources attached to session |
| `POST` | `/v1/sessions/{session_id}/resources` | AddResource | Attach file or github_repository mount (SDK method: `add`, not `create`) |
| `GET` | `/v1/sessions/{session_id}/resources/{resource_id}` | GetResource | Get a single resource |
| `POST` | `/v1/sessions/{session_id}/resources/{resource_id}` | UpdateResource | Update resource |
| `DELETE` | `/v1/sessions/{session_id}/resources/{resource_id}` | DeleteResource | Remove resource from session |
## Environments
| Method | Path | Operation | Description |
| -------- | ---------------------------------------------------------------- | -------------------- | ----------------------------------- |
| `POST` | `/v1/environments` | CreateEnvironment | Create environment |
| `GET` | `/v1/environments` | ListEnvironments | List environments |
| `GET` | `/v1/environments/{environment_id}` | GetEnvironment | Get environment details |
| `POST` | `/v1/environments/{environment_id}` | UpdateEnvironment | Update environment |
| `DELETE` | `/v1/environments/{environment_id}` | DeleteEnvironment | Delete environment. Returns 204. |
| `POST` | `/v1/environments/{environment_id}/archive` | ArchiveEnvironment | Archive environment. Makes it **read-only**; existing sessions continue, new sessions cannot reference it. No unarchive — this is the terminal state. |
## Vaults
Vaults store MCP credentials that Anthropic manages on your behalf — OAuth credentials with auto-refresh, or static bearer tokens. Attach to sessions via `vault_ids`. See `managed-agents-tools.md` §Vaults for the conceptual guide and credential shapes.
| Method | Path | Operation | Description |
| -------- | ------------------------------------------------ | ---------------- | ---------------------------------------- |
| `POST` | `/v1/vaults` | CreateVault | Create a vault |
| `GET` | `/v1/vaults` | ListVaults | List vaults |
| `GET` | `/v1/vaults/{vault_id}` | GetVault | Get vault details |
| `POST` | `/v1/vaults/{vault_id}` | UpdateVault | Update vault |
| `DELETE` | `/v1/vaults/{vault_id}` | DeleteVault | Delete vault |
| `POST` | `/v1/vaults/{vault_id}/archive` | ArchiveVault | Archive vault |
## Credentials
Credentials are individual secrets stored inside a vault.
| Method | Path | Operation | Description |
| -------- | ----------------------------------------------------------------- | ------------------ | ---------------------------- |
| `POST` | `/v1/vaults/{vault_id}/credentials` | CreateCredential | Create a credential |
| `GET` | `/v1/vaults/{vault_id}/credentials` | ListCredentials | List credentials in vault |
| `GET` | `/v1/vaults/{vault_id}/credentials/{credential_id}` | GetCredential | Get credential metadata |
| `POST` | `/v1/vaults/{vault_id}/credentials/{credential_id}` | UpdateCredential | Update credential |
| `DELETE` | `/v1/vaults/{vault_id}/credentials/{credential_id}` | DeleteCredential | Delete credential |
| `POST` | `/v1/vaults/{vault_id}/credentials/{credential_id}/archive` | ArchiveCredential | Archive credential |
## Files
| Method | Path | Operation | Description |
| -------- | ------------------------------------------------ | ---------------- | ---------------------------------------- |
| `POST` | `/v1/files` | UploadFile | Upload a file |
| `GET` | `/v1/files` | ListFiles | List files |
| `GET` | `/v1/files/{file_id}` | GetFile | Get file metadata (SDK method: `retrieve_metadata`) |
| `GET` | `/v1/files/{file_id}/content` | DownloadFile | Download file content |
| `DELETE` | `/v1/files/{file_id}` | DeleteFile | Delete a file |
## Skills
| Method | Path | Operation | Description |
| -------- | --------------------------------------------------------------- | ------------------ | ---------------------------- |
| `POST` | `/v1/skills` | CreateSkill | Create a skill |
| `GET` | `/v1/skills` | ListSkills | List skills |
| `GET` | `/v1/skills/{skill_id}` | GetSkill | Get skill details |
| `DELETE` | `/v1/skills/{skill_id}` | DeleteSkill | Delete a skill |
| `POST` | `/v1/skills/{skill_id}/versions` | CreateVersion | Create skill version |
| `GET` | `/v1/skills/{skill_id}/versions` | ListVersions | List skill versions |
| `GET` | `/v1/skills/{skill_id}/versions/{version}` | GetVersion | Get skill version |
| `DELETE` | `/v1/skills/{skill_id}/versions/{version}` | DeleteVersion | Delete skill version |
---
## Request/Response Schema Quick Reference
### CreateAgent Request Body
**Always start here.** `model`, `system`, `tools`, `mcp_servers`, `skills` are top-level fields on this object — they do NOT go on the session.
```json
{
"name": "string (required, 1-256 chars)",
"model": "claude-opus-4-7 (required — bare string, or {id, speed} object)",
"description": "string (optional, up to 2048 chars)",
"system": "string (optional, up to 100,000 chars)",
"tools": [
{ "type": "agent_toolset_20260401" }
],
"skills": [
{ "type": "anthropic", "skill_id": "xlsx" },
{ "type": "custom", "skill_id": "skill_abc123", "version": "1" }
],
"mcp_servers": [
{
"type": "url",
"name": "github",
"url": "https://api.githubcopilot.com/mcp/"
}
],
"metadata": {
"key": "value (max 16 pairs, keys ≤64 chars, values ≤512 chars)"
}
}
```
> Limits: `tools` max 50, `skills` max 64, `mcp_servers` max 20 (unique names).
### CreateSession Request Body
```json
{
"agent": "agent_abc123 (required — string shorthand for latest version, or {type: \"agent\", id, version} object)",
"environment_id": "env_abc123 (required)",
"title": "string (optional)",
"resources": [
{
"type": "github_repository",
"url": "https://github.com/owner/repo (required)",
"authorization_token": "ghp_... (required)",
"mount_path": "/workspace/repo (optional — defaults to /workspace/<repo-name>)",
"checkout": { "type": "branch", "name": "main" }
}
],
"vault_ids": ["vlt_abc123 (optional — MCP credentials with auto-refresh)"],
"metadata": {
"key": "value"
}
}
```
> The `agent` field accepts only a string ID or `{type: "agent", id, version}` — `model`/`system`/`tools` live on the agent, not here.
>
> **`checkout`** accepts `{type: "branch", name: "..."}` or `{type: "commit", sha: "..."}`. Omit for the repo's default branch.
### CreateEnvironment Request Body
```json
{
"name": "string (required)",
"description": "string (optional)",
"config": {
"type": "cloud",
"networking": {
"type": "unrestricted | limited (union — see SDK types)"
},
"packages": { }
},
"metadata": { "key": "value" }
}
```
### SendEvents Request Body
```json
{
"events": [
{
"type": "user.message",
"content": [
{
"type": "text",
"text": "Hello"
}
]
}
]
}
```
### Tool Result Event
```json
{
"type": "user.custom_tool_result",
"custom_tool_use_id": "sevt_abc123",
"content": [{ "type": "text", "text": "Result data" }],
"is_error": false
}
```
---
## Error Handling
Managed Agents endpoints use the standard Anthropic API error format. Errors are returned with an HTTP status code and a JSON body containing `type`, `error`, and `request_id`:
```json
{
"type": "error",
"error": {
"type": "invalid_request_error",
"message": "Description of what went wrong"
},
"request_id": "req_011CRv1W3XQ8XpFikNYG7RnE"
}
```
Include the `request_id` when reporting issues to Anthropic — it lets us trace the request end-to-end. The inner `error.type` is one of the following:
| Status | Error type | Description |
|---|---|---|
| 400 | `invalid_request_error` | The request was malformed or missing required parameters |
| 401 | `authentication_error` | Invalid or missing API key |
| 403 | `permission_error` | The API key doesn't have permission for this operation |
| 404 | `not_found_error` | The requested resource doesn't exist |
| 409 | `invalid_request_error` | The request conflicts with the resource's current state (e.g., sending to an archived session) |
| 413 | `request_too_large` | The request body exceeds the maximum allowed size |
| 429 | `rate_limit_error` | Too many requests — check rate limit headers for retry timing |
| 500 | `api_error` | An internal server error occurred |
| 529 | `overloaded_error` | The service is temporarily overloaded — retry with backoff |
Note that `409 Conflict` carries `error.type: "invalid_request_error"` (there is no separate `conflict_error` type); inspect both the HTTP status and the `message` to distinguish conflicts from other invalid requests.
---
## Rate Limits
Managed Agents endpoints have per-organization request-per-minute (RPM) limits, separate from your [Messages API token limits](https://platform.claude.com/docs/en/api/rate-limits). Model inference inside a session still draws from your organization's standard ITPM/OTPM limits.
| Endpoint group | Scope | RPM | Max concurrent |
|---|---|---|---|
| Create operations (Agents, Sessions, Vaults) | organization | 60 | — |
| All other operations (Agents, Sessions, Vaults) | organization | 600 | — |
| All operations (Environments) | organization | 60 | 5 |
Files and Skills endpoints use the standard tier-based [rate limits](https://platform.claude.com/docs/en/api/rate-limits).
When a limit is exceeded the API returns `429` with a `rate_limit_error` (see [Error Handling](#error-handling) for the response envelope) and a `retry-after` header indicating how many seconds to wait before retrying. The Anthropic SDK reads this header and retries automatically.
resources/bundled/skills/claude-api/shared/managed-agents-client-patterns.md
+209
View File
@@ -0,0 +1,209 @@
# Managed Agents — Common Client Patterns
Patterns you'll write on the client side when driving a Managed Agent session, grounded in working SDK examples.
Code samples are TypeScript — Python and cURL follow the same shape; see `python/managed-agents/README.md` and `curl/managed-agents.md` for equivalents.
---
## 1. Lossless stream reconnect
**Problem:** SSE has no replay. If the connection drops mid-session, a naive reconnect re-opens the stream from "now" and you silently miss every event emitted in between.
**Solution:** on reconnect, fetch the full event history via `events.list()` *before* consuming the live stream, and dedupe on event ID as the live stream catches up.
```ts
const seenEventIds = new Set<string>()
const stream = await client.beta.sessions.events.stream(session.id)
// Stream is now open and buffering server-side. Read history first.
for await (const event of client.beta.sessions.events.list(session.id)) {
seenEventIds.add(event.id)
handle(event)
}
// Tail the live stream. Dedupe only gates handle() — terminal checks must run
// even for already-seen events, or a terminal event that was in the history
// response gets skipped by `continue` and the loop never exits.
for await (const event of stream) {
if (!seenEventIds.has(event.id)) {
seenEventIds.add(event.id)
handle(event)
}
if (event.type === 'session.status_terminated') break
if (event.type === 'session.status_idle' && event.stop_reason.type !== 'requires_action') break
}
```
---
## 2. `processed_at` — queued vs processed
Every event on the stream carries `processed_at` (ISO 8601). For client-sent events (`user.message`, `user.interrupt`, `user.tool_confirmation`, `user.custom_tool_result`) it's `null` when the event has been queued but not yet picked up by the agent, and populated once the agent processes it. The same event appears on the stream twice — once with `processed_at: null`, once with a timestamp.
```ts
for await (const event of stream) {
if (event.type === 'user.message') {
if (event.processed_at == null) onQueued(event.id)
else onProcessed(event.id, event.processed_at)
}
}
```
Use this to drive pending → acknowledged UI state for anything you send. How you map a locally-rendered optimistic message to the server-assigned `event.id` is application-specific (typically via the return value of `events.send()` or FIFO ordering).
---
## 3. Interrupt a running session
Send `user.interrupt` as a normal event. The session keeps running until it reaches a safe boundary, then goes idle.
```ts
await client.beta.sessions.events.send(session.id, {
events: [{ type: 'user.interrupt' }],
})
// Drain until the session is truly done — see Pattern 5 for the full gate.
for await (const event of stream) {
if (event.type === 'session.status_terminated') break
if (
event.type === 'session.status_idle' &&
event.stop_reason.type !== 'requires_action'
) break
}
```
Reference: `interrupt.ts` — sends the interrupt the moment it sees `span.model_request_start`, drains to idle, then verifies via `sessions.retrieve()`.
---
## 4. `tool_confirmation` round-trip
When the agent has `permission_policy: { type: 'always_ask' }`, any call to that tool fires an `agent.tool_use` event with `evaluated_permission === 'ask'` and the session goes idle waiting for a decision. Respond with `user.tool_confirmation`.
```ts
for await (const event of stream) {
if (event.type === 'agent.tool_use' && event.evaluated_permission === 'ask') {
await client.beta.sessions.events.send(session.id, {
events: [{
type: 'user.tool_confirmation',
tool_use_id: event.id, // not a toolu_ id — use event.id
result: 'allow', // or 'deny'
// deny_message: '...', // optional, only with result: 'deny'
}],
})
}
}
```
Key points:
- `tool_use_id` is `event.id` (typically `sevt_...`), **not** a `toolu_...` ID.
- `result` is `'allow' | 'deny'`. Use `deny_message` to tell the model *why* you denied — it gets surfaced back to the agent.
- Multiple pending tools: respond once per `agent.tool_use` event with `evaluated_permission === 'ask'`.
Reference: `tool-permissions.ts`.
---
## 5. Correct idle-break gate
Do not break on `session.status_idle` alone. The session goes idle transiently — e.g. between parallel tool executions, while waiting for a `user.tool_confirmation`, or while awaiting a `user.custom_tool_result`. Break when idle with a terminal `stop_reason`, or on `session.status_terminated`.
```ts
for await (const event of stream) {
handle(event)
if (event.type === 'session.status_terminated') break
if (event.type === 'session.status_idle') {
if (event.stop_reason.type === 'requires_action') continue // waiting on you — handle it
break // end_turn or retries_exhausted — both terminal
}
}
```
`stop_reason.type` values on `session.status_idle`:
- `requires_action` — agent is waiting on a client-side event (tool confirmation, custom tool result). Handle it, don't break.
- `retries_exhausted` — terminal failure. Break, then check `sessions.retrieve()` for the error state.
- `end_turn` — normal completion.
---
## 6. Post-idle status-write race
The SSE stream emits `session.status_idle` slightly before the session's queryable status reflects it. Clients that break on idle and immediately call `sessions.delete()` or `sessions.archive()` will intermittently 400 with "cannot delete/archive while running."
Poll before cleanup:
```ts
let s
for (let i = 0; i < 10; i++) {
s = await client.beta.sessions.retrieve(session.id)
if (s.status !== 'running') break
await new Promise(r => setTimeout(r, 200))
}
if (s?.status !== 'running') {
await client.beta.sessions.archive(session.id)
} // else: still running after 2s — don't archive, let it settle or escalate
```
---
## 7. Stream-first, then send
Always open the stream **before** sending the kickoff event. Otherwise the agent may process the event and emit the first events before your consumer is attached, and you'll miss them.
```ts
const stream = await client.beta.sessions.events.stream(session.id)
await client.beta.sessions.events.send(session.id, {
events: [{ type: 'user.message', content: [{ type: 'text', text: 'Hello' }] }],
})
for await (const event of stream) { /* ... */ }
```
The `Promise.all([stream, send])` shape works too, but stream-first is simpler and has the same effect — the stream starts buffering the moment it's opened.
---
## 8. File-mount gotchas
**The mounted resource has a different `file_id` than the file you uploaded.** Session creation makes a session-scoped copy.
```ts
const uploaded = await client.beta.files.upload({ file, purpose: 'agent_resource' })
// uploaded.id → the original file
const session = await client.beta.sessions.create({
/* ... */
resources: [{ type: 'file', file_id: uploaded.id, mount_path: '/workspace/data.csv' }],
})
// session.resources[0].file_id !== uploaded.id ← different IDs
```
Delete the original via `files.delete(uploaded.id)`; the session-scoped copy is garbage-collected with the session. `mount_path` must be absolute — see `shared/managed-agents-environments.md`.
---
## 9. Secrets for non-MCP APIs and CLIs — keep them host-side via custom tools
**Problem:** you want the agent to call a third-party API or run a CLI that needs a secret (API key, token, service-account credential), but there is currently no way to set environment variables inside the session container, and vaults currently hold MCP credentials only — they are not exposed to the container's shell. So `curl`, installed CLIs, or SDK clients running via the `bash` tool have no first-class place to read a secret from.
**Solution:** move the authenticated call to your side. Declare a custom tool on the agent; when the agent emits `agent.custom_tool_use`, your orchestrator (the process reading the SSE stream) executes the call with its own credentials and responds with `user.custom_tool_result`. The container never sees the key.
```ts
// Agent template: declare the tool, no credentials
tools: [{ type: 'custom', name: 'linear_graphql', input_schema: { /* query, vars */ } }]
// Orchestrator: handle the call with host-side creds
for await (const event of stream) {
if (event.type === 'agent.custom_tool_use' && event.name === 'linear_graphql') {
const result = await linear.request(event.input.query, event.input.vars) // host's key
await client.beta.sessions.events.send(session.id, {
events: [{ type: 'user.custom_tool_result', tool_use_id: event.id, result }],
})
}
}
```
Same shape works for `gh` CLI, local eval scripts, or anything else that needs host-side auth or binaries.
**Security note:** this does not expose a public endpoint. `agent.custom_tool_use` arrives on the SSE stream your orchestrator already holds open with your Anthropic API key, and `user.custom_tool_result` goes back via `events.send()` under the same key. Your orchestrator is a client, not a server — nothing unauthenticated is listening.
**Do not embed API keys in the system prompt or user messages as a workaround.** Prompts and messages are stored in the session's event history, returned by `events.list()`, and included in compaction summaries — a secret placed there is durably persisted and readable via the API for the life of the session.
resources/bundled/skills/claude-api/shared/managed-agents-core.md
+218
View File
@@ -0,0 +1,218 @@
# Managed Agents — Core Concepts
## Architecture
Managed Agents is built around four core concepts:
| Concept | Endpoint | What it is |
|---|---|---|
| **Agent** | `/v1/agents` | A persisted, versioned object defining the agent's capabilities and persona: model, system prompt, tools, MCP servers, skills. **Must be created before starting a session.** See the Agents section below. |
| **Session** | `/v1/sessions` | A stateful interaction with an agent. References a pre-created agent by ID + an environment + initial instructions. Produces an event stream. |
| **Environment** | `/v1/environments` | A template defining the configuration for container provisioning. |
| **Container** | N/A | An isolated compute instance where the agent's **tools** execute (bash, file ops, code). The agent loop does not run here — it runs on Anthropic's orchestration layer and acts on the container via tool calls. |
```
┌─────────────────────────────────────┐
│ Anthropic orchestration layer │
Agent (config) ───────▶│ (agent loop: Claude + tool calls) │
└──────────────┬──────────────────────┘
│ tool calls
▼
Environment (template) ──▶ Container (tool execution workspace)
│
Session ─┤
├── Resources (files, repos — mounted at startup)
├── Vault IDs (MCP credential references)
└── Conversation (event stream in/out)
```
> **Agent creation is a prerequisite.** Sessions reference a pre-created agent by ID — `model`/`system`/`tools` live on the agent object, never on the session. Every flow starts with `POST /v1/agents`.
---
## Session Lifecycle
```
rescheduling → running ↔ idle → terminated
```
| Status | Description |
| -------------- | ------------------------------------------------------------------ |
| `idle` | Agent has finished the current task, and is awaiting input. It's either waiting for input to continue working via a `user.message` or blocked awaiting a `user.custom_tool_result` or `user.tool_confirmation`. The `stop_reason` attached contains more information about why the Agent has stopped working. |
| `running` | Session has starting running, and the Agent is actively doing work. |
| `rescheduling` | Session is (re)scheduling after a retryable error has occurred, ready to be picked up by the orchestration system. |
| `terminated` | Session has terminated, entering an irreversible and unusable state. |
- Events can be sent when the session is `running` or `idle`. Messages are queued and processed in order.
- The agent transitions `idle → running` when it receives a new event, then back to `idle` when done.
- Errors surface as `session.error` events in the stream, not as a status value.
### Built-in session features
- **Context compaction** — if you approach max context, the API automatically condenses session history to keep the interaction going
- **Prompt caching** — historical repeated tokens are cached, reducing processing time and cost
- **Extended thinking** — on by default, returned as `agent.thinking` events
### Session operations
| Operation | Notes |
|---|---|
| List / fetch | Paginated list or single resource by ID |
| Update | Only `title` is updatable |
| Archive | Session becomes **read-only**. Not reversible. |
| Delete | Permanently deletes session, event history, container, and checkpoints. |
---
## Sessions
A session is a running agent instance inside an environment.
### Session Object
Key fields returned by the API:
| Field | Type | Description |
| --------------- | -------- | --------------------------------------------------- |
| `type` | string | Always `"session"` |
| `id` | string | Unique session ID |
| `title` | string | Human-readable title |
| `status` | string | `idle`, `running`, `rescheduling`, `terminated` |
| `created_at` | string | ISO 8601 timestamp |
| `updated_at` | string | ISO 8601 timestamp |
| `archived_at` | string | ISO 8601 timestamp (nullable) |
| `environment_id` | string | Environment ID |
| `agent` | object | Agent configuration |
| `resources` | array | Attached files and repos |
| `metadata` | object | User-provided key-value pairs (max 8 keys) |
| `usage` | object | Token usage statistics |
### Creating a session
**A session is meaningless without an agent.** Sessions reference a pre-created agent by ID. Create the agent first via `agents.create()`, then reference it:
```ts
// 1. Create the agent (reusable, versioned)
const agent = await client.beta.agents.create(
{
name: "Coding Assistant",
model: "claude-opus-4-7",
system: "You are a helpful coding agent.",
tools: [{ type: "agent_toolset_20260401"}],
},
);
// 2. Start a session that references it
const session = await client.beta.sessions.create(
{
agent: agent.id, // string shorthand → latest version. Or: { type: "agent", id: agent.id, version: agent.version }
environment_id: environmentId,
title: "Hello World Session",
},
);
```
**Session creation parameters:**
| Field | Type | Required | Description |
| --------------- | -------- | -------- | ---------------------------------------------- |
| `agent` | string or object | **Yes** | String shorthand `"agent_abc123"` (latest version) or `{type: "agent", id, version}` |
| `environment_id`| string | **Yes** | Environment ID |
| `title` | string | No | Human-readable name (appears in logs/dashboards) |
| `resources` | array | No | Files or GitHub repos, mounted to the container at startup |
| `vault_ids` | array | No | Vault IDs (`vlt_*`) — MCP credentials with auto-refresh. See `shared/managed-agents-tools.md` → Vaults. |
| `metadata` | object | No | User-provided key-value pairs |
**Agent configuration fields** (passed to `agents.create()`, not `sessions.create()`):
| Field | Type | Required | Description |
| ------------- | -------- | -------- | ---------------------------------------------- |
| `name` | string | **Yes** | Human-readable name (1-256 chars) |
| `model` | string or object | **Yes** | Claude model ID (bare string, or `{id, speed}` object). All Claude 4.5+ models supported. |
| `system` | string | No | System prompt — defines the agent's behavior (up to 100K chars) |
| `tools` | array | No | Encompasses three kinds: (1) pre-built Claude Agent tools (`agent_toolset_20260401`), (2) MCP tools (`mcp_toolset`), and (3) custom client-side tools. Max 128. |
| `mcp_servers` | array | No | MCP server connections — standardized third-party capabilities (e.g. GitHub, Asana). Max 20, unique names. See `shared/managed-agents-tools.md` → MCP Servers. |
| `skills` | array | No | Customized "best-practices" context with progressive disclosure. Max 64. See `shared/managed-agents-tools.md` → Skills. |
| `description` | string | No | Description of the agent (up to 2048 chars) |
| `metadata` | object | No | Arbitrary key-value pairs (max 16, keys ≤64 chars, values ≤512 chars) |
---
## Agents
**This is where every Managed Agents flow begins.** The agent object is a persisted, versioned configuration — you create it once, then reference it by ID every time you start a session. No agent → no session.
### Agent Object
The API is **flat** — `model`, `system`, `tools` etc. are top-level fields, not wrapped in an `agent:{}` sub-object.
| Field | Type | Required | Description |
| ------------------ | -------- | -------- | -------------------------------------------------- |
| `name` | string | Yes | Human-readable name |
| `model` | string | Yes | Claude model ID |
| `system` | string | No | System prompt |
| `tools` | array | No | Agent toolset / MCP toolset / custom tools |
| `mcp_servers` | array | No | MCP server connections |
| `skills` | array | No | Skill references (max 64) |
| `description` | string | No | Description of the agent |
| `metadata` | object | No | Arbitrary key-value pairs |
### Lifecycle: create once, run many, update in place
The agent is a **persistent resource**, not a per-run parameter. The intended pattern:
```
┌─ setup (once) ─────────┐ ┌─ runtime (every invocation) ─┐
│ agents.create() │ │ sessions.create( │
│ → store agent_id │ ──→ │ agent={type:..., id: ID} │
│ in config/env/db │ │ ) │
└────────────────────────┘ └──────────────────────────────┘
```
**Anti-pattern:** calling `agents.create()` at the top of every script run. This accumulates orphaned agent objects, pays create latency on every invocation, and defeats the versioning model. If you see `agents.create()` in a function that's called per-request or per-cron-tick, that's wrong — hoist it to one-time setup and persist the ID.
### Versioning
Each `POST /v1/agents/{id}` (update) creates a new immutable version (numeric timestamp, e.g. `1772585501101368014`). The agent's history is append-only — you can't edit a past version.
**Why version:**
- **Reproducibility** — pin a session to a known-good config: `{type: "agent", id, version: 3}`
- **Safe iteration** — update the agent without breaking sessions already running on the old version
- **Rollback** — if a new system prompt regresses, pin new sessions back to the prior version while you debug
**`version` is optional.** Omit it (or use the string shorthand `agent="agent_abc123"`) to get the latest version at session-creation time. Pass it explicitly (`{type: "agent", id, version: N}`) to pin for reproducibility.
**Getting the version to pin:** `agents.create()` and `agents.update()` both return `version` in the response. Store it alongside `agent_id`. To fetch the current latest for an existing agent: `GET /v1/agents/{id}` → `.version`.
**When to update vs create new:** Update (`POST /v1/agents/{id}`) when it's conceptually the same agent with tweaked behavior (better prompt, extra tool). Create a new agent when it's a different persona/purpose. Rule of thumb: if you'd give it the same `name`, update.
### Agent Endpoints
| Operation | Method | Path |
| ---------------- | -------- | ------------------------------------- |
| Create | `POST` | `/v1/agents` |
| List | `GET` | `/v1/agents` |
| Get | `GET` | `/v1/agents/{id}` |
| Update | `POST` | `/v1/agents/{id}` |
| Archive | `POST` | `/v1/agents/{id}/archive` |
> ⚠️ **Archive is permanent.** Archiving makes the agent read-only: existing sessions continue to run, but **new sessions cannot reference it**, and there is no unarchive. Since agents have no `delete`, this is the terminal lifecycle state. Never archive a production agent as routine cleanup — confirm with the user first.
### Using an Agent in a Session
Reference the agent by string ID (latest version) or by object with an explicit version:
```python
# String shorthand — uses the agent's latest version
session = client.beta.sessions.create(
agent=agent.id,
environment_id=environment_id,
)
# Or pin to a specific version (int)
session = client.beta.sessions.create(
agent={"type": "agent", "id": agent.id, "version": agent.version},
environment_id=environment_id,
)
```
resources/bundled/skills/claude-api/shared/managed-agents-environments.md
+212
View File
@@ -0,0 +1,212 @@
# Managed Agents — Environments & Resources
## Environments
Creating a session requires an `environment_id`. Environments are **reusable configuration templates** for spinning up containers in Anthropic's infrastructure — you might create different environments for different use cases (e.g. data visualization vs web development, with different package sets). Anthropic handles scaling, container lifecycle, and work orchestration.
**Environment names must be unique.** Creating an environment with an existing name returns 409.
### Networking
| Network Policy | Description |
| ------------------------------- | ------------------------------------------------------------- |
| `unrestricted` | Full egress (except legal blocklist) |
| `package_managers_and_custom` | Package managers + custom `allowed_hosts` |
```json
{
"networking": {
"type": "package_managers_and_custom",
"allowed_hosts": ["api.example.com"]
}
}
```
**MCP caveat:** If using restricted networking, make sure `allowed_hosts` includes your MCP server domains. Otherwise the container can't reach them and tools silently fail.
### Creating an environment
The SDK adds `managed-agents-2026-04-01` automatically. TypeScript:
```ts
const env = await client.beta.environments.create({
name: "my_env",
config: {
type: "cloud",
networking: { type: "unrestricted" },
},
});
```
### Environment CRUD
| Operation | Method | Path | Notes |
| ---------------- | -------- | ------------------------------------------ | ----- |
| Create | `POST` | `/v1/environments` | |
| List | `GET` | `/v1/environments` | Paginated (`limit`, `after_id`, `before_id`) |
| Get | `GET` | `/v1/environments/{id}` | |
| Update | `POST` | `/v1/environments/{id}` | Changes apply only to **new** containers; existing sessions keep their original config |
| Delete | `DELETE` | `/v1/environments/{id}` | Returns 204. |
| Archive | `POST` | `/v1/environments/{id}/archive` | Makes it **read-only**; existing sessions continue, new sessions cannot reference it. No unarchive — terminal state. |
---
## Resources
Attach files and GitHub repositories to a session. **Session creation blocks until all resources are mounted** — the container won't go `running` until every file and repo is in place. Max **999 file resources** per session. Multiple GitHub repositories per session are supported.
### File Uploads (input — host → agent)
Upload a file first via the Files API, then reference by `file_id` + `mount_path`:
```ts
// 1. Upload
const file = await client.beta.files.upload({
file: fs.createReadStream("data.csv"),
purpose: "agent",
});
// 2. Attach as a session resource
const session = await client.beta.sessions.create({
agent: agent.id,
environment_id: envId,
resources: [
{ type: "file", file_id: file.id, mount_path: "/workspace/data.csv" }
],
});
```
**`mount_path` is required** and must be absolute. Parent directories are created automatically. Agent working directory defaults to `/workspace`. Files are mounted read-only — the agent writes modified versions to new paths.
### Session outputs (output — agent → host)
The agent can write files to `/mnt/session/outputs/` during a session. These are automatically captured by the Files API and can be listed and downloaded afterwards:
```ts
// After the turn completes, list output files scoped to this session:
for await (const f of client.beta.files.list({
scope_id: session.id,
betas: ["managed-agents-2026-04-01"],
})) {
console.log(f.filename, f.size_bytes);
const resp = await client.beta.files.download(f.id);
const text = await resp.text();
}
```
**Requirements:**
- The `write` tool (or `bash`) must be enabled for the agent to create output files.
- Session-scoped `files.list` / `files.download` captures outputs written to `/mnt/session/outputs/`.
- The filter parameter is **`scope_id`** (REST query param `?scope_id=<session_id>`). The SDK's files resource auto-adds only the `files-api-2025-04-14` header, so pass `betas: ["managed-agents-2026-04-01"]` explicitly (or both headers on raw HTTP) — without it the API may reject `scope_id` as an unknown field. Requires `@anthropic-ai/sdk` ≥ 0.88.0 / `anthropic` (Python) ≥ 0.92.0 — older versions don't type `scope_id`. The `ant` CLI does **not** expose this flag yet; use the SDK or curl.
- Pass the session ID returned by `sessions.create()` verbatim (e.g. `sesn_011CZx...`) — the API validates the prefix.
- There's a brief indexing lag (~1–3s) between `session.status_idle` and output files appearing in `files.list`. Retry once or twice if empty.
> **Fallback when `scope_id` filtering is unavailable** (older SDK, or endpoint returns an error): send a follow-up `user.message` asking the agent to `read` each file under `/mnt/session/outputs/` and return the contents. The agent streams the file bodies back as `agent.message` text. This works for text files only and costs output tokens — use it to unblock, not as the primary path.
This gives you a bidirectional file bridge: upload reference data in, download agent artifacts out.
### GitHub Repositories
Clones a GitHub repository into the session container during initialization, before the agent begins execution. The agent can read, edit, commit, and push via `bash` (`git`). Multiple repositories per session are supported — add one `resources` entry per repo. Repositories are cached, so future sessions that use the same repository start faster.
Repositories are attached for the lifetime of the session — to change which repositories are mounted, create a new session. You **can** rotate a repository's `authorization_token` on a running session via `client.beta.sessions.resources.update(resource_id, {session_id, authorization_token})`; the resource `id` is returned at session creation and by `resources.list()`.
**Fields:**
| Field | Required | Notes |
|---|---|---|
| `type` | ✅ | `"github_repository"` |
| `url` | ✅ | The GitHub repository URL |
| `authorization_token` | ✅ | GitHub Personal Access Token with repository access. **Never echoed in API responses.** |
| `mount_path` | ❌ | Path where the repository will be cloned. Defaults to `/workspace/<repo-name>`. |
| `checkout` | ❌ | `{type: "branch", name: "..."}` or `{type: "commit", sha: "..."}`. Defaults to the repo's default branch. |
**Token permission levels** (fine-grained PATs):
- `Contents: Read` — clone only
- `Contents: Read and write` — push changes and create pull requests
**How auth works:** `authorization_token` is never placed inside the container. `git pull` / `git push` and GitHub REST calls against the attached repository are routed through an Anthropic-side git proxy that injects the token after the request leaves the sandbox. Code running in the container — including anything the agent writes — cannot read or exfiltrate it.
> ‼️ **To generate pull requests** you also need GitHub **MCP server** access — the `github_repository` resource gives filesystem + git access only. See `shared/managed-agents-tools.md` → MCP Servers. The PR workflow is: edit files in the mounted repo → push branch via `bash` (authenticated via the git proxy using `authorization_token`) → create PR via the MCP `create_pull_request` tool (authenticated via the vault).
**TypeScript:**
```ts
// 1. Create the agent — declare GitHub MCP (no auth here)
const agent = await client.beta.agents.create(
{
name: 'GitHub Agent',
model: 'claude-opus-4-7',
mcp_servers: [
{ type: 'url', name: 'github', url: 'https://api.githubcopilot.com/mcp/' },
],
tools: [
{ type: 'agent_toolset_20260401', default_config: { enabled: true } },
{ type: 'mcp_toolset', mcp_server_name: 'github' },
],
},
);
// 2. Start a session — attach vault for MCP auth + mount the repo
const session = await client.beta.sessions.create({
agent: agent.id,
environment_id: envId,
vault_ids: [vaultId], // vault contains the GitHub MCP OAuth credential
resources: [
{
type: 'github_repository',
url: 'https://github.com/owner/repo',
authorization_token: process.env.GITHUB_TOKEN, // repo clone token (≠ MCP auth)
checkout: { type: 'branch', name: 'main' },
},
],
});
```
**Python:**
```python
import os
agent = client.beta.agents.create(
name="GitHub Agent",
model="claude-opus-4-7",
mcp_servers=[{
"type": "url",
"name": "github",
"url": "https://api.githubcopilot.com/mcp/",
}],
tools=[
{"type": "agent_toolset_20260401", "default_config": {"enabled": True}},
{"type": "mcp_toolset", "mcp_server_name": "github"},
],
)
session = client.beta.sessions.create(
agent=agent.id,
environment_id=env_id,
vault_ids=[vault_id], # vault contains the GitHub MCP OAuth credential
resources=[{
"type": "github_repository",
"url": "https://github.com/owner/repo",
"authorization_token": os.environ["GITHUB_TOKEN"], # repo clone token (≠ MCP auth)
"checkout": {"type": "branch", "name": "main"},
}],
)
```
---
## Files API
Upload and manage files for use as session resources, and download files the agent wrote to `/mnt/session/outputs/`.
| Operation | Method | Path | SDK |
| ---------------- | -------- | ------------------------------------- | --- |
| Upload | `POST` | `/v1/files` | `client.beta.files.upload({ file })` |
| List | `GET` | `/v1/files?scope_id=...` | `client.beta.files.list({ scope_id, betas: ["managed-agents-2026-04-01"] })` |
| Get Metadata | `GET` | `/v1/files/{id}` | `client.beta.files.retrieveMetadata(id)` |
| Download | `GET` | `/v1/files/{id}/content` | `client.beta.files.download(id)` → `Response` |
| Delete | `DELETE` | `/v1/files/{id}` | `client.beta.files.delete(id)` |
The `scope_id` filter on List scopes the results to files written to `/mnt/session/outputs/` by that session. Without the filter, you get all files uploaded to your account.
resources/bundled/skills/claude-api/shared/managed-agents-events.md
+189
View File
@@ -0,0 +1,189 @@
# Managed Agents — Events & Steering
## Events
### Sending Events
Send events to a session via `POST /v1/sessions/{id}/events`.
| Event Type | When to Send |
| ------------------------- | --------------------------------------------------- |
| `user.message` | Send a user message |
| `user.interrupt` | Interrupt the agent while it's running |
| `user.tool_confirmation` | Approve/deny a tool call (when `always_ask` policy) |
| `user.custom_tool_result` | Provide result for a custom tool call |
### Receiving Events
Two methods:
1. **Streaming (SSE)**: `GET /v1/sessions/{id}/events/stream` — real-time Server-Sent Events. **Long-lived** — the server sends periodic heartbeats to keep the connection alive.
2. **Polling**: `GET /v1/sessions/{id}/events` — paginated event list (query params: `limit` default 1000, `page`). **Returns immediately** — this is a plain paginated GET, not a long-poll.
All received events carry `id`, `type`, and `processed_at` (ISO 8601; `null` if not yet processed by the agent).
> ⚠️ **Robust polling (raw HTTP).** If you bypass the SDK and roll your own poll loop, don't rely on `requests` or `httpx` timeouts as wall-clock caps — they're **per-chunk** read timeouts, reset every time a byte arrives. A trickling response (heartbeats, a wedged chunked-encoding body, a misbehaving proxy) can keep the call blocked indefinitely even with `timeout=(5, 60)` or `httpx.Timeout(120)`. Neither library has a "total wall-clock" timeout built in. For a hard deadline: track `time.monotonic()` at the loop level and break/cancel if a single request exceeds your budget (e.g. via a watchdog thread, or `asyncio.wait_for()` around async httpx). **Prefer the SDK** — `client.beta.sessions.events.stream()` and `client.beta.sessions.events.list()` handle timeout + retry sanely.
>
> If `GET /v1/sessions/{id}/events` (paginated) ever hangs after headers, you've likely hit `GET /v1/sessions/{id}/events` by mistake or a server-side stall — report it; don't treat it as a client-config problem.
### Event Types (Received)
Event types use dot notation, grouped by namespace:
| Event Type | Description |
| --- | --- |
| `agent.message` | Agent text output |
| `agent.thinking` | Extended thinking blocks |
| `agent.tool_use` | Agent used a built-in tool (`agent_toolset_20260401`) |
| `agent.tool_result` | Result from a built-in tool |
| `agent.mcp_tool_use` | Agent used an MCP tool |
| `agent.mcp_tool_result` | Result from an MCP tool |
| `agent.custom_tool_use` | Agent invoked a custom tool — session goes idle, you respond with `user.custom_tool_result` |
| `agent.thread_context_compacted` | Conversation context was compacted |
| `session.status_idle` | Agent has finished the current task, and is awaiting input. It's either waiting for input to continue working via a `user.message` or blocked awaiting a `user.custom_tool_result` or `user.tool_confirmation`. The `stop_reason` attached contains more information about why the Agent has stopped working. |
| `session.status_running` | Session has starting running, and the Agent is actively doing work. |
| `session.status_rescheduled` | Session is (re)scheduling after a retryable error has occurred, ready to be picked up by the orchestration system. |
| `session.status_terminated` | Session has terminated, entering an irreversible and unusable state. |
| `session.error` | Error occurred during processing |
| `span.model_request_start` | Model inference started |
| `span.model_request_end` | Model inference completed |
The stream also echoes back user-sent events (`user.message`, `user.interrupt`, `user.tool_confirmation`, `user.custom_tool_result`).
---
## Steering Patterns
Practical patterns for driving a session via the events surface.
### Stream-first ordering
**Open the stream before sending events.** The stream only delivers events that occur *after* it's opened — it does not replay current state or historical events. If you send a message first and open the stream second, early events (including fast status transitions) arrive buffered in a single batch and you lose the ability to react to them in real time.
```ts
// ✅ Correct — stream and send concurrently
const [response] = await Promise.all([
streamEvents(sessionId), // opens SSE connection
sendMessage(sessionId, text),
]);
// ❌ Wrong — events before stream opens arrive as a single buffered batch
await sendMessage(sessionId, text);
const response = await streamEvents(sessionId);
```
**For full history,** use `GET /v1/sessions/{id}/events` (paginated list) — the stream only gives you live events from connection onward.
### Reconnecting after a dropped stream
**The SSE stream has no replay.** If your connection drops (httpx read timeout, network blip) and you reconnect, you only get events emitted *after* reconnection. Any events emitted during the gap are lost from the stream.
**The consolidation pattern:** on every (re)connect, overlap the stream with a history fetch and dedupe by event ID:
```python
def connect_with_consolidation(client, session_id):
# 1. Open the SSE stream first
stream = client.beta.sessions.events.stream(session_id=session_id)
# 2. Fetch history to cover any gap
history = client.beta.sessions.events.list(
session_id=session_id,
)
# 3. Yield history first, then stream — dedupe by event.id
seen = set()
for ev in history.data:
seen.add(ev.id)
yield ev
for ev in stream:
if ev.id not in seen:
seen.add(ev.id)
yield ev
```
### Message queuing
**You don't have to wait for a response before sending the next message.** User events are queued server-side and processed in order. This is useful for chat bridges where the user sends rapid follow-ups:
```ts
// All three go into one session; agent processes them in order
await sendMessage(sessionId, "Summarize the README");
await sendMessage(sessionId, "Actually also check the CONTRIBUTING guide");
await sendMessage(sessionId, "And compare the two");
// Stream once — agent responds to all three as a coherent turn
```
Events can be sent up to the Session at any time. There is no need to wait on a specific session status to enqueue new events via `client.beta.sessions.events.send()`
### Interrupt
An `interrupt` event **jumps the queue** (ahead of any pending user messages) and forces the session into `idle`. Use this for "stop" / "nevermind" / "cancel" commands:
```ts
await client.beta.sessions.events.send(sessionId, {
events: [{ type: 'interrupt' }],
});
```
The agent stops mid-task. It does not see the interrupt as a message — it just halts. Send a follow-up `user` event to explain what to do instead.
> **Note**: Interrupt events may have empty IDs in the current implementation. When troubleshooting, use the `processed_at` timestamp along with surrounding event IDs.
### Event payloads
some events carry useful metadata beyond the status change itself:
`session.status_idle` — includes a `stop_reason` field which elaborates on why the session stopped and what type of further action is required by the user.
```json
{
"id": "sevt_456",
"processed_at": "2026-04-07T04:27:43.197Z",
"stop_reason": {
"event_ids": [
"sevt_123"
],
"type": "requires_action"
},
"type": "status_idle"
}
```
`span.model_request_end` contains a `model_usage` field for cost tracking and efficiency analysis:
```json
{
"type": "span.model_request_end",
"id": "sevt_456",
"is_error": false,
"model_request_start_id": "sevt_123",
"model_usage": {
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 6656,
"input_tokens": 3571,
"output_tokens": 727
},
"processed_at": "2026-04-07T04:11:32.189Z"
}
```
**`agent.thread_context_compacted`** — emitted when the conversation history was summarized to fit context. Includes `pre_compaction_tokens` so you know how much was squeezed:
```json
{
"id": "sevt_abc123",
"processed_at": "2026-03-24T14:05:15.787Z",
"type": "agent.thread_context_compacted"
}
```
### Archive
When done with a session, archive it to free resources:
```ts
await client.beta.sessions.archive(sessionId);
```
> Archiving a **session** is routine cleanup — sessions are per-run and disposable. **Do not generalize this to agents or environments**: those are persistent, reusable resources, and archiving them is permanent (no unarchive; new sessions cannot reference them). See `shared/managed-agents-overview.md` → Common Pitfalls.
resources/bundled/skills/claude-api/shared/managed-agents-onboarding.md
+114
View File
@@ -0,0 +1,114 @@
# Managed Agents — Onboarding Flow
> **Invoked via `/claude-api managed-agents-onboard`?** You're in the right place. Run the interview below — don't summarize it back to the user, ask the questions.
Use this when a user wants to set up a Managed Agent from scratch. Three steps: **branch on know-vs-explore → configure the template → set up the session**. End by emitting working code.
> Read `shared/managed-agents-core.md` alongside this — it has full detail for each knob. This doc is the interview script, not the reference.
---
Claude Managed Agents is a hosted agent: Anthropic runs the agent loop on its orchestration layer and provisions a sandboxed container per session where the agent's tools execute. You supply the agent config and the environment config; the harness — event stream, sandbox orchestration, prompt caching, context compaction, and extended thinking — is handled for you.
**What you supply:**
- **An agent config** — tools, skills, model, system prompt. Reusable and versioned.
- **An environment config** — the sandbox your agent's tools execute in (networking, packages). Reusable across agents.
Each run of the agent is a **session**.
---
## 1. Know or explore?
Ask the user:
> Do you already know the agent you want to build, or would you like to explore some common patterns first?
### Explore path — show the patterns
Four shapes, same runtime code path (`sessions.create()` → `sessions.events.send()` → stream). Only the trigger and sink differ.
| Pattern | Trigger | Example |
|---|---|---|
| Event-triggered | Webhook | GitHub PR push → CMA (GitHub tool) → Slack | # <------ MC maybe delete?
| Scheduled | Cron | Daily brief: browser + GitHub + Jira → CMA → Slack | # <------ MC maybe delete?
| Fire-and-forget PR | Human | Slack slash-command → CMA (GitHub tool) → PR passing CI |
| Research + dashboard | Human | Topic → CMA (web search + `frontend-design` skill) → HTML dashboard |
Ask which shape fits, then continue with the Know path using it as the reference.
### Know path — configure template
Three rounds. Batch the questions in each round; don't ask them one at a time.
**Round A — Tools.** Start here; it's the most concrete part. Three types; ask which the user wants (any combination):
| Type | What it is | How to guide |
|---|---|---|
| **Prebuilt Claude Agent tools** (`agent_toolset_20260401`) | Ready-to-use: `bash`, `read`, `write`, `edit`, `glob`, `grep`, `web_fetch`, `web_search`. Enable all at once, or individually via `enabled: true/false`. | Recommend enabling the full toolset. List the 8 tools so the user knows what they're getting. Full detail: `shared/managed-agents-tools.md` → Agent Toolset. |
| **MCP tools** | Third-party integrations (GitHub, Linear, Asana, etc.) via `mcp_toolset`. Credentials live in a vault, not inline. | Ask which services. For each, walk through MCP server URL + vault credentials. Full detail: `shared/managed-agents-tools.md` → MCP Servers + Vaults. |
| **Custom tools** | The user's own app handles these tool calls — agent fires `agent.custom_tool_use`, the app sends a result message back. | Ask for each tool: name, description, input schema. The app code that handles the event is *their* code — don't generate it. Full detail: `shared/managed-agents-tools.md` → Custom Tools. |
**Round B — Skills, files, and repos.** What the agent has on hand when it starts.
*Skills* — two types; both work the same way — Claude auto-uses them when relevant. Max 64 per agent.
- [ ] **Pre-built Agent Skills**: `xlsx`, `docx`, `pptx`, `pdf`. Reference by name.
- [ ] **Custom Skills**: skills uploaded to the user's org via the Skills API. Reference by `skill_id` + optional `version`. If the skill doesn't exist yet, walk the user through `POST /v1/skills` + `POST /v1/skills/{id}/versions` (beta header `skills-2025-10-02`). Full detail: `shared/managed-agents-tools.md` → Skills + Skills API.
*GitHub repositories* — any repos the agent needs on-disk? For each:
- [ ] Repo URL (`https://github.com/org/repo`)
- [ ] `authorization_token` (PAT or GitHub App token scoped to the repo)
- [ ] Optional `mount_path` (defaults to `/workspace/<repo-name>`) and `checkout` (branch or SHA)
Emit as `resources: [{type: "github_repository", url, authorization_token, ...}]`. Full detail: `shared/managed-agents-environments.md` → GitHub Repositories.
> ‼️ **PR creation needs the GitHub MCP server too.** `github_repository` gives filesystem access only — to open PRs, also attach the GitHub MCP server in Round A and credential it via a vault. The workflow is: edit files in the mounted repo → push branch via `bash` → create PR via the MCP `create_pull_request` tool.
*Files* — any local files to seed the session with? For each:
- [ ] Upload via the Files API → persist `file_id`
- [ ] Choose a `mount_path` — absolute, e.g. `/workspace/data.csv` (parents auto-created; files mount read-only)
Emit as `resources: [{type: "file", file_id, mount_path}]`. Max 999 file resources. Agent working directory defaults to `/workspace`. Full detail: `shared/managed-agents-environments.md` → Files API.
**Round C — Environment + identity:**
- [ ] Networking: unrestricted internet from the container, or lock egress to specific hosts? (If locked, MCP server domains must be in `allowed_hosts` or tools silently fail.)
- [ ] Name?
- [ ] Job (one or two sentences — becomes the system prompt)?
- [ ] Model? (default `claude-opus-4-7`)
---
## 2. Set up the session
Per-run. Points at the agent + environment, attaches credentials, kicks off.
**Vault credentials** (if the agent declared MCP servers):
- [ ] Existing vault, or create one? (`client.beta.vaults.create()` + `vaults.credentials.create()`)
Credentials are write-only, matched to MCP servers by URL, auto-refreshed. See `shared/managed-agents-tools.md` → Vaults.
**Kickoff:**
- [ ] First message to the agent?
Session creation blocks until all resources mount. Open the event stream before sending the kickoff. Stream is SSE; break on `session.status_terminated`, or on `session.status_idle` with a terminal `stop_reason` — i.e. anything except `requires_action`, which fires transiently while the session waits on a tool confirmation or custom-tool result (see `shared/managed-agents-client-patterns.md` Pattern 5). Usage lands on `span.model_request_end`. Agent-written artifacts end up in `/mnt/session/outputs/` — download via `files.list({scope_id: session.id, betas: ["managed-agents-2026-04-01"]})`.
---
## 3. Emit the code
Go straight from the last interview answer to the code — no preamble about the setup-vs-runtime split, no "the critical thing to internalize…", no lecture about `agents.create()` being one-time. The two-block structure below already shows that; don't narrate it. Generate **two clearly-separated blocks** per language detected (Python/TS/cURL — see SKILL.md → Language Detection):
**Block 1 — Setup (run once, store the IDs):**
1. `environments.create()` → persist `env_id`
2. `agents.create()` with everything from §Round A–C → persist `agent_id` and `agent_version`
Label: `# ONE-TIME SETUP — run once, save the IDs to config/.env`
**Block 2 — Runtime (run on every invocation):**
1. Load `env_id` + `agent_id` from config/env
2. `sessions.create(agent=AGENT_ID, environment_id=ENV_ID, resources=[...], vault_ids=[...])`
3. Open stream, `events.send()` the kickoff, loop until `session.status_terminated` or `session.status_idle && stop_reason.type !== 'requires_action'` (see `shared/managed-agents-client-patterns.md` Pattern 5 for the full gate — do not break on bare `session.status_idle`)
> ⚠️ **Never emit `agents.create()` and `sessions.create()` in the same unguarded block.** That teaches the user to create a new agent on every run — the #1 anti-pattern. If they need a single script, wrap agent creation in `if not os.getenv("AGENT_ID"):`.
Pull exact syntax from `python/managed-agents/README.md`, `typescript/managed-agents/README.md`, or `curl/managed-agents.md`. Don't invent field names.
resources/bundled/skills/claude-api/shared/managed-agents-overview.md
+63
View File
@@ -0,0 +1,63 @@
# Managed Agents — Overview
Managed Agents provisions a container per session as the agent's workspace. The agent loop runs on Anthropic's orchestration layer; the container is where the agent's *tools* execute — bash commands, file operations, code. You create a persisted **Agent** config (model, system prompt, tools, MCP servers, skills), then start **Sessions** that reference it. The session streams events back to you; you send user messages and tool results in.
## ⚠️ THE MANDATORY FLOW: Agent (once) → Session (every run)
**Why agents are separate objects: versioning.** An agent is a persisted, versioned config — every update creates a new immutable version, and sessions pin to a version at creation time. This lets you iterate on the agent (tweak the prompt, add a tool) without breaking sessions already running, roll back if a change regresses, and A/B test versions side-by-side. None of that works if you `agents.create()` fresh on every run.
Every session references a pre-created `/v1/agents` object. Create the agent once, store the ID, and reuse it across runs.
| Step | Call | Frequency |
|---|---|---|
| 1 | `POST /v1/agents` — `model`, `system`, `tools`, `mcp_servers`, `skills` live here | **ONCE.** Store `agent.id` **and** `agent.version`. |
| 2 | `POST /v1/sessions` — `agent: "agent_abc123"` or `{type: "agent", id, version}` | **Every run.** String shorthand uses latest version. |
If you're about to write `sessions.create()` with `model`, `system`, or `tools` on the session body — **stop**. Those fields live on `agents.create()`. The session takes a *pointer* only.
**When generating code, separate setup from runtime.** `agents.create()` belongs in a setup script (or a guarded `if agent_id is None:` block), not at the top of the hot path. If the user's code calls `agents.create()` on every invocation, they're accumulating orphaned agents and paying the create latency for nothing. The correct shape is: create once → persist the ID (config file, env var, secrets manager) → every run loads the ID and calls `sessions.create()`.
**To change the agent's behavior, use `POST /v1/agents/{id}` — don't create a new one.** Each update bumps the version; running sessions keep their pinned version, new sessions get the latest (or pin explicitly via `{type: "agent", id, version}`). See `shared/managed-agents-core.md` → Agents → Versioning.
## Beta Headers
Managed Agents is in beta. The SDK sets required beta headers automatically:
| Beta Header | What it enables |
| ------------------------------ | ---------------------------------------------------- |
| `managed-agents-2026-04-01` | Agents, Environments, Sessions, Events, Session Resources, Vaults, Credentials |
| `skills-2025-10-02` | Skills API (for managing custom skill definitions) |
| `files-api-2025-04-14` | Files API for file uploads |
**Which beta header goes where:** The SDK sets `managed-agents-2026-04-01` automatically on `client.beta.{agents,environments,sessions,vaults}.*` calls, and `files-api-2025-04-14` / `skills-2025-10-02` automatically on `client.beta.files.*` / `client.beta.skills.*` calls. You do NOT need to add the Skills or Files beta header when calling Managed Agents endpoints. **Exception — session-scoped file listing:** `client.beta.files.list({scope_id: session.id})` is a Files endpoint that takes a Managed Agents parameter, so it needs **both** headers. Pass `betas: ["managed-agents-2026-04-01"]` explicitly on that call (the SDK adds the Files header; you add the Managed Agents one). See `shared/managed-agents-environments.md` → Session outputs.
## Reading Guide
| User wants to... | Read these files |
| -------------------------------------- | ------------------------------------------------------- |
| **Get started from scratch / "help me set up an agent"** | `shared/managed-agents-onboarding.md` — guided interview (WHERE→WHO→WHAT→WATCH), then emit code |
| Understand how the API works | `shared/managed-agents-core.md` |
| See the full endpoint reference | `shared/managed-agents-api-reference.md` |
| **Create an agent** (required first step) | `shared/managed-agents-core.md` (Agents section) + language file |
| Update/version an agent | `shared/managed-agents-core.md` (Agents → Versioning) — update, don't re-create |
| Create a session | `shared/managed-agents-core.md` + `{lang}/managed-agents/README.md` |
| Configure tools and permissions | `shared/managed-agents-tools.md` |
| Set up MCP servers | `shared/managed-agents-tools.md` (MCP Servers section) |
| Stream events / handle tool_use | `shared/managed-agents-events.md` + language file |
| Set up environments | `shared/managed-agents-environments.md` + language file |
| Upload files / attach repos | `shared/managed-agents-environments.md` (Resources) |
| Store MCP credentials | `shared/managed-agents-tools.md` (Vaults section) |
| Call a non-MCP API / CLI that needs a secret | `shared/managed-agents-client-patterns.md` Pattern 9 — no container env vars; vaults are MCP-only; keep the secret host-side via a custom tool |
## Common Pitfalls
- **Agent FIRST, then session — NO EXCEPTIONS** — the session's `agent` field accepts **only** a string ID or `{type: "agent", id, version}`. `model`, `system`, `tools`, `mcp_servers`, `skills` are **top-level fields on `POST /v1/agents`**, never on `sessions.create()`. If the user hasn't created an agent, that is step zero of every example.
- **Agent ONCE, not every run** — `agents.create()` is a setup step. Store the returned `agent_id` and reuse it; don't call `agents.create()` at the top of your hot path. If the agent's config needs to change, `POST /v1/agents/{id}` — each update creates a new version, and sessions can pin to a specific version for reproducibility.
- **MCP auth goes through vaults** — the agent's `mcp_servers` array declares `{type, name, url}` only (no auth). Credentials live in vaults (`client.beta.vaults.credentials.create`) and attach to sessions via `vault_ids`. Anthropic auto-refreshes OAuth tokens using the stored refresh token.
- **Stream to get events** — `GET /v1/sessions/{id}/events/stream` is the primary way to receive agent output in real-time.
- **SSE stream has no replay — reconnect with consolidation** — if the stream drops while a `agent.tool_use`, `agent.mcp_tool_use`, or `agent.custom_tool_use` is pending resolution (`user.tool_confirmation` for the first two, `user.custom_tool_result` for the last one), the session deadlocks (client disconnects → session idles → reconnect happens → no client resolution happens). On every (re)connect: open stream with `GET /v1/sessions/{id}/events/stream` , fetch `GET /v1/sessions/{id}/events`, dedupe by event ID, then proceed. See `shared/managed-agents-events.md` → Reconnecting after a dropped stream.
- **Don't trust HTTP-library timeouts as wall-clock caps** — `requests` `timeout=(c, r)` and `httpx.Timeout(n)` are *per-chunk* read timeouts; they reset every byte, so a trickling connection can block indefinitely. For a hard deadline on raw-HTTP polling, track `time.monotonic()` at the loop level and bail explicitly. Prefer the SDK's `sessions.events.stream()` / `session.events.list()` over hand-rolled HTTP. See `shared/managed-agents-events.md` → Receiving Events.
- **Messages queue** — you can send events while the session is `running` or `idle`; they're processed in order. No need to wait for a response before sending the next message.
- **Cloud environments only** — `config.type: "cloud"` is the only supported environment type.
- **Archive is permanent on every resource** — archiving an agent, environment, session, vault, or credential makes it read-only with no unarchive. For agents and environments specifically, archived resources cannot be referenced by new sessions (existing sessions continue). Do not call `.archive()` on a production agent or environment as cleanup — **always confirm with the user before archiving**.
resources/bundled/skills/claude-api/shared/managed-agents-tools.md
+315
View File
@@ -0,0 +1,315 @@
# Managed Agents — Tools & Skills
## Tools
### Server tools vs client tools
| Type | Who runs it | How it works |
|---|---|---|
| **Prebuilt Claude Agent tools** (`agent_toolset_20260401`) | Anthropic, on the session's container | File ops, bash, web search, etc. Enable all at once or configure individually with `enabled: true/false`. |
| **MCP tools** (`mcp_toolset`) | Anthropic, on the session's container | Capabilities exposed by connected MCP servers. Grant access per-server via the toolset. |
| **Custom tools** | **You** — your application handles the call and returns results | Agent emits a `agent.custom_tool_use` event, session goes `idle`, you send back a `user.custom_tool_result` event. |
**Recommendation:** Enable all prebuilt tools via `agent_toolset_20260401`, then disable individually as needed.
**Versioning:** The toolset is a versioned, static resource. When underlying tools change, a new toolset version is created (hence `_20260401`) so you always know exactly what you're getting.
### Agent Toolset
The `agent_toolset_20260401` provides these built-in tools:
| Tool | Description |
| ---------------------- | ---------------------------------------- |
| `bash` | Execute bash commands in a shell session |
| `read` | Read a file from the local filesystem, including text, images, PDFs, and Jupyter notebooks |
| `write` | Write a file to the local filesystem |
| `edit` | Perform string replacement in a file |
| `glob` | Fast file pattern matching using glob patterns |
| `grep` | Text search using regex patterns |
| `web_fetch` | Fetch content from a URL |
| `web_search` | Search the web for information |
Enable the full toolset:
```json
{
"tools": [
{ "type": "agent_toolset_20260401" }
]
}
```
### Per-Tool Configuration
Override defaults for individual tools. This example enables everything except bash:
```json
{
"tools": [
{
"type": "agent_toolset_20260401",
"default_config": { "enabled": true },
"configs": [
{ "name": "bash", "enabled": false }
]
}
]
}
```
| Field | Required | Description |
|---|---|---|
| `type` | ✅ | `"agent_toolset_20260401"` |
| `default_config` | ❌ | Applied to all tools. `{ "enabled": bool, "permission_policy": {...} }` |
| `configs` | ❌ | Per-tool overrides: `[{ "name": "...", "enabled": bool, "permission_policy": {...} }]` |
### Permission Policies
Control when server-executed tools (agent toolset + MCP) run automatically vs wait for approval. Does not apply to custom tools.
| Policy | Behavior |
|---|---|
| `always_allow` | Tool executes automatically (default) |
| `always_ask` | Session emits `session.status_idle` and pauses until you send a `tool_confirmation` event |
```json
{
"type": "agent_toolset_20260401",
"default_config": {
"enabled": true,
"permission_policy": { "type": "always_allow" }
},
"configs": [
{ "name": "bash", "permission_policy": { "type": "always_ask" } }
]
}
```
**Responding to `always_ask`:** Send a `user.tool_confirmation` event with `tool_use_id` from the triggering `agent_tool_use`/`mcp_tool_use` event:
```json
{ "type": "tool_confirmation", "tool_use_id": "sevt_abc123", "result": "allow" }
{ "type": "tool_confirmation", "tool_use_id": "sevt_def456", "result": "deny", "message": "Read .env.example instead" }
```
The optional `message` on a deny is delivered to the agent so it can adjust its approach.
To enable only specific tools, flip the default off and opt-in per tool:
```json
{
"tools": [
{
"type": "agent_toolset_20260401",
"default_config": { "enabled": false },
"configs": [
{ "name": "bash", "enabled": true },
{ "name": "read", "enabled": true }
]
}
]
}
```
### Custom Tools (Client-Side)
Custom tools are executed by **your application**, not Anthropic. The flow:
1. Agent decides to use the tool → session emits a `agent.custom_tool_use` event with inputs
2. Session goes `idle` waiting for you
3. Your application executes the tool
4. You send back a `user.custom_tool_result` event with the output
5. Session resumes `running`
No permission policy needed — you're the one executing.
```json
{
"tools": [
{
"type": "custom",
"name": "get_weather",
"description": "Fetch current weather for a city.",
"input_schema": {
"type": "object",
"properties": {
"city": { "type": "string", "description": "City name" }
},
"required": ["city"]
}
}
]
}
```
### MCP Servers
MCP (Model Context Protocol) servers expose standardized third-party capabilities (e.g. Asana, GitHub, Linear). **Configuration is split across agent and vault:**
1. **Agent creation** declares which servers to connect to (`type`, `name`, `url` — no auth). The agent's `mcp_servers` array has no auth field.
2. **Vault** stores the OAuth credentials. Attach via `vault_ids` on session create.
This keeps secrets out of reusable agent definitions. Each vault credential is tied to one MCP server URL; Anthropic matches credentials to servers by URL.
**Agent side — declare servers (no auth):**
| Field | Required | Description |
|---|---|---|
| `type` | ✅ | `"url"` |
| `name` | ✅ | Unique name — referenced by `mcp_toolset.mcp_server_name` |
| `url` | ✅ | The MCP server's endpoint URL (Streamable HTTP transport) |
```json
{
"mcp_servers": [
{ "type": "url", "name": "linear", "url": "https://mcp.linear.app/mcp" }
],
"tools": [
{ "type": "mcp_toolset", "mcp_server_name": "linear" }
]
}
```
**Session side — attach vault:**
```json
{
"agent": "agent_abc123",
"environment_id": "env_abc123",
"vault_ids": ["vlt_abc123"]
}
```
> 💡 **Per-tool enablement (empirical):** `mcp_toolset` has been observed accepting `default_config: {enabled: false}` + `configs: [{name, enabled: true}]` for an allowlist pattern. The API ref shows only the minimal `{type, mcp_server_name}` form.
> ⚠️ **MCP auth tokens ≠ REST API tokens.** Hosted MCP servers (`mcp.notion.com`, `mcp.linear.app`, etc.) typically require **OAuth bearer tokens**, not the service's native API keys. A Notion `ntn_` integration token authenticates against Notion's REST API but will **not** work as a vault credential for the Notion MCP server. These are different auth systems.
### Vaults — the MCP credential store
**Vaults** store OAuth credentials (access token + refresh token) that Anthropic auto-refreshes on your behalf via standard OAuth 2.0 `refresh_token` grant. This is the only way to authenticate MCP servers in the launch SDK.
#### Credentials and the sandbox
Vaults store credentials; those credentials **never enter the sandbox**. This is a deliberate security boundary — code running in the sandbox (including anything the agent writes) cannot read or exfiltrate a vaulted credential, even under prompt injection. Instead, credentials are injected by Anthropic-side proxies **after** a request leaves the sandbox:
- **MCP tool calls** are routed through an Anthropic-side proxy that fetches the credential from the vault and adds it to the outbound request.
- **Git operations on attached GitHub repositories** (`git pull`, `git push`, GitHub REST calls) are routed through a git proxy that injects the `github_repository` resource's `authorization_token` the same way.
**Not yet supported:** running other authenticated CLIs (e.g. `aws`, `gcloud`, `stripe`) directly inside the sandbox. There is currently no way to set container environment variables or expose vault credentials to arbitrary processes. If you need one of these today:
- **Prefer an MCP server** for that service if one exists — it gets the same vault-backed injection.
- **Otherwise, register a custom tool:** the agent emits `agent.custom_tool_use`, your orchestrator (which already holds the credential) executes the call and returns `user.custom_tool_result` over the same authenticated event stream. No public endpoint is exposed; the sandbox never sees the secret. See `shared/managed-agents-client-patterns.md` → Pattern 9.
**Do not put API keys in the system prompt or user messages as a workaround** — they persist in the session's event history.
> Formerly known internally as TATs (Tool/Tenant Access Tokens).
**Flow:**
1. Create a vault (`client.beta.vaults.create(...)`) — one per tenant/user, or one shared, depending on your model
2. Add MCP credentials to it (`client.beta.vaults.credentials.create(...)`) — each credential is tied to one MCP server URL
3. Reference the vault on session create via `vault_ids: ["vlt_..."]`
4. Anthropic auto-refreshes tokens before they expire; the agent uses the current access token when calling MCP tools
**Credential shape**:
```json
{
"display_name": "Notion (workspace-foo)",
"auth": {
"type": "mcp_oauth",
"mcp_server_url": "https://mcp.notion.com/mcp",
"access_token": "<current access token>",
"expires_at": "2026-04-02T14:00:00Z",
"refresh": {
"refresh_token": "<refresh token>",
"client_id": "<your OAuth client_id>",
"token_endpoint": "https://api.notion.com/v1/oauth/token",
"token_endpoint_auth": { "type": "none" }
}
}
}
```
The `refresh` block is what enables auto-refresh — `token_endpoint` is where Anthropic posts the `refresh_token` grant. `token_endpoint_auth` is a discriminated union:
| `type` | Shape | Use when |
|---|---|---|
| `"none"` | `{type: "none"}` | Public OAuth client (no secret) |
| `"client_secret_basic"` | `{type: "client_secret_basic", client_secret: "..."}` | Confidential client, secret via HTTP Basic auth |
| `"client_secret_post"` | `{type: "client_secret_post", client_secret: "..."}` | Confidential client, secret in request body |
Omit `refresh` entirely if you only have an access token with no refresh capability — it'll work until it expires, then the agent loses access.
> 💡 **Getting an OAuth token.** How you obtain the initial access and refresh tokens depends on the MCP server — consult its documentation. Once you have them, store them in a vault credential using the shape above; Anthropic auto-refreshes via the `refresh.token_endpoint` from there.
**Scoping:** Vaults are workspace-scoped. Anyone with developer+ role in the API workspace can create, read (metadata only — secrets are write-only), and attach vaults. `vault_ids` can be set at session **create** time but not via session update (the SDK docstring says "Not yet supported; requests setting this field are rejected").
---
## Skills
Skills are reusable, filesystem-based resources that provide your agent with domain-specific expertise: workflows, context, and best practices that transform general-purpose agents into specialists. Unlike prompts (conversation-level instructions for one-off tasks), skills load on-demand and eliminate the need to repeatedly provide the same guidance across multiple conversations.
Two types — both work the same way; the agent automatically uses them when relevant to the task at hand:
| Type | What it is |
|---|---|
| **Pre-built Anthropic skills** | Common document tasks (PowerPoint, Excel, Word, PDF). Reference by name (e.g. `xlsx`). |
| **Custom skills** | Skills you've created in your organization via the Skills API. Reference by `skill_id` + optional `version`. |
**Max 64 skills per agent.** Agent creation uses `managed-agents-2026-04-01`; the separate Skills API (for managing custom skill definitions) uses `skills-2025-10-02`.
### Enabling skills on a session
Skills are attached to the **agent** definition via `agents.create()`:
```ts
const agent = await client.beta.agents.create(
{
name: "Financial Agent",
model: "claude-opus-4-7",
system: "You are a financial analysis agent.",
skills: [
{ type: "anthropic", skill_id: "xlsx" },
{ type: "custom", skill_id: "skill_abc123", version: "latest" },
],
}
);
```
Python:
```python
agent = client.beta.agents.create(
name="Financial Agent",
model="claude-opus-4-7",
system="You are a financial analysis agent.",
skills=[
{"type": "anthropic", "skill_id": "xlsx"},
{"type": "custom", "skill_id": "skill_abc123", "version": "latest"},
]
)
```
**Skill reference fields:**
| Field | Anthropic skill | Custom skill |
|---|---|---|
| `type` | `"anthropic"` | `"custom"` |
| `skill_id` | Skill name (e.g. `"xlsx"`, `"docx"`, `"pptx"`, `"pdf"`) | Skill ID from Skills API (e.g. `"skill_abc123"`) |
| `version` | — | `"latest"` or a specific version number |
### Skills API
| Operation | Method | Path |
| --------------------- | -------- | ----------------------------------------------- |
| Create Skill | `POST` | `/v1/skills` |
| List Skills | `GET` | `/v1/skills` |
| Get Skill | `GET` | `/v1/skills/{id}` |
| Delete Skill | `DELETE` | `/v1/skills/{id}` |
| Create Version | `POST` | `/v1/skills/{id}/versions` |
| List Versions | `GET` | `/v1/skills/{id}/versions` |
| Get Version | `GET` | `/v1/skills/{id}/versions/{version}` |
| Delete Version | `DELETE` | `/v1/skills/{id}/versions/{version}` |
resources/bundled/skills/claude-api/shared/model-migration.md
+779
View File
@@ -0,0 +1,779 @@
# Model Migration Guide
How to move existing code to newer Claude models. Covers breaking changes, deprecated parameters, and drop-in replacements for retired models.
For the latest, authoritative version (with code samples in every supported language), WebFetch the **Migration Guide** URL from `shared/live-sources.md`. Use this file for the consolidated, skill-resident reference; fall back to the live docs whenever a model launch or breaking change may have shifted the picture.
**This file is large.** Use the section names below to jump (or `Grep` this file for the heading text). Read Step 0 and Step 1 first — they apply to every migration. Then read only the per-target section for the model you are migrating to.
| Section | When you need it |
|---|---|
| Step 0: Confirm the migration scope | Always — before any edits |
| Step 1: Classify each file | Always — decides whether to swap, add-alongside, or skip |
| Per-SDK Syntax Reference | Translate the Python examples in this guide to TypeScript / Go / Ruby / Java / C# / PHP |
| Destination Models / Retired Model Replacements | Picking a target model |
| Breaking Changes by Source Model | Migrating to Opus 4.6 / Sonnet 4.6 |
| Migrating to Opus 4.7 | Migrating to Opus 4.7 (breaking changes, silent defaults, behavioral shifts) |
| Opus 4.7 Migration Checklist | The required vs optional items for 4.7, tagged `[BLOCKS]` / `[TUNE]` |
| Verify the Migration | After edits — runtime spot-check |
**TL;DR:** Change the model ID string. If you were using `budget_tokens`, switch to `thinking: {type: "adaptive"}`. If you were using assistant prefills, they 400 on both Opus 4.6 and Sonnet 4.6 — switch to one of the prefill replacements (most often `output_config.format`; see the table in Breaking Changes by Source Model). If you're moving from Sonnet 4.5 to Sonnet 4.6, set `effort` explicitly — 4.6 defaults to `high`. Remove the `effort-2025-11-24` and `fine-grained-tool-streaming-2025-05-14` beta headers (GA on 4.6); remove `interleaved-thinking-2025-05-14` once you're on adaptive thinking (keep it only while using the transitional `budget_tokens` escape hatch). Then drop back from `client.beta.messages.create` to `client.messages.create`. Dial back any aggressive "CRITICAL: YOU MUST" tool instructions; 4.6 follows the system prompt much more closely.
---
## Step 0: Confirm the migration scope
**Before any Write, Edit, or MultiEdit call, confirm the scope.** If the user's request does not explicitly name a single file, a specific directory, or an explicit file list, **ask first — do not start editing**. This is non-negotiable: even imperative-sounding requests like "migrate my codebase", "move my project to X", "upgrade to Sonnet 4.6", or bare "migrate to Opus 4.7" leave the scope ambiguous and require a clarifying question. Phrases like "my project", "my code", "my codebase", "the whole thing", "everywhere", or "across the repo" are **ambiguous, not directive** — they tell you *what* to do but not *where*. Ask before doing.
Offer the common scopes explicitly and wait for the answer before touching any file:
1. The entire working directory
2. A specific subdirectory (e.g. `src/`, `app/`, `services/billing/`)
3. A specific file or a list of files
Surface this as a single clarifying question so the user can answer in one turn. **Proceed without asking only when the scope is already unambiguous** — the user named an exact file ("migrate `extract.py` to Sonnet 4.6"), pointed at a specific directory ("migrate everything under `services/billing/` to Opus 4.6"), listed specific files ("update `a.py` and `b.py`"), or already answered the scope question in an earlier turn. If you can answer the question "which files is this change going to touch?" with a precise list from the prompt alone, proceed. If not, ask.
**Worked example.** If the user says *"Move my project to Opus 4.6. I want adaptive thinking everywhere it makes sense."* you do not know whether "my project" means the whole working directory, just `src/`, just the production code, or something else — the `everywhere` makes the intent clear (update every call site *within scope*) but the scope itself is still not defined. Do not start editing. Respond with:
> Before I start editing, can you confirm the scope? I can migrate:
> 1. Every `.py` file in the working directory
> 2. Just the files under `src/` (production code)
> 3. A specific subdirectory or list of files you name
>
> Which one?
Then wait for the answer. The same applies to *"Migrate to Opus 4.7"* and bare *"Help me upgrade to Sonnet 4.6"* — ask before editing.
**Sizing the scope question (large repos).** Before asking, get a per-directory count so the user can pick concretely:
```sh
rg -l "<old-model-id>" --type-not md | cut -d/ -f1 | sort | uniq -c | sort -rn
```
Present the breakdown in your scope question (e.g. *"Found 217 references across 3 directories: api/ (130), api-go/ (62), routing/ (25). Which to migrate?"*). Also confirm `git status` is clean before surveying — unexpected modifications mean a concurrent process; stop and investigate before proceeding.
---
## Step 1: Classify each file
Not every file that contains the old model ID is a **caller** of the API. Before editing, classify each file into one of these buckets — the right action differs:
| # | Bucket | What it looks like | Action |
|---|---|---|---|
| 1 | **Calls the API/SDK** | `client.messages.create(model=…)`, `anthropic.Anthropic()`, request payloads | Swap the model ID **and** apply the breaking-change checklist for the target version (below). |
| 2 | **Defines or serves the model** | Model registries, OpenAPI specs, routing/queue configs, model-policy enums, generated catalogs | The old entry **stays** (the model is still served). Ask whether to (a) add the new model alongside, (b) leave alone, or (c) retire the old model — never blind-replace. **If you can't ask, default to (a): add the new model alongside and flag it** — replacing would de-register a model that's still in production. |
| 3 | **References the ID as an opaque string** | UI fallback constants, capability-gate substring checks, generic test fixtures, label parsers, env defaults | Usually swap the string and verify any parser/regex/substring match handles the new ID — but check the sub-cases below first. |
| 4 | **Suffixed variant ID** | `claude-<model>-<suffix>` like `-fast`, `-1024k`, `-200k`, `[1m]`, dated snapshots | These are deployment/routing identifiers, not the public model ID. **Do not assume a new-model equivalent exists.** Verify in the registry first; if absent, leave the string alone and flag it. |
**Bucket 3 sub-cases — before swapping a string reference, check:**
- **Capability gate** (e.g. `if 'opus-4-6' in model_id:` enables a feature) → **add the new ID alongside**, don't replace. The old model is still served and still has the capability, so replacing would silently disable the feature for any old-model traffic that still flows through. If you know no old-model traffic will hit this gate (single-caller codebase fully migrating), replacing is fine; if unsure, add alongside.
- **Registry-assert test** (e.g. `assert "claude-X" in supported_models`, `test_X_has_N_clusters`) → **add an assertion for the new model alongside; keep the old one.** The old model is still served, so its assertion stays valid — but the registry should also include the new model, so assert that too. Heuristic: if the test references multiple model versions in a list, it's a registry test; if one model in a struct compared only to itself, it's a generic fixture.
- **Frozen / generated snapshot** → **regenerate**, don't hand-edit.
- **Coupled to a definer** (e.g. an integration test that passes model authorization via a shared `conftest` seed list, or asserts on a billing-tier / rate-limit-group enum or a generated SKU/pricing catalog) → **verify the definer has a new-model entry first.** If not, add a seed entry (reusing the nearest existing tier as a placeholder); if you can't confidently do that, ask the user how to populate the definer. **Do not skip the test.** Swapping without populating the definer will make the test fail at runtime.
When migrating tests specifically: breaking parameters (`temperature`, `top_p`, `budget_tokens`) are usually absent — test fixtures rarely set sampling params on placeholder models. The breaking-change scan is still required, but expect mostly clean results.
**Find intentionally-flagged sync points first.** Many codebases tag spots that must change at every model launch with comment markers like `MODEL LAUNCH`, `KEEP IN SYNC`, `@model-update`, or similar. Grep for whatever convention the repo uses *before* the broad model-ID grep — those markers point at the load-bearing changes.
---
## Per-SDK Syntax Reference
Code examples in this guide are Python. **The same fields exist in every official Anthropic SDK** — Stainless generates all 7 from the same OpenAPI spec, so JSON field names map 1:1 with only case-convention differences. Use the rows below to translate the Python examples to the SDK you are migrating.
> **Verify type and method names against the SDK source before writing them into customer code.** WebFetch the relevant repository from the SDK source-code table in `shared/live-sources.md` (one row per SDK) and confirm the exact symbol — particularly for typed SDKs (Go, Java, C#) where union/builder names can differ from the JSON shape. Do not guess type names that aren't in the table below or in `<lang>/claude-api/README.md`.
### `thinking` — `budget_tokens` → adaptive
| SDK | Before | After |
|---|---|---|
| Python | `thinking={"type": "enabled", "budget_tokens": N}` | `thinking={"type": "adaptive"}` |
| TypeScript | `thinking: { type: 'enabled', budget_tokens: N }` | `thinking: { type: 'adaptive' }` |
| Go | `Thinking: anthropic.ThinkingConfigParamOfEnabled(N)` | `Thinking: anthropic.ThinkingConfigParamUnion{OfAdaptive: &anthropic.ThinkingConfigAdaptiveParam{}}` |
| Ruby | `thinking: { type: "enabled", budget_tokens: N }` | `thinking: { type: "adaptive" }` |
| Java | `.thinking(ThinkingConfigEnabled.builder().budgetTokens(N).build())` | `.thinking(ThinkingConfigAdaptive.builder().build())` |
| C# | `Thinking = new ThinkingConfigEnabled { BudgetTokens = N }` | `Thinking = new ThinkingConfigAdaptive()` |
| PHP | `thinking: ['type' => 'enabled', 'budget_tokens' => N]` | `thinking: ['type' => 'adaptive']` |
### Sampling parameters — `temperature` / `top_p` / `top_k`
(Remove the field entirely on Opus 4.7; on Claude 4.x keep at most one of `temperature` or `top_p`.)
| SDK | Field(s) to remove |
|---|---|
| Python | `temperature=…`, `top_p=…`, `top_k=…` |
| TypeScript | `temperature: …`, `top_p: …`, `top_k: …` |
| Go | `Temperature: anthropic.Float(…)`, `TopP: anthropic.Float(…)`, `TopK: anthropic.Int(…)` |
| Ruby | `temperature: …`, `top_p: …`, `top_k: …` |
| Java | `.temperature(…)`, `.topP(…)`, `.topK(…)` |
| C# | `Temperature = …`, `TopP = …`, `TopK = …` |
| PHP | `temperature: …`, `topP: …`, `topK: …` |
### Prefill replacement — structured outputs via `output_config.format`
| SDK | Remove (last assistant turn) | Add |
|---|---|---|
| Python | `{"role": "assistant", "content": "…"}` | `output_config={"format": {"type": "json_schema", "schema": SCHEMA}}` |
| TypeScript | `{ role: 'assistant', content: '…' }` | `output_config: { format: { type: 'json_schema', schema: SCHEMA } }` |
| Go | trailing `anthropic.MessageParam{Role: "assistant", …}` | `OutputConfig: anthropic.OutputConfigParam{Format: anthropic.JSONOutputFormatParam{…}}` |
| Ruby | `{ role: "assistant", content: "…" }` | `output_config: { format: { type: "json_schema", schema: SCHEMA } }` |
| Java | trailing `Message.builder().role(ASSISTANT)…` | `.outputConfig(OutputConfig.builder().format(JsonOutputFormat.builder()…build()).build())` |
| C# | trailing `new Message { Role = "assistant", … }` | `OutputConfig = new OutputConfig { Format = new JsonOutputFormat { … } }` |
| PHP | trailing `['role' => 'assistant', 'content' => '…']` | `outputConfig: ['format' => ['type' => 'json_schema', 'schema' => $SCHEMA]]` |
### `thinking.display` — opt back into summarized reasoning (Opus 4.7)
| SDK | Add |
|---|---|
| Python | `thinking={"type": "adaptive", "display": "summarized"}` |
| TypeScript | `thinking: { type: 'adaptive', display: 'summarized' }` |
| Go | `Thinking: anthropic.ThinkingConfigParamUnion{OfAdaptive: &anthropic.ThinkingConfigAdaptiveParam{Display: anthropic.ThinkingConfigAdaptiveDisplaySummarized}}` |
| Ruby | `thinking: { type: "adaptive", display: "summarized" }` (or `display_:` when constructing the model class directly) |
| Java | `.thinking(ThinkingConfigAdaptive.builder().display(ThinkingConfigAdaptive.Display.SUMMARIZED).build())` |
| C# | `Thinking = new ThinkingConfigAdaptive { Display = Display.Summarized }` |
| PHP | `thinking: ['type' => 'adaptive', 'display' => 'summarized']` |
For any field not in these tables, the JSON key in the Python example translates directly: `snake_case` for Python/TypeScript/Ruby, `camelCase` named args for PHP, `PascalCase` struct fields for Go/C#, `camelCase` builder methods for Java.
---
## Explain every change you make
Migration edits often look arbitrary to a user who hasn't read the release notes — a removed `temperature`, a deleted prefill, a rewritten system-prompt sentence. **For each edit, tell the user what you changed and why**, tied to the specific API or behavioral change that motivates it. Do this in your summary as you work, not just at the end.
Be especially explicit about **system-prompt edits**. Users are rightly protective of their prompts, and prompt-tuning changes are judgment calls (not hard API requirements). For any prompt edit:
- Quote the before and after text.
- State the behavioral shift that motivates it (e.g. *"Opus 4.7 calibrates response length to task complexity, so I added an explicit length instruction"*, or *"4.6 follows instructions more literally, so 'CRITICAL: YOU MUST use the search tool' will now overtrigger — softened to 'Use the search tool when…'"*).
- Make clear which prompt edits are **optional tuning** (tone, length, subagent guidance) versus which code edits are **required to avoid a 400** (sampling params, `budget_tokens`, prefills). Never present an optional prompt change as mandatory.
If you're applying several prompt-tuning edits at once, offer them as a short list the user can accept or decline item-by-item rather than silently rewriting their system prompt.
---
## Before You Migrate
1. **Confirm the target model ID.** Use only the exact strings from `shared/models.md` — do not append date suffixes to aliases (`claude-opus-4-6`, not `claude-opus-4-6-20251101`). Guessing an ID will 404.
2. **Check which features your code uses** with this checklist:
- `thinking: {type: "enabled", budget_tokens: N}` → migrate to adaptive thinking on Opus 4.6 / Sonnet 4.6 (still functional but deprecated)
- Assistant-turn prefills (`messages` ending with `role: "assistant"`) → must change on Opus 4.6 / Sonnet 4.6 (returns 400)
- `output_format` parameter on `messages.create()` → must change on all models (deprecated API-wide)
- `max_tokens > ~16000` → must stream on any model (above ~16K risks SDK HTTP timeouts). When streaming, Sonnet 4.6 / Haiku 4.5 cap at 64K and Opus 4.6 caps at 128K
- Beta headers `effort-2025-11-24`, `fine-grained-tool-streaming-2025-05-14`, `interleaved-thinking-2025-05-14` → GA on 4.6, remove them and switch from `client.beta.messages.create` to `client.messages.create`
- Moving Sonnet 4.5 → Sonnet 4.6 with no `effort` set → 4.6 defaults to `high`, which may change your latency/cost profile
- System prompts with `CRITICAL`, `MUST`, `If in doubt, use X` language → likely to overtrigger on 4.6 (see Prompt-Behavior Changes)
- Coming from 3.x / 4.0 / 4.1: also check sampling params (`temperature` + `top_p`), tool versions (`text_editor_20250728`), `refusal` + `model_context_window_exceeded` stop reasons, trailing-newline tool-param handling
3. **Test on a single request first.** Run one call against the new model, inspect the response, then roll out.
---
## Destination Models (recommended targets)
| If you're on… | Migrate to | Why |
| ------------------------------------- | ------------------ | ------------------------------------------------- |
| Opus 4.6 | `claude-opus-4-7` | Most capable model; adaptive thinking only; high-res vision; see Migrating to Opus 4.7 |
| Opus 4.0 / 4.1 / 4.5 / Opus 3 | `claude-opus-4-6` | Most intelligent 4.x before 4.7; adaptive thinking; 128K output |
| Sonnet 4.0 / 4.5 / 3.7 / 3.5 | `claude-sonnet-4-6`| Best speed / intelligence balance; adaptive thinking; 64K output |
| Haiku 3 / 3.5 | `claude-haiku-4-5` | Fastest and most cost-effective |
Default to the latest Opus for the caller's tier unless they explicitly chose otherwise. If you're moving from Opus 4.5 or older directly to Opus 4.7, apply the 4.6 migration first, then layer the Opus 4.7 changes on top (see Migrating to Opus 4.7 below).
---
## Retired Model Replacements
These models return 404 — update immediately:
| Retired model | Retired | Drop-in replacement |
| ----------------------------- | ------------- | -------------------- |
| `claude-3-7-sonnet-20250219` | Feb 19, 2026 | `claude-sonnet-4-6` |
| `claude-3-5-haiku-20241022` | Feb 19, 2026 | `claude-haiku-4-5` |
| `claude-3-opus-20240229` | Jan 5, 2026 | `claude-opus-4-7` |
| `claude-3-5-sonnet-20241022` | Oct 28, 2025 | `claude-sonnet-4-6` |
| `claude-3-5-sonnet-20240620` | Oct 28, 2025 | `claude-sonnet-4-6` |
| `claude-3-sonnet-20240229` | Jul 21, 2025 | `claude-sonnet-4-6` |
| `claude-2.1`, `claude-2.0` | Jul 21, 2025 | `claude-sonnet-4-6` |
## Deprecated Models (retiring soon)
| Model | Retires | Replacement |
| ----------------------------- | ------------- | -------------------- |
| `claude-3-haiku-20240307` | Apr 19, 2026 | `claude-haiku-4-5` |
| `claude-opus-4-20250514` | June 15, 2026 | `claude-opus-4-7` |
| `claude-sonnet-4-20250514` | June 15, 2026 | `claude-sonnet-4-6` |
---
## Breaking Changes by Source Model
### Migrating from Sonnet 4.5 to Sonnet 4.6 (effort default change)
Sonnet 4.5 had no `effort` parameter; Sonnet 4.6 defaults to `high`. If you just switch the model string and do nothing else, you may see noticeably higher latency and token usage. Set `effort` explicitly.
**Recommended starting points:**
| Workload | Start at | Notes |
| ------------------------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------- |
| Chat, classification, content generation | `low` | With `thinking: {"type": "disabled"}` you'll see similar or better performance vs. Sonnet 4.5 no-thinking |
| Most applications (balanced) | `medium` | The default sweet spot for quality vs. cost |
| Agentic coding, tool-heavy workflows | `medium` | Pair with adaptive thinking and a generous `max_tokens` (up to 64K with streaming — Sonnet 4.6's ceiling) |
| Autonomous multi-step agents, long-horizon loops | `high` | Scale down to `medium` if latency/tokens become a concern |
| Computer-use agents | `high` + adaptive | Sonnet 4.6's best computer-use accuracy is on adaptive + high |
For non-thinking chat workloads specifically:
```python
client.messages.create(
model="claude-sonnet-4-6",
max_tokens=8192,
thinking={"type": "disabled"},
output_config={"effort": "low"},
messages=[{"role": "user", "content": "..."}],
)
```
**When to use Opus 4.6 instead:** hardest and longest-horizon problems — large code migrations, deep research, extended autonomous work. Sonnet 4.6 wins on fast turnaround and cost efficiency.
### Migrating to Opus 4.6 / Sonnet 4.6 (from any older model)
**1. Manual extended thinking is deprecated — use adaptive thinking.**
`thinking: {type: "enabled", budget_tokens: N}` (manual extended thinking with a fixed token budget) is deprecated on Opus 4.6 and Sonnet 4.6. Replace it with `thinking: {type: "adaptive"}`, which lets Claude decide when and how much to think. Adaptive thinking also enables interleaved thinking automatically (no beta header needed).
```python
# Old (still works on older models, deprecated on 4.6)
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=16000,
thinking={"type": "enabled", "budget_tokens": 8000},
messages=[...]
)
# New (Opus 4.6 / Sonnet 4.6)
response = client.messages.create(
model="claude-opus-4-6", # or "claude-sonnet-4-6"
max_tokens=16000,
thinking={"type": "adaptive"},
output_config={"effort": "high"}, # optional: low | medium | high | max
messages=[...]
)
```
Adaptive thinking is the long-term target, and on internal evaluations it outperforms manual extended thinking. Move when you can.
**Transitional escape hatch:** manual extended thinking is still *functional* on Opus 4.6 and Sonnet 4.6 (deprecated, will be removed in a future release). If you need a hard ceiling while migrating — for example, to bound token spend on a runaway workload before you've tuned `effort` — you can keep `budget_tokens` around alongside an explicit `effort` value, then remove it in a follow-up. `budget_tokens` must be strictly less than `max_tokens`:
```python
# Transitional only — deprecated, plan to remove
client.messages.create(
model="claude-sonnet-4-6",
max_tokens=16384,
thinking={"type": "enabled", "budget_tokens": 8192}, # must be < max_tokens
output_config={"effort": "medium"},
messages=[...],
)
```
If the user asks for a "thinking budget" on 4.6, the preferred answer is `effort` — use `low`, `medium`, `high`, or `max` (Opus-tier only — not Sonnet or Haiku) rather than a token count.
**2. Effort parameter (Opus 4.5, Opus 4.6, Sonnet 4.6 only).**
Controls thinking depth and overall token spend. Goes inside `output_config`, not top-level. Default is `high`. `max` is Opus-tier only (Opus 4.6 and later — not Sonnet or Haiku). Errors on Sonnet 4.5 and Haiku 4.5.
```python
output_config={"effort": "medium"} # often the best cost / quality balance
```
### Migrating to the 4.6 family (Opus 4.6 and Sonnet 4.6)
**3. Assistant-turn prefills return 400 (Opus 4.6 and Sonnet 4.6).**
Prefilled responses on the final assistant turn are no longer supported on either Opus 4.6 or Sonnet 4.6 — both return a 400. Adding assistant messages *elsewhere* in the conversation (e.g., for few-shot examples) still works. Pick the replacement that matches what the prefill was doing:
| Prefill was used for | Replacement |
| -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Forcing JSON / YAML / schema output | `output_config.format` with a `json_schema` — see example below |
| Forcing a classification label | Tool with an enum field containing valid labels, or structured outputs |
| Skipping preambles (`Here is the summary:\n`) | System prompt instruction: *"Respond directly without preamble. Do not start with phrases like 'Here is...' or 'Based on...'."* |
| Steering around bad refusals | Usually no longer needed — 4.6 refuses far more appropriately. Plain user-turn prompting is sufficient. |
| Continuing an interrupted response | Move continuation into the user turn: *"Your previous response was interrupted and ended with `[last text]`. Continue from there."* |
| Injecting reminders / context hydration | Inject into the user turn instead. For complex agent harnesses, expose context via a tool call or during compaction. |
```python
# Old (fails on Opus 4.6 / Sonnet 4.6) — prefill forcing JSON shape
messages=[
{"role": "user", "content": "Extract the name."},
{"role": "assistant", "content": "{\"name\": \""},
]
# New — structured outputs replace the prefill
response = client.messages.create(
model="claude-opus-4-6",
max_tokens=1024,
output_config={"format": {"type": "json_schema", "schema": {...}}},
messages=[{"role": "user", "content": "Extract the name."}],
)
```
**4. Stream for `max_tokens > ~16K` (all models); Opus 4.6 alone reaches 128K.**
Non-streaming requests hit SDK HTTP timeouts at high `max_tokens`, regardless of model — stream for anything above ~16K output. The streamable ceiling differs by model: Sonnet 4.6 and Haiku 4.5 cap at 64K, and Opus 4.6 alone goes up to 128K.
```python
with client.messages.stream(model="claude-opus-4-6", max_tokens=64000, ...) as stream:
message = stream.get_final_message()
```
**5. Tool-call JSON escaping may differ (Opus 4.6 and Sonnet 4.6).**
Both 4.6 models can produce tool call `input` fields with Unicode or forward-slash escaping. Always parse with `json.loads()` / `JSON.parse()` — never raw-string-match the serialized input.
### All models
**6. `output_format` → `output_config.format` (API-wide).**
The old top-level `output_format` parameter on `messages.create()` is deprecated. Use `output_config.format` instead. This is not 4.6-specific — applies to every model.
---
## Beta Headers to Remove on 4.6
Several beta headers that were required on 4.5 are now GA on 4.6 and should be removed. Leaving them in is harmless but misleading; removing them also lets you move from `client.beta.messages.create(...)` back to `client.messages.create(...)`.
| Header | Status on 4.6 | Action |
| ----------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------- |
| `effort-2025-11-24` | Effort parameter is GA | Remove |
| `fine-grained-tool-streaming-2025-05-14` | GA | Remove |
| `interleaved-thinking-2025-05-14` | Adaptive thinking enables interleaved thinking automatically | Remove when using adaptive thinking; still functional on Sonnet 4.6 *with* manual extended thinking, but that path is deprecated |
| `token-efficient-tools-2025-02-19` | Built in to all Claude 4+ models | Remove (no effect) |
| `output-128k-2025-02-19` | Built in to Claude 4+ models | Remove (no effect) |
Once you remove all of these and finish moving to adaptive thinking, you can switch the SDK call site from the beta namespace back to the regular one:
```python
# Before
response = client.beta.messages.create(
model="claude-opus-4-5",
betas=["interleaved-thinking-2025-05-14", "effort-2025-11-24"],
...
)
# After
response = client.messages.create(
model="claude-opus-4-6",
thinking={"type": "adaptive"},
output_config={"effort": "high"},
...
)
```
---
## Additional Changes When Coming from 3.x / 4.0 / 4.1 → 4.6
If you're jumping from Opus 4.1, Sonnet 4, Sonnet 3.7, or an older Claude 3.x model directly to 4.6, apply everything above *plus* the items in this section. Users already on Opus 4.5 / Sonnet 4.5 can skip this.
**1. Sampling parameters: `temperature` OR `top_p`, not both.**
Passing both will error on every Claude 4+ model:
```python
# Old (3.x only — errors on 4+)
client.messages.create(temperature=0.7, top_p=0.9, ...)
# New
client.messages.create(temperature=0.7, ...) # or top_p, not both
```
**2. Update tool versions.**
Legacy tool versions are not supported on 4+. **Both the `type` and the `name` field change** — `text_editor_20250728` and `str_replace_based_edit_tool` are a pair; updating one without the other 400s. Also remove the `undo_edit` command from your text-editor integration:
| Old | New |
| ------------------------------------------------- | ------------------------------------------------------- |
| `text_editor_20250124` + `str_replace_editor` | `text_editor_20250728` + `str_replace_based_edit_tool` |
| `code_execution_*` (earlier versions) | `code_execution_20250825` |
| `undo_edit` command | *(no longer supported — delete call sites)* |
```python
# Before
tools = [{"type": "text_editor_20250124", "name": "str_replace_editor"}]
# After — BOTH fields change
tools = [{"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"}]
```
**3. Handle the `refusal` stop reason.**
Claude 4+ can return `stop_reason: "refusal"` on the response. If your code only handles `end_turn` / `tool_use` / `max_tokens`, add a branch:
```python
if response.stop_reason == "refusal":
# Surface the refusal to the user; do not retry with the same prompt
...
```
**4. Handle the `model_context_window_exceeded` stop reason (4.5+).**
Distinct from `max_tokens`: it means the model hit the *context window* limit, not the requested output cap. Handle both:
```python
if response.stop_reason == "model_context_window_exceeded":
# Context window exhausted — compact or split the conversation
...
elif response.stop_reason == "max_tokens":
# Requested output cap hit — retry with higher max_tokens or stream
...
```
**5. Trailing newlines preserved in tool call string parameters (4.5+).**
4.5 and 4.6 preserve trailing newlines that older models stripped. If your tool implementations do exact string matching against tool-call `input` values (e.g., `if name == "foo"`), verify they still match when the model sends `"foo\n"`. Normalizing with `.rstrip()` on the receiving side is usually the simplest fix.
**6. Haiku: rate limits reset between generations.**
Haiku 4.5 has its own rate-limit pool separate from Haiku 3 / 3.5. If you're ramping traffic as you migrate, check your tier's Haiku 4.5 limits at [API rate limits](https://platform.claude.com/docs/en/api/rate-limits) — a quota that comfortably served Haiku 3.5 traffic may need a tier bump for the same volume on 4.5.
---
## Prompt-Behavior Changes (Opus 4.5 / 4.6, Sonnet 4.6)
These don't break your code, but prompts that worked on 4.5-and-earlier may over- or under-trigger on 4.6. Tune as needed.
**1. Aggressive instructions cause overtriggering.** Opus 4.5 and 4.6 follow the system prompt much more closely than earlier models. Prompts written to *overcome* the old reluctance are now too aggressive:
| Before (worked on 4.0 / 4.5) | After (use on 4.6) |
| ------------------------------------------- | ----------------------------------------- |
| `CRITICAL: You MUST use this tool when...` | `Use this tool when...` |
| `Default to using [tool]` | `Use [tool] when it would improve X` |
| `If in doubt, use [tool]` | *(delete — no longer needed)* |
If the model is now overtriggering a tool or skill, the fix is almost always to dial back the language, not to add more guardrails.
**2. Overthinking and excessive exploration (Opus 4.6).** At higher `effort` settings, Opus 4.6 explores more before answering. If that burns too many thinking tokens, lower `effort` first (`medium` is often the sweet spot) before adding prose instructions to constrain reasoning.
**3. Overeager subagent spawning (Opus 4.6).** Opus 4.6 has a strong preference for delegating to subagents. If you see it spawning a subagent for something a direct `grep` or `read` would solve, add guidance: *"Use subagents only for parallel or independent workstreams. For single-file reads or sequential operations, work directly."*
**4. Overengineering (Opus 4.5 / 4.6).** Both models may add extra files, abstractions, or defensive error handling beyond what was asked. If you want minimal changes, prompt for it explicitly: *"Only make changes directly requested. Don't add helpers, abstractions, or error handling for scenarios that can't happen."*
**5. LaTeX math output (Opus 4.6).** Opus 4.6 defaults to LaTeX (`\frac{}{}`, `$...$`) for math and technical content. If you need plain text, instruct it explicitly: *"Format all math as plain text — no LaTeX, no `$`, no `\frac{}{}`. Use `/` for division and `^` for exponents."*
**6. Skipped verbal summaries (4.6 family).** The 4.6 models are more concise and may skip the summary paragraph after a tool call, jumping straight to the next action. If you rely on those summaries for visibility, add: *"After completing a task that involves tool use, provide a brief summary of what you did."*
**7. "Think" as a trigger word (Opus 4.5 with thinking disabled).** When `thinking` is off, Opus 4.5 is particularly sensitive to the word *think* and may reason more than you want. Use `consider`, `evaluate`, or `reason through` instead.
---
## Model-ID Rename Quick Reference
| Old string (migration source) | New string |
| ------------------------------ | ------------------ |
| `claude-opus-4-6` | `claude-opus-4-7` |
| `claude-opus-4-5` | `claude-opus-4-7` |
| `claude-opus-4-1` | `claude-opus-4-7` |
| `claude-opus-4-0` | `claude-opus-4-7` |
| `claude-sonnet-4-5` | `claude-sonnet-4-6`|
| `claude-sonnet-4-0` | `claude-sonnet-4-6`|
Older aliases (`claude-opus-4-5`, `claude-sonnet-4-5`, `claude-opus-4-1`, etc.) are still active and can be pinned if you need time before upgrading — see `shared/models.md` for the full legacy list.
---
## Migration Checklist
Every item is tagged: **`[BLOCKS]`** items cause a 400 error, infinite loop, silent timeout, or wrong tool selection if missed — apply these as code edits, not as suggestions. **`[TUNE]`** items are quality/cost adjustments.
For each file that calls `messages.create()` / equivalent SDK method:
- [ ] **[BLOCKS]** Update the `model=` string to the new alias
- [ ] **[BLOCKS]** Replace `budget_tokens` with `thinking={"type": "adaptive"}` (deprecated on Opus 4.6 / Sonnet 4.6)
- [ ] **[BLOCKS]** Move `format` from top-level `output_format` into `output_config.format`
- [ ] **[BLOCKS]** Remove any assistant-turn prefills if targeting Opus 4.6 or Sonnet 4.6 (see the prefill replacement table)
- [ ] **[BLOCKS]** Switch to streaming if `max_tokens > ~16000` (otherwise SDK HTTP timeout)
- [ ] **[TUNE]** Set `output_config={"effort": "..."}` explicitly — especially when moving Sonnet 4.5 → Sonnet 4.6 (4.6 defaults to `high`)
- [ ] **[TUNE]** Remove GA beta headers: `effort-2025-11-24`, `fine-grained-tool-streaming-2025-05-14`, `token-efficient-tools-2025-02-19`, `output-128k-2025-02-19`; remove `interleaved-thinking-2025-05-14` once on adaptive thinking
- [ ] **[TUNE]** Switch `client.beta.messages.create(...)` → `client.messages.create(...)` once all betas are removed
- [ ] **[TUNE]** Review system prompt for aggressive tool language (`CRITICAL:`, `MUST`, `If in doubt`) and dial it back
**Extra items when coming from 3.x / 4.0 / 4.1:**
- [ ] **[BLOCKS]** Remove either `temperature` or `top_p` (passing both 400s on Claude 4+)
- [ ] **[BLOCKS]** Update text-editor tool `type` to `text_editor_20250728`
- [ ] **[BLOCKS]** Update text-editor tool `name` to `str_replace_based_edit_tool` — **changing only the `type` and keeping `name: "str_replace_editor"` returns a 400**
- [ ] **[BLOCKS]** Update code-execution tool to `code_execution_20250825`
- [ ] **[BLOCKS]** Delete any `undo_edit` command call sites
- [ ] **[TUNE]** Add handling for `stop_reason == "refusal"`
- [ ] **[TUNE]** Add handling for `stop_reason == "model_context_window_exceeded"` (4.5+)
- [ ] **[TUNE]** Verify tool-param string matching tolerates trailing newlines (preserved on 4.5+)
- [ ] **[TUNE]** If moving to Haiku 4.5: review rate-limit tier (separate pool from Haiku 3.x)
**Verification:**
- [ ] Run one test request and inspect `response.stop_reason`, `response.usage`, and whether tool-use / thinking behavior matches expectations
For cached prompts: the render order and hash inputs did not change, so existing `cache_control` breakpoints keep working. However, **changing the model string invalidates the existing cache** — the first request on the new model will write the cache fresh.
---
## Migrating to Opus 4.7
> **Model ID `claude-opus-4-7` is authoritative as written here.** When the user asks to migrate to Opus 4.7, write `model="claude-opus-4-7"` exactly. Do **not** WebFetch to verify — this guide is the source of truth for migration target IDs. The corresponding entry exists in `shared/models.md`.
Claude Opus 4.7 is our most capable generally available model to date. It is highly autonomous and performs exceptionally well on long-horizon agentic work, knowledge work, vision tasks, and memory tasks. This section summarizes everything new at launch. It is layered on top of the 4.6 migration above — if the caller is jumping from Opus 4.5 or older, apply the 4.6 changes first, then apply this section.
**TL;DR for someone already on Opus 4.6:** update the model ID to `claude-opus-4-7`, strip any remaining `budget_tokens` and sampling parameters (both 400 on Opus 4.7), give `max_tokens` extra headroom and re-baseline with `count_tokens()` against the new model, opt back into `thinking.display: "summarized"` if reasoning is surfaced to users, and re-tune `effort` — it matters more on 4.7 than on any prior Opus.
### Breaking changes (will 400 on Opus 4.7)
**Extended thinking removed.**
`thinking: {type: "enabled", budget_tokens: N}` is no longer supported on Claude Opus 4.7 or later models and returns a 400 error. Switch to adaptive thinking (`thinking: {type: "adaptive"}`) and use the effort parameter to control thinking depth. Adaptive thinking is **off by default** on Claude Opus 4.7: requests with no `thinking` field run without thinking, matching Opus 4.6 behavior. Set `thinking: {type: "adaptive"}` explicitly to enable it.
```python
# Before (Opus 4.6)
client.messages.create(
model="claude-opus-4-6",
max_tokens=64000,
thinking={"type": "enabled", "budget_tokens": 32000},
messages=[{"role": "user", "content": "..."}],
)
# After (Opus 4.7)
client.messages.create(
model="claude-opus-4-7",
max_tokens=64000,
thinking={"type": "adaptive"},
output_config={"effort": "high"}, # or "max", "xhigh", "medium", "low"
messages=[{"role": "user", "content": "..."}],
)
```
If the caller wasn't using extended thinking, no change is required — thinking is off by default, or can be set explicitly with `thinking={"type": "disabled"}`.
Delete `budget_tokens` plumbing entirely. For the replacement `effort` value, see **Choosing an effort level on Opus 4.7** below — there is no exact 1:1 mapping from `budget_tokens`.
**Sampling parameters removed.**
The `temperature`, `top_p`, and `top_k` parameters are no longer accepted on Claude Opus 4.7. Requests that include them return a 400 error. Remove these fields from your request payloads. Prompting is the recommended way to guide model behavior on Claude Opus 4.7. If you were using `temperature = 0` for determinism, note that it never guaranteed identical outputs on prior models.
```python
# Before — errors on Opus 4.7
client.messages.create(temperature=0.7, top_p=0.9, ...)
# After
client.messages.create(...) # no sampling params
```
- **If the intent was determinism** — use `effort: "low"` with a tighter prompt.
- **If the intent was creative variance** — the prompt replacement depends on the use case; **ask the user** how they want variance elicited. If you can't ask, add a use-case-appropriate instruction along the lines of *"choose something off-distribution and interesting"* — e.g. for text generation, *"Vary your phrasing and structure across responses"*; for frontend/design, use the propose-4-directions approach under **Design and frontend coding** below.
### Choosing an effort level on Opus 4.7
`budget_tokens` controlled how much to *think*; `effort` controls how much to think *and* act, so there is no exact 1:1 mapping. **Use `xhigh` for best results in coding and agentic use cases, and a minimum of `high` for most intelligence-sensitive use cases.** Experiment with other levels to further tune token usage and intelligence:
| Level | Use when | Notes |
| --- | --- | --- |
| `max` | Intelligence-demanding tasks worth testing at the ceiling | Can deliver gains in some use cases but may show diminishing returns from increased token usage; can be prone to overthinking |
| `xhigh` | **Most coding and agentic use cases** | The best setting for these; used as the default in Claude Code |
| `high` | Intelligence-sensitive use cases generally | Balances token usage and intelligence; recommended minimum for most intelligence-sensitive work |
| `medium` | Cost-sensitive use cases that need to reduce token usage while trading off intelligence | |
| `low` | Short, scoped tasks and latency-sensitive workloads that are not intelligence-sensitive | |
### Silent default changes (no error, but behavior differs)
**Thinking content omitted by default.**
Thinking blocks still appear in the response stream on Claude Opus 4.7, but their `thinking` field is empty unless you explicitly opt in. This is a silent change from Claude Opus 4.6, where the default was to return summarized thinking text. To restore summarized thinking content on Claude Opus 4.7, set `thinking.display` to `"summarized"`. **The block-field name is unchanged** — it is still `block.thinking` on a `thinking`-type block; do not rename it.
**Detect this:** any code that reads `block.thinking` (or equivalent) from a `thinking`-type block and renders it in a UI, log, or trace. **The fix is the request parameter, not the response handling** — add `display: "summarized"` to the `thinking` parameter:
```python
thinking={"type": "adaptive", "display": "summarized"} # "display" is new on Opus 4.7; values: "omitted" (default) | "summarized"
```
The default is `"omitted"` on Claude Opus 4.7. If thinking content was never surfaced anywhere, no change needed. If your product streams reasoning to users, the new default appears as a long pause before output begins; set `display: "summarized"` to restore visible progress during thinking.
**Updated token counting.**
Claude Opus 4.7 and Claude Opus 4.6 count tokens differently. The same input text produces a higher token count on Claude Opus 4.7 than on Claude Opus 4.6, and `/v1/messages/count_tokens` will return a different number of tokens for Claude Opus 4.7 than it did for Claude Opus 4.6. The token efficiency of Claude Opus 4.7 can vary by workload shape. Prompting interventions, `task_budget`, and `effort` can help control costs and ensure appropriate token usage. Keep in mind that these controls may trade off model intelligence. **Update your `max_tokens` parameters to give additional headroom, including compaction triggers.** Claude Opus 4.7 provides a 1M context window at standard API pricing with no long-context premium.
What else to check:
- Client-side token estimators (tiktoken-style approximations) calibrated against 4.6
- Cost calculators that multiply tokens by a fixed per-token rate
- Rate-limit retry thresholds keyed to measured token counts
Re-baseline by re-running `client.messages.count_tokens()` against `claude-opus-4-7` on a representative sample of the caller's prompts. Do not apply a blanket multiplier. For cost-sensitive workloads, consider reducing `effort` by one level (e.g. `high` → `medium`). For agentic loops, consider adopting Task Budgets (below).
### New feature: Task Budgets (beta)
Opus 4.7 introduces **task budgets** — tell Claude how many tokens it has for a full agentic loop (thinking + tool calls + final output). The model sees a running countdown and uses it to prioritize work and wrap up gracefully as the budget is consumed.
This is a **suggestion the model is aware of**, not a hard cap. It is distinct from `max_tokens`, which remains the enforced per-response limit and is *not* surfaced to the model. Use `task_budget` when you want the model to self-moderate; use `max_tokens` as a hard ceiling to cap usage.
Requires beta header `task-budgets-2026-03-13`:
```python
client.beta.messages.create(
betas=["task-budgets-2026-03-13"],
model="claude-opus-4-7",
max_tokens=64000,
thinking={"type": "adaptive"},
output_config={
"effort": "high",
"task_budget": {"type": "tokens", "total": 128000},
},
messages=[...],
)
```
Set a generous budget for open-ended agentic tasks and tighten it for latency-sensitive ones. **Minimum `task_budget.total` is 20,000 tokens.** If the budget is too restrictive for the task, the model may complete it less thoroughly, referencing its budget as the constraint. **Do not add `task_budget` during a migration unless you are sure the budget value is right** — if you can run the workload and measure, do so; otherwise ask the user for the value rather than guessing. This is the primary lever for offsetting the token-counting shift on agentic workloads.
### Capability improvements
**High-resolution vision.** Opus 4.7 is the first Claude model with high-resolution image support. Maximum image resolution is **2576 pixels on the long edge** (up from 1568px on Opus 4.6 and prior). This unlocks gains on vision-heavy workloads, especially computer use and screenshot/artifact/document understanding. Coordinates returned by the model now map 1:1 to actual image pixels, so no scale-factor math is needed.
High-res support is **automatic on Opus 4.7** — no beta header, no client-side opt-in required. The model accepts larger inputs and returns pixel-accurate coordinates out of the box.
**Token cost.** Full-resolution images on Opus 4.7 can use up to ~3× more image tokens than on prior models (up to ~4784 tokens per image, vs. the previous ~1,600-token cap). If the extra fidelity isn't needed, downsample client-side before sending to control cost — but **do not add downsampling by default during a migration**. If you're not sure whether the pipeline needs the fidelity, ask the user rather than guessing. Use `count_tokens()` on representative images on Opus 4.7 to re-baseline before reacting to any measured cost shift.
Beyond resolution, Opus 4.7 also improves on low-level perception (pointing, measuring, counting) and natural-image bounding-box localization and detection.
**Knowledge work.** Meaningful gains on tasks where the model visually verifies its own output — `.docx` redlining, `.pptx` editing, and programmatic chart/figure analysis (e.g. pixel-level data transcription via image-processing libraries). If prompts have scaffolding like *"double-check the slide layout before returning"*, try removing it and re-baselining.
**Memory.** Opus 4.7 is better at writing and using file-system-based memory. If an agent maintains a scratchpad, notes file, or structured memory store across turns, that agent should improve at jotting down notes to itself and leveraging its notes in future tasks.
**User-facing progress updates.** Opus 4.7 provides more regular, higher-quality interim updates during long agentic traces. If the system prompt has scaffolding like *"After every 3 tool calls, summarize progress"*, try removing it to avoid excessive user-facing text. If the length or contents of Opus 4.7's updates are not well-calibrated to your use case, explicitly describe what these updates should look like in the prompt and provide examples.
### Real-time cybersecurity safeguards
Requests that involve prohibited or high-risk topics may lead to refusals.
### Fast Mode: not available on Opus 4.7
Opus 4.7 does not have a Fast Mode variant. **Opus 4.6 Fast remains supported**. Only surface this if the caller's code actually uses a Fast Mode model string (e.g. `claude-opus-4-6-fast`); if the word "fast" does not appear in the code, say nothing about Fast Mode.
When you see `model="claude-opus-4-6-fast"` (or similar), **the migration edit is**:
```python
# Opus 4.7 has no Fast Mode — keeping on 4.6 Fast (caller's choice to switch to standard Opus 4.7).
model="claude-opus-4-6-fast",
```
That is: leave the model string **unchanged**, add the comment above it, and tell the user their two options — (a) stay on Opus 4.6 Fast, which remains supported, or (b) move latency-tolerant traffic to standard Opus 4.7 for the intelligence gain. Do **not** rewrite the model string to `claude-opus-4-7` yourself; that silently trades latency for intelligence, which is the caller's decision.
### Behavioral shifts (prompt-tunable)
These don't break anything, but prompts tuned for Opus 4.6 may land differently. Opus 4.7 is more steerable than 4.6, so small prompt nudges usually close the gap.
**More literal instruction following.** Claude Opus 4.7 interprets prompts more literally and explicitly than Claude Opus 4.6, particularly at lower effort levels. It will not silently generalize an instruction from one item to another, and it will not infer requests you didn't make. The upside of this literalism is precision and less thrash. It generally performs better for API use cases with carefully tuned prompts, structured extraction, and pipelines where you want predictable behavior. A prompt and harness review may be especially helpful for migration to Claude Opus 4.7.
**Verbosity calibrates to task complexity.** Opus 4.7 scales response length to how complex it judges the task to be, rather than defaulting to a fixed verbosity — shorter answers on simple lookups, much longer on open-ended analysis. If the product depends on a particular length or style, tune the prompt explicitly. To reduce verbosity:
> *"Provide concise, focused responses. Skip non-essential context, and keep examples minimal."*
If you see specific kinds of over-verbosity (e.g. over-explaining), add instructions targeting those. Positive examples showing the desired level of concision tend to be more effective than negative examples or instructions telling the model what not to do. Do **not** assume existing "be concise" instructions should be removed — test first.
**Tone and writing style.** Opus 4.7 is more direct and opinionated, with less validation-forward phrasing and fewer emoji than Opus 4.6's warmer style. As with any new model, prose style on long-form writing may shift. If the product relies on a specific voice, re-evaluate style prompts against the new baseline. If a warmer or more conversational voice is wanted, specify it:
> *"Use a warm, collaborative tone. Acknowledge the user's framing before answering."*
**`effort` matters more than on any prior Opus.** Opus 4.7 respects `effort` levels more strictly, especially at the low end. At `low` and `medium` it scopes work to what was asked rather than going above and beyond — good for latency and cost, but on moderate tasks at `low` there is some risk of under-thinking.
- If shallow reasoning shows up on complex problems, raise `effort` to `high` or `xhigh` rather than prompting around it.
- If `effort` must stay `low` for latency, add targeted guidance: *"This task involves multi-step reasoning. Think carefully through the problem before responding."*
- **At `xhigh` or `max`, set a large `max_tokens`** so the model has room to think and act across tool calls and subagents. Start at 64K and tune from there. (`xhigh` is a new effort level on Opus 4.7, between `high` and `max`.)
Adaptive-thinking triggering is also steerable. If the model thinks more often than wanted — which can happen with large or complex system prompts — add: *"Thinking adds latency and should only be used when it will meaningfully improve answer quality — typically for problems that require multi-step reasoning. When in doubt, respond directly."*
**Uses tools less often by default.** Opus 4.7 tends to use tools less often than 4.6 and to use reasoning more. This produces better results in most cases, but for products that rely on tools (search/retrieval, function-calling, computer-use steps), it can drop tool-use rate. Two levers:
- **Raise `effort`** — `high` or `xhigh` show substantially more tool usage in agentic search and coding, and are especially useful for knowledge work.
- **Prompt for it** — be explicit in tool descriptions or the system prompt about when and how to use the tool, and encourage the model to err on the side of using it more often:
> *"When the answer depends on information not present in the conversation, you MUST call the `search` tool before answering — do not answer from prior knowledge."*
**Fewer subagents by default.** Opus 4.7 tends to spawn fewer subagents than 4.6. This is steerable — give explicit guidance on when delegation is desirable. For a coding agent, for example:
> *"Do NOT spawn a subagent for work you can complete directly in a single response (e.g. refactoring a function you can already see). Spawn multiple subagents in the same turn when fanning out across items or reading multiple files."*
**Design and frontend coding.** Opus 4.7 has stronger design instincts than 4.6, with a consistent default house style: warm cream/off-white backgrounds (around `#F4F1EA`), serif display type (Georgia, Fraunces, Playfair), italic word-accents, and a terracotta/amber accent. This reads well for editorial, hospitality, and portfolio briefs, but will feel off for dashboards, dev tools, fintech, healthcare, or enterprise apps — and it appears in slide decks as well as web UIs.
The default is persistent. Generic instructions ("don't use cream," "make it clean and minimal") tend to shift the model to a different fixed palette rather than producing variety. Two approaches work reliably:
1. **Specify a concrete alternative.** The model follows explicit specs precisely — give exact hex values, typefaces, and layout constraints.
2. **Have the model propose options before building.** This breaks the default and gives the user control:
> *"Before building, propose 4 distinct visual directions tailored to this brief (each as: bg hex / accent hex / typeface — one-line rationale). Ask the user to pick one, then implement only that direction."*
If the caller previously relied on `temperature` for design variety, use approach (2) — it produces meaningfully different directions across runs.
Opus 4.7 also requires less frontend-design prompting than previous models to avoid generic "AI slop" aesthetics. Where earlier models needed a lengthy anti-slop snippet, Opus 4.7 generates distinctive, creative frontends with a much shorter nudge. This snippet works well alongside the variety approaches above:
> *"NEVER use generic AI-generated aesthetics like overused font families (Inter, Roboto, Arial, system fonts), cliched color schemes (particularly purple gradients on white or dark backgrounds), predictable layouts and component patterns, and cookie-cutter design that lacks context-specific character. Use unique fonts, cohesive colors and themes, and animations for effects and micro-interactions."*
**Interactive coding products.** Opus 4.7's token usage and behavior can differ between autonomous, asynchronous coding agents with a single user turn and interactive, synchronous coding agents with multiple user turns. Specifically, it tends to use more tokens in interactive settings, primarily because it reasons more after user turns. This can improve long-horizon coherence, instruction following, and coding capabilities in long interactive coding sessions, but also comes with more token usage. To maximize both performance and token efficiency in coding products, use `effort: "xhigh"` or `"high"`, add autonomous features (like an auto mode), and reduce the number of human interactions required from users.
When limiting required user interactions, specify the task, intent, and relevant constraints upfront in the first human turn. Well-specified, clear, and accurate task descriptions upfront help maximize autonomy and intelligence while minimizing extra token usage after user turns — because Opus 4.7 is more autonomous than prior models, this usage pattern helps to maximize performance. In contrast, ambiguous or underspecified prompts conveyed progressively over multiple user turns tend to reduce token efficiency and sometimes performance.
**Code review.** Opus 4.7 is meaningfully better at finding bugs than prior models, with both higher recall and precision. However, if a code-review harness was tuned for an earlier model, it may initially show *lower* recall — this is likely a harness effect, not a capability regression. When a review prompt says "only report high-severity issues," "be conservative," or "don't nitpick," Opus 4.7 follows that instruction more faithfully than earlier models did: it investigates just as thoroughly, identifies the bugs, and then declines to report findings it judges to be below the stated bar. Precision rises, but measured recall can fall even though underlying bug-finding has improved.
Recommended prompt language:
> *"Report every issue you find, including ones you are uncertain about or consider low-severity. Do not filter for importance or confidence at this stage — a separate verification step will do that. Your goal here is coverage: it is better to surface a finding that later gets filtered out than to silently drop a bug. For each finding, include your confidence level and an estimated severity so a downstream filter can rank them."*
This can be used without an actual second step, but moving confidence filtering out of the finding step often helps. If the harness has a separate verification/dedup/ranking stage, tell the model explicitly that its job at the finding stage is coverage, not filtering. If single-pass self-filtering is wanted, be concrete about the bar rather than using qualitative terms like "important" — e.g. *"report any bugs that could cause incorrect behavior, a test failure, or a misleading result; only omit nits like pure style or naming preferences."* Iterate on prompts against a subset of evals to validate recall or F1 gains.
**Computer use.** Computer use works across resolutions up to the new 2576px / 3.75MP maximum. Sending images at **1080p** provides a good balance of performance and cost. For particularly cost-sensitive workloads, **720p** or **1366×768** are lower-cost options with strong performance. Test to find the ideal settings for the use case; experimenting with `effort` can also help tune behavior.
---
## Opus 4.7 Migration Checklist
Every item is tagged: **`[BLOCKS]`** items cause a 400 error, infinite loop, silent truncation, or empty output if missed — apply these as code edits, not as suggestions. **`[TUNE]`** items are quality/cost adjustments — surface them to the user as recommendations.
`[BLOCKS]` items prefixed with **"If…"** or **"At…"** are conditional. Before working through the list, **scan the file** for the conditions: does it surface thinking text to a UI/log? Does it set `output_config.effort` to `"x-high"` or `"max"`? Is it a security workload? Is it a multi-turn agentic loop? Apply only the items whose condition matches.
- [ ] **[BLOCKS]** Replace `thinking: {type: "enabled", budget_tokens: N}` with `thinking: {type: "adaptive"}` + `output_config.effort`; delete `budget_tokens` plumbing entirely
- [ ] **[BLOCKS]** Strip `temperature`, `top_p`, `top_k` from request construction
- [ ] **[BLOCKS]** If thinking content is surfaced to users or stored in logs: add `thinking.display: "summarized"` (otherwise the rendered text is empty)
- [ ] **[BLOCKS]** At `output_config.effort` of `xhigh` or `max`: set `max_tokens` ≥ 64000 (otherwise output truncates mid-thought)
- [ ] **[TUNE]** Give `max_tokens` and compaction triggers extra headroom; re-run `count_tokens()` against `claude-opus-4-7` on representative prompts to re-baseline (no blanket multiplier)
- [ ] **[TUNE]** Re-baseline cost and rate-limit dashboards *before* reacting to measured shifts
- [ ] **[TUNE]** Re-evaluate `effort` per route — use `xhigh` for coding/agentic and a minimum of `high` for most intelligence-sensitive work; it matters more on 4.7 than any prior Opus
- [ ] **[TUNE]** Multi-turn agentic loops: adopt the API-native Task Budgets (`output_config.task_budget`, beta `task-budgets-2026-03-13`, minimum 20k tokens) — this is for capping *cumulative* spend across a loop; per-turn depth is `effort`
- [ ] **[TUNE]** Check for ambiguous or underspecified instructions that relied on 4.6 generalizing intent, and update them to be clearer or more precise — 4.7 follows them literally
- [ ] **[TUNE]** Tool-use workloads: add explicit when/how-to-use guidance to tool descriptions (4.7 reaches for tools less often)
- [ ] **[TUNE]** Verbosity: test existing length instructions before changing them — 4.7 calibrates length to task complexity, so tune for the desired output rather than assuming a direction
- [ ] **[TUNE]** Remove forced-progress-update scaffolding (*"after every N tool calls…"*)
- [ ] **[TUNE]** Remove knowledge-work verification scaffolding (*"double-check the slide layout…"*) and re-baseline
- [ ] **[TUNE]** Add tone instruction if a warmer / more conversational voice is needed; re-evaluate style prompts on writing-heavy routes
- [ ] **[TUNE]** Subagent tool present: add explicit spawn / don't-spawn guidance
- [ ] **[TUNE]** Frontend/design output: specify a concrete palette/typeface, or have the model propose 4 visual directions before building (the default cream/serif house style is persistent)
- [ ] **[TUNE]** Interactive coding products: use `effort: "xhigh"` or `"high"`, add autonomous features (e.g. an auto mode) to reduce human interactions, and specify task/intent/constraints upfront in the first turn
- [ ] **[TUNE]** Code-review harnesses: remove or loosen "only report high-severity" / "be conservative" filters and have the model report every finding with confidence + severity; move filtering to a downstream step (4.7 follows severity filters more literally, which can depress measured recall)
- [ ] **[TUNE]** Vision-heavy pipelines (screenshots, charts, document understanding): leave images at native resolution up to 2576px long edge for the accuracy gain; remove any scale-factor math from coordinate handling (coords are now 1:1 with pixels). No beta header / opt-in needed — high-res is automatic on Opus 4.7.
- [ ] **[TUNE]** Computer-use pipelines: send screenshots at 1080p for a good performance/cost balance (720p or 1366×768 for cost-sensitive workloads); experiment with `effort` to tune behavior
- [ ] **[TUNE]** Cost-sensitive image pipelines: full-res images on 4.7 use up to ~4784 tokens vs ~1,600 on prior models (~3×). Downsampling client-side before upload avoids the increase, but **do not downsample by default** — if you're unsure whether fidelity is needed, ask the user. Re-baseline with `count_tokens()` on representative images before reacting to cost shifts.
---
## Verify the Migration
After updating, spot-check that the new model is actually being used. Replace `YOUR_TARGET_MODEL` with the model string you migrated to (e.g. `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5`) and keep the assertion prefix in sync:
```python
YOUR_TARGET_MODEL = "claude-opus-4-7" # or "claude-opus-4-6", "claude-sonnet-4-6", "claude-haiku-4-5"
response = client.messages.create(model=YOUR_TARGET_MODEL, max_tokens=64, messages=[...])
assert response.model.startswith(YOUR_TARGET_MODEL), response.model
```
For rate-limit headroom changes, pricing, or capability deltas (vision, structured outputs, effort support), query the Models API:
```python
m = client.models.retrieve(YOUR_TARGET_MODEL)
m.max_input_tokens, m.max_tokens
m.capabilities["effort"]["max"]["supported"]
```
See `shared/models.md` for the full capability lookup pattern.
resources/bundled/skills/claude-api/shared/models.md
+121
View File
@@ -0,0 +1,121 @@
# Claude Model Catalog
**Only use exact model IDs listed in this file.** Never guess or construct model IDs — incorrect IDs will cause API errors. Use aliases wherever available. For the latest information, WebFetch the Models Overview URL in `shared/live-sources.md`, or query the Models API directly (see Programmatic Model Discovery below).
## Programmatic Model Discovery
For **live** capability data — context window, max output tokens, feature support (thinking, vision, effort, structured outputs, etc.) — query the Models API instead of relying on the cached tables below. Use this when the user asks "what's the context window for X", "does model X support vision/thinking/effort", "which models support feature Y", or wants to select a model by capability at runtime.
```python
m = client.models.retrieve("claude-opus-4-7")
m.id # "claude-opus-4-7"
m.display_name # "Claude Opus 4.7"
m.max_input_tokens # context window (int)
m.max_tokens # max output tokens (int)
# capabilities is an untyped nested dict — bracket access, check ["supported"] at the leaf
caps = m.capabilities
caps["image_input"]["supported"] # vision
caps["thinking"]["types"]["adaptive"]["supported"] # adaptive thinking
caps["effort"]["max"]["supported"] # effort: max (also low/medium/high)
caps["structured_outputs"]["supported"]
caps["context_management"]["compact_20260112"]["supported"]
# filter across all models — iterate the page object directly (auto-paginates); do NOT use .data
[m for m in client.models.list()
if m.capabilities["thinking"]["types"]["adaptive"]["supported"]
and m.max_input_tokens >= 200_000]
```
Top-level fields (`id`, `display_name`, `max_input_tokens`, `max_tokens`) are typed attributes. `capabilities` is a dict — use bracket access, not attribute access. The API returns the full capability tree for every model with `supported: true/false` at each leaf, so bracket chains are safe without `.get()` guards. TypeScript SDK: same method names, also auto-paginates on iteration.
### Raw HTTP
```bash
curl https://api.anthropic.com/v1/models/claude-opus-4-7 \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01"
```
```json
{
"id": "claude-opus-4-7",
"display_name": "Claude Opus 4.7",
"max_input_tokens": 200000,
"max_tokens": 128000,
"capabilities": {
"image_input": {"supported": true},
"structured_outputs": {"supported": true},
"thinking": {"supported": true, "types": {"enabled": {"supported": false}, "adaptive": {"supported": true}}},
"effort": {"supported": true, "low": {"supported": true}, …, "max": {"supported": true}},
…
}
}
```
## Current Models (recommended)
| Friendly Name | Alias (use this) | Full ID | Context | Max Output | Status |
|-------------------|---------------------|-------------------------------|----------------|------------|--------|
| Claude Opus 4.7 | `claude-opus-4-7` | — | 1M | 128K | Active |
| Claude Opus 4.6 | `claude-opus-4-6` | — | 1M | 128K | Active |
| Claude Sonnet 4.6 | `claude-sonnet-4-6` | - | 1M | 64K | Active |
| Claude Haiku 4.5 | `claude-haiku-4-5` | `claude-haiku-4-5-20251001` | 200K | 64K | Active |
### Model Descriptions
- **Claude Opus 4.7** — The most capable Claude model to date — highly autonomous, strong on long-horizon agentic work, knowledge work, vision, and memory. Adaptive thinking only; sampling parameters and `budget_tokens` are removed. 1M context window at standard API pricing (no long-context premium) — see `shared/model-migration.md` → Migrating to Opus 4.7 for breaking changes.
- **Claude Opus 4.6** — Previous-generation Opus. Supports adaptive thinking (recommended), 128K max output tokens (requires streaming for large outputs). 1M context window.
- **Claude Sonnet 4.6** — Our best combination of speed and intelligence. Supports adaptive thinking (recommended). 1M context window. 64K max output tokens.
- **Claude Haiku 4.5** — Fastest and most cost-effective model for simple tasks.
## Legacy Models (still active)
| Friendly Name | Alias (use this) | Full ID | Status |
|-------------------|---------------------|-------------------------------|--------|
| Claude Opus 4.5 | `claude-opus-4-5` | `claude-opus-4-5-20251101` | Active |
| Claude Opus 4.1 | `claude-opus-4-1` | `claude-opus-4-1-20250805` | Active |
| Claude Sonnet 4.5 | `claude-sonnet-4-5` | `claude-sonnet-4-5-20250929` | Active |
| Claude Sonnet 4 | `claude-sonnet-4-0` | `claude-sonnet-4-20250514` | Active |
| Claude Opus 4 | `claude-opus-4-0` | `claude-opus-4-20250514` | Active |
## Deprecated Models (retiring soon)
| Friendly Name | Alias (use this) | Full ID | Status | Retires |
|-------------------|---------------------|-------------------------------|------------|--------------|
| Claude Haiku 3 | — | `claude-3-haiku-20240307` | Deprecated | Apr 19, 2026 |
## Retired Models (no longer available)
| Friendly Name | Full ID | Retired |
|-------------------|-------------------------------|-------------|
| Claude Sonnet 3.7 | `claude-3-7-sonnet-20250219` | Feb 19, 2026 |
| Claude Haiku 3.5 | `claude-3-5-haiku-20241022` | Feb 19, 2026 |
| Claude Opus 3 | `claude-3-opus-20240229` | Jan 5, 2026 |
| Claude Sonnet 3.5 | `claude-3-5-sonnet-20241022` | Oct 28, 2025 |
| Claude Sonnet 3.5 | `claude-3-5-sonnet-20240620` | Oct 28, 2025 |
| Claude Sonnet 3 | `claude-3-sonnet-20240229` | Jul 21, 2025 |
| Claude 2.1 | `claude-2.1` | Jul 21, 2025 |
| Claude 2.0 | `claude-2.0` | Jul 21, 2025 |
## Resolving User Requests
When a user asks for a model by name, use this table to find the correct model ID:
| User says... | Use this model ID |
|-------------------------------------------|--------------------------------|
| "opus", "most powerful" | `claude-opus-4-7` |
| "opus 4.7" | `claude-opus-4-7` |
| "opus 4.6" | `claude-opus-4-6` |
| "opus 4.5" | `claude-opus-4-5` |
| "opus 4.1" | `claude-opus-4-1` |
| "opus 4", "opus 4.0" | `claude-opus-4-0` |
| "sonnet", "balanced" | `claude-sonnet-4-6` |
| "sonnet 4.6" | `claude-sonnet-4-6` |
| "sonnet 4.5" | `claude-sonnet-4-5` |
| "sonnet 4", "sonnet 4.0" | `claude-sonnet-4-0` |
| "sonnet 3.7" | Retired — suggest `claude-sonnet-4-5` |
| "sonnet 3.5" | Retired — suggest `claude-sonnet-4-5` |
| "haiku", "fast", "cheap" | `claude-haiku-4-5` |
| "haiku 4.5" | `claude-haiku-4-5` |
| "haiku 3.5" | Retired — suggest `claude-haiku-4-5` |
| "haiku 3" | Deprecated — suggest `claude-haiku-4-5` |
resources/bundled/skills/claude-api/shared/prompt-caching.md
+171
View File
@@ -0,0 +1,171 @@
# Prompt Caching — Design & Optimization
This file covers how to design prompt-building code for effective caching. For language-specific syntax, see the `## Prompt Caching` section in each language's README or single-file doc.
## The one invariant everything follows from
**Prompt caching is a prefix match. Any change anywhere in the prefix invalidates everything after it.**
The cache key is derived from the exact bytes of the rendered prompt up to each `cache_control` breakpoint. A single byte difference at position N — a timestamp, a reordered JSON key, a different tool in the list — invalidates the cache for all breakpoints at positions ≥ N.
Render order is: `tools` → `system` → `messages`. A breakpoint on the last system block caches both tools and system together.
Design the prompt-building path around this constraint. Get the ordering right and most caching works for free. Get it wrong and no amount of `cache_control` markers will help.
---
## Workflow for optimizing existing code
When asked to add or optimize caching:
1. **Trace the prompt assembly path.** Find where `system`, `tools`, and `messages` are constructed. Identify every input that flows into them.
2. **Classify each input by stability:**
- Never changes → belongs early in the prompt, before any breakpoint
- Changes per-session → belongs after the global prefix, cache per-session
- Changes per-turn → belongs at the end, after the last breakpoint
- Changes per-request (timestamps, UUIDs, random IDs) → **eliminate or move to the very end**
3. **Check rendered order matches stability order.** Stable content must physically precede volatile content. If a timestamp is interpolated into the system prompt header, everything after it is uncacheable regardless of markers.
4. **Place breakpoints at stability boundaries.** See placement patterns below.
5. **Audit for silent invalidators.** See anti-patterns table.
---
## Placement patterns
### Large system prompt shared across many requests
Put a breakpoint on the last system text block. If there are tools, they render before system — the marker on the last system block caches tools + system together.
```json
"system": [
{"type": "text", "text": "<large shared prompt>", "cache_control": {"type": "ephemeral"}}
]
```
### Multi-turn conversations
Put a breakpoint on the last content block of the most-recently-appended turn. Each subsequent request reuses the entire prior conversation prefix. Earlier breakpoints remain valid read points, so hits accrue incrementally as the conversation grows.
```json
// Last content block of the last user turn
messages[-1].content[-1].cache_control = {"type": "ephemeral"}
```
### Shared prefix, varying suffix
Many requests share a large fixed preamble (few-shot examples, retrieved docs, instructions) but differ in the final question. Put the breakpoint at the end of the **shared** portion, not at the end of the whole prompt — otherwise every request writes a distinct cache entry and nothing is ever read.
```json
"messages": [{"role": "user", "content": [
{"type": "text", "text": "<shared context>", "cache_control": {"type": "ephemeral"}},
{"type": "text", "text": "<varying question>"} // no marker — differs every time
]}]
```
### Prompts that change from the beginning every time
Don't cache. If the first 1K tokens differ per request, there is no reusable prefix. Adding `cache_control` only pays the cache-write premium with zero reads. Leave it off.
---
## Architectural guidance
These are the decisions that matter more than marker placement. Fix these first.
**Keep the system prompt frozen.** Don't interpolate "current date: X", "mode: Y", "user name: Z" into the system prompt — those sit at the front of the prefix and invalidate everything downstream. Inject dynamic context as a user or assistant message later in `messages`. A message at turn 5 invalidates nothing before turn 5.
**Don't change tools or model mid-conversation.** Tools render at position 0; adding, removing, or reordering a tool invalidates the entire cache. Same for switching models (caches are model-scoped). If you need "modes", don't swap the tool set — give Claude a tool that records the mode transition, or pass the mode as message content. Serialize tools deterministically (sort by name).
**Fork operations must reuse the parent's exact prefix.** Side computations (summarization, compaction, sub-agents) often spin up a separate API call. If the fork rebuilds `system` / `tools` / `model` with any difference, it misses the parent's cache entirely. Copy the parent's `system`, `tools`, and `model` verbatim, then append fork-specific content at the end.
---
## Silent invalidators
When reviewing code, grep for these inside anything that feeds the prompt prefix:
| Pattern | Why it breaks caching |
|---|---|
| `datetime.now()` / `Date.now()` / `time.time()` in system prompt | Prefix changes every request |
| `uuid4()` / `crypto.randomUUID()` / request IDs early in content | Same — every request is unique |
| `json.dumps(d)` without `sort_keys=True` / iterating a `set` | Non-deterministic serialization → prefix bytes differ |
| f-string interpolating session/user ID into system prompt | Per-user prefix; no cross-user sharing |
| Conditional system sections (`if flag: system += ...`) | Every flag combination is a distinct prefix |
| `tools=build_tools(user)` where set varies per user | Tools render at position 0; nothing caches across users |
Fix by moving the dynamic piece after the last breakpoint, making it deterministic, or deleting it if it's not load-bearing.
---
## API reference
```json
"cache_control": {"type": "ephemeral"} // 5-minute TTL (default)
"cache_control": {"type": "ephemeral", "ttl": "1h"} // 1-hour TTL
```
- Max **4** `cache_control` breakpoints per request.
- Goes on any content block: system text blocks, tool definitions, message content blocks (`text`, `image`, `tool_use`, `tool_result`, `document`).
- Top-level `cache_control` on `messages.create()` auto-places on the last cacheable block — simplest option when you don't need fine-grained placement.
- Minimum cacheable prefix is model-dependent. Shorter prefixes silently won't cache even with a marker — no error, just `cache_creation_input_tokens: 0`:
| Model | Minimum |
|---|---:|
| Opus 4.7, Opus 4.6, Opus 4.5, Haiku 4.5 | 4096 tokens |
| Sonnet 4.6, Haiku 3.5, Haiku 3 | 2048 tokens |
| Sonnet 4.5, Sonnet 4.1, Sonnet 4, Sonnet 3.7 | 1024 tokens |
A 3K-token prompt caches on Sonnet 4.5 but silently won't on Opus 4.7.
**Economics:** Cache reads cost ~0.1× base input price. Cache writes cost **1.25× for 5-minute TTL, 2× for 1-hour TTL**. Break-even depends on TTL: with 5-minute TTL, two requests break even (1.25× + 0.1× = 1.35× vs 2× uncached); with 1-hour TTL, you need at least three requests (2× + 0.2× = 2.2× vs 3× uncached). The 1-hour TTL keeps entries alive across gaps in bursty traffic, but the doubled write cost means it needs more reads to pay off.
---
## Verifying cache hits
The response `usage` object reports cache activity:
| Field | Meaning |
|---|---|
| `cache_creation_input_tokens` | Tokens written to cache this request (you paid the ~1.25× write premium) |
| `cache_read_input_tokens` | Tokens served from cache this request (you paid ~0.1×) |
| `input_tokens` | Tokens processed at full price (not cached) |
If `cache_read_input_tokens` is zero across repeated requests with identical prefixes, a silent invalidator is at work — diff the rendered prompt bytes between two requests to find it.
**`input_tokens` is the uncached remainder only.** Total prompt size = `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`. If your agent ran for hours but `input_tokens` shows 4K, the rest was served from cache — check the sum, not the single field.
Language-specific access: `response.usage.cache_read_input_tokens` (Python/TS/Ruby), `$message->usage->cacheReadInputTokens` (PHP), `resp.Usage.CacheReadInputTokens` (Go/C#), `.usage().cacheReadInputTokens()` (Java).
---
## Invalidation hierarchy
Not every parameter change invalidates everything. The API has three cache tiers, and changes only invalidate their own tier and below:
| Change | Tools cache | System cache | Messages cache |
|---|:---:|:---:|:---:|
| Tool definitions (add/remove/reorder) | ❌ | ❌ | ❌ |
| Model switch | ❌ | ❌ | ❌ |
| `speed`, web-search, citations toggle | ✅ | ❌ | ❌ |
| System prompt content | ✅ | ❌ | ❌ |
| `tool_choice`, images, `thinking` enable/disable | ✅ | ✅ | ❌ |
| Message content | ✅ | ✅ | ❌ |
Implication: you can change `tool_choice` per-request or toggle `thinking` without losing the tools+system cache. Don't over-worry about these — only tool-definition and model changes force a full rebuild.
---
## 20-block lookback window
Each breakpoint walks backward **at most 20 content blocks** to find a prior cache entry. If a single turn adds more than 20 blocks (common in agentic loops with many tool_use/tool_result pairs), the next request's breakpoint won't find the previous cache and silently misses.
Fix: place an intermediate breakpoint every ~15 blocks in long turns, or put the marker on a block that's within 20 of the previous turn's last cached block.
---
## Concurrent-request timing
A cache entry becomes readable only after the first response **begins streaming**. N parallel requests with identical prefixes all pay full price — none can read what the others are still writing.
For fan-out patterns: send 1 request, await the first streamed token (not the full response), then fire the remaining N−1. They'll read the cache the first one just wrote.
resources/bundled/skills/claude-api/shared/tool-use-concepts.md
+327
View File
@@ -0,0 +1,327 @@
# Tool Use Concepts
This file covers the conceptual foundations of tool use with the Claude API. For language-specific code examples, see the `python/`, `typescript/`, or other language folders. For decision heuristics on which tools to expose, how to manage context in long-running agents, and caching strategy, see `agent-design.md`.
## User-Defined Tools
### Tool Definition Structure
> **Note:** When using the Tool Runner (beta), tool schemas are generated automatically from your function signatures (Python), Zod schemas (TypeScript), annotated classes (Java), `jsonschema` struct tags (Go), or `BaseTool` subclasses (Ruby). The raw JSON schema format below is for the manual approach — including PHP's `BetaRunnableTool`, which wraps a run closure around a hand-written schema — or SDKs without tool runner support.
Each tool requires a name, description, and JSON Schema for its inputs:
```json
{
"name": "get_weather",
"description": "Get current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City and state, e.g., San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit"
}
},
"required": ["location"]
}
}
```
**Best practices for tool definitions:**
- Use clear, descriptive names (e.g., `get_weather`, `search_database`, `send_email`)
- Write detailed descriptions — Claude uses these to decide when to use the tool
- Include descriptions for each property
- Use `enum` for parameters with a fixed set of values
- Mark truly required parameters in `required`; make others optional with defaults
---
### Tool Choice Options
Control when Claude uses tools:
| Value | Behavior |
| --------------------------------- | --------------------------------------------- |
| `{"type": "auto"}` | Claude decides whether to use tools (default) |
| `{"type": "any"}` | Claude must use at least one tool |
| `{"type": "tool", "name": "..."}` | Claude must use the specified tool |
| `{"type": "none"}` | Claude cannot use tools |
Any `tool_choice` value can also include `"disable_parallel_tool_use": true` to force Claude to use at most one tool per response. By default, Claude may request multiple tool calls in a single response.
---
### Tool Runner vs Manual Loop
**Tool Runner (Recommended):** The SDK's tool runner handles the agentic loop automatically — it calls the API, detects tool use requests, executes your tool functions, feeds results back to Claude, and repeats until Claude stops calling tools. Available in Python, TypeScript, Java, Go, Ruby, and PHP SDKs (beta). The Python SDK also provides MCP conversion helpers (`anthropic.lib.tools.mcp`) to convert MCP tools, prompts, and resources for use with the tool runner — see `python/claude-api/tool-use.md` for details.
**Manual Agentic Loop:** Use when you need fine-grained control over the loop (e.g., custom logging, conditional tool execution, human-in-the-loop approval). Loop until `stop_reason == "end_turn"`, always append the full `response.content` to preserve tool_use blocks, and ensure each `tool_result` includes the matching `tool_use_id`.
**Stop reasons for server-side tools:** When using server-side tools (code execution, web search, etc.), the API runs a server-side sampling loop. If this loop reaches its default limit of 10 iterations, the response will have `stop_reason: "pause_turn"`. To continue, re-send the user message and assistant response and make another API request — the server will resume where it left off. Do NOT add an extra user message like "Continue." — the API detects the trailing `server_tool_use` block and knows to resume automatically.
```python
# Handle pause_turn in your agentic loop
if response.stop_reason == "pause_turn":
messages = [
{"role": "user", "content": user_query},
{"role": "assistant", "content": response.content},
]
# Make another API request — server resumes automatically
response = client.messages.create(
model="claude-opus-4-7", messages=messages, tools=tools
)
```
Set a `max_continuations` limit (e.g., 5) to prevent infinite loops. For the full guide, see: `https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons`
> **Security:** The tool runner executes your tool functions automatically whenever Claude requests them. For tools with side effects (sending emails, modifying databases, financial transactions), validate inputs within your tool functions and consider requiring confirmation for destructive operations. Use the manual agentic loop if you need human-in-the-loop approval before each tool execution.
---
### Handling Tool Results
When Claude uses a tool, the response contains a `tool_use` block. You must:
1. Execute the tool with the provided input
2. Send the result back in a `tool_result` message
3. Continue the conversation
**Error handling in tool results:** When a tool execution fails, set `"is_error": true` and provide an informative error message. Claude will typically acknowledge the error and either try a different approach or ask for clarification.
**Multiple tool calls:** Claude can request multiple tools in a single response. Handle them all before continuing — send all results back in a single `user` message.
---
## Server-Side Tools: Code Execution
The code execution tool lets Claude run code in a secure, sandboxed container. Unlike user-defined tools, server-side tools run on Anthropic's infrastructure — you don't execute anything client-side. Just include the tool definition and Claude handles the rest.
### Key Facts
- Runs in an isolated container (1 CPU, 5 GiB RAM, 5 GiB disk)
- No internet access (fully sandboxed)
- Python 3.11 with data science libraries pre-installed
- Containers persist for 30 days and can be reused across requests
- Free when used with web search/web fetch tools; otherwise $0.05/hour after 1,550 free hours/month per organization
### Tool Definition
The tool requires no schema — just declare it in the `tools` array:
```json
{
"type": "code_execution_20260120",
"name": "code_execution"
}
```
Claude automatically gains access to `bash_code_execution` (run shell commands) and `text_editor_code_execution` (create/view/edit files).
### Pre-installed Python Libraries
- **Data science**: pandas, numpy, scipy, scikit-learn, statsmodels
- **Visualization**: matplotlib, seaborn
- **File processing**: openpyxl, xlsxwriter, pillow, pypdf, pdfplumber, python-docx, python-pptx
- **Math**: sympy, mpmath
- **Utilities**: tqdm, python-dateutil, pytz, sqlite3
Additional packages can be installed at runtime via `pip install`.
### Supported File Types for Upload
| Type | Extensions |
| ------ | ---------------------------------- |
| Data | CSV, Excel (.xlsx/.xls), JSON, XML |
| Images | JPEG, PNG, GIF, WebP |
| Text | .txt, .md, .py, .js, etc. |
### Container Reuse
Reuse containers across requests to maintain state (files, installed packages, variables). Extract the `container_id` from the first response and pass it to subsequent requests.
### Response Structure
The response contains interleaved text and tool result blocks:
- `text` — Claude's explanation
- `server_tool_use` — What Claude is doing
- `bash_code_execution_tool_result` — Code execution output (check `return_code` for success/failure)
- `text_editor_code_execution_tool_result` — File operation results
> **Security:** Always sanitize filenames with `os.path.basename()` / `path.basename()` before writing downloaded files to disk to prevent path traversal attacks. Write files to a dedicated output directory.
---
## Server-Side Tools: Web Search and Web Fetch
Web search and web fetch let Claude search the web and retrieve page content. They run server-side — just include the tool definitions and Claude handles queries, fetching, and result processing automatically.
### Tool Definitions
```json
[
{ "type": "web_search_20260209", "name": "web_search" },
{ "type": "web_fetch_20260209", "name": "web_fetch" }
]
```
### Dynamic Filtering (Opus 4.7 / Opus 4.6 / Sonnet 4.6)
The `web_search_20260209` and `web_fetch_20260209` versions support **dynamic filtering** — Claude writes and executes code to filter search results before they reach the context window, improving accuracy and token efficiency. Dynamic filtering is built into these tool versions and activates automatically; you do not need to separately declare the `code_execution` tool or pass any beta header.
```json
{
"tools": [
{ "type": "web_search_20260209", "name": "web_search" },
{ "type": "web_fetch_20260209", "name": "web_fetch" }
]
}
```
Without dynamic filtering, the previous `web_search_20250305` version is also available.
> **Note:** Only include the standalone `code_execution` tool when your application needs code execution for its own purposes (data analysis, file processing, visualization) independent of web search. Including it alongside `_20260209` web tools creates a second execution environment that can confuse the model.
---
## Server-Side Tools: Programmatic Tool Calling
With standard tool use, each tool call is a round trip: Claude calls, the result enters Claude's context, Claude reasons, then calls the next tool. Chained calls accumulate latency and tokens — most of that intermediate data is never needed again.
Programmatic tool calling lets Claude compose those calls into a script. The script runs in the code execution container; when it invokes a tool, the container pauses, the call executes, and the result returns to the running code (not to Claude's context). The script processes it with normal control flow. Only the final output returns to Claude. Use it when chaining many tool calls or when intermediate results are large and should be filtered before reaching the context window.
For full documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling`
---
## Server-Side Tools: Tool Search
The tool search tool lets Claude dynamically discover tools from large libraries without loading all definitions into the context window. Use it when you have many tools but only a few are relevant to any given request. Discovered tool schemas are appended to the request, not swapped in — this preserves the prompt cache (see `agent-design.md` §Caching for Agents).
For full documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool`
---
## Skills
Skills package task-specific instructions that Claude loads only when relevant. Each skill is a folder containing a `SKILL.md` file. The skill's short description sits in context by default; Claude reads the full file when the current task calls for it. Use skills to keep specialized instructions out of the base system prompt without losing discoverability.
For full documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/agents-and-tools/skills`
---
## Tool Use Examples
You can provide sample tool calls directly in your tool definitions to demonstrate usage patterns and reduce parameter errors. This helps Claude understand how to correctly format tool inputs, especially for tools with complex schemas.
For full documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/agents-and-tools/tool-use/implement-tool-use`
---
## Server-Side Tools: Computer Use
Computer use lets Claude interact with a desktop environment (screenshots, mouse, keyboard). It can be Anthropic-hosted (server-side, like code execution) or self-hosted (you provide the environment and execute actions client-side).
For full documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/agents-and-tools/computer-use/overview`
---
## Context Editing
Context editing clears stale tool results and thinking blocks from the transcript as a long-running agent accumulates turns. Unlike compaction (which summarizes), context editing prunes — the cleared content is removed, not replaced. Use it when old tool outputs are no longer relevant and you want to keep the transcript lean without losing the conversation structure. Thresholds for what to clear are configurable.
For full documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/build-with-claude/context-editing`
---
## Client-Side Tools: Memory
The memory tool enables Claude to store and retrieve information across conversations through a memory file directory. Claude can create, read, update, and delete files that persist between sessions.
### Key Facts
- Client-side tool — you control storage via your implementation
- Supports commands: `view`, `create`, `str_replace`, `insert`, `delete`, `rename`
- Operates on files in a `/memories` directory
- The Python, TypeScript, and Java SDKs provide helper classes/functions for implementing the memory backend
> **Security:** Never store API keys, passwords, tokens, or other secrets in memory files. Be cautious with personally identifiable information (PII) — check data privacy regulations (GDPR, CCPA) before persisting user data. The reference implementations have no built-in access control; in multi-user systems, implement per-user memory directories and authentication in your tool handlers.
For full implementation examples, use WebFetch:
- Docs: `https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool.md`
---
## Structured Outputs
Structured outputs constrain Claude's responses to follow a specific JSON schema, guaranteeing valid, parseable output. This is not a separate tool — it enhances the Messages API response format and/or tool parameter validation.
Two features are available:
- **JSON outputs** (`output_config.format`): Control Claude's response format
- **Strict tool use** (`strict: true`): Guarantee valid tool parameter schemas
**Supported models:** Claude Opus 4.7, Claude Sonnet 4.6, and Claude Haiku 4.5. Legacy models (Claude Opus 4.5, Claude Opus 4.1) also support structured outputs.
> **Recommended:** Use `client.messages.parse()` which automatically validates responses against your schema. When using `messages.create()` directly, use `output_config: {format: {...}}`. The `output_format` convenience parameter is also accepted by some SDK methods (e.g., `.parse()`), but `output_config.format` is the canonical API-level parameter.
### JSON Schema Limitations
**Supported:**
- Basic types: object, array, string, integer, number, boolean, null
- `enum`, `const`, `anyOf`, `allOf`, `$ref`/`$def`
- String formats: `date-time`, `time`, `date`, `duration`, `email`, `hostname`, `uri`, `ipv4`, `ipv6`, `uuid`
- `additionalProperties: false` (required for all objects)
**Not supported:**
- Recursive schemas
- Numerical constraints (`minimum`, `maximum`, `multipleOf`)
- String constraints (`minLength`, `maxLength`)
- Complex array constraints
- `additionalProperties` set to anything other than `false`
The Python and TypeScript SDKs automatically handle unsupported constraints by removing them from the schema sent to the API and validating them client-side.
### Important Notes
- **First request latency**: New schemas incur a one-time compilation cost. Subsequent requests with the same schema use a 24-hour cache.
- **Refusals**: If Claude refuses for safety reasons (`stop_reason: "refusal"`), the output may not match your schema.
- **Token limits**: If `stop_reason: "max_tokens"`, output may be incomplete. Increase `max_tokens`.
- **Incompatible with**: Citations (returns 400 error), message prefilling.
- **Works with**: Batches API, streaming, token counting, extended thinking.
---
## Tips for Effective Tool Use
1. **Provide detailed descriptions**: Claude relies heavily on descriptions to understand when and how to use tools
2. **Use specific tool names**: `get_current_weather` is better than `weather`
3. **Validate inputs**: Always validate tool inputs before execution
4. **Handle errors gracefully**: Return informative error messages so Claude can adapt
5. **Limit tool count**: Too many tools can confuse the model — keep the set focused
6. **Test tool interactions**: Verify Claude uses tools correctly in various scenarios
For detailed tool use documentation, use WebFetch:
- URL: `https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview`
resources/bundled/skills/claude-api/typescript/claude-api/README.md
+333
View File
@@ -0,0 +1,333 @@
# Claude API — TypeScript
## Installation
```bash
npm install @anthropic-ai/sdk
```
## Client Initialization
```typescript
import Anthropic from "@anthropic-ai/sdk";
// Default (uses ANTHROPIC_API_KEY env var)
const client = new Anthropic();
// Explicit API key
const client = new Anthropic({ apiKey: "your-api-key" });
```
---
## Basic Message Request
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [{ role: "user", content: "What is the capital of France?" }],
});
// response.content is ContentBlock[] — a discriminated union. Narrow by .type
// before accessing .text (TypeScript will error on content[0].text without this).
for (const block of response.content) {
if (block.type === "text") {
console.log(block.text);
}
}
```
---
## System Prompts
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
system:
"You are a helpful coding assistant. Always provide examples in Python.",
messages: [{ role: "user", content: "How do I read a JSON file?" }],
});
```
---
## Vision (Images)
### URL
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: [
{
type: "image",
source: { type: "url", url: "https://example.com/image.png" },
},
{ type: "text", text: "Describe this image" },
],
},
],
});
```
### Base64
```typescript
import fs from "fs";
const imageData = fs.readFileSync("image.png").toString("base64");
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: [
{
type: "image",
source: { type: "base64", media_type: "image/png", data: imageData },
},
{ type: "text", text: "What's in this image?" },
],
},
],
});
```
---
## Prompt Caching
**Caching is a prefix match** — any byte change anywhere in the prefix invalidates everything after it. For placement patterns, architectural guidance (frozen system prompt, deterministic tool order, where to put volatile content), and the silent-invalidator audit checklist, read `shared/prompt-caching.md`.
### Automatic Caching (Recommended)
Use top-level `cache_control` to automatically cache the last cacheable block in the request:
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
cache_control: { type: "ephemeral" }, // auto-caches the last cacheable block
system: "You are an expert on this large document...",
messages: [{ role: "user", content: "Summarize the key points" }],
});
```
### Manual Cache Control
For fine-grained control, add `cache_control` to specific content blocks:
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
system: [
{
type: "text",
text: "You are an expert on this large document...",
cache_control: { type: "ephemeral" }, // default TTL is 5 minutes
},
],
messages: [{ role: "user", content: "Summarize the key points" }],
});
// With explicit TTL (time-to-live)
const response2 = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
system: [
{
type: "text",
text: "You are an expert on this large document...",
cache_control: { type: "ephemeral", ttl: "1h" }, // 1 hour TTL
},
],
messages: [{ role: "user", content: "Summarize the key points" }],
});
```
### Verifying Cache Hits
```typescript
console.log(response.usage.cache_creation_input_tokens); // tokens written to cache (~1.25x cost)
console.log(response.usage.cache_read_input_tokens); // tokens served from cache (~0.1x cost)
console.log(response.usage.input_tokens); // uncached tokens (full cost)
```
If `cache_read_input_tokens` is zero across repeated identical-prefix requests, a silent invalidator is at work — `Date.now()` or a UUID in the system prompt, non-deterministic key ordering, or a varying tool set. See `shared/prompt-caching.md` for the full audit table.
---
## Extended Thinking
> **Opus 4.7, Opus 4.6, and Sonnet 4.6:** Use adaptive thinking. `budget_tokens` is removed on Opus 4.7 (400 if sent); deprecated on Opus 4.6 and Sonnet 4.6.
> **Older models:** Use `thinking: {type: "enabled", budget_tokens: N}` (must be < `max_tokens`, min 1024).
```typescript
// Opus 4.7 / 4.6: adaptive thinking (recommended)
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
thinking: { type: "adaptive" },
output_config: { effort: "high" }, // low | medium | high | max
messages: [
{ role: "user", content: "Solve this math problem step by step..." },
],
});
for (const block of response.content) {
if (block.type === "thinking") {
console.log("Thinking:", block.thinking);
} else if (block.type === "text") {
console.log("Response:", block.text);
}
}
```
---
## Error Handling
Use the SDK's typed exception classes — never check error messages with string matching:
```typescript
import Anthropic from "@anthropic-ai/sdk";
try {
const response = await client.messages.create({...});
} catch (error) {
if (error instanceof Anthropic.BadRequestError) {
console.error("Bad request:", error.message);
} else if (error instanceof Anthropic.AuthenticationError) {
console.error("Invalid API key");
} else if (error instanceof Anthropic.RateLimitError) {
console.error("Rate limited - retry later");
} else if (error instanceof Anthropic.APIError) {
console.error(`API error ${error.status}:`, error.message);
}
}
```
All classes extend `Anthropic.APIError` with a typed `status` field. Check from most specific to least specific. See [shared/error-codes.md](../../shared/error-codes.md) for the full error code reference.
---
## Multi-Turn Conversations
The API is stateless — send the full conversation history each time. Use `Anthropic.MessageParam[]` to type the messages array:
```typescript
const messages: Anthropic.MessageParam[] = [
{ role: "user", content: "My name is Alice." },
{ role: "assistant", content: "Hello Alice! Nice to meet you." },
{ role: "user", content: "What's my name?" },
];
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: messages,
});
```
**Rules:**
- Consecutive same-role messages are allowed — the API combines them into a single turn
- First message must be `user`
- Use SDK types (`Anthropic.MessageParam`, `Anthropic.Message`, `Anthropic.Tool`, etc.) for all API data structures — don't redefine equivalent interfaces
---
### Compaction (long conversations)
> **Beta, Opus 4.7, Opus 4.6, and Sonnet 4.6.** When conversations approach the 200K context window, compaction automatically summarizes earlier context server-side. The API returns a `compaction` block; you must pass it back on subsequent requests — append `response.content`, not just the text.
```typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const messages: Anthropic.Beta.BetaMessageParam[] = [];
async function chat(userMessage: string): Promise<string> {
messages.push({ role: "user", content: userMessage });
const response = await client.beta.messages.create({
betas: ["compact-2026-01-12"],
model: "claude-opus-4-7",
max_tokens: 16000,
messages,
context_management: {
edits: [{ type: "compact_20260112" }],
},
});
// Append full content — compaction blocks must be preserved
messages.push({ role: "assistant", content: response.content });
const textBlock = response.content.find(
(b): b is Anthropic.Beta.BetaTextBlock => b.type === "text",
);
return textBlock?.text ?? "";
}
// Compaction triggers automatically when context grows large
console.log(await chat("Help me build a Python web scraper"));
console.log(await chat("Add support for JavaScript-rendered pages"));
console.log(await chat("Now add rate limiting and error handling"));
```
---
## Stop Reasons
The `stop_reason` field in the response indicates why the model stopped generating:
| Value | Meaning |
| --------------- | --------------------------------------------------------------- |
| `end_turn` | Claude finished its response naturally |
| `max_tokens` | Hit the `max_tokens` limit — increase it or use streaming |
| `stop_sequence` | Hit a custom stop sequence |
| `tool_use` | Claude wants to call a tool — execute it and continue |
| `pause_turn` | Model paused and can be resumed (agentic flows) |
| `refusal` | Claude refused for safety reasons — output may not match schema |
---
## Cost Optimization Strategies
### 1. Use Prompt Caching for Repeated Context
```typescript
// Automatic caching (simplest — caches the last cacheable block)
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
cache_control: { type: "ephemeral" },
system: largeDocumentText, // e.g., 50KB of context
messages: [{ role: "user", content: "Summarize the key points" }],
});
// First request: full cost
// Subsequent requests: ~90% cheaper for cached portion
```
### 2. Use Token Counting Before Requests
```typescript
const countResponse = await client.messages.countTokens({
model: "claude-opus-4-7",
messages: messages,
system: system,
});
const estimatedInputCost = countResponse.input_tokens * 0.000005; // $5/1M tokens
console.log(`Estimated input cost: $${estimatedInputCost.toFixed(4)}`);
```
resources/bundled/skills/claude-api/typescript/claude-api/batches.md
+106
View File
@@ -0,0 +1,106 @@
# Message Batches API — TypeScript
The Batches API (`POST /v1/messages/batches`) processes Messages API requests asynchronously at 50% of standard prices.
## Key Facts
- Up to 100,000 requests or 256 MB per batch
- Most batches complete within 1 hour; maximum 24 hours
- Results available for 29 days after creation
- 50% cost reduction on all token usage
- All Messages API features supported (vision, tools, caching, etc.)
---
## Create a Batch
```typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const messageBatch = await client.messages.batches.create({
requests: [
{
custom_id: "request-1",
params: {
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{ role: "user", content: "Summarize climate change impacts" },
],
},
},
{
custom_id: "request-2",
params: {
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{ role: "user", content: "Explain quantum computing basics" },
],
},
},
],
});
console.log(`Batch ID: ${messageBatch.id}`);
console.log(`Status: ${messageBatch.processing_status}`);
```
---
## Poll for Completion
```typescript
let batch;
while (true) {
batch = await client.messages.batches.retrieve(messageBatch.id);
if (batch.processing_status === "ended") break;
console.log(
`Status: ${batch.processing_status}, processing: ${batch.request_counts.processing}`,
);
await new Promise((resolve) => setTimeout(resolve, 60_000));
}
console.log("Batch complete!");
console.log(`Succeeded: ${batch.request_counts.succeeded}`);
console.log(`Errored: ${batch.request_counts.errored}`);
```
---
## Retrieve Results
```typescript
for await (const result of await client.messages.batches.results(
messageBatch.id,
)) {
switch (result.result.type) {
case "succeeded":
console.log(
`[${result.custom_id}] ${result.result.message.content[0].text.slice(0, 100)}`,
);
break;
case "errored":
if (result.result.error.type === "invalid_request") {
console.log(`[${result.custom_id}] Validation error - fix and retry`);
} else {
console.log(`[${result.custom_id}] Server error - safe to retry`);
}
break;
case "expired":
console.log(`[${result.custom_id}] Expired - resubmit`);
break;
}
}
```
---
## Cancel a Batch
```typescript
const cancelled = await client.messages.batches.cancel(messageBatch.id);
console.log(`Status: ${cancelled.processing_status}`); // "canceling"
```
resources/bundled/skills/claude-api/typescript/claude-api/files-api.md
+98
View File
@@ -0,0 +1,98 @@
# Files API — TypeScript
The Files API uploads files for use in Messages API requests. Reference files via `file_id` in content blocks, avoiding re-uploads across multiple API calls.
**Beta:** Pass `betas: ["files-api-2025-04-14"]` in your API calls (the SDK sets the required header automatically).
## Key Facts
- Maximum file size: 500 MB
- Total storage: 100 GB per organization
- Files persist until deleted
- File operations (upload, list, delete) are free; content used in messages is billed as input tokens
- Not available on Amazon Bedrock or Google Vertex AI
---
## Upload a File
```typescript
import Anthropic, { toFile } from "@anthropic-ai/sdk";
import fs from "fs";
const client = new Anthropic();
const uploaded = await client.beta.files.upload({
file: await toFile(fs.createReadStream("report.pdf"), undefined, {
type: "application/pdf",
}),
betas: ["files-api-2025-04-14"],
});
console.log(`File ID: ${uploaded.id}`);
console.log(`Size: ${uploaded.size_bytes} bytes`);
```
---
## Use a File in Messages
### PDF / Text Document
```typescript
const response = await client.beta.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: [
{ type: "text", text: "Summarize the key findings in this report." },
{
type: "document",
source: { type: "file", file_id: uploaded.id },
title: "Q4 Report",
citations: { enabled: true },
},
],
},
],
betas: ["files-api-2025-04-14"],
});
console.log(response.content[0].text);
```
---
## Manage Files
### List Files
```typescript
const files = await client.beta.files.list({
betas: ["files-api-2025-04-14"],
});
for (const f of files.data) {
console.log(`${f.id}: ${f.filename} (${f.size_bytes} bytes)`);
}
```
### Delete a File
```typescript
await client.beta.files.delete("file_011CNha8iCJcU1wXNR6q4V8w", {
betas: ["files-api-2025-04-14"],
});
```
### Download a File
```typescript
const response = await client.beta.files.download(
"file_011CNha8iCJcU1wXNR6q4V8w",
{ betas: ["files-api-2025-04-14"] },
);
const content = Buffer.from(await response.arrayBuffer());
await fs.promises.writeFile("output.txt", content);
```
resources/bundled/skills/claude-api/typescript/claude-api/streaming.md
+178
View File
@@ -0,0 +1,178 @@
# Streaming — TypeScript
## Quick Start
```typescript
const stream = client.messages.stream({
model: "claude-opus-4-7",
max_tokens: 64000,
messages: [{ role: "user", content: "Write a story" }],
});
for await (const event of stream) {
if (
event.type === "content_block_delta" &&
event.delta.type === "text_delta"
) {
process.stdout.write(event.delta.text);
}
}
```
---
## Handling Different Content Types
> **Opus 4.7 / Opus 4.6:** Use `thinking: {type: "adaptive"}`. On older models, use `thinking: {type: "enabled", budget_tokens: N}` instead.
```typescript
const stream = client.messages.stream({
model: "claude-opus-4-7",
max_tokens: 64000,
thinking: { type: "adaptive" },
messages: [{ role: "user", content: "Analyze this problem" }],
});
for await (const event of stream) {
switch (event.type) {
case "content_block_start":
switch (event.content_block.type) {
case "thinking":
console.log("\n[Thinking...]");
break;
case "text":
console.log("\n[Response:]");
break;
}
break;
case "content_block_delta":
switch (event.delta.type) {
case "thinking_delta":
process.stdout.write(event.delta.thinking);
break;
case "text_delta":
process.stdout.write(event.delta.text);
break;
}
break;
}
}
```
---
## Streaming with Tool Use (Tool Runner)
Use the tool runner with `stream: true`. The outer loop iterates over tool runner iterations (messages), the inner loop processes stream events:
```typescript
import Anthropic from "@anthropic-ai/sdk";
import { betaZodTool } from "@anthropic-ai/sdk/helpers/beta/zod";
import { z } from "zod";
const client = new Anthropic();
const getWeather = betaZodTool({
name: "get_weather",
description: "Get current weather for a location",
inputSchema: z.object({
location: z.string().describe("City and state, e.g., San Francisco, CA"),
}),
run: async ({ location }) => `72°F and sunny in ${location}`,
});
const runner = client.beta.messages.toolRunner({
model: "claude-opus-4-7",
max_tokens: 64000,
tools: [getWeather],
messages: [
{ role: "user", content: "What's the weather in Paris and London?" },
],
stream: true,
});
// Outer loop: each tool runner iteration
for await (const messageStream of runner) {
// Inner loop: stream events for this iteration
for await (const event of messageStream) {
switch (event.type) {
case "content_block_delta":
switch (event.delta.type) {
case "text_delta":
process.stdout.write(event.delta.text);
break;
case "input_json_delta":
// Tool input being streamed
break;
}
break;
}
}
}
```
---
## Getting the Final Message
```typescript
const stream = client.messages.stream({
model: "claude-opus-4-7",
max_tokens: 64000,
messages: [{ role: "user", content: "Hello" }],
});
for await (const event of stream) {
// Process events...
}
const finalMessage = await stream.finalMessage();
console.log(`Tokens used: ${finalMessage.usage.output_tokens}`);
```
---
## Stream Event Types
| Event Type | Description | When it fires |
| --------------------- | --------------------------- | --------------------------------- |
| `message_start` | Contains message metadata | Once at the beginning |
| `content_block_start` | New content block beginning | When a text/tool_use block starts |
| `content_block_delta` | Incremental content update | For each token/chunk |
| `content_block_stop` | Content block complete | When a block finishes |
| `message_delta` | Message-level updates | Contains `stop_reason`, usage |
| `message_stop` | Message complete | Once at the end |
## Best Practices
1. **Always flush output** — Use `process.stdout.write()` for immediate display
2. **Handle partial responses** — If the stream is interrupted, you may have incomplete content
3. **Track token usage** — The `message_delta` event contains usage information
4. **Use `finalMessage()`** — Get the complete `Anthropic.Message` object even when streaming. Don't wrap `.on()` events in `new Promise()` — `finalMessage()` handles all completion/error/abort states internally
5. **Buffer for web UIs** — Consider buffering a few tokens before rendering to avoid excessive DOM updates
6. **Use `stream.on("text", ...)` for deltas** — The `text` event provides just the delta string, simpler than manually filtering `content_block_delta` events
7. **For agentic loops with streaming** — See the [Streaming Manual Loop](./tool-use.md#streaming-manual-loop) section in tool-use.md for combining `stream()` + `finalMessage()` with a tool-use loop
## Raw SSE Format
If using raw HTTP (not SDKs), the stream returns Server-Sent Events:
```
event: message_start
data: {"type":"message_start","message":{"id":"msg_...","type":"message",...}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":12}}
event: message_stop
data: {"type":"message_stop"}
```
resources/bundled/skills/claude-api/typescript/claude-api/tool-use.md
+527
View File
@@ -0,0 +1,527 @@
# Tool Use — TypeScript
For conceptual overview (tool definitions, tool choice, tips), see [shared/tool-use-concepts.md](../../shared/tool-use-concepts.md).
## Tool Runner (Recommended)
**Beta:** The tool runner is in beta in the TypeScript SDK.
Use `betaZodTool` with Zod schemas to define tools with a `run` function, then pass them to `client.beta.messages.toolRunner()`:
```typescript
import Anthropic from "@anthropic-ai/sdk";
import { betaZodTool } from "@anthropic-ai/sdk/helpers/beta/zod";
import { z } from "zod";
const client = new Anthropic();
const getWeather = betaZodTool({
name: "get_weather",
description: "Get current weather for a location",
inputSchema: z.object({
location: z.string().describe("City and state, e.g., San Francisco, CA"),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
run: async (input) => {
// Your implementation here
return `72°F and sunny in ${input.location}`;
},
});
// The tool runner handles the agentic loop and returns the final message
const finalMessage = await client.beta.messages.toolRunner({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: [getWeather],
messages: [{ role: "user", content: "What's the weather in Paris?" }],
});
console.log(finalMessage.content);
```
**Key benefits of the tool runner:**
- No manual loop — the SDK handles calling tools and feeding results back
- Type-safe tool inputs via Zod schemas
- Tool schemas are generated automatically from Zod definitions
- Iteration stops automatically when Claude has no more tool calls
---
## Manual Agentic Loop
Use this when you need fine-grained control (custom logging, conditional tool execution, streaming individual iterations, human-in-the-loop approval):
```typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const tools: Anthropic.Tool[] = [...]; // Your tool definitions
let messages: Anthropic.MessageParam[] = [{ role: "user", content: userInput }];
while (true) {
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: tools,
messages: messages,
});
if (response.stop_reason === "end_turn") break;
// Server-side tool hit iteration limit; append assistant turn and re-send to continue
if (response.stop_reason === "pause_turn") {
messages.push({ role: "assistant", content: response.content });
continue;
}
const toolUseBlocks = response.content.filter(
(b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
);
messages.push({ role: "assistant", content: response.content });
const toolResults: Anthropic.ToolResultBlockParam[] = [];
for (const tool of toolUseBlocks) {
const result = await executeTool(tool.name, tool.input);
toolResults.push({
type: "tool_result",
tool_use_id: tool.id,
content: result,
});
}
messages.push({ role: "user", content: toolResults });
}
```
### Streaming Manual Loop
Use `client.messages.stream()` + `finalMessage()` instead of `.create()` when you need streaming within a manual loop. Text deltas are streamed on each iteration; `finalMessage()` collects the complete `Message` so you can inspect `stop_reason` and extract tool-use blocks:
```typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const tools: Anthropic.Tool[] = [...];
let messages: Anthropic.MessageParam[] = [{ role: "user", content: userInput }];
while (true) {
const stream = client.messages.stream({
model: "claude-opus-4-7",
max_tokens: 64000,
tools,
messages,
});
// Stream text deltas on each iteration
stream.on("text", (delta) => {
process.stdout.write(delta);
});
// finalMessage() resolves with the complete Message — no need to
// manually wire up .on("message") / .on("error") / .on("abort")
const message = await stream.finalMessage();
if (message.stop_reason === "end_turn") break;
// Server-side tool hit iteration limit; append assistant turn and re-send to continue
if (message.stop_reason === "pause_turn") {
messages.push({ role: "assistant", content: message.content });
continue;
}
const toolUseBlocks = message.content.filter(
(b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
);
messages.push({ role: "assistant", content: message.content });
const toolResults: Anthropic.ToolResultBlockParam[] = [];
for (const tool of toolUseBlocks) {
const result = await executeTool(tool.name, tool.input);
toolResults.push({
type: "tool_result",
tool_use_id: tool.id,
content: result,
});
}
messages.push({ role: "user", content: toolResults });
}
```
> **Important:** Don't wrap `.on()` events in `new Promise()` to collect the final message — use `stream.finalMessage()` instead. The SDK handles all error/abort/completion states internally.
> **Error handling in the loop:** Use the SDK's typed exceptions (e.g., `Anthropic.RateLimitError`, `Anthropic.APIError`) — see [Error Handling](./README.md#error-handling) for examples. Don't check error messages with string matching.
> **SDK types:** Use `Anthropic.MessageParam`, `Anthropic.Tool`, `Anthropic.ToolUseBlock`, `Anthropic.ToolResultBlockParam`, `Anthropic.Message`, etc. for all API-related data structures. Don't redefine equivalent interfaces.
---
## Handling Tool Results
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: tools,
messages: [{ role: "user", content: "What's the weather in Paris?" }],
});
for (const block of response.content) {
if (block.type === "tool_use") {
const result = await executeTool(block.name, block.input);
const followup = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: tools,
messages: [
{ role: "user", content: "What's the weather in Paris?" },
{ role: "assistant", content: response.content },
{
role: "user",
content: [
{ type: "tool_result", tool_use_id: block.id, content: result },
],
},
],
});
}
}
```
---
## Tool Choice
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: tools,
tool_choice: { type: "tool", name: "get_weather" },
messages: [{ role: "user", content: "What's the weather in Paris?" }],
});
```
---
## Server-Side Tools
Version-suffixed `type` literals; `name` is fixed per interface. Pass plain object literals — the `ToolUnion` type is satisfied structurally. **The `name`/`type` pair must match the interface**: mixing `str_replace_based_edit_tool` (20250728 name) with `text_editor_20250124` (which expects `str_replace_editor`) is a TS2322.
**Don't type-annotate as `Tool[]`** — `Tool` is just the custom-tool variant. Let structural typing infer from the `tools` param, or annotate as `Anthropic.Messages.ToolUnion[]` if you must:
```typescript
// ✓ let inference work — no annotation
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: [
{ type: "text_editor_20250728", name: "str_replace_based_edit_tool" },
{ type: "bash_20250124", name: "bash" },
{ type: "web_search_20260209", name: "web_search" },
{ type: "code_execution_20260120", name: "code_execution" },
],
messages: [{ role: "user", content: "..." }],
});
// ✗ this is a TS2352 — Tool is the CUSTOM tool variant only
// const tools: Anthropic.Tool[] = [{ type: "text_editor_20250728", ... }]
```
| Interface | `name` | `type` |
|---|---|---|
| `ToolTextEditor20250124` | `str_replace_editor` | `text_editor_20250124` |
| `ToolTextEditor20250429` | `str_replace_based_edit_tool` | `text_editor_20250429` |
| `ToolTextEditor20250728` | `str_replace_based_edit_tool` | `text_editor_20250728` |
| `ToolBash20250124` | `bash` | `bash_20250124` |
| `WebSearchTool20260209` | `web_search` | `web_search_20260209` |
| `WebFetchTool20260209` | `web_fetch` | `web_fetch_20260209` |
| `CodeExecutionTool20260120` | `code_execution` | `code_execution_20260120` |
**Don't mix beta and non-beta types**: if you call `client.beta.messages.create()`, the response `content` is `BetaContentBlock[]` — you cannot pass that to a non-beta `ContentBlockParam[]` without narrowing each element.
---
## Code Execution
### Basic Usage
```typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content:
"Calculate the mean and standard deviation of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]",
},
],
tools: [{ type: "code_execution_20260120", name: "code_execution" }],
});
```
### Reading Local Files (ESM note)
`__dirname` doesn't exist in ES modules. For script-relative paths use `import.meta.url`:
```typescript
import { readFileSync } from "fs";
import { fileURLToPath } from "url";
import { dirname, join } from "path";
const __dirname = dirname(fileURLToPath(import.meta.url));
const pdfBytes = readFileSync(join(__dirname, "sample.pdf"));
```
Or use a CWD-relative path if the script runs from a known directory: `readFileSync("./sample.pdf")`.
### Upload Files for Analysis
```typescript
import Anthropic, { toFile } from "@anthropic-ai/sdk";
import { createReadStream } from "fs";
const client = new Anthropic();
// 1. Upload a file
const uploaded = await client.beta.files.upload({
file: await toFile(createReadStream("sales_data.csv"), undefined, {
type: "text/csv",
}),
betas: ["files-api-2025-04-14"],
});
// 2. Pass to code execution
// Code execution is GA; Files API is still beta (pass via RequestOptions)
const response = await client.messages.create(
{
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: [
{
type: "text",
text: "Analyze this sales data. Show trends and create a visualization.",
},
{ type: "container_upload", file_id: uploaded.id },
],
},
],
tools: [{ type: "code_execution_20260120", name: "code_execution" }],
},
{ headers: { "anthropic-beta": "files-api-2025-04-14" } },
);
```
### Retrieve Generated Files
```typescript
import path from "path";
import fs from "fs";
const OUTPUT_DIR = "./claude_outputs";
await fs.promises.mkdir(OUTPUT_DIR, { recursive: true });
for (const block of response.content) {
if (block.type === "bash_code_execution_tool_result") {
const result = block.content;
if (result.type === "bash_code_execution_result" && result.content) {
for (const fileRef of result.content) {
if (fileRef.type === "bash_code_execution_output") {
const metadata = await client.beta.files.retrieveMetadata(
fileRef.file_id,
);
const downloadResponse = await client.beta.files.download(fileRef.file_id);
const fileBytes = Buffer.from(await downloadResponse.arrayBuffer());
const safeName = path.basename(metadata.filename);
if (!safeName || safeName === "." || safeName === "..") {
console.warn(`Skipping invalid filename: ${metadata.filename}`);
continue;
}
const outputPath = path.join(OUTPUT_DIR, safeName);
await fs.promises.writeFile(outputPath, fileBytes);
console.log(`Saved: ${outputPath}`);
}
}
}
}
}
```
### Container Reuse
```typescript
// First request: set up environment
const response1 = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: "Install tabulate and create data.json with sample user data",
},
],
tools: [{ type: "code_execution_20260120", name: "code_execution" }],
});
// Reuse container
// container is nullable — set only when using server-side code execution
const containerId = response1.container!.id;
const response2 = await client.messages.create({
container: containerId,
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: "Read data.json and display as a formatted table",
},
],
tools: [{ type: "code_execution_20260120", name: "code_execution" }],
});
```
---
## Memory Tool
### Basic Usage
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: "Remember that my preferred language is TypeScript.",
},
],
tools: [{ type: "memory_20250818", name: "memory" }],
});
```
### SDK Memory Helper
Use `betaMemoryTool` with a `MemoryToolHandlers` implementation:
```typescript
import {
betaMemoryTool,
type MemoryToolHandlers,
} from "@anthropic-ai/sdk/helpers/beta/memory";
const handlers: MemoryToolHandlers = {
async view(command) { ... },
async create(command) { ... },
async str_replace(command) { ... },
async insert(command) { ... },
async delete(command) { ... },
async rename(command) { ... },
};
const memory = betaMemoryTool(handlers);
const runner = client.beta.messages.toolRunner({
model: "claude-opus-4-7",
max_tokens: 16000,
tools: [memory],
messages: [{ role: "user", content: "Remember my preferences" }],
});
for await (const message of runner) {
console.log(message);
}
```
For full implementation examples, use WebFetch:
- `https://github.com/anthropics/anthropic-sdk-typescript/blob/main/examples/tools-helpers-memory.ts`
---
## Structured Outputs
### JSON Outputs (Zod — Recommended)
```typescript
import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
const ContactInfoSchema = z.object({
name: z.string(),
email: z.string(),
plan: z.string(),
interests: z.array(z.string()),
demo_requested: z.boolean(),
});
const client = new Anthropic();
const response = await client.messages.parse({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content:
"Extract: Jane Doe (jane@co.com) wants Enterprise, interested in API and SDKs, wants a demo.",
},
],
output_config: {
format: zodOutputFormat(ContactInfoSchema),
},
});
// parsed_output is null if parsing failed — assert or guard
console.log(response.parsed_output!.name); // "Jane Doe"
```
### Strict Tool Use
```typescript
const response = await client.messages.create({
model: "claude-opus-4-7",
max_tokens: 16000,
messages: [
{
role: "user",
content: "Book a flight to Tokyo for 2 passengers on March 15",
},
],
tools: [
{
name: "book_flight",
description: "Book a flight to a destination",
strict: true,
input_schema: {
type: "object",
properties: {
destination: { type: "string" },
date: { type: "string", format: "date" },
passengers: {
type: "integer",
enum: [1, 2, 3, 4, 5, 6, 7, 8],
},
},
required: ["destination", "date", "passengers"],
additionalProperties: false,
},
},
],
});
```
resources/bundled/skills/claude-api/typescript/managed-agents/README.md
+360
View File
@@ -0,0 +1,360 @@
# Managed Agents — TypeScript
> **Bindings not shown here:** This README covers the most common managed-agents flows for TypeScript. If you need a class, method, namespace, field, or behavior that isn't shown, WebFetch the TypeScript SDK repo **or the relevant docs page** from `shared/live-sources.md` rather than guess. Do not extrapolate from cURL shapes or another language's SDK.
> **Agents are persistent — create once, reference by ID.** Store the agent ID returned by `agents.create` and pass it to every subsequent `sessions.create`; do not call `agents.create` in the request path. The Anthropic CLI is one convenient way to create agents and environments from version-controlled YAML — its URL is in `shared/live-sources.md`. The examples below show in-code creation for completeness; in production the create call belongs in setup, not in the request path.
## Installation
```bash
npm install @anthropic-ai/sdk
```
## Client Initialization
```typescript
import Anthropic from "@anthropic-ai/sdk";
// Default (uses ANTHROPIC_API_KEY env var)
const client = new Anthropic();
// Explicit API key
const client = new Anthropic({ apiKey: "your-api-key" });
```
---
## Create an Environment
```typescript
const environment = await client.beta.environments.create(
{
name: "my-dev-env",
config: {
type: "cloud",
networking: { type: "unrestricted" },
},
},
);
console.log(environment.id); // env_...
```
---
## Create an Agent (required first step)
> ⚠️ **There is no inline agent config.** `model`/`system`/`tools` live on the agent object, not the session. Always start with `agents.create()` — the session only takes `agent: { type: "agent", id: agent.id }`.
### Minimal
```typescript
// 1. Create the agent (reusable, versioned)
const agent = await client.beta.agents.create(
{
name: "Coding Assistant",
model: "claude-opus-4-7",
tools: [{ type: "agent_toolset_20260401", default_config: { enabled: true } }],
},
);
// 2. Start a session
const session = await client.beta.sessions.create(
{
agent: { type: "agent", id: agent.id, version: agent.version },
environment_id: environment.id,
},
);
console.log(session.id, session.status);
```
### With system prompt and custom tools
```typescript
const agent = await client.beta.agents.create(
{
name: "Code Reviewer",
model: "claude-opus-4-7",
system: "You are a senior code reviewer.",
tools: [
{ type: "agent_toolset_20260401", default_config: { enabled: true } },
{
type: "custom",
name: "run_tests",
description: "Run the test suite",
input_schema: {
type: "object",
properties: {
test_path: { type: "string", description: "Path to test file" },
},
required: ["test_path"],
},
},
],
},
);
const session = await client.beta.sessions.create(
{
agent: { type: "agent", id: agent.id, version: agent.version },
environment_id: environment.id,
title: "Code review session",
resources: [
{
type: "github_repository",
url: "https://github.com/owner/repo",
mount_path: "/workspace/repo",
authorization_token: process.env.GITHUB_TOKEN,
branch: "main",
},
],
},
);
```
---
## Send a User Message
```typescript
await client.beta.sessions.events.send(
session.id,
{
events: [
{
type: "user.message",
content: [{ type: "text", text: "Review the auth module" }],
},
],
},
);
```
> 💡 **Stream-first:** Open the stream *before* (or concurrently with) sending the message. The stream only delivers events that occur after it opens — stream-after-send means early events arrive buffered in one batch. See [Steering Patterns](../../shared/managed-agents-events.md#steering-patterns).
---
## Stream Events (SSE)
```typescript
// Stream-first: open stream and send concurrently
const [events] = await Promise.all([
collectStream(session.id),
client.beta.sessions.events.send(
session.id,
{ events: [{ type: "user.message", content: [{ type: "text", text: "..." }] }] },
),
]);
// Standalone stream iteration:
const stream = await client.beta.sessions.stream(
session.id,
);
for await (const event of stream) {
switch (event.type) {
case "agent.message":
for (const block of event.content) {
if (block.type === "text") {
process.stdout.write(block.text);
}
}
break;
case "agent.custom_tool_use":
// Custom tool invocation — session is now idle
console.log(`\nCustom tool call: ${event.tool_name}`);
console.log(`Input: ${JSON.stringify(event.input)}`);
break;
case "session.status_idle":
console.log("\n--- Agent idle ---");
break;
case "session.status_terminated":
console.log("\n--- Session terminated ---");
break;
}
}
```
---
## Provide Custom Tool Result
```typescript
await client.beta.sessions.events.send(
session.id,
{
events: [
{
type: "user.custom_tool_result",
custom_tool_use_id: "sevt_abc123",
content: [{ type: "text", text: "All 42 tests passed." }],
},
],
},
);
```
---
## Poll Events
```typescript
const events = await client.beta.sessions.events.list(
session.id,
);
for (const event of events.data) {
console.log(`${event.type}: ${event.id}`);
}
```
---
## Full Streaming Loop with Custom Tools
```typescript
function runCustomTool(toolName: string, toolInput: unknown): string {
if (toolName === "run_tests") {
// Your tool implementation here
return "All tests passed.";
}
return `Unknown tool: ${toolName}`;
}
async function runSession(client: Anthropic, sessionId: string) {
while (true) {
const stream = await client.beta.sessions.stream(
sessionId,
);
const toolCalls: Array<{ custom_tool_use_id: string; tool_name: string; input: unknown }> = [];
for await (const event of stream) {
if (event.type === "agent.message") {
for (const block of event.content) {
if (block.type === "text") {
process.stdout.write(block.text);
}
}
} else if (event.type === "agent.custom_tool_use") {
toolCalls.push({
id: event.id,
tool_name: event.tool_name,
input: event.input,
});
} else if (event.type === "session.status_idle") {
break;
} else if (event.type === "session.status_terminated") {
return;
}
}
if (toolCalls.length === 0) break;
// Process custom tool calls
const results = toolCalls.map((call) => ({
type: "user.custom_tool_result" as const,
custom_tool_use_id: call.id,
content: [{ type: "text" as const, text: runCustomTool(call.tool_name, call.input) }],
}));
await client.beta.sessions.events.send(
sessionId,
{ events: results },
);
}
}
```
---
## Upload a File
```typescript
import fs from "fs";
const file = await client.beta.files.upload({
file: fs.createReadStream("data.csv"),
purpose: "agent",
});
// Use in a session
const session = await client.beta.sessions.create(
{
agent: { type: "agent", id: agent.id, version: agent.version },
environment_id: environment.id,
resources: [{ type: "file", file_id: file.id, mount_path: "/workspace/data.csv" }],
},
);
```
---
## List and Download Session Files
List files the agent wrote to `/mnt/session/outputs/` during a session, then download them.
```typescript
import fs from "fs";
// List files associated with a session
const files = await client.beta.files.list({
scope_id: session.id,
betas: ["managed-agents-2026-04-01"],
});
for (const f of files.data) {
console.log(f.filename, f.size_bytes);
// Download and save to disk
const resp = await client.beta.files.download(f.id);
const buffer = Buffer.from(await resp.arrayBuffer());
fs.writeFileSync(f.filename, buffer);
}
```
> 💡 There's a brief indexing lag (~1–3s) between `session.status_idle` and output files appearing in `files.list`. Retry once or twice if the list is empty.
---
## Session Management
```typescript
// Get session details
const session = await client.beta.sessions.retrieve("sesn_011CZxAbc123Def456");
console.log(session.status, session.usage);
// List sessions
const sessions = await client.beta.sessions.list();
// Delete a session
await client.beta.sessions.delete("sesn_011CZxAbc123Def456");
// Archive a session
await client.beta.sessions.archive("sesn_011CZxAbc123Def456");
```
---
## MCP Server Integration
```typescript
// Agent declares MCP server (no auth here — auth goes in a vault)
const agent = await client.beta.agents.create({
name: "MCP Agent",
model: "claude-opus-4-7",
mcp_servers: [
{ type: "url", name: "my-tools", url: "https://my-mcp-server.example.com/sse" },
],
tools: [
{ type: "agent_toolset_20260401", default_config: { enabled: true } },
{ type: "mcp_toolset", mcp_server_name: "my-tools" },
],
});
// Session attaches vault(s) containing credentials for those MCP server URLs
const session = await client.beta.sessions.create({
agent: agent.id,
environment_id: environment.id,
vault_ids: [vault.id],
});
```
See `shared/managed-agents-tools.md` §Vaults for creating vaults and adding credentials.
resources/bundled/skills/create-skill/LICENSE.txt
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
resources/bundled/skills/create-skill/SKILL.md
+456
View File
@@ -0,0 +1,456 @@
---
name: create-skill
description: Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
license: Complete terms in LICENSE.txt
---
# Skill Creator
A skill for creating new skills and iteratively improving them.
At a high level, the process of creating a skill goes like this:
- Decide what you want the skill to do and roughly how it should do it
- Write a draft of the skill
- Create a few test prompts and run the agent with access to the skill on them
- Help the user evaluate the results both qualitatively and quantitatively
- While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist)
- Use the `eval-viewer/generate_review.py` script to show the user the results for them to look at, and also let them look at the quantitative metrics
- Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)
- Repeat until you're satisfied
- Expand the test set and try again at larger scale
Your job when using this skill is to figure out where the user is in this process and then jump in and help them progress through these stages. So for instance, maybe they're like "I want to make a skill for X". You can help narrow down what they mean, write a draft, write the test cases, figure out how they want to evaluate, run all the prompts, and repeat.
On the other hand, maybe they already have a draft of the skill. In this case you can go straight to the eval/iterate part of the loop.
Of course, you should always be flexible and if the user is like "I don't need to run a bunch of evaluations, just vibe with me", you can do that instead.
Then after the skill is done (but again, the order is flexible), you can also run the skill description improver, which we have a whole separate script for, to optimize the triggering of the skill.
Cool? Cool.
## Communicating with the user
The skill creator is liable to be used by people across a wide range of familiarity with coding jargon. There's a trend now where the power of AI agents is inspiring non-developers to open up their terminals. On the other hand, the bulk of users are probably fairly computer-literate.
So please pay attention to context cues to understand how to phrase your communication! In the default case, just to give you some idea:
- "evaluation" and "benchmark" are borderline, but OK
- for "JSON" and "assertion" you want to see serious cues from the user that they know what those things are before using them without explaining them
It's OK to briefly explain terms if you're in doubt, and feel free to clarify terms with a short definition if you're unsure if the user will get it.
---
## Creating a skill
### Capture Intent
Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the conversation history first — the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill the gaps, and should confirm before proceeding to the next step.
1. What should this skill enable the agent to do?
2. When should this skill trigger? (what user phrases/contexts)
3. What's the expected output format?
4. Should we set up test cases to verify the skill works? Skills with objectively verifiable outputs (file transforms, data extraction, code generation, fixed workflow steps) benefit from test cases. Skills with subjective outputs (writing style, art) often don't need them. Suggest the appropriate default based on the skill type, but let the user decide.
### Interview and Research
Proactively ask questions about edge cases, input/output formats, example files, success criteria, and dependencies. Wait to write test prompts until you've got this part ironed out.
Check available MCPs - if useful for research (searching docs, finding similar skills, looking up best practices), research in parallel via subagents if available, otherwise inline. Come prepared with context to reduce burden on the user.
### Write the SKILL.md
Based on the user interview, fill in these components:
- **name**: Skill identifier
- **description**: When to trigger, what it does. This is the primary triggering mechanism - include both what the skill does AND specific contexts for when to use it. All "when to use" info goes here, not in the body. Note: currently agents have a tendency to "undertrigger" skills -- to not use them when they'd be useful. To combat this, please make the skill descriptions a little bit "pushy". So for instance, instead of "How to build a simple fast dashboard to display internal data.", you might write "How to build a simple fast dashboard to display internal data. Make sure to use this skill whenever the user mentions dashboards, data visualization, internal metrics, or wants to display any kind of company data, even if they don't explicitly ask for a 'dashboard.'"
- **compatibility**: Required tools, dependencies (optional, rarely needed)
- **the rest of the skill :)**
### Skill Writing Guide
#### Anatomy of a Skill
```
skill-name/
├── SKILL.md (required)
│ ├── YAML frontmatter (name, description required)
│ └── Markdown instructions
└── Bundled Resources (optional)
├── scripts/ - Executable code for deterministic/repetitive tasks
├── references/ - Docs loaded into context as needed
└── assets/ - Files used in output (templates, icons, fonts)
```
#### Progressive Disclosure
Skills use a three-level loading system:
1. **Metadata** (name + description) - Always in context (~100 words)
2. **SKILL.md body** - In context whenever skill triggers (<500 lines ideal)
3. **Bundled resources** - As needed (unlimited, scripts can execute without loading)
These word counts are approximate and you can feel free to go longer if needed.
**Key patterns:**
- Keep SKILL.md under 500 lines; if you're approaching this limit, add an additional layer of hierarchy along with clear pointers about where the model using the skill should go next to follow up.
- Reference files clearly from SKILL.md with guidance on when to read them
- For large reference files (>300 lines), include a table of contents
**Domain organization**: When a skill supports multiple domains/frameworks, organize by variant:
```
cloud-deploy/
├── SKILL.md (workflow + selection)
└── references/
├── aws.md
├── gcp.md
└── azure.md
```
The agent reads only the relevant reference file.
#### Principle of Lack of Surprise
This goes without saying, but skills must not contain malware, exploit code, or any content that could compromise system security. A skill's contents should not surprise the user in their intent if described. Don't go along with requests to create misleading skills or skills designed to facilitate unauthorized access, data exfiltration, or other malicious activities. Things like a "roleplay as an XYZ" are OK though.
#### Writing Patterns
Prefer using the imperative form in instructions.
**Defining output formats** - You can do it like this:
```markdown
## Report structure
ALWAYS use this exact template:
# [Title]
## Executive summary
## Key findings
## Recommendations
```
**Examples pattern** - It's useful to include examples. You can format them like this (but if "Input" and "Output" are in the examples you might want to deviate a little):
```markdown
## Commit message format
**Example 1:**
Input: Added user authentication with JWT tokens
Output: feat(auth): implement JWT-based authentication
```
### Writing Style
Try to explain to the model why things are important in lieu of heavy-handed musty MUSTs. Use theory of mind and try to make the skill general and not super-narrow to specific examples. Start by writing a draft and then look at it with fresh eyes and improve it.
### Test Cases
After writing the skill draft, come up with 2-3 realistic test prompts — the kind of thing a real user would actually say. Share them with the user: [you don't have to use this exact language] "Here are a few test cases I'd like to try. Do these look right, or do you want to add more?" Then run them.
Save test cases to `evals/evals.json`. Don't write assertions yet — just the prompts. You'll draft assertions in the next step while the runs are in progress.
```json
{
"skill_name": "example-skill",
"evals": [
{
"id": 1,
"prompt": "User's task prompt",
"expected_output": "Description of expected result",
"files": []
}
]
}
```
See `references/schemas.md` for the full schema (including the `assertions` field, which you'll add later).
## Running and evaluating test cases
This section is one continuous sequence — don't stop partway through. Do NOT use `/skill-test` or any other testing skill.
Put results in `<skill-name>-workspace/` as a sibling to the skill directory. Within the workspace, organize results by iteration (`iteration-1/`, `iteration-2/`, etc.) and within that, each test case gets a directory (`eval-0/`, `eval-1/`, etc.). Don't create all of this upfront — just create directories as you go.
### Step 1: Spawn all runs (with-skill AND baseline) in the same turn
For each test case, spawn two subagents in the same turn — one with the skill, one without. This is important: don't spawn the with-skill runs first and then come back for baselines later. Launch everything at once so it all finishes around the same time.
**With-skill run:**
```
Execute this task:
- Skill path: <path-to-skill>
- Task: <eval prompt>
- Input files: <eval files if any, or "none">
- Save outputs to: <workspace>/iteration-<N>/eval-<ID>/with_skill/outputs/
- Outputs to save: <what the user cares about — e.g., "the .docx file", "the final CSV">
```
**Baseline run** (same prompt, but the baseline depends on context):
- **Creating a new skill**: no skill at all. Same prompt, no skill path, save to `without_skill/outputs/`.
- **Improving an existing skill**: the old version. Before editing, snapshot the skill (`cp -r <skill-path> <workspace>/skill-snapshot/`), then point the baseline subagent at the snapshot. Save to `old_skill/outputs/`.
Write an `eval_metadata.json` for each test case (assertions can be empty for now). Give each eval a descriptive name based on what it's testing — not just "eval-0". Use this name for the directory too. If this iteration uses new or modified eval prompts, create these files for each new eval directory — don't assume they carry over from previous iterations.
```json
{
"eval_id": 0,
"eval_name": "descriptive-name-here",
"prompt": "The user's task prompt",
"assertions": []
}
```
### Step 2: While runs are in progress, draft assertions
Don't just wait for the runs to finish — you can use this time productively. Draft quantitative assertions for each test case and explain them to the user. If assertions already exist in `evals/evals.json`, review them and explain what they check.
Good assertions are objectively verifiable and have descriptive names — they should read clearly in the benchmark viewer so someone glancing at the results immediately understands what each one checks. Subjective skills (writing style, design quality) are better evaluated qualitatively — don't force assertions onto things that need human judgment.
Update the `eval_metadata.json` files and `evals/evals.json` with the assertions once drafted. Also explain to the user what they'll see in the viewer — both the qualitative outputs and the quantitative benchmark.
### Step 3: As runs complete, capture timing data
When each subagent task completes, you receive a notification containing `total_tokens` and `duration_ms`. Save this data immediately to `timing.json` in the run directory:
```json
{
"total_tokens": 84852,
"duration_ms": 23332,
"total_duration_seconds": 23.3
}
```
This is the only opportunity to capture this data — it comes through the task notification and isn't persisted elsewhere. Process each notification as it arrives rather than trying to batch them.
### Step 4: Grade, aggregate, and launch the viewer
Once all runs are done:
1. **Grade each run** — spawn a grader subagent (or grade inline) that reads `agents/grader.md` and evaluates each assertion against the outputs. Save results to `grading.json` in each run directory. The grading.json expectations array must use the fields `text`, `passed`, and `evidence` (not `name`/`met`/`details` or other variants) — the viewer depends on these exact field names. For assertions that can be checked programmatically, write and run a script rather than eyeballing it — scripts are faster, more reliable, and can be reused across iterations.
2. **Aggregate into benchmark** — run the aggregation script from the skill-creator directory:
```bash
python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <name>
```
This produces `benchmark.json` and `benchmark.md` with pass_rate, time, and tokens for each configuration, with mean ± stddev and the delta. If generating benchmark.json manually, see `references/schemas.md` for the exact schema the viewer expects.
Put each with_skill version before its baseline counterpart.
3. **Do an analyst pass** — read the benchmark data and surface patterns the aggregate stats might hide. See `agents/analyzer.md` (the "Analyzing Benchmark Results" section) for what to look for — things like assertions that always pass regardless of skill (non-discriminating), high-variance evals (possibly flaky), and time/token tradeoffs.
4. **Launch the viewer** with both qualitative outputs and quantitative data:
```bash
nohup python <skill-creator-path>/eval-viewer/generate_review.py \
<workspace>/iteration-N \
--skill-name "my-skill" \
--benchmark <workspace>/iteration-N/benchmark.json \
> /dev/null 2>&1 &
VIEWER_PID=$!
```
For iteration 2+, also pass `--previous-workspace <workspace>/iteration-<N-1>`.
**Headless environments:** If `webbrowser.open()` is not available or the environment has no display, use `--static <output_path>` to write a standalone HTML file instead of starting a server. Feedback will be downloaded as a `feedback.json` file when the user clicks "Submit All Reviews". After download, copy `feedback.json` into the workspace directory for the next iteration to pick up.
Note: please use generate_review.py to create the viewer; there's no need to write custom HTML.
5. **Tell the user** something like: "I've opened the results in your browser. There are two tabs — 'Outputs' lets you click through each test case and leave feedback, 'Benchmark' shows the quantitative comparison. When you're done, come back here and let me know."
### What the user sees in the viewer
The "Outputs" tab shows one test case at a time:
- **Prompt**: the task that was given
- **Output**: the files the skill produced, rendered inline where possible
- **Previous Output** (iteration 2+): collapsed section showing last iteration's output
- **Formal Grades** (if grading was run): collapsed section showing assertion pass/fail
- **Feedback**: a textbox that auto-saves as they type
- **Previous Feedback** (iteration 2+): their comments from last time, shown below the textbox
The "Benchmark" tab shows the stats summary: pass rates, timing, and token usage for each configuration, with per-eval breakdowns and analyst observations.
Navigation is via prev/next buttons or arrow keys. When done, they click "Submit All Reviews" which saves all feedback to `feedback.json`.
### Step 5: Read the feedback
When the user tells you they're done, read `feedback.json`:
```json
{
"reviews": [
{"run_id": "eval-0-with_skill", "feedback": "the chart is missing axis labels", "timestamp": "..."},
{"run_id": "eval-1-with_skill", "feedback": "", "timestamp": "..."},
{"run_id": "eval-2-with_skill", "feedback": "perfect, love this", "timestamp": "..."}
],
"status": "complete"
}
```
Empty feedback means the user thought it was fine. Focus your improvements on the test cases where the user had specific complaints.
Kill the viewer server when you're done with it:
```bash
kill $VIEWER_PID 2>/dev/null
```
---
## Improving the skill
This is the heart of the loop. You've run the test cases, the user has reviewed the results, and now you need to make the skill better based on their feedback.
### How to think about improvements
1. **Generalize from the feedback.** The big picture thing that's happening here is that we're trying to create skills that can be used a million times (maybe literally, maybe even more who knows) across many different prompts. Here you and the user are iterating on only a few examples over and over again because it helps move faster. The user knows these examples in and out and it's quick for them to assess new outputs. But if the skill you and the user are codeveloping works only for those examples, it's useless. Rather than put in fiddly overfitty changes, or oppressively constrictive MUSTs, if there's some stubborn issue, you might try branching out and using different metaphors, or recommending different patterns of working. It's relatively cheap to try and maybe you'll land on something great.
2. **Keep the prompt lean.** Remove things that aren't pulling their weight. Make sure to read the transcripts, not just the final outputs — if it looks like the skill is making the model waste a bunch of time doing things that are unproductive, you can try getting rid of the parts of the skill that are making it do that and seeing what happens.
3. **Explain the why.** Try hard to explain the **why** behind everything you're asking the model to do. Today's LLMs are *smart*. They have good theory of mind and when given a good harness can go beyond rote instructions and really make things happen. Even if the feedback from the user is terse or frustrated, try to actually understand the task and why the user is writing what they wrote, and what they actually wrote, and then transmit this understanding into the instructions. If you find yourself writing ALWAYS or NEVER in all caps, or using super rigid structures, that's a yellow flag — if possible, reframe and explain the reasoning so that the model understands why the thing you're asking for is important. That's a more humane, powerful, and effective approach.
4. **Look for repeated work across test cases.** Read the transcripts from the test runs and notice if the subagents all independently wrote similar helper scripts or took the same multi-step approach to something. If all 3 test cases resulted in the subagent writing a `create_docx.py` or a `build_chart.py`, that's a strong signal the skill should bundle that script. Write it once, put it in `scripts/`, and tell the skill to use it. This saves every future invocation from reinventing the wheel.
This task is pretty important and your thinking time is not the blocker; take your time and really mull things over. I'd suggest writing a draft revision and then looking at it anew and making improvements. Really do your best to get into the head of the user and understand what they want and need.
### The iteration loop
After improving the skill:
1. Apply your improvements to the skill
2. Rerun all test cases into a new `iteration-<N+1>/` directory, including baseline runs. If you're creating a new skill, the baseline is always `without_skill` (no skill) — that stays the same across iterations. If you're improving an existing skill, use your judgment on what makes sense as the baseline: the original version the user came in with, or the previous iteration.
3. Launch the reviewer with `--previous-workspace` pointing at the previous iteration
4. Wait for the user to review and tell you they're done
5. Read the new feedback, improve again, repeat
Keep going until:
- The user says they're happy
- The feedback is all empty (everything looks good)
- You're not making meaningful progress
---
## Advanced: Blind comparison
For situations where you want a more rigorous comparison between two versions of a skill (e.g., the user asks "is the new version actually better?"), there's a blind comparison system. Read `agents/comparator.md` and `agents/analyzer.md` for the details. The basic idea is: give two outputs to an independent agent without telling it which is which, and let it judge quality. Then analyze why the winner won.
This is optional, requires subagents, and most users won't need it. The human review loop is usually sufficient.
---
## Description Optimization
The description field in SKILL.md frontmatter is the primary mechanism that determines whether the agent invokes a skill. After creating or improving a skill, offer to optimize the description for better triggering accuracy.
### Step 1: Generate trigger eval queries
Create 20 eval queries — a mix of should-trigger and should-not-trigger. Save as JSON:
```json
[
{"query": "the user prompt", "should_trigger": true},
{"query": "another prompt", "should_trigger": false}
]
```
The queries must be realistic and something a user would actually type. Not abstract requests, but requests that are concrete and specific and have a good amount of detail. For instance, file paths, personal context about the user's job or situation, column names and values, company names, URLs. A little bit of backstory. Some might be in lowercase or contain abbreviations or typos or casual speech. Use a mix of different lengths, and focus on edge cases rather than making them clear-cut (the user will get a chance to sign off on them).
Bad: `"Format this data"`, `"Extract text from PDF"`, `"Create a chart"`
Good: `"ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column that shows the profit margin as a percentage. The revenue is in column C and costs are in column D i think"`
For the **should-trigger** queries (8-10), think about coverage. You want different phrasings of the same intent — some formal, some casual. Include cases where the user doesn't explicitly name the skill or file type but clearly needs it. Throw in some uncommon use cases and cases where this skill competes with another but should win.
For the **should-not-trigger** queries (8-10), the most valuable ones are the near-misses — queries that share keywords or concepts with the skill but actually need something different. Think adjacent domains, ambiguous phrasing where a naive keyword match would trigger but shouldn't, and cases where the query touches on something the skill does but in a context where another tool is more appropriate.
The key thing to avoid: don't make should-not-trigger queries obviously irrelevant. "Write a fibonacci function" as a negative test for a PDF skill is too easy — it doesn't test anything. The negative cases should be genuinely tricky.
### Step 2: Review with user
Present the eval set to the user for review using the HTML template:
1. Read the template from `assets/eval_review.html`
2. Replace the placeholders:
- `__EVAL_DATA_PLACEHOLDER__` → the JSON array of eval items (no quotes around it — it's a JS variable assignment)
- `__SKILL_NAME_PLACEHOLDER__` → the skill's name
- `__SKILL_DESCRIPTION_PLACEHOLDER__` → the skill's current description
3. Write to a temp file (e.g., `/tmp/eval_review_<skill-name>.html`) and open it: `open /tmp/eval_review_<skill-name>.html`
4. The user can edit queries, toggle should-trigger, add/remove entries, then click "Export Eval Set"
5. The file downloads to `~/Downloads/eval_set.json` — check the Downloads folder for the most recent version in case there are multiple (e.g., `eval_set (1).json`)
This step matters — bad eval queries lead to bad descriptions.
### Step 3: Run the optimization loop
Tell the user: "This will take some time — I'll run the optimization loop in the background and check on it periodically."
Save the eval set to the workspace, then run in the background:
```bash
python -m scripts.run_loop \
--eval-set <path-to-trigger-eval.json> \
--skill-path <path-to-skill> \
--model <model-id-powering-this-session> \
--max-iterations 5 \
--verbose
```
Use the model ID from your system prompt (the one powering the current session) so the triggering test matches what the user actually experiences.
While it runs, periodically tail the output to give the user updates on which iteration it's on and what the scores look like.
This handles the full optimization loop automatically. It splits the eval set into 60% train and 40% held-out test, evaluates the current description (running each query 3 times to get a reliable trigger rate), then calls the model to propose improvements based on what failed. It re-evaluates each new description on both train and test, iterating up to 5 times. When it's done, it opens an HTML report in the browser showing the results per iteration and returns JSON with `best_description` — selected by test score rather than train score to avoid overfitting.
### How skill triggering works
Understanding the triggering mechanism helps design better eval queries. Skills appear in the agent's `available_skills` list with their name + description, and the agent decides whether to consult a skill based on that description. The important thing to know is that the agent only consults skills for tasks it can't easily handle on its own — simple, one-step queries like "read this PDF" may not trigger a skill even if the description matches perfectly, because the agent can handle them directly with basic tools. Complex, multi-step, or specialized queries reliably trigger skills when the description matches.
This means your eval queries should be substantive enough that the agent would actually benefit from consulting a skill. Simple queries like "read file X" are poor test cases — they won't trigger skills regardless of description quality.
### Step 4: Apply the result
Take `best_description` from the JSON output and update the skill's SKILL.md frontmatter. Show the user before/after and report the scores.
---
### Package and Present (only if `present_files` tool is available)
Check whether you have access to the `present_files` tool. If you don't, skip this step. If you do, package the skill and present the .skill file to the user:
```bash
python -m scripts.package_skill <path/to/skill-folder>
```
After packaging, direct the user to the resulting `.skill` file path so they can install it.
---
### Updating an existing skill
The user might be asking you to update an existing skill, not create a new one. In this case:
- **Preserve the original name.** Note the skill's directory name and `name` frontmatter field -- use them unchanged. E.g., if the installed skill is `research-helper`, output `research-helper.skill` (not `research-helper-v2`).
- **Copy to a writeable location before editing.** The installed skill path may be read-only. Copy to `/tmp/skill-name/`, edit there, and package from the copy.
- **If packaging manually, stage in `/tmp/` first**, then copy to the output directory -- direct writes may fail due to permissions.
---
## Reference files
The agents/ directory contains instructions for specialized subagents. Read them when you need to spawn the relevant subagent.
- `agents/grader.md` — How to evaluate assertions against outputs
- `agents/comparator.md` — How to do blind A/B comparison between two outputs
- `agents/analyzer.md` — How to analyze why one version beat another
The references/ directory has additional documentation:
- `references/schemas.md` — JSON structures for evals.json, grading.json, etc.
---
Repeating one more time the core loop here for emphasis:
- Figure out what the skill is about
- Draft or edit the skill
- Run the agent with access to the skill on test prompts
- With the user, evaluate the outputs:
- Create benchmark.json and run `eval-viewer/generate_review.py` to help the user review them
- Run quantitative evals
- Repeat until you and the user are satisfied
- Package the final skill and return it to the user.
Please add steps to your TodoList, if you have such a thing, to make sure you don't forget. Specifically put "Create evals JSON and run `eval-viewer/generate_review.py` so human can review test cases" in your TodoList to make sure it happens.
Good luck!
resources/bundled/skills/create-skill/agents/analyzer.md
+274
View File
@@ -0,0 +1,274 @@
# Post-hoc Analyzer Agent
Analyze blind comparison results to understand WHY the winner won and generate improvement suggestions.
## Role
After the blind comparator determines a winner, the Post-hoc Analyzer "unblids" the results by examining the skills and transcripts. The goal is to extract actionable insights: what made the winner better, and how can the loser be improved?
## Inputs
You receive these parameters in your prompt:
- **winner**: "A" or "B" (from blind comparison)
- **winner_skill_path**: Path to the skill that produced the winning output
- **winner_transcript_path**: Path to the execution transcript for the winner
- **loser_skill_path**: Path to the skill that produced the losing output
- **loser_transcript_path**: Path to the execution transcript for the loser
- **comparison_result_path**: Path to the blind comparator's output JSON
- **output_path**: Where to save the analysis results
## Process
### Step 1: Read Comparison Result
1. Read the blind comparator's output at comparison_result_path
2. Note the winning side (A or B), the reasoning, and any scores
3. Understand what the comparator valued in the winning output
### Step 2: Read Both Skills
1. Read the winner skill's SKILL.md and key referenced files
2. Read the loser skill's SKILL.md and key referenced files
3. Identify structural differences:
- Instructions clarity and specificity
- Script/tool usage patterns
- Example coverage
- Edge case handling
### Step 3: Read Both Transcripts
1. Read the winner's transcript
2. Read the loser's transcript
3. Compare execution patterns:
- How closely did each follow their skill's instructions?
- What tools were used differently?
- Where did the loser diverge from optimal behavior?
- Did either encounter errors or make recovery attempts?
### Step 4: Analyze Instruction Following
For each transcript, evaluate:
- Did the agent follow the skill's explicit instructions?
- Did the agent use the skill's provided tools/scripts?
- Were there missed opportunities to leverage skill content?
- Did the agent add unnecessary steps not in the skill?
Score instruction following 1-10 and note specific issues.
### Step 5: Identify Winner Strengths
Determine what made the winner better:
- Clearer instructions that led to better behavior?
- Better scripts/tools that produced better output?
- More comprehensive examples that guided edge cases?
- Better error handling guidance?
Be specific. Quote from skills/transcripts where relevant.
### Step 6: Identify Loser Weaknesses
Determine what held the loser back:
- Ambiguous instructions that led to suboptimal choices?
- Missing tools/scripts that forced workarounds?
- Gaps in edge case coverage?
- Poor error handling that caused failures?
### Step 7: Generate Improvement Suggestions
Based on the analysis, produce actionable suggestions for improving the loser skill:
- Specific instruction changes to make
- Tools/scripts to add or modify
- Examples to include
- Edge cases to address
Prioritize by impact. Focus on changes that would have changed the outcome.
### Step 8: Write Analysis Results
Save structured analysis to `{output_path}`.
## Output Format
Write a JSON file with this structure:
```json
{
"comparison_summary": {
"winner": "A",
"winner_skill": "path/to/winner/skill",
"loser_skill": "path/to/loser/skill",
"comparator_reasoning": "Brief summary of why comparator chose winner"
},
"winner_strengths": [
"Clear step-by-step instructions for handling multi-page documents",
"Included validation script that caught formatting errors",
"Explicit guidance on fallback behavior when OCR fails"
],
"loser_weaknesses": [
"Vague instruction 'process the document appropriately' led to inconsistent behavior",
"No script for validation, agent had to improvise and made errors",
"No guidance on OCR failure, agent gave up instead of trying alternatives"
],
"instruction_following": {
"winner": {
"score": 9,
"issues": [
"Minor: skipped optional logging step"
]
},
"loser": {
"score": 6,
"issues": [
"Did not use the skill's formatting template",
"Invented own approach instead of following step 3",
"Missed the 'always validate output' instruction"
]
}
},
"improvement_suggestions": [
{
"priority": "high",
"category": "instructions",
"suggestion": "Replace 'process the document appropriately' with explicit steps: 1) Extract text, 2) Identify sections, 3) Format per template",
"expected_impact": "Would eliminate ambiguity that caused inconsistent behavior"
},
{
"priority": "high",
"category": "tools",
"suggestion": "Add validate_output.py script similar to winner skill's validation approach",
"expected_impact": "Would catch formatting errors before final output"
},
{
"priority": "medium",
"category": "error_handling",
"suggestion": "Add fallback instructions: 'If OCR fails, try: 1) different resolution, 2) image preprocessing, 3) manual extraction'",
"expected_impact": "Would prevent early failure on difficult documents"
}
],
"transcript_insights": {
"winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script -> Fixed 2 issues -> Produced output",
"loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods -> No validation -> Output had errors"
}
}
```
## Guidelines
- **Be specific**: Quote from skills and transcripts, don't just say "instructions were unclear"
- **Be actionable**: Suggestions should be concrete changes, not vague advice
- **Focus on skill improvements**: The goal is to improve the losing skill, not critique the agent
- **Prioritize by impact**: Which changes would most likely have changed the outcome?
- **Consider causation**: Did the skill weakness actually cause the worse output, or is it incidental?
- **Stay objective**: Analyze what happened, don't editorialize
- **Think about generalization**: Would this improvement help on other evals too?
## Categories for Suggestions
Use these categories to organize improvement suggestions:
| Category | Description |
|----------|-------------|
| `instructions` | Changes to the skill's prose instructions |
| `tools` | Scripts, templates, or utilities to add/modify |
| `examples` | Example inputs/outputs to include |
| `error_handling` | Guidance for handling failures |
| `structure` | Reorganization of skill content |
| `references` | External docs or resources to add |
## Priority Levels
- **high**: Would likely change the outcome of this comparison
- **medium**: Would improve quality but may not change win/loss
- **low**: Nice to have, marginal improvement
---
# Analyzing Benchmark Results
When analyzing benchmark results, the analyzer's purpose is to **surface patterns and anomalies** across multiple runs, not suggest skill improvements.
## Role
Review all benchmark run results and generate freeform notes that help the user understand skill performance. Focus on patterns that wouldn't be visible from aggregate metrics alone.
## Inputs
You receive these parameters in your prompt:
- **benchmark_data_path**: Path to the in-progress benchmark.json with all run results
- **skill_path**: Path to the skill being benchmarked
- **output_path**: Where to save the notes (as JSON array of strings)
## Process
### Step 1: Read Benchmark Data
1. Read the benchmark.json containing all run results
2. Note the configurations tested (with_skill, without_skill)
3. Understand the run_summary aggregates already calculated
### Step 2: Analyze Per-Assertion Patterns
For each expectation across all runs:
- Does it **always pass** in both configurations? (may not differentiate skill value)
- Does it **always fail** in both configurations? (may be broken or beyond capability)
- Does it **always pass with skill but fail without**? (skill clearly adds value here)
- Does it **always fail with skill but pass without**? (skill may be hurting)
- Is it **highly variable**? (flaky expectation or non-deterministic behavior)
### Step 3: Analyze Cross-Eval Patterns
Look for patterns across evals:
- Are certain eval types consistently harder/easier?
- Do some evals show high variance while others are stable?
- Are there surprising results that contradict expectations?
### Step 4: Analyze Metrics Patterns
Look at time_seconds, tokens, tool_calls:
- Does the skill significantly increase execution time?
- Is there high variance in resource usage?
- Are there outlier runs that skew the aggregates?
### Step 5: Generate Notes
Write freeform observations as a list of strings. Each note should:
- State a specific observation
- Be grounded in the data (not speculation)
- Help the user understand something the aggregate metrics don't show
Examples:
- "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value"
- "Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure that may be flaky"
- "Without-skill runs consistently fail on table extraction expectations (0% pass rate)"
- "Skill adds 13s average execution time but improves pass rate by 50%"
- "Token usage is 80% higher with skill, primarily due to script output parsing"
- "All 3 without-skill runs for eval 1 produced empty output"
### Step 6: Write Notes
Save notes to `{output_path}` as a JSON array of strings:
```json
[
"Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value",
"Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure",
"Without-skill runs consistently fail on table extraction expectations",
"Skill adds 13s average execution time but improves pass rate by 50%"
]
```
## Guidelines
**DO:**
- Report what you observe in the data
- Be specific about which evals, expectations, or runs you're referring to
- Note patterns that aggregate metrics would hide
- Provide context that helps interpret the numbers
**DO NOT:**
- Suggest improvements to the skill (that's for the improvement step, not benchmarking)
- Make subjective quality judgments ("the output was good/bad")
- Speculate about causes without evidence
- Repeat information already in the run_summary aggregates
resources/bundled/skills/create-skill/agents/comparator.md
+202
View File
@@ -0,0 +1,202 @@
# Blind Comparator Agent
Compare two outputs WITHOUT knowing which skill produced them.
## Role
The Blind Comparator judges which output better accomplishes the eval task. You receive two outputs labeled A and B, but you do NOT know which skill produced which. This prevents bias toward a particular skill or approach.
Your judgment is based purely on output quality and task completion.
## Inputs
You receive these parameters in your prompt:
- **output_a_path**: Path to the first output file or directory
- **output_b_path**: Path to the second output file or directory
- **eval_prompt**: The original task/prompt that was executed
- **expectations**: List of expectations to check (optional - may be empty)
## Process
### Step 1: Read Both Outputs
1. Examine output A (file or directory)
2. Examine output B (file or directory)
3. Note the type, structure, and content of each
4. If outputs are directories, examine all relevant files inside
### Step 2: Understand the Task
1. Read the eval_prompt carefully
2. Identify what the task requires:
- What should be produced?
- What qualities matter (accuracy, completeness, format)?
- What would distinguish a good output from a poor one?
### Step 3: Generate Evaluation Rubric
Based on the task, generate a rubric with two dimensions:
**Content Rubric** (what the output contains):
| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) |
|-----------|----------|----------------|---------------|
| Correctness | Major errors | Minor errors | Fully correct |
| Completeness | Missing key elements | Mostly complete | All elements present |
| Accuracy | Significant inaccuracies | Minor inaccuracies | Accurate throughout |
**Structure Rubric** (how the output is organized):
| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) |
|-----------|----------|----------------|---------------|
| Organization | Disorganized | Reasonably organized | Clear, logical structure |
| Formatting | Inconsistent/broken | Mostly consistent | Professional, polished |
| Usability | Difficult to use | Usable with effort | Easy to use |
Adapt criteria to the specific task. For example:
- PDF form → "Field alignment", "Text readability", "Data placement"
- Document → "Section structure", "Heading hierarchy", "Paragraph flow"
- Data output → "Schema correctness", "Data types", "Completeness"
### Step 4: Evaluate Each Output Against the Rubric
For each output (A and B):
1. **Score each criterion** on the rubric (1-5 scale)
2. **Calculate dimension totals**: Content score, Structure score
3. **Calculate overall score**: Average of dimension scores, scaled to 1-10
### Step 5: Check Assertions (if provided)
If expectations are provided:
1. Check each expectation against output A
2. Check each expectation against output B
3. Count pass rates for each output
4. Use expectation scores as secondary evidence (not the primary decision factor)
### Step 6: Determine the Winner
Compare A and B based on (in priority order):
1. **Primary**: Overall rubric score (content + structure)
2. **Secondary**: Assertion pass rates (if applicable)
3. **Tiebreaker**: If truly equal, declare a TIE
Be decisive - ties should be rare. One output is usually better, even if marginally.
### Step 7: Write Comparison Results
Save results to a JSON file at the path specified (or `comparison.json` if not specified).
## Output Format
Write a JSON file with this structure:
```json
{
"winner": "A",
"reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.",
"rubric": {
"A": {
"content": {
"correctness": 5,
"completeness": 5,
"accuracy": 4
},
"structure": {
"organization": 4,
"formatting": 5,
"usability": 4
},
"content_score": 4.7,
"structure_score": 4.3,
"overall_score": 9.0
},
"B": {
"content": {
"correctness": 3,
"completeness": 2,
"accuracy": 3
},
"structure": {
"organization": 3,
"formatting": 2,
"usability": 3
},
"content_score": 2.7,
"structure_score": 2.7,
"overall_score": 5.4
}
},
"output_quality": {
"A": {
"score": 9,
"strengths": ["Complete solution", "Well-formatted", "All fields present"],
"weaknesses": ["Minor style inconsistency in header"]
},
"B": {
"score": 5,
"strengths": ["Readable output", "Correct basic structure"],
"weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"]
}
},
"expectation_results": {
"A": {
"passed": 4,
"total": 5,
"pass_rate": 0.80,
"details": [
{"text": "Output includes name", "passed": true},
{"text": "Output includes date", "passed": true},
{"text": "Format is PDF", "passed": true},
{"text": "Contains signature", "passed": false},
{"text": "Readable text", "passed": true}
]
},
"B": {
"passed": 3,
"total": 5,
"pass_rate": 0.60,
"details": [
{"text": "Output includes name", "passed": true},
{"text": "Output includes date", "passed": false},
{"text": "Format is PDF", "passed": true},
{"text": "Contains signature", "passed": false},
{"text": "Readable text", "passed": true}
]
}
}
}
```
If no expectations were provided, omit the `expectation_results` field entirely.
## Field Descriptions
- **winner**: "A", "B", or "TIE"
- **reasoning**: Clear explanation of why the winner was chosen (or why it's a tie)
- **rubric**: Structured rubric evaluation for each output
- **content**: Scores for content criteria (correctness, completeness, accuracy)
- **structure**: Scores for structure criteria (organization, formatting, usability)
- **content_score**: Average of content criteria (1-5)
- **structure_score**: Average of structure criteria (1-5)
- **overall_score**: Combined score scaled to 1-10
- **output_quality**: Summary quality assessment
- **score**: 1-10 rating (should match rubric overall_score)
- **strengths**: List of positive aspects
- **weaknesses**: List of issues or shortcomings
- **expectation_results**: (Only if expectations provided)
- **passed**: Number of expectations that passed
- **total**: Total number of expectations
- **pass_rate**: Fraction passed (0.0 to 1.0)
- **details**: Individual expectation results
## Guidelines
- **Stay blind**: DO NOT try to infer which skill produced which output. Judge purely on output quality.
- **Be specific**: Cite specific examples when explaining strengths and weaknesses.
- **Be decisive**: Choose a winner unless outputs are genuinely equivalent.
- **Output quality first**: Assertion scores are secondary to overall task completion.
- **Be objective**: Don't favor outputs based on style preferences; focus on correctness and completeness.
- **Explain your reasoning**: The reasoning field should make it clear why you chose the winner.
- **Handle edge cases**: If both outputs fail, pick the one that fails less badly. If both are excellent, pick the one that's marginally better.
resources/bundled/skills/create-skill/agents/grader.md
+223
View File
@@ -0,0 +1,223 @@
# Grader Agent
Evaluate expectations against an execution transcript and outputs.
## Role
The Grader reviews a transcript and output files, then determines whether each expectation passes or fails. Provide clear evidence for each judgment.
You have two jobs: grade the outputs, and critique the evals themselves. A passing grade on a weak assertion is worse than useless — it creates false confidence. When you notice an assertion that's trivially satisfied, or an important outcome that no assertion checks, say so.
## Inputs
You receive these parameters in your prompt:
- **expectations**: List of expectations to evaluate (strings)
- **transcript_path**: Path to the execution transcript (markdown file)
- **outputs_dir**: Directory containing output files from execution
## Process
### Step 1: Read the Transcript
1. Read the transcript file completely
2. Note the eval prompt, execution steps, and final result
3. Identify any issues or errors documented
### Step 2: Examine Output Files
1. List files in outputs_dir
2. Read/examine each file relevant to the expectations. If outputs aren't plain text, use the inspection tools provided in your prompt — don't rely solely on what the transcript says the executor produced.
3. Note contents, structure, and quality
### Step 3: Evaluate Each Assertion
For each expectation:
1. **Search for evidence** in the transcript and outputs
2. **Determine verdict**:
- **PASS**: Clear evidence the expectation is true AND the evidence reflects genuine task completion, not just surface-level compliance
- **FAIL**: No evidence, or evidence contradicts the expectation, or the evidence is superficial (e.g., correct filename but empty/wrong content)
3. **Cite the evidence**: Quote the specific text or describe what you found
### Step 4: Extract and Verify Claims
Beyond the predefined expectations, extract implicit claims from the outputs and verify them:
1. **Extract claims** from the transcript and outputs:
- Factual statements ("The form has 12 fields")
- Process claims ("Used pypdf to fill the form")
- Quality claims ("All fields were filled correctly")
2. **Verify each claim**:
- **Factual claims**: Can be checked against the outputs or external sources
- **Process claims**: Can be verified from the transcript
- **Quality claims**: Evaluate whether the claim is justified
3. **Flag unverifiable claims**: Note claims that cannot be verified with available information
This catches issues that predefined expectations might miss.
### Step 5: Read User Notes
If `{outputs_dir}/user_notes.md` exists:
1. Read it and note any uncertainties or issues flagged by the executor
2. Include relevant concerns in the grading output
3. These may reveal problems even when expectations pass
### Step 6: Critique the Evals
After grading, consider whether the evals themselves could be improved. Only surface suggestions when there's a clear gap.
Good suggestions test meaningful outcomes — assertions that are hard to satisfy without actually doing the work correctly. Think about what makes an assertion *discriminating*: it passes when the skill genuinely succeeds and fails when it doesn't.
Suggestions worth raising:
- An assertion that passed but would also pass for a clearly wrong output (e.g., checking filename existence but not file content)
- An important outcome you observed — good or bad — that no assertion covers at all
- An assertion that can't actually be verified from the available outputs
Keep the bar high. The goal is to flag things the eval author would say "good catch" about, not to nitpick every assertion.
### Step 7: Write Grading Results
Save results to `{outputs_dir}/../grading.json` (sibling to outputs_dir).
## Grading Criteria
**PASS when**:
- The transcript or outputs clearly demonstrate the expectation is true
- Specific evidence can be cited
- The evidence reflects genuine substance, not just surface compliance (e.g., a file exists AND contains correct content, not just the right filename)
**FAIL when**:
- No evidence found for the expectation
- Evidence contradicts the expectation
- The expectation cannot be verified from available information
- The evidence is superficial — the assertion is technically satisfied but the underlying task outcome is wrong or incomplete
- The output appears to meet the assertion by coincidence rather than by actually doing the work
**When uncertain**: The burden of proof to pass is on the expectation.
### Step 8: Read Executor Metrics and Timing
1. If `{outputs_dir}/metrics.json` exists, read it and include in grading output
2. If `{outputs_dir}/../timing.json` exists, read it and include timing data
## Output Format
Write a JSON file with this structure:
```json
{
"expectations": [
{
"text": "The output includes the name 'John Smith'",
"passed": true,
"evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'"
},
{
"text": "The spreadsheet has a SUM formula in cell B10",
"passed": false,
"evidence": "No spreadsheet was created. The output was a text file."
},
{
"text": "The assistant used the skill's OCR script",
"passed": true,
"evidence": "Transcript Step 2 shows: 'Tool: Bash - python ocr_script.py image.png'"
}
],
"summary": {
"passed": 2,
"failed": 1,
"total": 3,
"pass_rate": 0.67
},
"execution_metrics": {
"tool_calls": {
"Read": 5,
"Write": 2,
"Bash": 8
},
"total_tool_calls": 15,
"total_steps": 6,
"errors_encountered": 0,
"output_chars": 12450,
"transcript_chars": 3200
},
"timing": {
"executor_duration_seconds": 165.0,
"grader_duration_seconds": 26.0,
"total_duration_seconds": 191.0
},
"claims": [
{
"claim": "The form has 12 fillable fields",
"type": "factual",
"verified": true,
"evidence": "Counted 12 fields in field_info.json"
},
{
"claim": "All required fields were populated",
"type": "quality",
"verified": false,
"evidence": "Reference section was left blank despite data being available"
}
],
"user_notes_summary": {
"uncertainties": ["Used 2023 data, may be stale"],
"needs_review": [],
"workarounds": ["Fell back to text overlay for non-fillable fields"]
},
"eval_feedback": {
"suggestions": [
{
"assertion": "The output includes the name 'John Smith'",
"reason": "A hallucinated document that mentions the name would also pass — consider checking it appears as the primary contact with matching phone and email from the input"
},
{
"reason": "No assertion checks whether the extracted phone numbers match the input — I observed incorrect numbers in the output that went uncaught"
}
],
"overall": "Assertions check presence but not correctness. Consider adding content verification."
}
}
```
## Field Descriptions
- **expectations**: Array of graded expectations
- **text**: The original expectation text
- **passed**: Boolean - true if expectation passes
- **evidence**: Specific quote or description supporting the verdict
- **summary**: Aggregate statistics
- **passed**: Count of passed expectations
- **failed**: Count of failed expectations
- **total**: Total expectations evaluated
- **pass_rate**: Fraction passed (0.0 to 1.0)
- **execution_metrics**: Copied from executor's metrics.json (if available)
- **output_chars**: Total character count of output files (proxy for tokens)
- **transcript_chars**: Character count of transcript
- **timing**: Wall clock timing from timing.json (if available)
- **executor_duration_seconds**: Time spent in executor subagent
- **total_duration_seconds**: Total elapsed time for the run
- **claims**: Extracted and verified claims from the output
- **claim**: The statement being verified
- **type**: "factual", "process", or "quality"
- **verified**: Boolean - whether the claim holds
- **evidence**: Supporting or contradicting evidence
- **user_notes_summary**: Issues flagged by the executor
- **uncertainties**: Things the executor wasn't sure about
- **needs_review**: Items requiring human attention
- **workarounds**: Places where the skill didn't work as expected
- **eval_feedback**: Improvement suggestions for the evals (only when warranted)
- **suggestions**: List of concrete suggestions, each with a `reason` and optionally an `assertion` it relates to
- **overall**: Brief assessment — can be "No suggestions, evals look solid" if nothing to flag
## Guidelines
- **Be objective**: Base verdicts on evidence, not assumptions
- **Be specific**: Quote the exact text that supports your verdict
- **Be thorough**: Check both transcript and output files
- **Be consistent**: Apply the same standard to each expectation
- **Explain failures**: Make it clear why evidence was insufficient
- **No partial credit**: Each expectation is pass or fail, not partial
resources/bundled/skills/create-skill/assets/eval_review.html
+146
View File
@@ -0,0 +1,146 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Eval Set Review - __SKILL_NAME_PLACEHOLDER__</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Poppins:wght@500;600&family=Lora:wght@400;500&display=swap" rel="stylesheet">
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: 'Lora', Georgia, serif; background: #faf9f5; padding: 2rem; color: #141413; }
h1 { font-family: 'Poppins', sans-serif; margin-bottom: 0.5rem; font-size: 1.5rem; }
.description { color: #b0aea5; margin-bottom: 1.5rem; font-style: italic; max-width: 900px; }
.controls { margin-bottom: 1rem; display: flex; gap: 0.5rem; }
.btn { font-family: 'Poppins', sans-serif; padding: 0.5rem 1rem; border: none; border-radius: 6px; cursor: pointer; font-size: 0.875rem; font-weight: 500; }
.btn-add { background: #6a9bcc; color: white; }
.btn-add:hover { background: #5889b8; }
.btn-export { background: #d97757; color: white; }
.btn-export:hover { background: #c4613f; }
table { width: 100%; max-width: 1100px; border-collapse: collapse; background: white; border-radius: 6px; overflow: hidden; box-shadow: 0 1px 3px rgba(0,0,0,0.08); }
th { font-family: 'Poppins', sans-serif; background: #141413; color: #faf9f5; padding: 0.75rem 1rem; text-align: left; font-size: 0.875rem; }
td { padding: 0.75rem 1rem; border-bottom: 1px solid #e8e6dc; vertical-align: top; }
tr:nth-child(even) td { background: #faf9f5; }
tr:hover td { background: #f3f1ea; }
.section-header td { background: #e8e6dc; font-family: 'Poppins', sans-serif; font-weight: 500; font-size: 0.8rem; color: #141413; text-transform: uppercase; letter-spacing: 0.05em; }
.query-input { width: 100%; padding: 0.4rem; border: 1px solid #e8e6dc; border-radius: 4px; font-size: 0.875rem; font-family: 'Lora', Georgia, serif; resize: vertical; min-height: 60px; }
.query-input:focus { outline: none; border-color: #d97757; box-shadow: 0 0 0 2px rgba(217,119,87,0.15); }
.toggle { position: relative; display: inline-block; width: 44px; height: 24px; }
.toggle input { opacity: 0; width: 0; height: 0; }
.toggle .slider { position: absolute; inset: 0; background: #b0aea5; border-radius: 24px; cursor: pointer; transition: 0.2s; }
.toggle .slider::before { content: ""; position: absolute; width: 18px; height: 18px; left: 3px; bottom: 3px; background: white; border-radius: 50%; transition: 0.2s; }
.toggle input:checked + .slider { background: #d97757; }
.toggle input:checked + .slider::before { transform: translateX(20px); }
.btn-delete { background: #c44; color: white; padding: 0.3rem 0.6rem; border: none; border-radius: 4px; cursor: pointer; font-size: 0.75rem; font-family: 'Poppins', sans-serif; }
.btn-delete:hover { background: #a33; }
.summary { margin-top: 1rem; color: #b0aea5; font-size: 0.875rem; }
</style>
</head>
<body>
<h1>Eval Set Review: <span id="skill-name">__SKILL_NAME_PLACEHOLDER__</span></h1>
<p class="description">Current description: <span id="skill-desc">__SKILL_DESCRIPTION_PLACEHOLDER__</span></p>
<div class="controls">
<button class="btn btn-add" onclick="addRow()">+ Add Query</button>
<button class="btn btn-export" onclick="exportEvalSet()">Export Eval Set</button>
</div>
<table>
<thead>
<tr>
<th style="width:65%">Query</th>
<th style="width:18%">Should Trigger</th>
<th style="width:10%">Actions</th>
</tr>
</thead>
<tbody id="eval-body"></tbody>
</table>
<p class="summary" id="summary"></p>
<script>
const EVAL_DATA = __EVAL_DATA_PLACEHOLDER__;
let evalItems = [...EVAL_DATA];
function render() {
const tbody = document.getElementById('eval-body');
tbody.innerHTML = '';
// Sort: should-trigger first, then should-not-trigger
const sorted = evalItems
.map((item, origIdx) => ({ ...item, origIdx }))
.sort((a, b) => (b.should_trigger ? 1 : 0) - (a.should_trigger ? 1 : 0));
let lastGroup = null;
sorted.forEach(item => {
const group = item.should_trigger ? 'trigger' : 'no-trigger';
if (group !== lastGroup) {
const headerRow = document.createElement('tr');
headerRow.className = 'section-header';
headerRow.innerHTML = `<td colspan="3">${item.should_trigger ? 'Should Trigger' : 'Should NOT Trigger'}</td>`;
tbody.appendChild(headerRow);
lastGroup = group;
}
const idx = item.origIdx;
const tr = document.createElement('tr');
tr.innerHTML = `
<td><textarea class="query-input" onchange="updateQuery(${idx}, this.value)">${escapeHtml(item.query)}</textarea></td>
<td>
<label class="toggle">
<input type="checkbox" ${item.should_trigger ? 'checked' : ''} onchange="updateTrigger(${idx}, this.checked)">
<span class="slider"></span>
</label>
<span style="margin-left:8px;font-size:0.8rem;color:#b0aea5">${item.should_trigger ? 'Yes' : 'No'}</span>
</td>
<td><button class="btn-delete" onclick="deleteRow(${idx})">Delete</button></td>
`;
tbody.appendChild(tr);
});
updateSummary();
}
function escapeHtml(text) {
const div = document.createElement('div');
div.textContent = text;
return div.innerHTML;
}
function updateQuery(idx, value) { evalItems[idx].query = value; updateSummary(); }
function updateTrigger(idx, value) { evalItems[idx].should_trigger = value; render(); }
function deleteRow(idx) { evalItems.splice(idx, 1); render(); }
function addRow() {
evalItems.push({ query: '', should_trigger: true });
render();
const inputs = document.querySelectorAll('.query-input');
inputs[inputs.length - 1].focus();
}
function updateSummary() {
const trigger = evalItems.filter(i => i.should_trigger).length;
const noTrigger = evalItems.filter(i => !i.should_trigger).length;
document.getElementById('summary').textContent =
`${evalItems.length} queries total: ${trigger} should trigger, ${noTrigger} should not trigger`;
}
function exportEvalSet() {
const valid = evalItems.filter(i => i.query.trim() !== '');
const data = valid.map(i => ({ query: i.query.trim(), should_trigger: i.should_trigger }));
const blob = new Blob([JSON.stringify(data, null, 2)], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'eval_set.json';
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
}
render();
</script>
</body>
</html>
resources/bundled/skills/create-skill/eval-viewer/generate_review.py
+471
View File
@@ -0,0 +1,471 @@
#!/usr/bin/env python3
"""Generate and serve a review page for eval results.
Reads the workspace directory, discovers runs (directories with outputs/),
embeds all output data into a self-contained HTML page, and serves it via
a tiny HTTP server. Feedback auto-saves to feedback.json in the workspace.
Usage:
python generate_review.py <workspace-path> [--port PORT] [--skill-name NAME]
python generate_review.py <workspace-path> --previous-feedback /path/to/old/feedback.json
No dependencies beyond the Python stdlib are required.
"""
import argparse
import base64
import json
import mimetypes
import os
import re
import signal
import subprocess
import sys
import time
import webbrowser
from functools import partial
from http.server import HTTPServer, BaseHTTPRequestHandler
from pathlib import Path
# Files to exclude from output listings
METADATA_FILES = {"transcript.md", "user_notes.md", "metrics.json"}
# Extensions we render as inline text
TEXT_EXTENSIONS = {
".txt", ".md", ".json", ".csv", ".py", ".js", ".ts", ".tsx", ".jsx",
".yaml", ".yml", ".xml", ".html", ".css", ".sh", ".rb", ".go", ".rs",
".java", ".c", ".cpp", ".h", ".hpp", ".sql", ".r", ".toml",
}
# Extensions we render as inline images
IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".gif", ".svg", ".webp"}
# MIME type overrides for common types
MIME_OVERRIDES = {
".svg": "image/svg+xml",
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
}
def get_mime_type(path: Path) -> str:
ext = path.suffix.lower()
if ext in MIME_OVERRIDES:
return MIME_OVERRIDES[ext]
mime, _ = mimetypes.guess_type(str(path))
return mime or "application/octet-stream"
def find_runs(workspace: Path) -> list[dict]:
"""Recursively find directories that contain an outputs/ subdirectory."""
runs: list[dict] = []
_find_runs_recursive(workspace, workspace, runs)
runs.sort(key=lambda r: (r.get("eval_id", float("inf")), r["id"]))
return runs
def _find_runs_recursive(root: Path, current: Path, runs: list[dict]) -> None:
if not current.is_dir():
return
outputs_dir = current / "outputs"
if outputs_dir.is_dir():
run = build_run(root, current)
if run:
runs.append(run)
return
skip = {"node_modules", ".git", "__pycache__", "skill", "inputs"}
for child in sorted(current.iterdir()):
if child.is_dir() and child.name not in skip:
_find_runs_recursive(root, child, runs)
def build_run(root: Path, run_dir: Path) -> dict | None:
"""Build a run dict with prompt, outputs, and grading data."""
prompt = ""
eval_id = None
# Try eval_metadata.json
for candidate in [run_dir / "eval_metadata.json", run_dir.parent / "eval_metadata.json"]:
if candidate.exists():
try:
metadata = json.loads(candidate.read_text())
prompt = metadata.get("prompt", "")
eval_id = metadata.get("eval_id")
except (json.JSONDecodeError, OSError):
pass
if prompt:
break
# Fall back to transcript.md
if not prompt:
for candidate in [run_dir / "transcript.md", run_dir / "outputs" / "transcript.md"]:
if candidate.exists():
try:
text = candidate.read_text()
match = re.search(r"## Eval Prompt\n\n([\s\S]*?)(?=\n##|$)", text)
if match:
prompt = match.group(1).strip()
except OSError:
pass
if prompt:
break
if not prompt:
prompt = "(No prompt found)"
run_id = str(run_dir.relative_to(root)).replace("/", "-").replace("\\", "-")
# Collect output files
outputs_dir = run_dir / "outputs"
output_files: list[dict] = []
if outputs_dir.is_dir():
for f in sorted(outputs_dir.iterdir()):
if f.is_file() and f.name not in METADATA_FILES:
output_files.append(embed_file(f))
# Load grading if present
grading = None
for candidate in [run_dir / "grading.json", run_dir.parent / "grading.json"]:
if candidate.exists():
try:
grading = json.loads(candidate.read_text())
except (json.JSONDecodeError, OSError):
pass
if grading:
break
return {
"id": run_id,
"prompt": prompt,
"eval_id": eval_id,
"outputs": output_files,
"grading": grading,
}
def embed_file(path: Path) -> dict:
"""Read a file and return an embedded representation."""
ext = path.suffix.lower()
mime = get_mime_type(path)
if ext in TEXT_EXTENSIONS:
try:
content = path.read_text(errors="replace")
except OSError:
content = "(Error reading file)"
return {
"name": path.name,
"type": "text",
"content": content,
}
elif ext in IMAGE_EXTENSIONS:
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "image",
"mime": mime,
"data_uri": f"data:{mime};base64,{b64}",
}
elif ext == ".pdf":
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "pdf",
"data_uri": f"data:{mime};base64,{b64}",
}
elif ext == ".xlsx":
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "xlsx",
"data_b64": b64,
}
else:
# Binary / unknown — base64 download link
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "binary",
"mime": mime,
"data_uri": f"data:{mime};base64,{b64}",
}
def load_previous_iteration(workspace: Path) -> dict[str, dict]:
"""Load previous iteration's feedback and outputs.
Returns a map of run_id -> {"feedback": str, "outputs": list[dict]}.
"""
result: dict[str, dict] = {}
# Load feedback
feedback_map: dict[str, str] = {}
feedback_path = workspace / "feedback.json"
if feedback_path.exists():
try:
data = json.loads(feedback_path.read_text())
feedback_map = {
r["run_id"]: r["feedback"]
for r in data.get("reviews", [])
if r.get("feedback", "").strip()
}
except (json.JSONDecodeError, OSError, KeyError):
pass
# Load runs (to get outputs)
prev_runs = find_runs(workspace)
for run in prev_runs:
result[run["id"]] = {
"feedback": feedback_map.get(run["id"], ""),
"outputs": run.get("outputs", []),
}
# Also add feedback for run_ids that had feedback but no matching run
for run_id, fb in feedback_map.items():
if run_id not in result:
result[run_id] = {"feedback": fb, "outputs": []}
return result
def generate_html(
runs: list[dict],
skill_name: str,
previous: dict[str, dict] | None = None,
benchmark: dict | None = None,
) -> str:
"""Generate the complete standalone HTML page with embedded data."""
template_path = Path(__file__).parent / "viewer.html"
template = template_path.read_text()
# Build previous_feedback and previous_outputs maps for the template
previous_feedback: dict[str, str] = {}
previous_outputs: dict[str, list[dict]] = {}
if previous:
for run_id, data in previous.items():
if data.get("feedback"):
previous_feedback[run_id] = data["feedback"]
if data.get("outputs"):
previous_outputs[run_id] = data["outputs"]
embedded = {
"skill_name": skill_name,
"runs": runs,
"previous_feedback": previous_feedback,
"previous_outputs": previous_outputs,
}
if benchmark:
embedded["benchmark"] = benchmark
data_json = json.dumps(embedded)
return template.replace("/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};")
# ---------------------------------------------------------------------------
# HTTP server (stdlib only, zero dependencies)
# ---------------------------------------------------------------------------
def _kill_port(port: int) -> None:
"""Kill any process listening on the given port."""
try:
result = subprocess.run(
["lsof", "-ti", f":{port}"],
capture_output=True, text=True, timeout=5,
)
for pid_str in result.stdout.strip().split("\n"):
if pid_str.strip():
try:
os.kill(int(pid_str.strip()), signal.SIGTERM)
except (ProcessLookupError, ValueError):
pass
if result.stdout.strip():
time.sleep(0.5)
except subprocess.TimeoutExpired:
pass
except FileNotFoundError:
print("Note: lsof not found, cannot check if port is in use", file=sys.stderr)
class ReviewHandler(BaseHTTPRequestHandler):
"""Serves the review HTML and handles feedback saves.
Regenerates the HTML on each page load so that refreshing the browser
picks up new eval outputs without restarting the server.
"""
def __init__(
self,
workspace: Path,
skill_name: str,
feedback_path: Path,
previous: dict[str, dict],
benchmark_path: Path | None,
*args,
**kwargs,
):
self.workspace = workspace
self.skill_name = skill_name
self.feedback_path = feedback_path
self.previous = previous
self.benchmark_path = benchmark_path
super().__init__(*args, **kwargs)
def do_GET(self) -> None:
if self.path == "/" or self.path == "/index.html":
# Regenerate HTML on each request (re-scans workspace for new outputs)
runs = find_runs(self.workspace)
benchmark = None
if self.benchmark_path and self.benchmark_path.exists():
try:
benchmark = json.loads(self.benchmark_path.read_text())
except (json.JSONDecodeError, OSError):
pass
html = generate_html(runs, self.skill_name, self.previous, benchmark)
content = html.encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(content)))
self.end_headers()
self.wfile.write(content)
elif self.path == "/api/feedback":
data = b"{}"
if self.feedback_path.exists():
data = self.feedback_path.read_bytes()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
else:
self.send_error(404)
def do_POST(self) -> None:
if self.path == "/api/feedback":
length = int(self.headers.get("Content-Length", 0))
body = self.rfile.read(length)
try:
data = json.loads(body)
if not isinstance(data, dict) or "reviews" not in data:
raise ValueError("Expected JSON object with 'reviews' key")
self.feedback_path.write_text(json.dumps(data, indent=2) + "\n")
resp = b'{"ok":true}'
self.send_response(200)
except (json.JSONDecodeError, OSError, ValueError) as e:
resp = json.dumps({"error": str(e)}).encode()
self.send_response(500)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(resp)))
self.end_headers()
self.wfile.write(resp)
else:
self.send_error(404)
def log_message(self, format: str, *args: object) -> None:
# Suppress request logging to keep terminal clean
pass
def main() -> None:
parser = argparse.ArgumentParser(description="Generate and serve eval review")
parser.add_argument("workspace", type=Path, help="Path to workspace directory")
parser.add_argument("--port", "-p", type=int, default=3117, help="Server port (default: 3117)")
parser.add_argument("--skill-name", "-n", type=str, default=None, help="Skill name for header")
parser.add_argument(
"--previous-workspace", type=Path, default=None,
help="Path to previous iteration's workspace (shows old outputs and feedback as context)",
)
parser.add_argument(
"--benchmark", type=Path, default=None,
help="Path to benchmark.json to show in the Benchmark tab",
)
parser.add_argument(
"--static", "-s", type=Path, default=None,
help="Write standalone HTML to this path instead of starting a server",
)
args = parser.parse_args()
workspace = args.workspace.resolve()
if not workspace.is_dir():
print(f"Error: {workspace} is not a directory", file=sys.stderr)
sys.exit(1)
runs = find_runs(workspace)
if not runs:
print(f"No runs found in {workspace}", file=sys.stderr)
sys.exit(1)
skill_name = args.skill_name or workspace.name.replace("-workspace", "")
feedback_path = workspace / "feedback.json"
previous: dict[str, dict] = {}
if args.previous_workspace:
previous = load_previous_iteration(args.previous_workspace.resolve())
benchmark_path = args.benchmark.resolve() if args.benchmark else None
benchmark = None
if benchmark_path and benchmark_path.exists():
try:
benchmark = json.loads(benchmark_path.read_text())
except (json.JSONDecodeError, OSError):
pass
if args.static:
html = generate_html(runs, skill_name, previous, benchmark)
args.static.parent.mkdir(parents=True, exist_ok=True)
args.static.write_text(html)
print(f"\n Static viewer written to: {args.static}\n")
sys.exit(0)
# Kill any existing process on the target port
port = args.port
_kill_port(port)
handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path)
try:
server = HTTPServer(("127.0.0.1", port), handler)
except OSError:
# Port still in use after kill attempt — find a free one
server = HTTPServer(("127.0.0.1", 0), handler)
port = server.server_address[1]
url = f"http://localhost:{port}"
print(f"\n Eval Viewer")
print(f" ─────────────────────────────────")
print(f" URL: {url}")
print(f" Workspace: {workspace}")
print(f" Feedback: {feedback_path}")
if previous:
print(f" Previous: {args.previous_workspace} ({len(previous)} runs)")
if benchmark_path:
print(f" Benchmark: {benchmark_path}")
print(f"\n Press Ctrl+C to stop.\n")
webbrowser.open(url)
try:
server.serve_forever()
except KeyboardInterrupt:
print("\nStopped.")
server.server_close()
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/eval-viewer/viewer.html
+1325
View File
File diff suppressed because it is too large Load Diff
resources/bundled/skills/create-skill/references/schemas.md
+430
View File
@@ -0,0 +1,430 @@
# JSON Schemas
This document defines the JSON schemas used by skill-creator.
---
## evals.json
Defines the evals for a skill. Located at `evals/evals.json` within the skill directory.
```json
{
"skill_name": "example-skill",
"evals": [
{
"id": 1,
"prompt": "User's example prompt",
"expected_output": "Description of expected result",
"files": ["evals/files/sample1.pdf"],
"expectations": [
"The output includes X",
"The skill used script Y"
]
}
]
}
```
**Fields:**
- `skill_name`: Name matching the skill's frontmatter
- `evals[].id`: Unique integer identifier
- `evals[].prompt`: The task to execute
- `evals[].expected_output`: Human-readable description of success
- `evals[].files`: Optional list of input file paths (relative to skill root)
- `evals[].expectations`: List of verifiable statements
---
## history.json
Tracks version progression in Improve mode. Located at workspace root.
```json
{
"started_at": "2026-01-15T10:30:00Z",
"skill_name": "pdf",
"current_best": "v2",
"iterations": [
{
"version": "v0",
"parent": null,
"expectation_pass_rate": 0.65,
"grading_result": "baseline",
"is_current_best": false
},
{
"version": "v1",
"parent": "v0",
"expectation_pass_rate": 0.75,
"grading_result": "won",
"is_current_best": false
},
{
"version": "v2",
"parent": "v1",
"expectation_pass_rate": 0.85,
"grading_result": "won",
"is_current_best": true
}
]
}
```
**Fields:**
- `started_at`: ISO timestamp of when improvement started
- `skill_name`: Name of the skill being improved
- `current_best`: Version identifier of the best performer
- `iterations[].version`: Version identifier (v0, v1, ...)
- `iterations[].parent`: Parent version this was derived from
- `iterations[].expectation_pass_rate`: Pass rate from grading
- `iterations[].grading_result`: "baseline", "won", "lost", or "tie"
- `iterations[].is_current_best`: Whether this is the current best version
---
## grading.json
Output from the grader agent. Located at `<run-dir>/grading.json`.
```json
{
"expectations": [
{
"text": "The output includes the name 'John Smith'",
"passed": true,
"evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'"
},
{
"text": "The spreadsheet has a SUM formula in cell B10",
"passed": false,
"evidence": "No spreadsheet was created. The output was a text file."
}
],
"summary": {
"passed": 2,
"failed": 1,
"total": 3,
"pass_rate": 0.67
},
"execution_metrics": {
"tool_calls": {
"Read": 5,
"Write": 2,
"Bash": 8
},
"total_tool_calls": 15,
"total_steps": 6,
"errors_encountered": 0,
"output_chars": 12450,
"transcript_chars": 3200
},
"timing": {
"executor_duration_seconds": 165.0,
"grader_duration_seconds": 26.0,
"total_duration_seconds": 191.0
},
"claims": [
{
"claim": "The form has 12 fillable fields",
"type": "factual",
"verified": true,
"evidence": "Counted 12 fields in field_info.json"
}
],
"user_notes_summary": {
"uncertainties": ["Used 2023 data, may be stale"],
"needs_review": [],
"workarounds": ["Fell back to text overlay for non-fillable fields"]
},
"eval_feedback": {
"suggestions": [
{
"assertion": "The output includes the name 'John Smith'",
"reason": "A hallucinated document that mentions the name would also pass"
}
],
"overall": "Assertions check presence but not correctness."
}
}
```
**Fields:**
- `expectations[]`: Graded expectations with evidence
- `summary`: Aggregate pass/fail counts
- `execution_metrics`: Tool usage and output size (from executor's metrics.json)
- `timing`: Wall clock timing (from timing.json)
- `claims`: Extracted and verified claims from the output
- `user_notes_summary`: Issues flagged by the executor
- `eval_feedback`: (optional) Improvement suggestions for the evals, only present when the grader identifies issues worth raising
---
## metrics.json
Output from the executor agent. Located at `<run-dir>/outputs/metrics.json`.
```json
{
"tool_calls": {
"Read": 5,
"Write": 2,
"Bash": 8,
"Edit": 1,
"Glob": 2,
"Grep": 0
},
"total_tool_calls": 18,
"total_steps": 6,
"files_created": ["filled_form.pdf", "field_values.json"],
"errors_encountered": 0,
"output_chars": 12450,
"transcript_chars": 3200
}
```
**Fields:**
- `tool_calls`: Count per tool type
- `total_tool_calls`: Sum of all tool calls
- `total_steps`: Number of major execution steps
- `files_created`: List of output files created
- `errors_encountered`: Number of errors during execution
- `output_chars`: Total character count of output files
- `transcript_chars`: Character count of transcript
---
## timing.json
Wall clock timing for a run. Located at `<run-dir>/timing.json`.
**How to capture:** When a subagent task completes, the task notification includes `total_tokens` and `duration_ms`. Save these immediately — they are not persisted anywhere else and cannot be recovered after the fact.
```json
{
"total_tokens": 84852,
"duration_ms": 23332,
"total_duration_seconds": 23.3,
"executor_start": "2026-01-15T10:30:00Z",
"executor_end": "2026-01-15T10:32:45Z",
"executor_duration_seconds": 165.0,
"grader_start": "2026-01-15T10:32:46Z",
"grader_end": "2026-01-15T10:33:12Z",
"grader_duration_seconds": 26.0
}
```
---
## benchmark.json
Output from Benchmark mode. Located at `benchmarks/<timestamp>/benchmark.json`.
```json
{
"metadata": {
"skill_name": "pdf",
"skill_path": "/path/to/pdf",
"executor_model": "claude-sonnet-4-20250514",
"analyzer_model": "most-capable-model",
"timestamp": "2026-01-15T10:30:00Z",
"evals_run": [1, 2, 3],
"runs_per_configuration": 3
},
"runs": [
{
"eval_id": 1,
"eval_name": "Ocean",
"configuration": "with_skill",
"run_number": 1,
"result": {
"pass_rate": 0.85,
"passed": 6,
"failed": 1,
"total": 7,
"time_seconds": 42.5,
"tokens": 3800,
"tool_calls": 18,
"errors": 0
},
"expectations": [
{"text": "...", "passed": true, "evidence": "..."}
],
"notes": [
"Used 2023 data, may be stale",
"Fell back to text overlay for non-fillable fields"
]
}
],
"run_summary": {
"with_skill": {
"pass_rate": {"mean": 0.85, "stddev": 0.05, "min": 0.80, "max": 0.90},
"time_seconds": {"mean": 45.0, "stddev": 12.0, "min": 32.0, "max": 58.0},
"tokens": {"mean": 3800, "stddev": 400, "min": 3200, "max": 4100}
},
"without_skill": {
"pass_rate": {"mean": 0.35, "stddev": 0.08, "min": 0.28, "max": 0.45},
"time_seconds": {"mean": 32.0, "stddev": 8.0, "min": 24.0, "max": 42.0},
"tokens": {"mean": 2100, "stddev": 300, "min": 1800, "max": 2500}
},
"delta": {
"pass_rate": "+0.50",
"time_seconds": "+13.0",
"tokens": "+1700"
}
},
"notes": [
"Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value",
"Eval 3 shows high variance (50% ± 40%) - may be flaky or model-dependent",
"Without-skill runs consistently fail on table extraction expectations",
"Skill adds 13s average execution time but improves pass rate by 50%"
]
}
```
**Fields:**
- `metadata`: Information about the benchmark run
- `skill_name`: Name of the skill
- `timestamp`: When the benchmark was run
- `evals_run`: List of eval names or IDs
- `runs_per_configuration`: Number of runs per config (e.g. 3)
- `runs[]`: Individual run results
- `eval_id`: Numeric eval identifier
- `eval_name`: Human-readable eval name (used as section header in the viewer)
- `configuration`: Must be `"with_skill"` or `"without_skill"` (the viewer uses this exact string for grouping and color coding)
- `run_number`: Integer run number (1, 2, 3...)
- `result`: Nested object with `pass_rate`, `passed`, `total`, `time_seconds`, `tokens`, `errors`
- `run_summary`: Statistical aggregates per configuration
- `with_skill` / `without_skill`: Each contains `pass_rate`, `time_seconds`, `tokens` objects with `mean` and `stddev` fields
- `delta`: Difference strings like `"+0.50"`, `"+13.0"`, `"+1700"`
- `notes`: Freeform observations from the analyzer
**Important:** The viewer reads these field names exactly. Using `config` instead of `configuration`, or putting `pass_rate` at the top level of a run instead of nested under `result`, will cause the viewer to show empty/zero values. Always reference this schema when generating benchmark.json manually.
---
## comparison.json
Output from blind comparator. Located at `<grading-dir>/comparison-N.json`.
```json
{
"winner": "A",
"reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.",
"rubric": {
"A": {
"content": {
"correctness": 5,
"completeness": 5,
"accuracy": 4
},
"structure": {
"organization": 4,
"formatting": 5,
"usability": 4
},
"content_score": 4.7,
"structure_score": 4.3,
"overall_score": 9.0
},
"B": {
"content": {
"correctness": 3,
"completeness": 2,
"accuracy": 3
},
"structure": {
"organization": 3,
"formatting": 2,
"usability": 3
},
"content_score": 2.7,
"structure_score": 2.7,
"overall_score": 5.4
}
},
"output_quality": {
"A": {
"score": 9,
"strengths": ["Complete solution", "Well-formatted", "All fields present"],
"weaknesses": ["Minor style inconsistency in header"]
},
"B": {
"score": 5,
"strengths": ["Readable output", "Correct basic structure"],
"weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"]
}
},
"expectation_results": {
"A": {
"passed": 4,
"total": 5,
"pass_rate": 0.80,
"details": [
{"text": "Output includes name", "passed": true}
]
},
"B": {
"passed": 3,
"total": 5,
"pass_rate": 0.60,
"details": [
{"text": "Output includes name", "passed": true}
]
}
}
}
```
---
## analysis.json
Output from post-hoc analyzer. Located at `<grading-dir>/analysis.json`.
```json
{
"comparison_summary": {
"winner": "A",
"winner_skill": "path/to/winner/skill",
"loser_skill": "path/to/loser/skill",
"comparator_reasoning": "Brief summary of why comparator chose winner"
},
"winner_strengths": [
"Clear step-by-step instructions for handling multi-page documents",
"Included validation script that caught formatting errors"
],
"loser_weaknesses": [
"Vague instruction 'process the document appropriately' led to inconsistent behavior",
"No script for validation, agent had to improvise"
],
"instruction_following": {
"winner": {
"score": 9,
"issues": ["Minor: skipped optional logging step"]
},
"loser": {
"score": 6,
"issues": [
"Did not use the skill's formatting template",
"Invented own approach instead of following step 3"
]
}
},
"improvement_suggestions": [
{
"priority": "high",
"category": "instructions",
"suggestion": "Replace 'process the document appropriately' with explicit steps",
"expected_impact": "Would eliminate ambiguity that caused inconsistent behavior"
}
],
"transcript_insights": {
"winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script",
"loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods"
}
}
```
resources/bundled/skills/create-skill/scripts/__init__.py
View File
resources/bundled/skills/create-skill/scripts/aggregate_benchmark.py
+401
View File
@@ -0,0 +1,401 @@
#!/usr/bin/env python3
"""
Aggregate individual run results into benchmark summary statistics.
Reads grading.json files from run directories and produces:
- run_summary with mean, stddev, min, max for each metric
- delta between with_skill and without_skill configurations
Usage:
python aggregate_benchmark.py <benchmark_dir>
Example:
python aggregate_benchmark.py benchmarks/2026-01-15T10-30-00/
The script supports two directory layouts:
Workspace layout (from skill-creator iterations):
<benchmark_dir>/
└── eval-N/
├── with_skill/
│ ├── run-1/grading.json
│ └── run-2/grading.json
└── without_skill/
├── run-1/grading.json
└── run-2/grading.json
Legacy layout (with runs/ subdirectory):
<benchmark_dir>/
└── runs/
└── eval-N/
├── with_skill/
│ └── run-1/grading.json
└── without_skill/
└── run-1/grading.json
"""
import argparse
import json
import math
import sys
from datetime import datetime, timezone
from pathlib import Path
def calculate_stats(values: list[float]) -> dict:
"""Calculate mean, stddev, min, max for a list of values."""
if not values:
return {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0}
n = len(values)
mean = sum(values) / n
if n > 1:
variance = sum((x - mean) ** 2 for x in values) / (n - 1)
stddev = math.sqrt(variance)
else:
stddev = 0.0
return {
"mean": round(mean, 4),
"stddev": round(stddev, 4),
"min": round(min(values), 4),
"max": round(max(values), 4)
}
def load_run_results(benchmark_dir: Path) -> dict:
"""
Load all run results from a benchmark directory.
Returns dict keyed by config name (e.g. "with_skill"/"without_skill",
or "new_skill"/"old_skill"), each containing a list of run results.
"""
# Support both layouts: eval dirs directly under benchmark_dir, or under runs/
runs_dir = benchmark_dir / "runs"
if runs_dir.exists():
search_dir = runs_dir
elif list(benchmark_dir.glob("eval-*")):
search_dir = benchmark_dir
else:
print(f"No eval directories found in {benchmark_dir} or {benchmark_dir / 'runs'}")
return {}
results: dict[str, list] = {}
for eval_idx, eval_dir in enumerate(sorted(search_dir.glob("eval-*"))):
metadata_path = eval_dir / "eval_metadata.json"
if metadata_path.exists():
try:
with open(metadata_path) as mf:
eval_id = json.load(mf).get("eval_id", eval_idx)
except (json.JSONDecodeError, OSError):
eval_id = eval_idx
else:
try:
eval_id = int(eval_dir.name.split("-")[1])
except ValueError:
eval_id = eval_idx
# Discover config directories dynamically rather than hardcoding names
for config_dir in sorted(eval_dir.iterdir()):
if not config_dir.is_dir():
continue
# Skip non-config directories (inputs, outputs, etc.)
if not list(config_dir.glob("run-*")):
continue
config = config_dir.name
if config not in results:
results[config] = []
for run_dir in sorted(config_dir.glob("run-*")):
run_number = int(run_dir.name.split("-")[1])
grading_file = run_dir / "grading.json"
if not grading_file.exists():
print(f"Warning: grading.json not found in {run_dir}")
continue
try:
with open(grading_file) as f:
grading = json.load(f)
except json.JSONDecodeError as e:
print(f"Warning: Invalid JSON in {grading_file}: {e}")
continue
# Extract metrics
result = {
"eval_id": eval_id,
"run_number": run_number,
"pass_rate": grading.get("summary", {}).get("pass_rate", 0.0),
"passed": grading.get("summary", {}).get("passed", 0),
"failed": grading.get("summary", {}).get("failed", 0),
"total": grading.get("summary", {}).get("total", 0),
}
# Extract timing — check grading.json first, then sibling timing.json
timing = grading.get("timing", {})
result["time_seconds"] = timing.get("total_duration_seconds", 0.0)
timing_file = run_dir / "timing.json"
if result["time_seconds"] == 0.0 and timing_file.exists():
try:
with open(timing_file) as tf:
timing_data = json.load(tf)
result["time_seconds"] = timing_data.get("total_duration_seconds", 0.0)
result["tokens"] = timing_data.get("total_tokens", 0)
except json.JSONDecodeError:
pass
# Extract metrics if available
metrics = grading.get("execution_metrics", {})
result["tool_calls"] = metrics.get("total_tool_calls", 0)
if not result.get("tokens"):
result["tokens"] = metrics.get("output_chars", 0)
result["errors"] = metrics.get("errors_encountered", 0)
# Extract expectations — viewer requires fields: text, passed, evidence
raw_expectations = grading.get("expectations", [])
for exp in raw_expectations:
if "text" not in exp or "passed" not in exp:
print(f"Warning: expectation in {grading_file} missing required fields (text, passed, evidence): {exp}")
result["expectations"] = raw_expectations
# Extract notes from user_notes_summary
notes_summary = grading.get("user_notes_summary", {})
notes = []
notes.extend(notes_summary.get("uncertainties", []))
notes.extend(notes_summary.get("needs_review", []))
notes.extend(notes_summary.get("workarounds", []))
result["notes"] = notes
results[config].append(result)
return results
def aggregate_results(results: dict) -> dict:
"""
Aggregate run results into summary statistics.
Returns run_summary with stats for each configuration and delta.
"""
run_summary = {}
configs = list(results.keys())
for config in configs:
runs = results.get(config, [])
if not runs:
run_summary[config] = {
"pass_rate": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0},
"time_seconds": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0},
"tokens": {"mean": 0, "stddev": 0, "min": 0, "max": 0}
}
continue
pass_rates = [r["pass_rate"] for r in runs]
times = [r["time_seconds"] for r in runs]
tokens = [r.get("tokens", 0) for r in runs]
run_summary[config] = {
"pass_rate": calculate_stats(pass_rates),
"time_seconds": calculate_stats(times),
"tokens": calculate_stats(tokens)
}
# Calculate delta between the first two configs (if two exist)
if len(configs) >= 2:
primary = run_summary.get(configs[0], {})
baseline = run_summary.get(configs[1], {})
else:
primary = run_summary.get(configs[0], {}) if configs else {}
baseline = {}
delta_pass_rate = primary.get("pass_rate", {}).get("mean", 0) - baseline.get("pass_rate", {}).get("mean", 0)
delta_time = primary.get("time_seconds", {}).get("mean", 0) - baseline.get("time_seconds", {}).get("mean", 0)
delta_tokens = primary.get("tokens", {}).get("mean", 0) - baseline.get("tokens", {}).get("mean", 0)
run_summary["delta"] = {
"pass_rate": f"{delta_pass_rate:+.2f}",
"time_seconds": f"{delta_time:+.1f}",
"tokens": f"{delta_tokens:+.0f}"
}
return run_summary
def generate_benchmark(benchmark_dir: Path, skill_name: str = "", skill_path: str = "") -> dict:
"""
Generate complete benchmark.json from run results.
"""
results = load_run_results(benchmark_dir)
run_summary = aggregate_results(results)
# Build runs array for benchmark.json
runs = []
for config in results:
for result in results[config]:
runs.append({
"eval_id": result["eval_id"],
"configuration": config,
"run_number": result["run_number"],
"result": {
"pass_rate": result["pass_rate"],
"passed": result["passed"],
"failed": result["failed"],
"total": result["total"],
"time_seconds": result["time_seconds"],
"tokens": result.get("tokens", 0),
"tool_calls": result.get("tool_calls", 0),
"errors": result.get("errors", 0)
},
"expectations": result["expectations"],
"notes": result["notes"]
})
# Determine eval IDs from results
eval_ids = sorted(set(
r["eval_id"]
for config in results.values()
for r in config
))
benchmark = {
"metadata": {
"skill_name": skill_name or "<skill-name>",
"skill_path": skill_path or "<path/to/skill>",
"executor_model": "<model-name>",
"analyzer_model": "<model-name>",
"timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"evals_run": eval_ids,
"runs_per_configuration": 3
},
"runs": runs,
"run_summary": run_summary,
"notes": [] # To be filled by analyzer
}
return benchmark
def generate_markdown(benchmark: dict) -> str:
"""Generate human-readable benchmark.md from benchmark data."""
metadata = benchmark["metadata"]
run_summary = benchmark["run_summary"]
# Determine config names (excluding "delta")
configs = [k for k in run_summary if k != "delta"]
config_a = configs[0] if len(configs) >= 1 else "config_a"
config_b = configs[1] if len(configs) >= 2 else "config_b"
label_a = config_a.replace("_", " ").title()
label_b = config_b.replace("_", " ").title()
lines = [
f"# Skill Benchmark: {metadata['skill_name']}",
"",
f"**Model**: {metadata['executor_model']}",
f"**Date**: {metadata['timestamp']}",
f"**Evals**: {', '.join(map(str, metadata['evals_run']))} ({metadata['runs_per_configuration']} runs each per configuration)",
"",
"## Summary",
"",
f"| Metric | {label_a} | {label_b} | Delta |",
"|--------|------------|---------------|-------|",
]
a_summary = run_summary.get(config_a, {})
b_summary = run_summary.get(config_b, {})
delta = run_summary.get("delta", {})
# Format pass rate
a_pr = a_summary.get("pass_rate", {})
b_pr = b_summary.get("pass_rate", {})
lines.append(f"| Pass Rate | {a_pr.get('mean', 0)*100:.0f}% ± {a_pr.get('stddev', 0)*100:.0f}% | {b_pr.get('mean', 0)*100:.0f}% ± {b_pr.get('stddev', 0)*100:.0f}% | {delta.get('pass_rate', '—')} |")
# Format time
a_time = a_summary.get("time_seconds", {})
b_time = b_summary.get("time_seconds", {})
lines.append(f"| Time | {a_time.get('mean', 0):.1f}s ± {a_time.get('stddev', 0):.1f}s | {b_time.get('mean', 0):.1f}s ± {b_time.get('stddev', 0):.1f}s | {delta.get('time_seconds', '—')}s |")
# Format tokens
a_tokens = a_summary.get("tokens", {})
b_tokens = b_summary.get("tokens", {})
lines.append(f"| Tokens | {a_tokens.get('mean', 0):.0f} ± {a_tokens.get('stddev', 0):.0f} | {b_tokens.get('mean', 0):.0f} ± {b_tokens.get('stddev', 0):.0f} | {delta.get('tokens', '—')} |")
# Notes section
if benchmark.get("notes"):
lines.extend([
"",
"## Notes",
""
])
for note in benchmark["notes"]:
lines.append(f"- {note}")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(
description="Aggregate benchmark run results into summary statistics"
)
parser.add_argument(
"benchmark_dir",
type=Path,
help="Path to the benchmark directory"
)
parser.add_argument(
"--skill-name",
default="",
help="Name of the skill being benchmarked"
)
parser.add_argument(
"--skill-path",
default="",
help="Path to the skill being benchmarked"
)
parser.add_argument(
"--output", "-o",
type=Path,
help="Output path for benchmark.json (default: <benchmark_dir>/benchmark.json)"
)
args = parser.parse_args()
if not args.benchmark_dir.exists():
print(f"Directory not found: {args.benchmark_dir}")
sys.exit(1)
# Generate benchmark
benchmark = generate_benchmark(args.benchmark_dir, args.skill_name, args.skill_path)
# Determine output paths
output_json = args.output or (args.benchmark_dir / "benchmark.json")
output_md = output_json.with_suffix(".md")
# Write benchmark.json
with open(output_json, "w") as f:
json.dump(benchmark, f, indent=2)
print(f"Generated: {output_json}")
# Write benchmark.md
markdown = generate_markdown(benchmark)
with open(output_md, "w") as f:
f.write(markdown)
print(f"Generated: {output_md}")
# Print summary
run_summary = benchmark["run_summary"]
configs = [k for k in run_summary if k != "delta"]
delta = run_summary.get("delta", {})
print(f"\nSummary:")
for config in configs:
pr = run_summary[config]["pass_rate"]["mean"]
label = config.replace("_", " ").title()
print(f" {label}: {pr*100:.1f}% pass rate")
print(f" Delta: {delta.get('pass_rate', '—')}")
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/scripts/generate_report.py
+326
View File
@@ -0,0 +1,326 @@
#!/usr/bin/env python3
"""Generate an HTML report from run_loop.py output.
Takes the JSON output from run_loop.py and generates a visual HTML report
showing each description attempt with check/x for each test case.
Distinguishes between train and test queries.
"""
import argparse
import html
import json
import sys
from pathlib import Path
def generate_html(data: dict, auto_refresh: bool = False, skill_name: str = "") -> str:
"""Generate HTML report from loop output data. If auto_refresh is True, adds a meta refresh tag."""
history = data.get("history", [])
holdout = data.get("holdout", 0)
title_prefix = html.escape(skill_name + " \u2014 ") if skill_name else ""
# Get all unique queries from train and test sets, with should_trigger info
train_queries: list[dict] = []
test_queries: list[dict] = []
if history:
for r in history[0].get("train_results", history[0].get("results", [])):
train_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)})
if history[0].get("test_results"):
for r in history[0].get("test_results", []):
test_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)})
refresh_tag = ' <meta http-equiv="refresh" content="5">\n' if auto_refresh else ""
html_parts = ["""<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
""" + refresh_tag + """ <title>""" + title_prefix + """Skill Description Optimization</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Poppins:wght@500;600&family=Lora:wght@400;500&display=swap" rel="stylesheet">
<style>
body {
font-family: 'Lora', Georgia, serif;
max-width: 100%;
margin: 0 auto;
padding: 20px;
background: #faf9f5;
color: #141413;
}
h1 { font-family: 'Poppins', sans-serif; color: #141413; }
.explainer {
background: white;
padding: 15px;
border-radius: 6px;
margin-bottom: 20px;
border: 1px solid #e8e6dc;
color: #b0aea5;
font-size: 0.875rem;
line-height: 1.6;
}
.summary {
background: white;
padding: 15px;
border-radius: 6px;
margin-bottom: 20px;
border: 1px solid #e8e6dc;
}
.summary p { margin: 5px 0; }
.best { color: #788c5d; font-weight: bold; }
.table-container {
overflow-x: auto;
width: 100%;
}
table {
border-collapse: collapse;
background: white;
border: 1px solid #e8e6dc;
border-radius: 6px;
font-size: 12px;
min-width: 100%;
}
th, td {
padding: 8px;
text-align: left;
border: 1px solid #e8e6dc;
white-space: normal;
word-wrap: break-word;
}
th {
font-family: 'Poppins', sans-serif;
background: #141413;
color: #faf9f5;
font-weight: 500;
}
th.test-col {
background: #6a9bcc;
}
th.query-col { min-width: 200px; }
td.description {
font-family: monospace;
font-size: 11px;
word-wrap: break-word;
max-width: 400px;
}
td.result {
text-align: center;
font-size: 16px;
min-width: 40px;
}
td.test-result {
background: #f0f6fc;
}
.pass { color: #788c5d; }
.fail { color: #c44; }
.rate {
font-size: 9px;
color: #b0aea5;
display: block;
}
tr:hover { background: #faf9f5; }
.score {
display: inline-block;
padding: 2px 6px;
border-radius: 4px;
font-weight: bold;
font-size: 11px;
}
.score-good { background: #eef2e8; color: #788c5d; }
.score-ok { background: #fef3c7; color: #d97706; }
.score-bad { background: #fceaea; color: #c44; }
.train-label { color: #b0aea5; font-size: 10px; }
.test-label { color: #6a9bcc; font-size: 10px; font-weight: bold; }
.best-row { background: #f5f8f2; }
th.positive-col { border-bottom: 3px solid #788c5d; }
th.negative-col { border-bottom: 3px solid #c44; }
th.test-col.positive-col { border-bottom: 3px solid #788c5d; }
th.test-col.negative-col { border-bottom: 3px solid #c44; }
.legend { font-family: 'Poppins', sans-serif; display: flex; gap: 20px; margin-bottom: 10px; font-size: 13px; align-items: center; }
.legend-item { display: flex; align-items: center; gap: 6px; }
.legend-swatch { width: 16px; height: 16px; border-radius: 3px; display: inline-block; }
.swatch-positive { background: #141413; border-bottom: 3px solid #788c5d; }
.swatch-negative { background: #141413; border-bottom: 3px solid #c44; }
.swatch-test { background: #6a9bcc; }
.swatch-train { background: #141413; }
</style>
</head>
<body>
<h1>""" + title_prefix + """Skill Description Optimization</h1>
<div class="explainer">
<strong>Optimizing your skill's description.</strong> This page updates automatically as the agent tests different versions of your skill's description. Each row is an iteration — a new description attempt. The columns show test queries: green checkmarks mean the skill triggered correctly (or correctly didn't trigger), red crosses mean it got it wrong. The "Train" score shows performance on queries used to improve the description; the "Test" score shows performance on held-out queries the optimizer hasn't seen. When it's done, the agent will apply the best-performing description to your skill.
</div>
"""]
# Summary section
best_test_score = data.get('best_test_score')
best_train_score = data.get('best_train_score')
html_parts.append(f"""
<div class="summary">
<p><strong>Original:</strong> {html.escape(data.get('original_description', 'N/A'))}</p>
<p class="best"><strong>Best:</strong> {html.escape(data.get('best_description', 'N/A'))}</p>
<p><strong>Best Score:</strong> {data.get('best_score', 'N/A')} {'(test)' if best_test_score else '(train)'}</p>
<p><strong>Iterations:</strong> {data.get('iterations_run', 0)} | <strong>Train:</strong> {data.get('train_size', '?')} | <strong>Test:</strong> {data.get('test_size', '?')}</p>
</div>
""")
# Legend
html_parts.append("""
<div class="legend">
<span style="font-weight:600">Query columns:</span>
<span class="legend-item"><span class="legend-swatch swatch-positive"></span> Should trigger</span>
<span class="legend-item"><span class="legend-swatch swatch-negative"></span> Should NOT trigger</span>
<span class="legend-item"><span class="legend-swatch swatch-train"></span> Train</span>
<span class="legend-item"><span class="legend-swatch swatch-test"></span> Test</span>
</div>
""")
# Table header
html_parts.append("""
<div class="table-container">
<table>
<thead>
<tr>
<th>Iter</th>
<th>Train</th>
<th>Test</th>
<th class="query-col">Description</th>
""")
# Add column headers for train queries
for qinfo in train_queries:
polarity = "positive-col" if qinfo["should_trigger"] else "negative-col"
html_parts.append(f' <th class="{polarity}">{html.escape(qinfo["query"])}</th>\n')
# Add column headers for test queries (different color)
for qinfo in test_queries:
polarity = "positive-col" if qinfo["should_trigger"] else "negative-col"
html_parts.append(f' <th class="test-col {polarity}">{html.escape(qinfo["query"])}</th>\n')
html_parts.append(""" </tr>
</thead>
<tbody>
""")
# Find best iteration for highlighting
if test_queries:
best_iter = max(history, key=lambda h: h.get("test_passed") or 0).get("iteration")
else:
best_iter = max(history, key=lambda h: h.get("train_passed", h.get("passed", 0))).get("iteration")
# Add rows for each iteration
for h in history:
iteration = h.get("iteration", "?")
train_passed = h.get("train_passed", h.get("passed", 0))
train_total = h.get("train_total", h.get("total", 0))
test_passed = h.get("test_passed")
test_total = h.get("test_total")
description = h.get("description", "")
train_results = h.get("train_results", h.get("results", []))
test_results = h.get("test_results", [])
# Create lookups for results by query
train_by_query = {r["query"]: r for r in train_results}
test_by_query = {r["query"]: r for r in test_results} if test_results else {}
# Compute aggregate correct/total runs across all retries
def aggregate_runs(results: list[dict]) -> tuple[int, int]:
correct = 0
total = 0
for r in results:
runs = r.get("runs", 0)
triggers = r.get("triggers", 0)
total += runs
if r.get("should_trigger", True):
correct += triggers
else:
correct += runs - triggers
return correct, total
train_correct, train_runs = aggregate_runs(train_results)
test_correct, test_runs = aggregate_runs(test_results)
# Determine score classes
def score_class(correct: int, total: int) -> str:
if total > 0:
ratio = correct / total
if ratio >= 0.8:
return "score-good"
elif ratio >= 0.5:
return "score-ok"
return "score-bad"
train_class = score_class(train_correct, train_runs)
test_class = score_class(test_correct, test_runs)
row_class = "best-row" if iteration == best_iter else ""
html_parts.append(f""" <tr class="{row_class}">
<td>{iteration}</td>
<td><span class="score {train_class}">{train_correct}/{train_runs}</span></td>
<td><span class="score {test_class}">{test_correct}/{test_runs}</span></td>
<td class="description">{html.escape(description)}</td>
""")
# Add result for each train query
for qinfo in train_queries:
r = train_by_query.get(qinfo["query"], {})
did_pass = r.get("pass", False)
triggers = r.get("triggers", 0)
runs = r.get("runs", 0)
icon = "✓" if did_pass else "✗"
css_class = "pass" if did_pass else "fail"
html_parts.append(f' <td class="result {css_class}">{icon}<span class="rate">{triggers}/{runs}</span></td>\n')
# Add result for each test query (with different background)
for qinfo in test_queries:
r = test_by_query.get(qinfo["query"], {})
did_pass = r.get("pass", False)
triggers = r.get("triggers", 0)
runs = r.get("runs", 0)
icon = "✓" if did_pass else "✗"
css_class = "pass" if did_pass else "fail"
html_parts.append(f' <td class="result test-result {css_class}">{icon}<span class="rate">{triggers}/{runs}</span></td>\n')
html_parts.append(" </tr>\n")
html_parts.append(""" </tbody>
</table>
</div>
""")
html_parts.append("""
</body>
</html>
""")
return "".join(html_parts)
def main():
parser = argparse.ArgumentParser(description="Generate HTML report from run_loop output")
parser.add_argument("input", help="Path to JSON output from run_loop.py (or - for stdin)")
parser.add_argument("-o", "--output", default=None, help="Output HTML file (default: stdout)")
parser.add_argument("--skill-name", default="", help="Skill name to include in the report title")
args = parser.parse_args()
if args.input == "-":
data = json.load(sys.stdin)
else:
data = json.loads(Path(args.input).read_text())
html_output = generate_html(data, skill_name=args.skill_name)
if args.output:
Path(args.output).write_text(html_output)
print(f"Report written to {args.output}", file=sys.stderr)
else:
print(html_output)
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/scripts/improve_description.py
+246
View File
@@ -0,0 +1,246 @@
#!/usr/bin/env python3
"""Improve a skill description based on eval results.
Takes eval results (from run_eval.py) and generates an improved description
by calling `claude -p` as a subprocess (same auth pattern as run_eval.py).
"""
import argparse
import json
import os
import re
import subprocess
import sys
from pathlib import Path
from scripts.utils import parse_skill_md
def _call_claude(prompt: str, model: str | None, timeout: int = 300) -> str:
"""Run `claude -p` with the prompt on stdin and return the text response.
Prompt goes over stdin (not argv) because it embeds the full SKILL.md
body and can easily exceed comfortable argv length.
"""
cmd = ["claude", "-p", "--output-format", "text"]
if model:
cmd.extend(["--model", model])
# Remove CLAUDECODE env var to allow nesting claude -p inside an
# interactive session. The guard is for interactive terminal conflicts;
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}
result = subprocess.run(
cmd,
input=prompt,
capture_output=True,
text=True,
env=env,
timeout=timeout,
)
if result.returncode != 0:
raise RuntimeError(
f"claude -p exited {result.returncode}\nstderr: {result.stderr}"
)
return result.stdout
def improve_description(
skill_name: str,
skill_content: str,
current_description: str,
eval_results: dict,
history: list[dict],
model: str,
test_results: dict | None = None,
log_dir: Path | None = None,
iteration: int | None = None,
) -> str:
"""Call the agent to improve the description based on eval results."""
failed_triggers = [
r for r in eval_results["results"]
if r["should_trigger"] and not r["pass"]
]
false_triggers = [
r for r in eval_results["results"]
if not r["should_trigger"] and not r["pass"]
]
# Build scores summary
train_score = f"{eval_results['summary']['passed']}/{eval_results['summary']['total']}"
if test_results:
test_score = f"{test_results['summary']['passed']}/{test_results['summary']['total']}"
scores_summary = f"Train: {train_score}, Test: {test_score}"
else:
scores_summary = f"Train: {train_score}"
prompt = f"""You are optimizing a skill description for a skill called "{skill_name}". A "skill" is sort of like a prompt, but with progressive disclosure -- there's a title and description that the agent sees when deciding whether to use the skill, and then if it does use the skill, it reads the .md file which has lots more details and potentially links to other resources in the skill folder like helper files and scripts and additional documentation or examples.
The description appears in the agent's "available_skills" list. When a user sends a query, the agent decides whether to invoke the skill based solely on the title and on this description. Your goal is to write a description that triggers for relevant queries, and doesn't trigger for irrelevant ones.
Here's the current description:
<current_description>
"{current_description}"
</current_description>
Current scores ({scores_summary}):
<scores_summary>
"""
if failed_triggers:
prompt += "FAILED TO TRIGGER (should have triggered but didn't):\n"
for r in failed_triggers:
prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n'
prompt += "\n"
if false_triggers:
prompt += "FALSE TRIGGERS (triggered but shouldn't have):\n"
for r in false_triggers:
prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n'
prompt += "\n"
if history:
prompt += "PREVIOUS ATTEMPTS (do NOT repeat these — try something structurally different):\n\n"
for h in history:
train_s = f"{h.get('train_passed', h.get('passed', 0))}/{h.get('train_total', h.get('total', 0))}"
test_s = f"{h.get('test_passed', '?')}/{h.get('test_total', '?')}" if h.get('test_passed') is not None else None
score_str = f"train={train_s}" + (f", test={test_s}" if test_s else "")
prompt += f'<attempt {score_str}>\n'
prompt += f'Description: "{h["description"]}"\n'
if "results" in h:
prompt += "Train results:\n"
for r in h["results"]:
status = "PASS" if r["pass"] else "FAIL"
prompt += f' [{status}] "{r["query"][:80]}" (triggered {r["triggers"]}/{r["runs"]})\n'
if h.get("note"):
prompt += f'Note: {h["note"]}\n'
prompt += "</attempt>\n\n"
prompt += f"""</scores_summary>
Skill content (for context on what the skill does):
<skill_content>
{skill_content}
</skill_content>
Based on the failures, write a new and improved description that is more likely to trigger correctly. When I say "based on the failures", it's a bit of a tricky line to walk because we don't want to overfit to the specific cases you're seeing. So what I DON'T want you to do is produce an ever-expanding list of specific queries that this skill should or shouldn't trigger for. Instead, try to generalize from the failures to broader categories of user intent and situations where this skill would be useful or not useful. The reason for this is twofold:
1. Avoid overfitting
2. The list might get loooong and it's injected into ALL queries and there might be a lot of skills, so we don't want to blow too much space on any given description.
Concretely, your description should not be more than about 100-200 words, even if that comes at the cost of accuracy. There is a hard limit of 1024 characters — descriptions over that will be truncated, so stay comfortably under it.
Here are some tips that we've found to work well in writing these descriptions:
- The skill should be phrased in the imperative -- "Use this skill for" rather than "this skill does"
- The skill description should focus on the user's intent, what they are trying to achieve, vs. the implementation details of how the skill works.
- The description competes with other skills for the agent's attention — make it distinctive and immediately recognizable.
- If you're getting lots of failures after repeated attempts, change things up. Try different sentence structures or wordings.
I'd encourage you to be creative and mix up the style in different iterations since you'll have multiple opportunities to try different approaches and we'll just grab the highest-scoring one at the end.
Please respond with only the new description text in <new_description> tags, nothing else."""
text = _call_claude(prompt, model)
match = re.search(r"<new_description>(.*?)</new_description>", text, re.DOTALL)
description = match.group(1).strip().strip('"') if match else text.strip().strip('"')
transcript: dict = {
"iteration": iteration,
"prompt": prompt,
"response": text,
"parsed_description": description,
"char_count": len(description),
"over_limit": len(description) > 1024,
}
# Safety net: the prompt already states the 1024-char hard limit, but if
# the model blew past it anyway, make one fresh single-turn call that
# quotes the too-long version and asks for a shorter rewrite. (The old
# SDK path did this as a true multi-turn; `claude -p` is one-shot, so we
# inline the prior output into the new prompt instead.)
if len(description) > 1024:
shorten_prompt = (
f"{prompt}\n\n"
f"---\n\n"
f"A previous attempt produced this description, which at "
f"{len(description)} characters is over the 1024-character hard limit:\n\n"
f'"{description}"\n\n'
f"Rewrite it to be under 1024 characters while keeping the most "
f"important trigger words and intent coverage. Respond with only "
f"the new description in <new_description> tags."
)
shorten_text = _call_claude(shorten_prompt, model)
match = re.search(r"<new_description>(.*?)</new_description>", shorten_text, re.DOTALL)
shortened = match.group(1).strip().strip('"') if match else shorten_text.strip().strip('"')
transcript["rewrite_prompt"] = shorten_prompt
transcript["rewrite_response"] = shorten_text
transcript["rewrite_description"] = shortened
transcript["rewrite_char_count"] = len(shortened)
description = shortened
transcript["final_description"] = description
if log_dir:
log_dir.mkdir(parents=True, exist_ok=True)
log_file = log_dir / f"improve_iter_{iteration or 'unknown'}.json"
log_file.write_text(json.dumps(transcript, indent=2))
return description
def main():
parser = argparse.ArgumentParser(description="Improve a skill description based on eval results")
parser.add_argument("--eval-results", required=True, help="Path to eval results JSON (from run_eval.py)")
parser.add_argument("--skill-path", required=True, help="Path to skill directory")
parser.add_argument("--history", default=None, help="Path to history JSON (previous attempts)")
parser.add_argument("--model", required=True, help="Model for improvement")
parser.add_argument("--verbose", action="store_true", help="Print thinking to stderr")
args = parser.parse_args()
skill_path = Path(args.skill_path)
if not (skill_path / "SKILL.md").exists():
print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
sys.exit(1)
eval_results = json.loads(Path(args.eval_results).read_text())
history = []
if args.history:
history = json.loads(Path(args.history).read_text())
name, _, content = parse_skill_md(skill_path)
current_description = eval_results["description"]
if args.verbose:
print(f"Current: {current_description}", file=sys.stderr)
print(f"Score: {eval_results['summary']['passed']}/{eval_results['summary']['total']}", file=sys.stderr)
new_description = improve_description(
skill_name=name,
skill_content=content,
current_description=current_description,
eval_results=eval_results,
history=history,
model=args.model,
)
if args.verbose:
print(f"Improved: {new_description}", file=sys.stderr)
# Output as JSON with both the new description and updated history
output = {
"description": new_description,
"history": history + [{
"description": current_description,
"passed": eval_results["summary"]["passed"],
"failed": eval_results["summary"]["failed"],
"total": eval_results["summary"]["total"],
"results": eval_results["results"],
}],
}
print(json.dumps(output, indent=2))
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/scripts/package_skill.py
Executable
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""
Skill Packager - Creates a distributable .skill file of a skill folder
Usage:
python utils/package_skill.py <path/to/skill-folder> [output-directory]
Example:
python utils/package_skill.py skills/public/my-skill
python utils/package_skill.py skills/public/my-skill ./dist
"""
import fnmatch
import sys
import zipfile
from pathlib import Path
from scripts.quick_validate import validate_skill
# Patterns to exclude when packaging skills.
EXCLUDE_DIRS = {"__pycache__", "node_modules"}
EXCLUDE_GLOBS = {"*.pyc"}
EXCLUDE_FILES = {".DS_Store"}
# Directories excluded only at the skill root (not when nested deeper).
ROOT_EXCLUDE_DIRS = {"evals"}
def should_exclude(rel_path: Path) -> bool:
"""Check if a path should be excluded from packaging."""
parts = rel_path.parts
if any(part in EXCLUDE_DIRS for part in parts):
return True
# rel_path is relative to skill_path.parent, so parts[0] is the skill
# folder name and parts[1] (if present) is the first subdir.
if len(parts) > 1 and parts[1] in ROOT_EXCLUDE_DIRS:
return True
name = rel_path.name
if name in EXCLUDE_FILES:
return True
return any(fnmatch.fnmatch(name, pat) for pat in EXCLUDE_GLOBS)
def package_skill(skill_path, output_dir=None):
"""
Package a skill folder into a .skill file.
Args:
skill_path: Path to the skill folder
output_dir: Optional output directory for the .skill file (defaults to current directory)
Returns:
Path to the created .skill file, or None if error
"""
skill_path = Path(skill_path).resolve()
# Validate skill folder exists
if not skill_path.exists():
print(f"❌ Error: Skill folder not found: {skill_path}")
return None
if not skill_path.is_dir():
print(f"❌ Error: Path is not a directory: {skill_path}")
return None
# Validate SKILL.md exists
skill_md = skill_path / "SKILL.md"
if not skill_md.exists():
print(f"❌ Error: SKILL.md not found in {skill_path}")
return None
# Run validation before packaging
print("🔍 Validating skill...")
valid, message = validate_skill(skill_path)
if not valid:
print(f"❌ Validation failed: {message}")
print(" Please fix the validation errors before packaging.")
return None
print(f"✅ {message}\n")
# Determine output location
skill_name = skill_path.name
if output_dir:
output_path = Path(output_dir).resolve()
output_path.mkdir(parents=True, exist_ok=True)
else:
output_path = Path.cwd()
skill_filename = output_path / f"{skill_name}.skill"
# Create the .skill file (zip format)
try:
with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf:
# Walk through the skill directory, excluding build artifacts
for file_path in skill_path.rglob('*'):
if not file_path.is_file():
continue
arcname = file_path.relative_to(skill_path.parent)
if should_exclude(arcname):
print(f" Skipped: {arcname}")
continue
zipf.write(file_path, arcname)
print(f" Added: {arcname}")
print(f"\n✅ Successfully packaged skill to: {skill_filename}")
return skill_filename
except Exception as e:
print(f"❌ Error creating .skill file: {e}")
return None
def main():
if len(sys.argv) < 2:
print("Usage: python utils/package_skill.py <path/to/skill-folder> [output-directory]")
print("\nExample:")
print(" python utils/package_skill.py skills/public/my-skill")
print(" python utils/package_skill.py skills/public/my-skill ./dist")
sys.exit(1)
skill_path = sys.argv[1]
output_dir = sys.argv[2] if len(sys.argv) > 2 else None
print(f"📦 Packaging skill: {skill_path}")
if output_dir:
print(f" Output directory: {output_dir}")
print()
result = package_skill(skill_path, output_dir)
if result:
sys.exit(0)
else:
sys.exit(1)
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/scripts/quick_validate.py
Executable
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env python3
"""
Quick validation script for skills - minimal version
"""
import sys
import os
import re
import yaml
from pathlib import Path
def validate_skill(skill_path):
"""Basic validation of a skill"""
skill_path = Path(skill_path)
# Check SKILL.md exists
skill_md = skill_path / 'SKILL.md'
if not skill_md.exists():
return False, "SKILL.md not found"
# Read and validate frontmatter
content = skill_md.read_text()
if not content.startswith('---'):
return False, "No YAML frontmatter found"
# Extract frontmatter
match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
if not match:
return False, "Invalid frontmatter format"
frontmatter_text = match.group(1)
# Parse YAML frontmatter
try:
frontmatter = yaml.safe_load(frontmatter_text)
if not isinstance(frontmatter, dict):
return False, "Frontmatter must be a YAML dictionary"
except yaml.YAMLError as e:
return False, f"Invalid YAML in frontmatter: {e}"
# Define allowed properties
ALLOWED_PROPERTIES = {'name', 'description', 'license', 'allowed-tools', 'metadata', 'compatibility'}
# Check for unexpected properties (excluding nested keys under metadata)
unexpected_keys = set(frontmatter.keys()) - ALLOWED_PROPERTIES
if unexpected_keys:
return False, (
f"Unexpected key(s) in SKILL.md frontmatter: {', '.join(sorted(unexpected_keys))}. "
f"Allowed properties are: {', '.join(sorted(ALLOWED_PROPERTIES))}"
)
# Check required fields
if 'name' not in frontmatter:
return False, "Missing 'name' in frontmatter"
if 'description' not in frontmatter:
return False, "Missing 'description' in frontmatter"
# Extract name for validation
name = frontmatter.get('name', '')
if not isinstance(name, str):
return False, f"Name must be a string, got {type(name).__name__}"
name = name.strip()
if name:
# Check naming convention (kebab-case: lowercase with hyphens)
if not re.match(r'^[a-z0-9-]+$', name):
return False, f"Name '{name}' should be kebab-case (lowercase letters, digits, and hyphens only)"
if name.startswith('-') or name.endswith('-') or '--' in name:
return False, f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens"
# Check name length (max 64 characters per spec)
if len(name) > 64:
return False, f"Name is too long ({len(name)} characters). Maximum is 64 characters."
# Extract and validate description
description = frontmatter.get('description', '')
if not isinstance(description, str):
return False, f"Description must be a string, got {type(description).__name__}"
description = description.strip()
if description:
# Check for angle brackets
if '<' in description or '>' in description:
return False, "Description cannot contain angle brackets (< or >)"
# Check description length (max 1024 characters per spec)
if len(description) > 1024:
return False, f"Description is too long ({len(description)} characters). Maximum is 1024 characters."
# Validate compatibility field if present (optional)
compatibility = frontmatter.get('compatibility', '')
if compatibility:
if not isinstance(compatibility, str):
return False, f"Compatibility must be a string, got {type(compatibility).__name__}"
if len(compatibility) > 500:
return False, f"Compatibility is too long ({len(compatibility)} characters). Maximum is 500 characters."
return True, "Skill is valid!"
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python quick_validate.py <skill_directory>")
sys.exit(1)
valid, message = validate_skill(sys.argv[1])
print(message)
sys.exit(0 if valid else 1)
resources/bundled/skills/create-skill/scripts/run_eval.py
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env python3
"""Run trigger evaluation for a skill description.
Tests whether a skill's description causes the agent to trigger (read the skill)
for a set of queries. Outputs results as JSON.
"""
import argparse
import json
import os
import select
import subprocess
import sys
import time
import uuid
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path
from scripts.utils import parse_skill_md
def find_project_root() -> Path:
"""Find the project root by walking up from cwd looking for .claude/.
Mimics how the agent discovers its project root, so the command file
we create ends up where claude -p will look for it.
"""
current = Path.cwd()
for parent in [current, *current.parents]:
if (parent / ".claude").is_dir():
return parent
return current
def run_single_query(
query: str,
skill_name: str,
skill_description: str,
timeout: int,
project_root: str,
model: str | None = None,
) -> bool:
"""Run a single query and return whether the skill was triggered.
Creates a command file in .claude/commands/ so it appears in the agent's
available_skills list, then runs `claude -p` with the raw query.
Uses --include-partial-messages to detect triggering early from
stream events (content_block_start) rather than waiting for the
full assistant message, which only arrives after tool execution.
"""
unique_id = uuid.uuid4().hex[:8]
clean_name = f"{skill_name}-skill-{unique_id}"
project_commands_dir = Path(project_root) / ".claude" / "commands"
command_file = project_commands_dir / f"{clean_name}.md"
try:
project_commands_dir.mkdir(parents=True, exist_ok=True)
# Use YAML block scalar to avoid breaking on quotes in description
indented_desc = "\n ".join(skill_description.split("\n"))
command_content = (
f"---\n"
f"description: |\n"
f" {indented_desc}\n"
f"---\n\n"
f"# {skill_name}\n\n"
f"This skill handles: {skill_description}\n"
)
command_file.write_text(command_content)
cmd = [
"claude",
"-p", query,
"--output-format", "stream-json",
"--verbose",
"--include-partial-messages",
]
if model:
cmd.extend(["--model", model])
# Remove CLAUDECODE env var to allow nesting claude -p inside an
# interactive session. The guard is for interactive terminal conflicts;
# programmatic subprocess usage is safe.
env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}
process = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
cwd=project_root,
env=env,
)
triggered = False
start_time = time.time()
buffer = ""
# Track state for stream event detection
pending_tool_name = None
accumulated_json = ""
try:
while time.time() - start_time < timeout:
if process.poll() is not None:
remaining = process.stdout.read()
if remaining:
buffer += remaining.decode("utf-8", errors="replace")
break
ready, _, _ = select.select([process.stdout], [], [], 1.0)
if not ready:
continue
chunk = os.read(process.stdout.fileno(), 8192)
if not chunk:
break
buffer += chunk.decode("utf-8", errors="replace")
while "\n" in buffer:
line, buffer = buffer.split("\n", 1)
line = line.strip()
if not line:
continue
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
# Early detection via stream events
if event.get("type") == "stream_event":
se = event.get("event", {})
se_type = se.get("type", "")
if se_type == "content_block_start":
cb = se.get("content_block", {})
if cb.get("type") == "tool_use":
tool_name = cb.get("name", "")
if tool_name in ("Skill", "Read"):
pending_tool_name = tool_name
accumulated_json = ""
else:
return False
elif se_type == "content_block_delta" and pending_tool_name:
delta = se.get("delta", {})
if delta.get("type") == "input_json_delta":
accumulated_json += delta.get("partial_json", "")
if clean_name in accumulated_json:
return True
elif se_type in ("content_block_stop", "message_stop"):
if pending_tool_name:
return clean_name in accumulated_json
if se_type == "message_stop":
return False
# Fallback: full assistant message
elif event.get("type") == "assistant":
message = event.get("message", {})
for content_item in message.get("content", []):
if content_item.get("type") != "tool_use":
continue
tool_name = content_item.get("name", "")
tool_input = content_item.get("input", {})
if tool_name == "Skill" and clean_name in tool_input.get("skill", ""):
triggered = True
elif tool_name == "Read" and clean_name in tool_input.get("file_path", ""):
triggered = True
return triggered
elif event.get("type") == "result":
return triggered
finally:
# Clean up process on any exit path (return, exception, timeout)
if process.poll() is None:
process.kill()
process.wait()
return triggered
finally:
if command_file.exists():
command_file.unlink()
def run_eval(
eval_set: list[dict],
skill_name: str,
description: str,
num_workers: int,
timeout: int,
project_root: Path,
runs_per_query: int = 1,
trigger_threshold: float = 0.5,
model: str | None = None,
) -> dict:
"""Run the full eval set and return results."""
results = []
with ProcessPoolExecutor(max_workers=num_workers) as executor:
future_to_info = {}
for item in eval_set:
for run_idx in range(runs_per_query):
future = executor.submit(
run_single_query,
item["query"],
skill_name,
description,
timeout,
str(project_root),
model,
)
future_to_info[future] = (item, run_idx)
query_triggers: dict[str, list[bool]] = {}
query_items: dict[str, dict] = {}
for future in as_completed(future_to_info):
item, _ = future_to_info[future]
query = item["query"]
query_items[query] = item
if query not in query_triggers:
query_triggers[query] = []
try:
query_triggers[query].append(future.result())
except Exception as e:
print(f"Warning: query failed: {e}", file=sys.stderr)
query_triggers[query].append(False)
for query, triggers in query_triggers.items():
item = query_items[query]
trigger_rate = sum(triggers) / len(triggers)
should_trigger = item["should_trigger"]
if should_trigger:
did_pass = trigger_rate >= trigger_threshold
else:
did_pass = trigger_rate < trigger_threshold
results.append({
"query": query,
"should_trigger": should_trigger,
"trigger_rate": trigger_rate,
"triggers": sum(triggers),
"runs": len(triggers),
"pass": did_pass,
})
passed = sum(1 for r in results if r["pass"])
total = len(results)
return {
"skill_name": skill_name,
"description": description,
"results": results,
"summary": {
"total": total,
"passed": passed,
"failed": total - passed,
},
}
def main():
parser = argparse.ArgumentParser(description="Run trigger evaluation for a skill description")
parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file")
parser.add_argument("--skill-path", required=True, help="Path to skill directory")
parser.add_argument("--description", default=None, help="Override description to test")
parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers")
parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds")
parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query")
parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold")
parser.add_argument("--model", default=None, help="Model to use for claude -p (default: user's configured model)")
parser.add_argument("--verbose", action="store_true", help="Print progress to stderr")
args = parser.parse_args()
eval_set = json.loads(Path(args.eval_set).read_text())
skill_path = Path(args.skill_path)
if not (skill_path / "SKILL.md").exists():
print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
sys.exit(1)
name, original_description, content = parse_skill_md(skill_path)
description = args.description or original_description
project_root = find_project_root()
if args.verbose:
print(f"Evaluating: {description}", file=sys.stderr)
output = run_eval(
eval_set=eval_set,
skill_name=name,
description=description,
num_workers=args.num_workers,
timeout=args.timeout,
project_root=project_root,
runs_per_query=args.runs_per_query,
trigger_threshold=args.trigger_threshold,
model=args.model,
)
if args.verbose:
summary = output["summary"]
print(f"Results: {summary['passed']}/{summary['total']} passed", file=sys.stderr)
for r in output["results"]:
status = "PASS" if r["pass"] else "FAIL"
rate_str = f"{r['triggers']}/{r['runs']}"
print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:70]}", file=sys.stderr)
print(json.dumps(output, indent=2))
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/scripts/run_loop.py
+328
View File
@@ -0,0 +1,328 @@
#!/usr/bin/env python3
"""Run the eval + improve loop until all pass or max iterations reached.
Combines run_eval.py and improve_description.py in a loop, tracking history
and returning the best description found. Supports train/test split to prevent
overfitting.
"""
import argparse
import json
import random
import sys
import tempfile
import time
import webbrowser
from pathlib import Path
from scripts.generate_report import generate_html
from scripts.improve_description import improve_description
from scripts.run_eval import find_project_root, run_eval
from scripts.utils import parse_skill_md
def split_eval_set(eval_set: list[dict], holdout: float, seed: int = 42) -> tuple[list[dict], list[dict]]:
"""Split eval set into train and test sets, stratified by should_trigger."""
random.seed(seed)
# Separate by should_trigger
trigger = [e for e in eval_set if e["should_trigger"]]
no_trigger = [e for e in eval_set if not e["should_trigger"]]
# Shuffle each group
random.shuffle(trigger)
random.shuffle(no_trigger)
# Calculate split points
n_trigger_test = max(1, int(len(trigger) * holdout))
n_no_trigger_test = max(1, int(len(no_trigger) * holdout))
# Split
test_set = trigger[:n_trigger_test] + no_trigger[:n_no_trigger_test]
train_set = trigger[n_trigger_test:] + no_trigger[n_no_trigger_test:]
return train_set, test_set
def run_loop(
eval_set: list[dict],
skill_path: Path,
description_override: str | None,
num_workers: int,
timeout: int,
max_iterations: int,
runs_per_query: int,
trigger_threshold: float,
holdout: float,
model: str,
verbose: bool,
live_report_path: Path | None = None,
log_dir: Path | None = None,
) -> dict:
"""Run the eval + improvement loop."""
project_root = find_project_root()
name, original_description, content = parse_skill_md(skill_path)
current_description = description_override or original_description
# Split into train/test if holdout > 0
if holdout > 0:
train_set, test_set = split_eval_set(eval_set, holdout)
if verbose:
print(f"Split: {len(train_set)} train, {len(test_set)} test (holdout={holdout})", file=sys.stderr)
else:
train_set = eval_set
test_set = []
history = []
exit_reason = "unknown"
for iteration in range(1, max_iterations + 1):
if verbose:
print(f"\n{'='*60}", file=sys.stderr)
print(f"Iteration {iteration}/{max_iterations}", file=sys.stderr)
print(f"Description: {current_description}", file=sys.stderr)
print(f"{'='*60}", file=sys.stderr)
# Evaluate train + test together in one batch for parallelism
all_queries = train_set + test_set
t0 = time.time()
all_results = run_eval(
eval_set=all_queries,
skill_name=name,
description=current_description,
num_workers=num_workers,
timeout=timeout,
project_root=project_root,
runs_per_query=runs_per_query,
trigger_threshold=trigger_threshold,
model=model,
)
eval_elapsed = time.time() - t0
# Split results back into train/test by matching queries
train_queries_set = {q["query"] for q in train_set}
train_result_list = [r for r in all_results["results"] if r["query"] in train_queries_set]
test_result_list = [r for r in all_results["results"] if r["query"] not in train_queries_set]
train_passed = sum(1 for r in train_result_list if r["pass"])
train_total = len(train_result_list)
train_summary = {"passed": train_passed, "failed": train_total - train_passed, "total": train_total}
train_results = {"results": train_result_list, "summary": train_summary}
if test_set:
test_passed = sum(1 for r in test_result_list if r["pass"])
test_total = len(test_result_list)
test_summary = {"passed": test_passed, "failed": test_total - test_passed, "total": test_total}
test_results = {"results": test_result_list, "summary": test_summary}
else:
test_results = None
test_summary = None
history.append({
"iteration": iteration,
"description": current_description,
"train_passed": train_summary["passed"],
"train_failed": train_summary["failed"],
"train_total": train_summary["total"],
"train_results": train_results["results"],
"test_passed": test_summary["passed"] if test_summary else None,
"test_failed": test_summary["failed"] if test_summary else None,
"test_total": test_summary["total"] if test_summary else None,
"test_results": test_results["results"] if test_results else None,
# For backward compat with report generator
"passed": train_summary["passed"],
"failed": train_summary["failed"],
"total": train_summary["total"],
"results": train_results["results"],
})
# Write live report if path provided
if live_report_path:
partial_output = {
"original_description": original_description,
"best_description": current_description,
"best_score": "in progress",
"iterations_run": len(history),
"holdout": holdout,
"train_size": len(train_set),
"test_size": len(test_set),
"history": history,
}
live_report_path.write_text(generate_html(partial_output, auto_refresh=True, skill_name=name))
if verbose:
def print_eval_stats(label, results, elapsed):
pos = [r for r in results if r["should_trigger"]]
neg = [r for r in results if not r["should_trigger"]]
tp = sum(r["triggers"] for r in pos)
pos_runs = sum(r["runs"] for r in pos)
fn = pos_runs - tp
fp = sum(r["triggers"] for r in neg)
neg_runs = sum(r["runs"] for r in neg)
tn = neg_runs - fp
total = tp + tn + fp + fn
precision = tp / (tp + fp) if (tp + fp) > 0 else 1.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 1.0
accuracy = (tp + tn) / total if total > 0 else 0.0
print(f"{label}: {tp+tn}/{total} correct, precision={precision:.0%} recall={recall:.0%} accuracy={accuracy:.0%} ({elapsed:.1f}s)", file=sys.stderr)
for r in results:
status = "PASS" if r["pass"] else "FAIL"
rate_str = f"{r['triggers']}/{r['runs']}"
print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:60]}", file=sys.stderr)
print_eval_stats("Train", train_results["results"], eval_elapsed)
if test_summary:
print_eval_stats("Test ", test_results["results"], 0)
if train_summary["failed"] == 0:
exit_reason = f"all_passed (iteration {iteration})"
if verbose:
print(f"\nAll train queries passed on iteration {iteration}!", file=sys.stderr)
break
if iteration == max_iterations:
exit_reason = f"max_iterations ({max_iterations})"
if verbose:
print(f"\nMax iterations reached ({max_iterations}).", file=sys.stderr)
break
# Improve the description based on train results
if verbose:
print(f"\nImproving description...", file=sys.stderr)
t0 = time.time()
# Strip test scores from history so improvement model can't see them
blinded_history = [
{k: v for k, v in h.items() if not k.startswith("test_")}
for h in history
]
new_description = improve_description(
skill_name=name,
skill_content=content,
current_description=current_description,
eval_results=train_results,
history=blinded_history,
model=model,
log_dir=log_dir,
iteration=iteration,
)
improve_elapsed = time.time() - t0
if verbose:
print(f"Proposed ({improve_elapsed:.1f}s): {new_description}", file=sys.stderr)
current_description = new_description
# Find the best iteration by TEST score (or train if no test set)
if test_set:
best = max(history, key=lambda h: h["test_passed"] or 0)
best_score = f"{best['test_passed']}/{best['test_total']}"
else:
best = max(history, key=lambda h: h["train_passed"])
best_score = f"{best['train_passed']}/{best['train_total']}"
if verbose:
print(f"\nExit reason: {exit_reason}", file=sys.stderr)
print(f"Best score: {best_score} (iteration {best['iteration']})", file=sys.stderr)
return {
"exit_reason": exit_reason,
"original_description": original_description,
"best_description": best["description"],
"best_score": best_score,
"best_train_score": f"{best['train_passed']}/{best['train_total']}",
"best_test_score": f"{best['test_passed']}/{best['test_total']}" if test_set else None,
"final_description": current_description,
"iterations_run": len(history),
"holdout": holdout,
"train_size": len(train_set),
"test_size": len(test_set),
"history": history,
}
def main():
parser = argparse.ArgumentParser(description="Run eval + improve loop")
parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file")
parser.add_argument("--skill-path", required=True, help="Path to skill directory")
parser.add_argument("--description", default=None, help="Override starting description")
parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers")
parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds")
parser.add_argument("--max-iterations", type=int, default=5, help="Max improvement iterations")
parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query")
parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold")
parser.add_argument("--holdout", type=float, default=0.4, help="Fraction of eval set to hold out for testing (0 to disable)")
parser.add_argument("--model", required=True, help="Model for improvement")
parser.add_argument("--verbose", action="store_true", help="Print progress to stderr")
parser.add_argument("--report", default="auto", help="Generate HTML report at this path (default: 'auto' for temp file, 'none' to disable)")
parser.add_argument("--results-dir", default=None, help="Save all outputs (results.json, report.html, log.txt) to a timestamped subdirectory here")
args = parser.parse_args()
eval_set = json.loads(Path(args.eval_set).read_text())
skill_path = Path(args.skill_path)
if not (skill_path / "SKILL.md").exists():
print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
sys.exit(1)
name, _, _ = parse_skill_md(skill_path)
# Set up live report path
if args.report != "none":
if args.report == "auto":
timestamp = time.strftime("%Y%m%d_%H%M%S")
live_report_path = Path(tempfile.gettempdir()) / f"skill_description_report_{skill_path.name}_{timestamp}.html"
else:
live_report_path = Path(args.report)
# Open the report immediately so the user can watch
live_report_path.write_text("<html><body><h1>Starting optimization loop...</h1><meta http-equiv='refresh' content='5'></body></html>")
webbrowser.open(str(live_report_path))
else:
live_report_path = None
# Determine output directory (create before run_loop so logs can be written)
if args.results_dir:
timestamp = time.strftime("%Y-%m-%d_%H%M%S")
results_dir = Path(args.results_dir) / timestamp
results_dir.mkdir(parents=True, exist_ok=True)
else:
results_dir = None
log_dir = results_dir / "logs" if results_dir else None
output = run_loop(
eval_set=eval_set,
skill_path=skill_path,
description_override=args.description,
num_workers=args.num_workers,
timeout=args.timeout,
max_iterations=args.max_iterations,
runs_per_query=args.runs_per_query,
trigger_threshold=args.trigger_threshold,
holdout=args.holdout,
model=args.model,
verbose=args.verbose,
live_report_path=live_report_path,
log_dir=log_dir,
)
# Save JSON output
json_output = json.dumps(output, indent=2)
print(json_output)
if results_dir:
(results_dir / "results.json").write_text(json_output)
# Write final HTML report (without auto-refresh)
if live_report_path:
live_report_path.write_text(generate_html(output, auto_refresh=False, skill_name=name))
print(f"\nReport: {live_report_path}", file=sys.stderr)
if results_dir and live_report_path:
(results_dir / "report.html").write_text(generate_html(output, auto_refresh=False, skill_name=name))
if results_dir:
print(f"Results saved to: {results_dir}", file=sys.stderr)
if __name__ == "__main__":
main()
resources/bundled/skills/create-skill/scripts/utils.py
+47
View File
@@ -0,0 +1,47 @@
"""Shared utilities for skill-creator scripts."""
from pathlib import Path
def parse_skill_md(skill_path: Path) -> tuple[str, str, str]:
"""Parse a SKILL.md file, returning (name, description, full_content)."""
content = (skill_path / "SKILL.md").read_text()
lines = content.split("\n")
if lines[0].strip() != "---":
raise ValueError("SKILL.md missing frontmatter (no opening ---)")
end_idx = None
for i, line in enumerate(lines[1:], start=1):
if line.strip() == "---":
end_idx = i
break
if end_idx is None:
raise ValueError("SKILL.md missing frontmatter (no closing ---)")
name = ""
description = ""
frontmatter_lines = lines[1:end_idx]
i = 0
while i < len(frontmatter_lines):
line = frontmatter_lines[i]
if line.startswith("name:"):
name = line[len("name:"):].strip().strip('"').strip("'")
elif line.startswith("description:"):
value = line[len("description:"):].strip()
# Handle YAML multiline indicators (>, |, >-, |-)
if value in (">", "|", ">-", "|-"):
continuation_lines: list[str] = []
i += 1
while i < len(frontmatter_lines) and (frontmatter_lines[i].startswith(" ") or frontmatter_lines[i].startswith("\t")):
continuation_lines.append(frontmatter_lines[i].strip())
i += 1
description = " ".join(continuation_lines)
continue
else:
description = value.strip('"').strip("'")
i += 1
return name, description, content
resources/bundled/skills/create-tab-config/SKILL.md
+29
View File
@@ -0,0 +1,29 @@
---
name: create-tab-config
description: Create new Warp tab config TOML files from natural-language requests. Use when the user wants a new tab config, a new tab layout, or asks for a slash command to generate a tab config.
---
# create-tab-config
Create a new Warp tab config based on what the user wants.
## Required context
- Use the `tab-configs` skill as the canonical source of truth for:
- schema details
- validation rules
- examples
- common layout patterns
## Workflow
1. Understand what the user wants to create.
2. If important details are missing, use the `ask_user_question` tool to clarify them before writing anything. Do not guess about layout, commands, directories, parameters, or close-time behavior.
3. Generate valid TOML that matches the `tab-configs` schema.
4. Determine the correct tab config directory for the user's Warp build.
Tab configs live under `~/.warp/tab_configs/` for standard builds. Non-stable builds use a channel-specific variant (e.g. `~/.warp-<channel>/tab_configs/`).
To find the correct directory, run `ls -d ~/.warp*/` to list the available Warp data directories and pick the one that corresponds to the running build. When in doubt, ask the user which build they are using.
Create the `tab_configs/` subdirectory if it does not exist.
Write the file using a descriptive snake_case filename ending in `.toml`.
5. If the intended filename might conflict with an existing config and it is unclear whether to overwrite or create a new file, use the `ask_user_question` tool.
6. Briefly explain what you created, including the layout and any commands or parameters.
resources/bundled/skills/feedback/SKILL.md
+240
View File
@@ -0,0 +1,240 @@
---
name: feedback
description: Turn rough feedback about the Warp app into a filed GitHub issue or duplicate-issue response for `warpdotdev/warp`. Use when the user shares a Warp bug report, regression, confusing UX note, feature gap, or short complaint and wants it clarified into reproduction steps, expected vs actual behavior, concrete impact, and grounded source references from local Warp repos when available.
---
# Feedback
Turn rough Warp app feedback into a crisp filed issue or duplicate-issue response for `warpdotdev/warp`.
Treat Warp client, Warp app, Warp terminal, and Warp UX feedback as `warpdotdev/warp` unless the user clearly asks for a different destination.
## Overview
- Use the `gh` CLI to search for and fetch code from `warpdotdev/warp` when product or implementation context would improve the report.
- If those repos are not available, draft the issue from the user's report alone rather than blocking on more context.
- This skill is strictly for issue filing and duplicate detection. Never modify code, generate patches, propose implementation diffs, or open a pull request as part of this workflow.
- If you cannot file an issue, say so explicitly in the response instead of attempting another side effect.
- The helper script applies the `in-app-feedback` label to filed issues for tracking.
## Code access boundaries
This skill runs in environments where Warp source code may be present in the current working directory or on disk. The following rules apply unconditionally regardless of what source code is visible locally:
- **Never write, edit, or delete any source file.** Do not use Edit, Write, or any tool that modifies files on disk, even if asked to do so as part of filing feedback or "while you're in the code."
- **Never create or modify any git artifact.** Do not stage files, create commits, create branches, produce patches, or modify any git state.
- **Local source code is read-only context at most.** You may read local Warp source files (e.g., with Read or grep) only to find concrete file paths, symbol names, or setting names that would make a source reference more precise. Never read local files to produce a code fix or diff.
- **Prefer `gh` CLI for code lookups.** Use `gh` to search and fetch code from `warpdotdev/warp` rather than reading the local checkout when both are available.
- **The presence of local source code is not an invitation to fix it.** Observing that you are inside a Warp source directory changes nothing about the permitted outputs of this skill: issue filed, duplicate found, or explicit refusal.
Load the bundled reference files only when relevant:
- platform and OS-version resolution, plus operating-system-specific behavior: `references/platforms.md`
- logs and crash artifacts: `references/logs.md`
- output calibration examples: `references/examples.md`
## Workflow
### 1. Confirm scope and classify the report
- This skill only handles feedback about the Warp product that could plausibly be addressed by a code or docs change to the Warp client, server, or SDKs. Before drafting anything, verify the request is in scope.
- **Decline and exit the skill (do not call the helper script) when the report is clearly out of scope.** Out-of-scope categories include, but are not limited to:
- Account, billing, subscription, plan, credits, refund, or invoice questions.
- Login, authentication, SSO, password, or session-expiry problems.
- Requests to contact human support, sales, or legal.
- General venting, praise, or commentary with no actionable product signal.
- Questions about third-party tools or the user's own shell, machine, or network configuration that are not about Warp's behavior.
- Anything the user explicitly says is not about Warp, or that they just want to talk through.
- When you decline, respond in one or two sentences that (a) say you won't file an issue, (b) name the reason in plain language, and (c) point the user at the right channel: account/billing/support concerns go to the in-app Help menu or `support@warp.dev`, community discussion goes to the Warp Slack community, and security reports go to `security@warp.dev`. Do not apologize performatively and do not offer to retry the same flow.
- Only if the request is in scope, classify it as `bug`, `regression`, `ux issue`, or `feature request` before drafting.
### 2. Ask only for missing facts that materially improve the draft
- Before drafting, decide whether the report already contains the minimum actionable information: what the user was doing, what they expected, and what happened (for bugs, regressions, and UX issues) or what they want to be able to do and why (for feature requests). If any of those pieces is missing, run a single focused clarifying round.
- Use the `ask_user_question` tool for that round. Ask 3-4 high-value multiple-choice questions in a single call, focused on user experience and expectations: what the user was trying to do, what felt confusing or broken, what they expected to happen instead, where in the product they hit the issue, and how much it blocked them.
- Follow the tool guidance where possible: only ask when necessary, do not add labels like `Select One` or `Select All that Apply`, and if fixed options are too limiting, include an `Other` option. If the user skips a question, proceed with your best judgment on what they did answer.
- **Run at most one clarifying round.** If after that round the minimum actionable information is still missing, decline to file rather than drafting a weak issue. Tell the user in one or two sentences exactly which specifics would unblock a future report (for example: "A short description of what you were doing when it happened and what you expected instead would let us turn this into an actionable bug report."). Do not file a placeholder issue just to close the loop.
- For bugs and regressions, first read `references/platforms.md` and try to resolve Warp version and operating system from the bundled version metadata and available context. Ask for reproduction steps only when they are not already clear, and for regressions in particular only when the flow is not readily available from the report or supporting context.
- For crashes, startup failures, rendering bugs, sync issues, or hard-to-reproduce regressions, ask for logs or crash artifacts only when they are likely to help. Read `references/logs.md` only when needed.
- If operating system version, Warp version, or operating-system-specific behavior is relevant, read `references/platforms.md` and follow the bundled metadata guidance there yourself when possible. Ask the user only if you still cannot determine the necessary platform details.
### 3. Check whether the feature or capability is already supported
- Before concluding that something is missing from Warp (feature requests, "it doesn't do X" complaints, "I wish it could Y" asks, or any UX complaint that could be explained by an existing setting or workflow), you **must** consult the docs first.
- Call the `search_warp_documentation` tool with the user's own phrasing. If the first query is vague or returns nothing actionable, try one shorter variant that keeps the same user-visible problem.
- If the search returns a clear match, respond with a concise, direct answer that cites the docs page (title + URL) and explains how the existing functionality addresses the user's ask. Do not file an issue and do not invoke the helper script.
- If the search returns an ambiguous or partial match, briefly summarize what does exist and ask one clarifying question about whether that satisfies the user's intent before deciding whether to file.
- If the search turns up nothing relevant, proceed to step 4. Do not invent workarounds, and do not imply a feature is missing when the docs already answer the question.
- Docs-first checking applies primarily to feature-request-shaped reports. For reproducible bugs and regressions, skip ahead to step 4 unless docs would clarify whether the current behavior is intended.
### 4. Ground the report in product and code context when helpful
- Search the `warpdotdev/warp` repo via the `gh` CLI for matching product language, expected workflows, setting names, or UX intent when that context would make the draft more actionable.
- Search the `warpdotdev/warp` repo via the `gh` CLI for matching components, settings surfaces, feature flags, and likely code paths when implementation context would help triage.
- Add source references only when they point to real files, symbols, settings names, or spec text that plausibly relate to the feedback.
- Never invent a root cause just to make the report sound complete.
### 5. Draft the issue
- Keep the title concrete and user-visible.
- Rewrite rough notes into a polished issue body with the shared section structure below.
- Preserve the user's meaning while making the report easier for an engineer to act on.
- If the exact reproduction steps are still uncertain, write the best-supported scenario and call out what is still unknown.
- Make the title specific enough that it can be used as the primary duplicate-detection query.
### 6. Handle image attachments, if present
- If the user's query includes one or more image attachments visible to you as multimodal context, apply the rules in this step in addition to the normal drafting workflow.
- Incorporate what you can see in each image into the drafted issue body. At minimum, describe the relevant visual content in prose in the `Problem`, `Actual behavior`, or similar section so the report remains coherent even if the images do not end up attached to the filed issue.
- In the `Artifacts` section, emit one entry per attached image, in the order you encountered them. Each entry must include both a **caption** describing what the screenshot depicts and a drop-target placeholder, so the issue body still conveys what each screenshot was meant to show even if the user never uploads the image. Use this format per image, numbered starting at 1:
```md
**Screenshot 1:** <one-sentence caption describing what this screenshot shows, written from what you saw in multimodal context>.
_Paste screenshot 1 here_
```
Captions should be concrete (for example, "Agent footer with the Send button misaligned below the text input") rather than generic ("a screenshot" or "the bug"). Do not invent details that aren't visible; if an image is ambiguous or unreadable, say so in the caption.
- When invoking the helper script (see the `Output` section below), pass `--use browser` instead of `--use gh`. This makes the script skip `gh issue create` and open the prefilled new-issue page in the browser so the user can upload images through GitHub's native drag-and-drop.
- In your final response, explicitly instruct the user to paste or drag each attached image into the body at the corresponding `_Paste screenshot N here_` line(s) and then submit the issue. Reference the count of attached images so the user knows how many to paste. Do not claim the issue has been filed until the user submits.
- If the user's query has no image attachments, do not add captions or placeholders, pass `--use gh` as usual, and do not add drag-and-drop instructions to your final response.
### 7. Check for likely duplicates before filing
- Before invoking `scripts/file_feedback_issue.py`, search issues in `warpdotdev/warp` for likely title matches using the drafted title as the primary query.
- Use a lightweight title-based check only. Prefer precision over recall, and do not run a broad semantic fishing expedition.
- Start with the exact drafted title. If the exact title returns no clear title match, try one shorter normalized variant that removes filler words while preserving the same user-visible problem.
- A suitable command is:
```bash
GH_PAGER=cat gh issue list \
--repo warpdotdev/warp \
--state all \
--limit 10 \
--search "<title> in:title" \
--json number,title,url,state
```
- Treat a result as a duplicate candidate only when the existing issue title clearly refers to the same underlying problem or request.
- If you find a clear title match, do not file a new issue. Respond by pointing the user to the existing issue and explain briefly why it appears to match.
- If no clear title match is found, proceed to file the new issue.
## Issue Structure
Use these sections in order when they apply:
- Summary
- Problem
- Reproduction steps or desired workflow
- Artifacts
- Warp version
- Operating system
For bugs, regressions, and UX issues, also include:
- Expected behavior
- Actual behavior
If you found grounded repo evidence, append:
- Possible source references
Section rules:
- If a required field is unknown, say `Unknown`.
- If an optional section does not apply, omit it.
- If no artifacts are attached, say `None attached`.
- For feature requests, the `Problem` section can describe the current friction or missing capability, and the `Reproduction steps or desired workflow` section can describe the desired flow instead of literal repro steps.
- For feature requests, `Expected behavior` and `Actual behavior` are usually omitted unless they genuinely clarify the request.
## Source Reference Rules
- Prefer concrete file paths, symbols, settings names, and spec headings over broad guesses like "probably in the terminal code."
- Cite spec references when they clarify the intended workflow, wording, or product expectation.
- Cite implementation references when they localize the likely surface area or affected component.
- Keep each reference brief and explain why it is relevant.
- Omit references that are speculative, weakly related, or only tangentially connected.
## Writing Rules
- Turn vague feedback into specific behavior without changing the user's meaning.
- Convert fuzzy narratives into numbered reproduction steps when the sequence can be inferred responsibly.
- Call out severity, frequency, and regression context when available.
- Be explicit about uncertainty or missing details instead of smoothing them over.
- Do not mention this skill, private internal discussion, or speculative implementation theories in the issue body.
- Do not pad the issue with source references unless they genuinely improve debugging or triage.
- Never treat this workflow as permission to implement a fix. This skill may only file an issue, decline to file, or point to an existing issue.
- Never suggest code changes, patches, or implementation diffs in the issue body, in your response, or as a follow-up action — even informally or "as a starting point." If the user asks for a fix, decline and redirect them to the filed issue.
- **Refuse to file in three situations:** (1) the report is out of scope per step 1, (2) the minimum actionable information is still missing after the single clarifying round in step 2, or (3) step 3 found a clear docs match for what the user is asking about. In each case, explain the decision in one or two plain sentences and do not call the helper script. Never imply a feature is missing when docs already answer the question, and never file a placeholder issue just to acknowledge the user.
## Output
Use the bundled helper script `scripts/file_feedback_issue.py` to file the issue in `warpdotdev/warp` instead of calling `gh` directly. The script requires a `--use` flag that selects the filing method explicitly:
- `--use gh`: creates the issue with `gh issue create`. Requires `gh` to be installed and authenticated for `github.com`. Prints a `created` result with `issue_url` on success, or `unavailable` when `gh` is missing or unauthenticated. Does not silently fall back to the browser.
- `--use browser`: opens the prefilled new-issue page in the browser so the user can upload image attachments via GitHub's web UI. Prints a `browser_opened` result on success. If the browser cannot be opened, automatically falls back to `gh issue create` and prints a `created` result with `browser_unavailable: true`; if both are unavailable, prints `failed`. Use this whenever the user attached one or more images to the query.
- The script always targets `warpdotdev/warp` on `github.com`.
Write the final body to a temporary UTF-8 file and pass the final title directly as an argument. When the user has no image attachments:
```bash
python3 scripts/file_feedback_issue.py \
--use gh \
--title "<title>" \
--body-file <body-file>
```
When the user has one or more image attachments:
```bash
# Opens the prefilled new-issue page in the browser so the user can drop
# their images into the issue body via GitHub's web UI.
python3 scripts/file_feedback_issue.py \
--use browser \
--title "<title>" \
--body-file <body-file>
```
The title and body should be structured as follows:
Issue title: `<title>`
Issue body:
```md
<!-- warp-feedback-skill:v1 -->
## Summary
...
## Problem
...
<!-- Include these sections for bugs, regressions, and UX issues when applicable. -->
## Expected behavior
...
## Actual behavior
...
## Reproduction steps or desired workflow
1. ...
## Artifacts
...
## Warp version
...
## Operating system
...
<!-- Omit this section if there are no grounded references. -->
## Possible source references
- path or symbol: why it may be relevant
```
After completing the duplicate-check and filing workflow:
- If the duplicate-check step finds an existing matching issue, respond with the existing issue link and a brief summary (2–4 sentences max) explaining that a likely duplicate already exists, including the matching title, whether that issue is open or closed, and why it appears to match. Do not create another issue.
- If the JSON result has `status: "created"` and `browser_unavailable: true`, explicitly tell the user that the browser could not be opened (include the `message` field from the result) and that the issue was filed programmatically with the available text contents. Make clear that image attachments were not uploaded to the issue. Then provide the issue link and a brief summary (3–5 sentences max) of what was filed.
- If the JSON result has `status: "created"` (without `browser_unavailable`), respond with only the created issue link and a brief summary (3–5 sentences max) of what was filed: the classification, the core problem, and any notable missing details.
- If the JSON result has `status: "unavailable"`, say that you could not file the issue because the GitHub CLI was not installed or authenticated, and include the returned message.
- If the JSON result has `status: "browser_opened"` (seen when `--use browser` was used for image-bearing feedback), do not claim the issue has been filed. Pass the returned `message` to the user, and explicitly instruct them to paste or drag each attached image into the placeholder line(s) in the issue body and then submit the issue. When the result includes a `body` field (the drafted body was too long to prefill in the URL), surface that body so the user can paste it into the issue form before attaching images.
- If the JSON result has `status: "failed"`, say that you could not file the issue because the filing flow failed, and include the returned `error` or `gh_error` message. When `--use browser` was used and filing failed (meaning both the browser and the `gh` CLI fallback were unavailable), make it explicit that image attachments were not handed off and no issue was filed.
- If issue filing is not possible for any other reason, say explicitly that no issue was filed.
Read `references/examples.md` only if you need a compact example of the expected polish level.
resources/bundled/skills/feedback/references/examples.md
+39
View File
@@ -0,0 +1,39 @@
# Examples
Use these only to calibrate tone and structure. Do not copy them mechanically.
## Example 1: Bug
Rough feedback:
`Warp loses my selected theme after restart on macOS. I changed it twice and it keeps going back.`
Draft shape:
- classification: `bug`
- title: `Theme selection resets after restarting Warp on macOS`
- missing details: Warp version, macOS version, whether the issue reproduces every restart
Key body moves:
- turn the complaint into a concrete restart flow
- state the expected persisted theme behavior
- mark unknown version details as `Unknown`
## Example 2: Feature Request
Rough feedback:
`The command palette is too hard to scan. I want better grouping for settings-related actions.`
Draft shape:
- classification: `feature request`
- title: `Improve Command Palette grouping for settings-related actions`
- missing details: concrete commands that are hardest to find, screenshots if the user has them
Key body moves:
- describe the current friction instead of inventing reproduction steps
- focus the request on the desired workflow and user impact
- draft immediately unless a specific missing fact blocks clarity
resources/bundled/skills/feedback/references/logs.md
+33
View File
@@ -0,0 +1,33 @@
# Log And Crash Artifact Guidance
Use this only for crashes, startup failures, rendering bugs, sync issues, or hard-to-reproduce regressions.
- Ask for logs only when they are likely to improve the report.
- Note in the issue that logs or crash reports were attached, but do not claim they contain console input or output.
- In the `Artifacts` section, mention the exact file names or bundles that were attached.
macOS paths and commands:
- Logs live under `~/Library/Logs/`
- Stable app logs are typically `~/Library/Logs/warp.log*`
- Preview app logs are typically `~/Library/Logs/warp_preview.log*`
- Stable zip command: `zip -j ~/Desktop/warp-logs.zip ~/Library/Logs/warp.log*`
- Preview zip command: `zip -j ~/Desktop/warp_preview-logs.zip ~/Library/Logs/warp_preview.log*`
- If Warp still opens, the user can search `View Warp Logs` in the Command Palette
- Crash reports may also exist under `~/Library/Logs/DiagnosticReports/` as Warp `.ips` files
Linux paths:
- Logs live under Warp's state directory.
- Stable app logs are typically `~/.local/state/warp-terminal/warp.log*`
- Preview app logs are typically `~/.local/state/warp-terminal-preview/warp_preview.log*`
- If the exact channel is unclear, ask the user to open the nearest `warp*.log*` files under `~/.local/state/`
Windows paths:
- Logs live under Warp's local app data state directory.
- Stable app logs are typically `%LOCALAPPDATA%\warp\Warp\data\logs\warp.log*`
- Preview app logs are typically `%LOCALAPPDATA%\warp\WarpPreview\data\logs\warp_preview.log*`
- If the exact channel is unclear, ask the user to look under `%LOCALAPPDATA%\warp\` for the relevant `Warp*` folder and attach the matching `warp*.log*` files from its `data\logs\` directory
If no artifacts are available, say so plainly instead of implying they were checked.
resources/bundled/skills/feedback/references/platforms.md
+29
View File
@@ -0,0 +1,29 @@
# Platform Guidance
Use this only when the operating system, Warp version, or operating-system-specific behavior is relevant and missing.
- Resolve the execution surface first: packaged native Warp app or web session.
- Prefer the bundled helper scripts and metadata files over prose or ad hoc shell inspection.
- If the user already gave a sufficiently specific OS version or Warp version, do not ask again.
- Include both OS name and version in the `Operating system` section when available.
- Include the `Warp version` section when available, and note when the report is about a web session rather than a packaged native install.
## Operating system
Resolve the OS from the machine where the reported behavior actually happens. Do not substitute the OS of a different host, container, or remote target unless that is where the bug occurs.
Run the bundled helper script when you need to resolve OS name and version:
```bash
python3 scripts/resolve_platform.py
```
Use the script output directly when filling `Operating system`. Ask the user only if Python is unavailable or the output still does not identify the relevant environment precisely enough.
## Warp version
For packaged native Warp installs, read the bundled version metadata file directly:
The bundled version metadata file lives at `../../metadata/version.json` relative to the skill root. Read its `warp_version` field and use that value directly.
Use the file contents directly when filling `Warp version`. Ask the user only if Python is unavailable, the bundled metadata file is missing or unreadable, or the report is about a browser or web session rather than a packaged native install.
- Browser or web session with no local Warp executable: use the version or build identifier from the session URL or surrounding session metadata when present. If there is no concrete version string, record that it was a web session and leave `Warp version` as `Unknown` rather than guessing.
resources/bundled/skills/feedback/scripts/file_feedback_issue.py
+308
View File
@@ -0,0 +1,308 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import os
import platform
import urllib.parse
import webbrowser
from pathlib import Path
import shutil
import subprocess
import sys
DEFAULT_REPO = "warpdotdev/warp"
DEFAULT_HOSTNAME = "github.com"
FEEDBACK_LABEL = "in-app-feedback"
# GitHub's new-issue page accepts a prefilled title and body via query
# parameters, but browsers and intermediate servers commonly cap URLs around
# 8 KB. Keep a conservative threshold that leaves headroom for the base URL
# and percent-encoding overhead.
MAX_PREFILL_URL_LENGTH = 8000
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=(
"File a GitHub issue in warpdotdev/warp. The caller must choose "
"the filing method via --use: `gh` to create the issue directly with the "
"gh CLI, or `browser` to open the prefilled new-issue page in the browser "
"(used when attachments require manual upload via GitHub's web UI)."
)
)
parser.add_argument(
"--use",
dest="use_method",
required=True,
choices=["gh", "browser"],
help=(
"Filing method. `gh` creates the issue via the gh CLI (requires gh to be "
"installed and authenticated). `browser` opens the prefilled new-issue "
"page in the user's browser so attachments can be uploaded via GitHub's "
"web UI."
),
)
parser.add_argument("--title", required=True, help="Issue title.")
parser.add_argument(
"--body-file",
type=Path,
required=True,
help="Path to a UTF-8 file containing the issue body.",
)
return parser.parse_args()
def read_text(path: Path, field_name: str) -> str:
try:
return path.read_text(encoding="utf-8")
except OSError as exc:
raise SystemExit(f"Failed to read {field_name} from {path}: {exc}") from exc
def normalize_title(title: str) -> str:
normalized_title = " ".join(title.splitlines()).strip()
if not normalized_title:
raise SystemExit("Issue title must not be empty.")
return normalized_title
def normalize_body(body: str) -> str:
normalized_body = body.rstrip("\n")
if not normalized_body.strip():
raise SystemExit("Issue body must not be empty.")
return normalized_body
def gh_path_if_authenticated() -> str | None:
gh_path = shutil.which("gh")
if gh_path is None:
return None
auth_result = subprocess.run(
[gh_path, "auth", "status", "--hostname", DEFAULT_HOSTNAME],
capture_output=True,
text=True,
check=False,
)
if auth_result.returncode != 0:
return None
return gh_path
def create_issue_with_gh(
gh_path: str,
title: str,
body: str,
) -> tuple[str | None, str | None]:
command = [
gh_path, "issue", "create",
"--repo", DEFAULT_REPO,
"--title", title,
"--body", body,
"--label", FEEDBACK_LABEL,
]
result = subprocess.run(command, capture_output=True, text=True, check=False)
if result.returncode != 0:
error_output = "\n".join(part for part in [result.stdout.strip(), result.stderr.strip()] if part).strip()
return None, error_output or "gh issue create failed"
issue_url = result.stdout.strip().splitlines()[-1].strip()
if not issue_url:
return None, "gh issue create succeeded but did not return an issue URL"
return issue_url, None
def build_new_issue_url(title: str, body: str | None) -> str:
"""Build a GitHub new-issue URL with the provided title and optional body prefilled."""
base = f"https://{DEFAULT_HOSTNAME}/{DEFAULT_REPO}/issues/new"
params: list[tuple[str, str]] = [("title", title)]
if body is not None:
params.append(("body", body))
return f"{base}?{urllib.parse.urlencode(params, quote_via=urllib.parse.quote)}"
def browser_is_available() -> tuple[bool, str | None]:
"""Return whether opening a browser is likely to succeed on this system.
On macOS and Windows a GUI session is effectively always present for the
user running this script. On Linux and other unix-likes ``webbrowser.open``
can return True without actually opening a browser when no display server is
running (for example, in a headless SSH session), so we require ``DISPLAY``
or ``WAYLAND_DISPLAY`` to be set.
"""
system = platform.system()
if system in ("Darwin", "Windows"):
return True, None
if os.environ.get("DISPLAY") or os.environ.get("WAYLAND_DISPLAY"):
return True, None
return (
False,
(
"No graphical display is available (neither DISPLAY nor WAYLAND_DISPLAY "
"is set), so the GitHub new-issue page cannot be opened in a browser."
),
)
def open_in_browser(url: str) -> bool:
try:
return webbrowser.open(url, new=2)
except webbrowser.Error:
return False
def fallback_to_browser(title: str, body: str) -> int:
"""Open the GitHub new-issue page with a prefilled title (and body when it fits).
Intended for the caller-selected ``--use browser`` path, which the feedback skill
uses when image attachments need to be uploaded via GitHub's web UI. When the full
prefill URL would exceed ``MAX_PREFILL_URL_LENGTH``, the body is omitted from the
URL and surfaced in the JSON result under a ``body`` field so the caller can tell
the user to paste it into the issue form manually.
When the browser cannot be opened, the function attempts a ``gh issue create``
fallback so the issue is still filed with the available text contents. The returned
payload includes ``browser_unavailable: true`` when that fallback is used, so the
caller can inform the user that the browser could not be opened and images were not
uploaded. If both the browser and ``gh`` are unavailable, filing fails.
"""
full_url = build_new_issue_url(title, body)
body_fits_in_url = len(full_url) <= MAX_PREFILL_URL_LENGTH
url_to_open = full_url if body_fits_in_url else build_new_issue_url(title, None)
browser_available, browser_unavailable_reason = browser_is_available()
base_payload: dict[str, object] = {
"method": "browser",
"repo": DEFAULT_REPO,
"url": url_to_open,
}
if not body_fits_in_url:
# Surface the body so the caller can instruct the user to paste it in.
base_payload["body"] = body
browser_failure_reason: str | None = None
if not browser_available:
browser_failure_reason = browser_unavailable_reason or "No display available."
elif not open_in_browser(url_to_open):
browser_failure_reason = "Unable to open a web browser for the prefilled new-issue page."
if browser_failure_reason is not None:
# Browser unavailable — try gh CLI as a fallback so the issue is still filed.
gh_path = gh_path_if_authenticated()
if gh_path is not None:
issue_url, _gh_error = create_issue_with_gh(gh_path, title, body)
if issue_url is not None:
print_result({
"status": "created",
"method": "gh",
"repo": DEFAULT_REPO,
"issue_url": issue_url,
"browser_unavailable": True,
"message": (
f"{browser_failure_reason} "
"The issue was filed programmatically with the available text contents. "
"Image attachments were not uploaded."
),
})
return 0
base_payload["status"] = "failed"
base_payload["error"] = (
browser_failure_reason
+ " Image attachments could not be handed off through the browser flow; "
"no issue has been filed."
)
print_result(base_payload)
return 1
# The browser path is only used by the skill when image attachments are
# present, so the user-facing message always references pasting/dropping them.
if body_fits_in_url:
message = (
"Opened the GitHub new-issue page in your browser with the title and body prefilled. "
"Paste or drag your attached screenshot(s) into the body at the placeholder line(s), "
"then submit the issue."
)
else:
message = (
"Opened the GitHub new-issue page in your browser with the title prefilled. "
"The drafted body was too long to include in the URL; paste the body (returned in "
"this result's `body` field) into the issue form first, then paste or drag your "
"attached screenshot(s) into the placeholder line(s), then submit."
)
base_payload["status"] = "browser_opened"
base_payload["message"] = message
print_result(base_payload)
return 0
def print_result(payload: dict[str, object]) -> None:
json.dump(payload, sys.stdout)
sys.stdout.write("\n")
def file_with_gh(title: str, body: str) -> int:
"""File the issue via the gh CLI. Returns status `unavailable` when gh isn't
installed or not authenticated; the caller is responsible for choosing a
different --use method in that case (no automatic fallback happens here).
"""
gh_path = gh_path_if_authenticated()
if gh_path is None:
print_result(
{
"status": "unavailable",
"method": "gh",
"repo": DEFAULT_REPO,
"message": f"GitHub CLI is not installed or not authenticated for {DEFAULT_HOSTNAME}.",
}
)
return 0
issue_url, gh_error = create_issue_with_gh(gh_path, title, body)
if issue_url is not None:
print_result(
{
"status": "created",
"method": "gh",
"repo": DEFAULT_REPO,
"issue_url": issue_url,
}
)
return 0
print_result(
{
"status": "failed",
"method": "gh",
"repo": DEFAULT_REPO,
"gh_error": gh_error,
}
)
return 1
def main() -> int:
args = parse_args()
title = normalize_title(args.title)
body = normalize_body(read_text(args.body_file, "body"))
if args.use_method == "browser":
return fallback_to_browser(title, body)
# argparse enforces choices=["gh", "browser"], so the remaining case is "gh".
return file_with_gh(title, body)
if __name__ == "__main__":
raise SystemExit(main())
resources/bundled/skills/feedback/scripts/resolve_platform.py
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
from __future__ import annotations
import json
from pathlib import Path
import platform
import subprocess
import sys
def linux_info() -> dict[str, str]:
fields: dict[str, str] = {}
os_release = Path("/etc/os-release")
if os_release.exists():
for line in os_release.read_text(encoding="utf-8", errors="replace").splitlines():
if "=" not in line or line.startswith("#"):
continue
key, value = line.split("=", 1)
fields[key] = value.strip().strip(chr(34))
return {
"os": fields.get("PRETTY_NAME") or fields.get("NAME") or "Linux",
"os_version": fields.get("VERSION_ID") or fields.get("VERSION") or platform.release(),
"kernel": platform.release(),
}
def mac_info() -> dict[str, str]:
build = None
result = subprocess.run(
["sw_vers", "-buildVersion"],
capture_output=True,
text=True,
check=False,
)
if result.returncode == 0:
build = result.stdout.strip() or None
return {
"os": "macOS",
"os_version": platform.mac_ver()[0] or None,
"os_build": build,
}
def windows_info() -> dict[str, str]:
version = sys.getwindowsversion()
return {
"os": "Windows",
"os_version": f"{version.major}.{version.minor}.{version.build}",
}
def main() -> int:
system = platform.system()
if system == "Darwin":
info = mac_info()
elif system == "Linux":
info = linux_info()
elif system == "Windows":
info = windows_info()
else:
info = {
"os": system or "Unknown",
"os_version": platform.version() or None,
}
json.dump({k: v for k, v in info.items() if v}, sys.stdout, sort_keys=True)
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())
resources/bundled/skills/feedback/scripts/test_file_feedback_issue.py
+365
View File
@@ -0,0 +1,365 @@
#!/usr/bin/env python3
"""Unit tests for file_feedback_issue.py.
Run with:
python3 resources/channel-gated-skills/dogfood/feedback/scripts/test_file_feedback_issue.py
"""
from __future__ import annotations
import contextlib
import importlib.util
import io
import json
import sys
import unittest
import urllib.parse
from contextlib import redirect_stdout
from pathlib import Path
from unittest import mock
SCRIPT_DIR = Path(__file__).resolve().parent
MODULE_PATH = SCRIPT_DIR / "file_feedback_issue.py"
def load_module():
spec = importlib.util.spec_from_file_location("file_feedback_issue", MODULE_PATH)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
ffi = load_module()
class BuildNewIssueUrlTests(unittest.TestCase):
def test_url_includes_repo_and_title(self):
url = ffi.build_new_issue_url("Hello world", None)
self.assertTrue(url.startswith(f"https://{ffi.DEFAULT_HOSTNAME}/{ffi.DEFAULT_REPO}/issues/new?"))
parsed = urllib.parse.urlparse(url)
qs = urllib.parse.parse_qs(parsed.query)
self.assertEqual(qs["title"], ["Hello world"])
self.assertNotIn("body", qs)
def test_url_includes_body_when_provided(self):
url = ffi.build_new_issue_url("T", "body text with spaces & symbols?")
parsed = urllib.parse.urlparse(url)
qs = urllib.parse.parse_qs(parsed.query)
self.assertEqual(qs["title"], ["T"])
self.assertEqual(qs["body"], ["body text with spaces & symbols?"])
def test_special_characters_are_percent_encoded(self):
url = ffi.build_new_issue_url("crash: `foo`", "<script>alert(1)</script>")
# Spaces should be %20, not +, because we use quote_via=quote.
self.assertIn("%20", url)
self.assertNotIn("+", url.split("?", 1)[1])
# Angle brackets and backticks are percent-encoded.
self.assertIn("%3C", url)
self.assertIn("%3E", url)
class FallbackToBrowserTests(unittest.TestCase):
def _run_fallback(
self,
title,
body,
open_ok=True,
browser_available=(True, None),
gh_path=None,
gh_create_result=(None, "gh not configured"),
):
with mock.patch.object(
ffi, "open_in_browser", return_value=open_ok
) as open_mock, mock.patch.object(
ffi, "browser_is_available", return_value=browser_available
), mock.patch.object(
ffi, "gh_path_if_authenticated", return_value=gh_path
), mock.patch.object(
ffi, "create_issue_with_gh", return_value=gh_create_result
):
buf = io.StringIO()
with redirect_stdout(buf):
rc = ffi.fallback_to_browser(title, body)
payload = json.loads(buf.getvalue().strip())
return rc, payload, open_mock
def test_short_body_is_embedded_in_url(self):
rc, payload, open_mock = self._run_fallback("t", "small body")
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "browser_opened")
self.assertEqual(payload["method"], "browser")
# Short bodies fit in the URL; no `body` field is surfaced separately.
self.assertNotIn("body", payload)
open_mock.assert_called_once()
called_url = open_mock.call_args.args[0]
parsed = urllib.parse.urlparse(called_url)
qs = urllib.parse.parse_qs(parsed.query)
self.assertEqual(qs["title"], ["t"])
self.assertEqual(qs["body"], ["small body"])
def test_long_body_surfaces_body_in_payload_and_opens_title_only_url(self):
huge_body = "A" * (ffi.MAX_PREFILL_URL_LENGTH + 500)
rc, payload, open_mock = self._run_fallback("t", huge_body)
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "browser_opened")
# Long bodies are returned in the payload so the caller can instruct
# the user to paste them; no clipboard copy happens.
self.assertEqual(payload["body"], huge_body)
called_url = open_mock.call_args.args[0]
parsed = urllib.parse.urlparse(called_url)
qs = urllib.parse.parse_qs(parsed.query)
self.assertEqual(qs["title"], ["t"])
self.assertNotIn("body", qs)
def test_browser_failure_falls_back_to_gh_when_available(self):
rc, payload, _ = self._run_fallback(
"t",
"small body",
open_ok=False,
gh_path="/usr/bin/gh",
gh_create_result=("https://github.com/warpdotdev/warp/issues/7", None),
)
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "created")
self.assertEqual(payload["method"], "gh")
self.assertTrue(payload["issue_url"].endswith("/issues/7"))
self.assertTrue(payload.get("browser_unavailable"))
self.assertIn("attachment", payload["message"].lower())
def test_browser_failure_reports_failed_when_gh_also_unavailable(self):
rc, payload, open_mock = self._run_fallback("t", "small body", open_ok=False)
# Filing failed, so exit code must be non-zero for shell callers that
# check `$?` to distinguish "issue filed" from "issue not filed".
self.assertEqual(rc, 1)
self.assertEqual(payload["status"], "failed")
self.assertEqual(payload["method"], "browser")
self.assertIn("url", payload)
# Failure messaging must acknowledge attachments so the user understands
# why the image workflow couldn't complete.
self.assertIn("attachment", payload["error"].lower())
def test_browser_unavailable_falls_back_to_gh_when_available(self):
rc, payload, open_mock = self._run_fallback(
"t",
"small body",
browser_available=(False, "No DISPLAY"),
gh_path="/usr/bin/gh",
gh_create_result=("https://github.com/warpdotdev/warp/issues/8", None),
)
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "created")
self.assertEqual(payload["method"], "gh")
self.assertTrue(payload["issue_url"].endswith("/issues/8"))
self.assertTrue(payload.get("browser_unavailable"))
self.assertIn("No DISPLAY", payload["message"])
self.assertIn("attachment", payload["message"].lower())
open_mock.assert_not_called()
def test_browser_unavailable_reports_failed_when_gh_also_unavailable(self):
rc, payload, open_mock = self._run_fallback(
"t",
"small body",
browser_available=(False, "No DISPLAY"),
)
self.assertEqual(rc, 1)
self.assertEqual(payload["status"], "failed")
self.assertEqual(payload["method"], "browser")
self.assertIn("No DISPLAY", payload["error"])
self.assertIn("attachment", payload["error"].lower())
open_mock.assert_not_called()
class BrowserIsAvailableTests(unittest.TestCase):
def test_darwin_is_always_available(self):
with mock.patch.object(ffi.platform, "system", return_value="Darwin"), \
mock.patch.dict(ffi.os.environ, {}, clear=True):
ok, reason = ffi.browser_is_available()
self.assertTrue(ok)
self.assertIsNone(reason)
def test_windows_is_always_available(self):
with mock.patch.object(ffi.platform, "system", return_value="Windows"), \
mock.patch.dict(ffi.os.environ, {}, clear=True):
ok, reason = ffi.browser_is_available()
self.assertTrue(ok)
self.assertIsNone(reason)
def test_linux_without_display_is_unavailable(self):
with mock.patch.object(ffi.platform, "system", return_value="Linux"), \
mock.patch.dict(ffi.os.environ, {}, clear=True):
ok, reason = ffi.browser_is_available()
self.assertFalse(ok)
self.assertIsNotNone(reason)
def test_linux_with_display_is_available(self):
with mock.patch.object(ffi.platform, "system", return_value="Linux"), \
mock.patch.dict(ffi.os.environ, {"DISPLAY": ":0"}, clear=True):
ok, reason = ffi.browser_is_available()
self.assertTrue(ok)
self.assertIsNone(reason)
def test_linux_with_wayland_display_is_available(self):
with mock.patch.object(ffi.platform, "system", return_value="Linux"), \
mock.patch.dict(ffi.os.environ, {"WAYLAND_DISPLAY": "wayland-0"}, clear=True):
ok, reason = ffi.browser_is_available()
self.assertTrue(ok)
self.assertIsNone(reason)
class FileWithGhTests(unittest.TestCase):
def _run(self, **mocks):
patches = []
for attr, value in mocks.items():
patches.append(mock.patch.object(ffi, attr, return_value=value))
buf = io.StringIO()
with contextlib.ExitStack() as stack:
for p in patches:
stack.enter_context(p)
with redirect_stdout(buf):
rc = ffi.file_with_gh("hello", "body")
return rc, json.loads(buf.getvalue().strip())
def test_reports_unavailable_when_gh_missing(self):
rc, payload = self._run(gh_path_if_authenticated=None)
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "unavailable")
self.assertEqual(payload["method"], "gh")
self.assertIn("message", payload)
def test_creates_when_gh_available(self):
rc, payload = self._run(
gh_path_if_authenticated="/usr/bin/gh",
create_issue_with_gh=(
"https://github.com/warpdotdev/warp/issues/42",
None,
),
)
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "created")
self.assertEqual(payload["method"], "gh")
self.assertTrue(payload["issue_url"].endswith("/issues/42"))
def test_reports_failed_when_gh_create_fails(self):
rc, payload = self._run(
gh_path_if_authenticated="/usr/bin/gh",
create_issue_with_gh=(None, "gh failed to create issue"),
)
# Filing failed, so exit code must be non-zero.
self.assertEqual(rc, 1)
self.assertEqual(payload["status"], "failed")
self.assertEqual(payload["method"], "gh")
self.assertEqual(payload["gh_error"], "gh failed to create issue")
class MainTests(unittest.TestCase):
"""Tests for the --use dispatch in main().
The caller must pick a filing method explicitly; main() does not fall back
between the two paths.
"""
def setUp(self):
self.body_file = SCRIPT_DIR / "_tmp_body.txt"
self.body_file.write_text("my body", encoding="utf-8")
def tearDown(self):
self.body_file.unlink(missing_ok=True)
def _run_main(self, argv_extras):
argv = [
"file_feedback_issue.py",
"--title",
"hello",
"--body-file",
str(self.body_file),
*argv_extras,
]
buf = io.StringIO()
with mock.patch.object(sys, "argv", argv):
with redirect_stdout(buf):
rc = ffi.main()
return rc, json.loads(buf.getvalue().strip())
def test_use_gh_creates_when_gh_available(self):
with mock.patch.object(
ffi, "gh_path_if_authenticated", return_value="/usr/bin/gh"
), mock.patch.object(
ffi,
"create_issue_with_gh",
return_value=(
"https://github.com/warpdotdev/warp/issues/999",
None,
),
):
rc, payload = self._run_main(["--use", "gh"])
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "created")
self.assertEqual(payload["method"], "gh")
self.assertTrue(payload["issue_url"].endswith("/issues/999"))
def test_use_gh_reports_unavailable_and_does_not_open_browser(self):
with mock.patch.object(
ffi, "gh_path_if_authenticated", return_value=None
), mock.patch.object(ffi, "open_in_browser", return_value=True) as open_mock:
rc, payload = self._run_main(["--use", "gh"])
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "unavailable")
self.assertEqual(payload["method"], "gh")
# Critical invariant: --use gh must not silently fall back to the browser.
open_mock.assert_not_called()
def test_use_browser_does_not_touch_gh(self):
with mock.patch.object(
ffi, "gh_path_if_authenticated"
) as gh_mock, mock.patch.object(
ffi, "create_issue_with_gh"
) as create_mock, mock.patch.object(
ffi, "browser_is_available", return_value=(True, None)
), mock.patch.object(ffi, "open_in_browser", return_value=True) as open_mock:
rc, payload = self._run_main(["--use", "browser"])
self.assertEqual(rc, 0)
self.assertEqual(payload["status"], "browser_opened")
self.assertEqual(payload["method"], "browser")
open_mock.assert_called_once()
gh_mock.assert_not_called()
create_mock.assert_not_called()
def test_missing_use_flag_exits_with_argparse_error(self):
stderr = io.StringIO()
with mock.patch.object(
sys,
"argv",
[
"file_feedback_issue.py",
"--title",
"hello",
"--body-file",
str(self.body_file),
],
), mock.patch.object(sys, "stderr", stderr), contextlib.suppress(SystemExit):
ffi.main()
self.assertIn("--use", stderr.getvalue())
def test_invalid_use_value_is_rejected(self):
stderr = io.StringIO()
with mock.patch.object(
sys,
"argv",
[
"file_feedback_issue.py",
"--use",
"carrier-pigeon",
"--title",
"hello",
"--body-file",
str(self.body_file),
],
), mock.patch.object(sys, "stderr", stderr), contextlib.suppress(SystemExit):
ffi.main()
self.assertIn("invalid choice", stderr.getvalue())
if __name__ == "__main__":
unittest.main()
resources/bundled/skills/modify-settings/SKILL.md
+73
View File
@@ -0,0 +1,73 @@
---
name: modify-settings
description: View or modify Warp application settings using the bundled JSON schema for guidance
---
# modify-settings
Use this skill when the user wants to view, change, or troubleshoot Warp application settings.
## Settings Schema
A JSON schema describing all available settings is bundled at:
```
{{settings_schema_path}}
```
The schema follows JSON Schema draft 2020-12, with settings organized hierarchically under `properties`. Each setting includes:
- **`description`** — what the setting controls
- **`type`** — the value type (`string`, `boolean`, `integer`, etc.)
- **`default`** — the default value
- **`enum`** or **`oneOf`** — valid values, when the setting is constrained
### Finding a setting
Use `grep` to do an initial broad search for candidate key names:
```sh
grep -i "font" {{settings_schema_path}}
```
Once you have a candidate key name, run the bundled script to get the **full dotted path**, the setting's properties, and any parent context. This is critical — the schema has multiple sections with similar names (e.g. several `input` keys), so never assume the nesting from grep output alone.
```sh
python3 <skill_dir>/scripts/find_setting.py {{settings_schema_path}} <key_name>
```
The output gives you the unambiguous full path (e.g. `properties.appearance.properties.input.properties.input_mode`) and the setting's full definition including valid values.
## Settings File
The user's settings are stored in a TOML file at:
```
{{settings_file_path}}
```
Settings use dotted TOML section headers matching the schema hierarchy. Always trace the **full** nesting path from the schema to the TOML — each intermediate `properties` key becomes a section level. For example:
A property at `properties.appearance.properties.font_size` (one level deep) corresponds to:
```toml
[appearance]
font_size = 14
```
A property at `properties.appearance.properties.themes.properties.theme` (two levels deep) corresponds to:
```toml
[appearance.themes]
theme = "light"
```
A common mistake is to stop one level too early — always count the full depth before writing the TOML section header.
If the file does not exist yet, create it. Warp hot-reloads this file, so changes take effect immediately.
## Workflow
1. **Find the setting** — use `grep` to identify candidate key names, then run the Python path-tracing script to get the full dotted path and the setting's valid values. Never rely on grep output alone to infer nesting.
2. **Read current value** — check the settings file to see whether the setting is already configured.
3. **Apply the change** — add or update the setting in the TOML file with a valid value from the schema.
resources/bundled/skills/modify-settings/scripts/find_setting.py
+42
View File
@@ -0,0 +1,42 @@
"""
Find a setting's full dotted path and properties in the Warp settings JSON schema.
Usage:
python3 find_setting.py <settings_schema_path> <key_name>
Example:
python3 find_setting.py /path/to/settings_schema.json input_mode
"""
import json
import sys
def find_key(obj, target, path=""):
found = False
if isinstance(obj, dict):
for k, v in obj.items():
new_path = f"{path}.{k}" if path else k
if k == target:
print(f"Path: {new_path}")
print(json.dumps(v, indent=2))
print()
found = True
if find_key(v, target, new_path):
found = True
return found
if __name__ == "__main__":
if len(sys.argv) != 3:
print(f"Usage: {sys.argv[0]} <settings_schema_path> <key_name>")
sys.exit(1)
schema_path, target_key = sys.argv[1], sys.argv[2]
with open(schema_path) as f:
schema = json.load(f)
if not find_key(schema, target_key):
print(f"No setting found matching key: {target_key}", file=sys.stderr)
sys.exit(1)
resources/bundled/skills/oz-platform/SKILL.md
+273
View File
@@ -0,0 +1,273 @@
---
name: oz-platform
description: Use Warp's REST API and command line to run, configure, and inspect Oz cloud agents
---
# oz-platform
Use the Oz REST API and CLI to:
* Spawn cloud agents
* Get the status of a cloud agent
* Schedule cloud agents to run repeatedly
* Create and manage the environments in which cloud agents run
* Provide secrets for cloud agents to use
## Command Line
The Oz CLI is installed as `{{warp_cli_binary_name}}`. To get help output, use `{{warp_cli_binary_name}} help` or `{{warp_cli_binary_name}} help <subcommand>`.
Prefer `--output-format text` to review the response, or `--output-format json` to parse fields with `jq`.
You can find more information at https://docs.warp.dev/reference/cli.
The most important commands are:
* `{{warp_cli_binary_name}} agent run-cloud`: Spawn a new cloud agent. You can configure the prompt, model, environment, and other settings.
* `{{warp_cli_binary_name}} run list` and `{{warp_cli_binary_name}} run get <run-id>`: List all cloud agent runs, and get details about a particular run.
* `{{warp_cli_binary_name}} environment list` and `{{warp_cli_binary_name}} environment get`: List available environments, and get more information about a particular environment.
* `{{warp_cli_binary_name}} schedule list` and `{{warp_cli_binary_name}} schedule get`: List scheduled tasks with most recent runs, and get more information about a particular scheduled run.
Most subcommands support the `--output-format json` flag to produce JSON output, which you can pipe into `jq` or other commands.
### Examples
Start a cloud agent, and then monitor its status:
```sh
$ {{warp_cli_binary_name}} agent run-cloud --prompt "Update the login error to be more specific" --environment UA17BXYZ
# ...
Spawned agent with run ID: 5972cca4-a410-42af-930a-e56bc23e07ac
```
```sh
$ {{warp_cli_binary_name}} run get 5972cca4-a410-42af-930a-e56bc23e07ac
# ...
```
Schedule an agent to summarize feedback every day at 8am UTC:
```sh
$ {{warp_cli_binary_name}} schedule create --cron "0 8 * * *" \
--prompt "Collect all feedback from new GitHub issues and provide a summary report" \
--environment UA17BXYZ
```
Create a secret for cloud agents to use:
```sh
$ {{warp_cli_binary_name}} secret create JIRA_API_KEY --team --value-file jira_key.txt --description "API key to access Jira"
```
## REST API
Oz has a REST API for starting and inspecting cloud agents.
All API requests require authentication using an API key. The user can generate API keys in their Warp settings, on the `Platform` page (accessible via `{{warp_url_scheme}}://settings/platform`).
You can find the full OpenAPI specification here: https://docs.warp.dev/reference/api-and-sdk
### TypeScript / JavaScript SDK
The TypeScript SDK is available via NPM. It is fully async, and works with Node, Bun, and Deno.
* Package link: https://www.npmjs.com/package/oz-agent-sdk
* Source Code: https://github.com/warpdotdev/oz-sdk-typescript
* API reference: https://raw.githubusercontent.com/warpdotdev/oz-sdk-typescript/HEAD/api.md
### Python SDK
The Python SDK is available from PyPi. It can be used synchronously or asynchronously.
* Package link: https://pypi.org/project/oz-agent-sdk/
* Source Code: https://github.com/warpdotdev/oz-sdk-python
* API reference: https://raw.githubusercontent.com/warpdotdev/oz-sdk-python/refs/heads/main/api.md
### API Examples
```sh
curl -L -X POST {{warp_server_url}}/api/v1/agent/run \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"prompt": "Update the login error to be more specific",
"config": {
"environment_id": "UA17BXYZ"
}
}'
```
```sh
curl -L -X GET {{warp_server_url}}/api/v1/agent/runs/5972cca4-a410-42af-930a-e56bc23e07ac \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json'
```
## GitHub Actions Integration
You can trigger Oz cloud agents from GitHub Actions workflows. This enables automation like:
* Triaging issues when they're created or labeled
* Running checks on pull requests
* Scheduling periodic tasks via workflow dispatch
The agent will have access to the `gh` CLI to communicate back to the repository. Prefer prompting the agent to use `gh` vs. requiring the agent to respond with structured output for the GitHub workflow to parse.
### Action Setup
Use `warpdotdev/oz-agent-action@main` in your workflow. Required inputs:
* `prompt`: The task description for the agent
* `warp_api_key`: API key (store in GitHub secrets, e.g., `${{ secrets.WARP_API_KEY }}`)
* `profile`: Optional agent profile identifier (can use repo variable, e.g., `${{ vars.WARP_AGENT_PROFILE || '' }}`)
The action outputs `agent_output` with the agent's response.
### Minimal Workflow Example
```yaml
name: Run Oz Agent
on:
issues:
types: [opened, labeled]
jobs:
agent:
runs-on: ubuntu-latest
permissions:
contents: write
issues: write
pull-requests: write
steps:
- uses: actions/checkout@v6
- uses: warpdotdev/oz-agent-action@main
id: agent
with:
prompt: |
Analyze the GitHub issue and provide a summary.
Issue: ${{ github.event.issue.title }}
${{ github.event.issue.body }}
Respond to the issue with a comment containing your summary using the `gh` CLI.
warp_api_key: ${{ secrets.WARP_API_KEY }}
profile: ${{ vars.WARP_AGENT_PROFILE || '' }}
- name: Use Agent Output
run: echo "${{ steps.agent.outputs.agent_output }}"
```
### Common Patterns
**Conditional steps**: Use `if: steps.agent.outputs.agent_output` to branch on agent results.
**Templating**: Use `actions/github-script@v7` to construct dynamic prompts from issue templates, repo context, or code.
**Error handling**: Check action success with `if: success()` or `if: failure()`.
**Git operations**: The action runs with checked-out code and Git credentials, so agents can commit and push changes.
## Environments
All cloud agents run in an environment. The environment defines:
* Which programs are preinstalled for the agent (based on a Docker image)
* The Git repositories to check out before the agent starts
* Setup commands to run, such as `npm install` or `cargo fetch`
You should almost always run cloud agents in an environment. Otherwise, they may not have the necessary code or tools available.
Cloud agents run in a sandbox, so they _can_ install additional programs into their environment. They also have Git credentials to create PRs and push branches.
Cloud environments DO NOT store secret values, like API keys. Use the `{{warp_cli_binary_name}} secret` commands instead.
## Using Third-Party Coding CLIs
Oz environments support running third-party coding agent CLIs such as Claude Code, Codex, Gemini CLI, Amp, Copilot CLI, and OpenCode. The `-agents` tagged variants of prebuilt Oz Docker images (e.g. `warpdotdev/dev-rust:1.85-agents`) come with the most popular CLIs preinstalled. Base tags (without `-agents`) do not include coding agent CLIs.
For detailed per-CLI documentation (installation, authentication, non-interactive flags, and artifact reporting), see [references/third-party-clis.md](./references/third-party-clis.md).
### For Interactive Agents: Launching Cloud Agents with Third-Party CLIs
When you are an interactive agent launching a cloud agent to use a third-party CLI:
1. **Environment Selection**: First, ask the user which environment to use. Present the public `-agents` image options from [warpdotdev/oz-dev-environments](https://github.com/warpdotdev/oz-dev-environments):
- `warpdotdev/dev-base:latest-agents`
- `warpdotdev/dev-go:1.23-agents`
- `warpdotdev/dev-rust:1.83-agents`
- `warpdotdev/dev-rust:1.85-agents`
- `warpdotdev/dev-java:21-agents`
- `warpdotdev/dev-dotnet:8.0-agents`
- `warpdotdev/dev-ruby:3.3-agents`
- `warpdotdev/dev-web:latest-agents`
- `warpdotdev/dev-full:latest-agents`
Also ask if they want to use an existing environment instead. To list their environments:
```sh
{{warp_cli_binary_name}} environment list --output-format text
```
If they choose a public image without an existing environment, create one with `{{warp_cli_binary_name}} environment create ...`
2. **Prompt Construction**: Construct a simple prompt that delegates CLI invocation to the cloud agent:
```sh
{{warp_cli_binary_name}} agent run-cloud \
--environment <ENV_ID> \
--prompt 'Read the oz-platform skill for instructions on using [CLI name] to solve: <task description>'
```
**Do not** include the CLI command syntax in the prompt. The cloud agent will read the oz-platform skill and follow those instructions.
**Example**:
```sh
$ {{warp_cli_binary_name}} agent run-cloud \
--environment <ENV_ID> \
--prompt 'Read the oz-platform skill for instructions on using Claude Code to solve: Summarize the architecture of this project'
```
### For Cloud Agents: Invoking Third-Party CLIs
When you are a cloud agent instructed to use a third-party CLI:
1. **Environment**: You are already running in an environment with the CLI preinstalled (if it's in an `-agents` image).
2. **Authentication**: API keys are available as environment variables (e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`). These were configured as Oz secrets by the user.
3. **Task Delegation (IMPORTANT)**: The user's task should be completed **entirely by the third-party CLI**. Do NOT use Warp's built-in tools to complete the task yourself:
- Do NOT use `edit_files`, `read_files`, `grep`, `codebase_semantic_search`, or other Warp coding tools to perform the user's task
- The third-party CLI should do all the coding, file editing, searching, and analysis work
- Your role is to:
- Set up the CLI (e.g., authenticate if needed)
- Construct the prompt for the CLI with the user's task
- Run the CLI and monitor its execution
- Debug any issues with the CLI itself
- Report artifacts back to Warp (see below)
4. **CLI Invocation**: Read [references/third-party-clis.md](./references/third-party-clis.md) for detailed instructions on:
- Non-interactive mode flags for each CLI (e.g. `claude -p`, `codex exec`, `gemini -p`)
- Authentication setup steps if needed (e.g. Codex requires `printenv OPENAI_API_KEY | codex login --with-api-key`)
- Useful flags and options
- Example commands
5. **Artifact Reporting**: When the third-party CLI creates a PR, parse its output for the PR URL and branch name, then call `report_pr` to register the artifact in the Warp UI.
**Example workflow**:
```sh
# 1. Read this skill and references/third-party-clis.md to understand CLI usage
# 2. Set up authentication if needed (e.g., for Codex)
# For Claude Code, ANTHROPIC_API_KEY is already available
# 3. Run the CLI with the user's task - let it do ALL the work
$ claude -p "Summarize the architecture of this project"
# 4. If a PR was created, parse the CLI output and report the artifact
# Example: report_pr(pr_url="https://github.com/...", branch="feature-branch")
```
**What NOT to do**:
```sh
# ❌ Don't read files yourself to help the CLI
$ read_files ...
# ❌ Don't search the codebase yourself
$ grep ...
# ❌ Don't edit files yourself
$ edit_files ...
# ✅ Instead, let the third-party CLI handle everything
$ claude -p "Complete the entire task: <user's task>"
```
resources/bundled/skills/oz-platform/references/third-party-clis.md
+217
View File
@@ -0,0 +1,217 @@
# Third-Party Coding Agent CLIs
**Context**: This reference is for cloud agents who need to invoke third-party coding CLIs. If you are an interactive agent launching a cloud agent, see the "For Interactive Agents" section in the main oz-platform skill instead.
This reference documents third-party coding agent CLIs that Oz cloud agents can install and run. Each CLI has its own authentication, non-interactive mode, and nuances.
The `-agents` tagged variants of prebuilt Oz Docker images (e.g. `warpdotdev/dev-base:latest-agents`, `warpdotdev/dev-rust:1.85-agents`) include Claude Code, Codex, and Gemini CLI preinstalled. Base tags (without `-agents`) do not include coding agent CLIs.
> **Note**: Commands below that reference the Warp CLI should use the CLI binary name from the parent skill (the `{{warp_cli_binary_name}}` value).
## Reporting Artifacts
When a third-party CLI creates a PR or branch, Oz does not automatically detect it. You must explicitly register these artifacts so they appear in the Warp UI.
**Pull Requests**: After a third-party CLI creates a PR, call the `report_pr` tool with the `pr_url` and `branch` parameters. Parse the CLI's output to extract the PR URL and branch name.
**How to prompt for this**: When writing prompts that instruct an Oz agent to delegate work to a third-party CLI, always include instructions to:
1. Tell the third-party CLI to print the PR URL and branch name in its output.
2. After the CLI finishes, parse the output and call `report_pr` to register the artifact.
**Plans**: Plan artifacts are registered automatically by the Oz agent when it creates plans using its built-in plan tools. No special handling is needed.
---
## Claude Code
- **Package**: `npm install -g @anthropic-ai/claude-code`
- **Auth**: `ANTHROPIC_API_KEY` environment variable (automatic).
- **Non-interactive mode**: `claude -p "<prompt>"`
- **Useful flags**: `--output-format json`, `--allowedTools "Read,Write,Bash(...)"`, `--append-system-prompt`
### Oz secret
```sh
{{warp_cli_binary_name}} secret create ANTHROPIC_API_KEY --team --value-file key.txt --description "Anthropic API key for Claude Code"
```
### Example prompt
```
Run: claude -p "Analyze the codebase and list potential security issues" --output-format json
```
### Artifact reporting prompt
```
Run Claude Code to implement the feature:
claude -p "Create a branch, implement the login fix, commit, push, and create a PR using gh CLI. Print the full PR URL and branch name at the end."
After Claude Code finishes, find the PR URL and branch name from its output.
Then call the report_pr tool with those values so the PR appears as an artifact.
```
---
## Codex CLI
- **Package**: `npm install -g @openai/codex`
- **Auth**: Requires an explicit login step before use:
```
printenv OPENAI_API_KEY | codex login --with-api-key
```
Alternatively, `CODEX_API_KEY` can be set directly.
- **Recommended**: Add `printenv OPENAI_API_KEY | codex login --with-api-key` as an environment setup command so authentication happens automatically before the agent starts.
- **Non-interactive mode**: `codex exec "<prompt>"`
- **Regional endpoint**: Set `OPENAI_BASE_URL` if needed (e.g. `https://us.api.openai.com/v1`).
- **Useful flags**: `--full-auto`, `--sandbox workspace-write`, `--json`, `--skip-git-repo-check`
### Oz secret
```sh
{{warp_cli_binary_name}} secret create OPENAI_API_KEY --team --value-file key.txt --description "OpenAI API key for Codex CLI"
```
### Example prompt
```
First authenticate Codex: printenv OPENAI_API_KEY | codex login --with-api-key
Then run: codex exec "Refactor the utils module to reduce duplication"
```
### Artifact reporting prompt
```
Authenticate Codex: printenv OPENAI_API_KEY | codex login --with-api-key
Then run: codex exec --full-auto "Create a branch, fix the bug, commit, push, and create a PR. Print the PR URL and branch name."
After Codex finishes, parse the PR URL and branch name from its output and call report_pr.
```
---
## Gemini CLI
- **Package**: `npm install -g @google/gemini-cli`
- **Auth**: `GEMINI_API_KEY` environment variable (automatic). Obtain from [Google AI Studio](https://aistudio.google.com/apikey).
- **Non-interactive mode**: `gemini -p "<prompt>"` (headless mode)
- **Useful flags**: `--output-format json`, `--yolo` (auto-approve tool actions)
### Oz secret
```sh
{{warp_cli_binary_name}} secret create GEMINI_API_KEY --team --value-file key.txt --description "Gemini API key for Gemini CLI"
```
### Example prompt
```
Run: gemini -p "Review the test suite and suggest missing edge cases" --output-format json
```
### Artifact reporting prompt
```
Run Gemini CLI:
gemini -p "Create a branch, implement the change, commit, push, and create a PR using gh CLI. Print the full PR URL and branch name at the end." --yolo
After it finishes, parse the PR URL and branch from the output and call report_pr.
```
---
## Amp
- **Package**: `npm install -g @sourcegraph/amp`
- **Auth**: `AMP_API_KEY` environment variable. Obtain from [ampcode.com/settings](https://ampcode.com/settings). In isolated mode, uses `ANTHROPIC_API_KEY` instead.
- **Non-interactive mode**: `amp -x "<prompt>"` (execute mode)
- **Useful flags**: `--dangerously-allow-all` (skip tool approval prompts)
### Oz secret
```sh
{{warp_cli_binary_name}} secret create AMP_API_KEY --team --value-file key.txt --description "Amp API key"
```
### Example prompt
```
Run: amp -x "List all TODO comments in the codebase and group them by priority"
```
### Artifact reporting prompt
```
Run Amp:
amp --dangerously-allow-all -x "Create a branch, implement the fix, commit, push, and create a PR using gh CLI. Print the full PR URL and branch name."
After Amp finishes, parse the PR URL and branch from the output and call report_pr.
```
---
## Copilot CLI
- **Binary**: `copilot` (standalone, from [github/copilot-cli](https://github.com/github/copilot-cli))
- **Auth**: `GH_TOKEN` or `GITHUB_TOKEN` environment variable with a fine-grained PAT that has the **Copilot Requests** permission. Also supports `COPILOT_GITHUB_TOKEN`.
- **Non-interactive mode**: `copilot -p "<prompt>"`
- **Useful flags**: `--allow-all-tools`
- **Note**: The `gh copilot` extension (distinct from standalone `copilot`) requires OAuth and does **not** work with PATs.
- **Not preinstalled** in Oz images. Install via setup commands or GitHub releases.
### Oz secret
```sh
{{warp_cli_binary_name}} secret create GH_TOKEN --team --value-file token.txt --description "GitHub PAT with Copilot Requests permission"
```
### Example prompt
```
Run: copilot -p "Review the latest changes and suggest improvements" --allow-all-tools
```
### Artifact reporting prompt
```
Run Copilot CLI:
copilot -p "Create a branch, implement the fix, commit, push, and create a PR using gh CLI. Print the full PR URL and branch name at the end." --allow-all-tools
After Copilot finishes, parse the PR URL and branch from the output and call report_pr.
```
---
## OpenCode
- **Install**: `curl -fsSL https://opencode.ai/install | bash` (or via binary release)
- **Auth**: Uses provider-specific API keys via environment variables (e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`). Also reads from `.env` files. Run `opencode auth login` to configure interactively.
- **Non-interactive mode**: `opencode run "<prompt>"` or `opencode -p "<prompt>"`
- **Useful flags**: `-f json` (JSON output), `-q` (quiet/no spinner)
- **Not preinstalled** in Oz images. Install via setup commands.
### Example prompt
```
Run: opencode run "Explain the architecture of this project" -q
```
---
## Droid (Factory)
- **Install**: `curl -fsSL https://app.factory.ai/cli | sh`
- **Auth**: Requires a Factory account. Use `/login` in the CLI or generate an API key from Factory Settings. **Headless env-var auth is not yet confirmed** — this CLI may require interactive login.
- **Non-interactive mode**: `droid exec "<prompt>"`
- **Useful flags**: `--auto low|medium|high` (permission tier), `--skip-permissions-unsafe`
- **Status**: **Not currently supported** for headless Oz environments due to unclear non-interactive auth. Excluded from prebuilt images.
---
## Quick Reference
| CLI | Command | Auth Env Var | Non-Interactive Flag | Preinstalled |
|-----|---------|-------------|---------------------|-------------|
| Claude Code | `claude` | `ANTHROPIC_API_KEY` | `-p` | Yes |
| Codex | `codex` | `OPENAI_API_KEY` | `exec` | Yes |
| Gemini CLI | `gemini` | `GEMINI_API_KEY` | `-p` | Yes |
| Amp | `amp` | `AMP_API_KEY` | `-x` | No |
| Copilot CLI | `copilot` | `GH_TOKEN` / `GITHUB_TOKEN` | `-p` | No |
| OpenCode | `opencode` | Provider-specific | `run` / `-p` | No |
| Droid | `droid` | N/A (interactive login) | `exec` | No |
resources/bundled/skills/pr-comments/SKILL.md
+65
View File
@@ -0,0 +1,65 @@
---
name: pr-comments
description: "Fetch and display GitHub PR review comments for the current branch."
---
# Fetch PR Comments
Fetch all review comments from the current branch's GitHub PR and display them via `insert_code_review_comments`.
## Procedure
1. Run the bundled script (must be inside a git repo with an open PR on the current branch).
Use `do_not_summarize_output: true` when running this shell command so the JSON output is not truncated.
```bash
python3 <skill_dir>/scripts/fetch_github_review_comments.py
```
The script prints JSON to stdout.
If the script fails to fetch comments, run the fallback `gh` commands instead.
2. Call `insert_code_review_comments` with the three top-level fields from the JSON output:
- `local_repository_path`
- `base_branch`
- `comments`
3. Stop and wait for the user. After displaying each batch of comments, you MUST ask the user how they would like to proceed. Do NOT take any further action until the user provides explicit instructions unless the user explicitly asks you to.
Do NOT make code changes in response to the fetched comments unless the user tells you to. Do NOT impersonate the user by submitting review responses.
Your role when fetching and displaying comments is purely informational — present the comments and wait for direction.
## What the Script Handles
- Fetches issue comments, diff comments, and reviews via `gh api --paginate`
- Trims large diff hunks to a window around the commented line
- Sets `reply_metadata` on reply comments
- Sets `location_metadata` on top-level diff comments (filepath, trimmed diff hunk, line, side)
- PR-level comments (issue comments and reviews) have neither location nor reply metadata
## Script fallback commands
If the script fails to fetch comments, follow these steps to fetch comments directly from the GitHub API:
1. Use the GitHub cli to find the PR number, owner name, repo name, and PR base branch for the current branch.
2. Use the GitHub /repos/{owner_login}/{repo_name}/issues/{pr_number}/comments endpoint to fetch PR-level comments.
3. Use the GitHub /repos/{owner_login}/{repo_name}/pulls/{pr_number}/comments endpoint to fetch line- and file-attached review comments. Remove location metadata and diff hunks from thread replies.
4. Use the GitHub /repos/{owner_login}/{repo_name}/pulls/{pr_number}/reviews endpoint with a filter to fetch code reviews with comment text.
5. Invoke the `insert_code_review_comments` tool to send the comments to the user. Include all PR-, review-, file- and line-level comments. If there are no comments on the PR, use the tool to return an empty list. DO NOT read out the comment contents without the tool.
Ensure the pager is not used by clearing the GH_PAGER environment variable. For example, on MacOS using zsh, use:
```sh
$ GH_PAGER="" gh pr view --json number,headRepository,headRepositoryOwner,baseRefName
$ GH_PAGER="" gh api /repos/{owner_login}/{repo_name}/issues/{pr_number}/comments --jq '.[] | {id, html_url, user_login: .user.login, body, created_at, updated_at}'
$ GH_PAGER="" gh api /repos/{owner_login}/{repo_name}/pulls/{pr_number}/comments --jq '.[] | {id, html_url, diff_hunk, path, user_login: .user.login, body, created_at, updated_at, start_line, original_start_line, start_side, line, original_line, side, in_reply_to_id, subject_type} | if .in_reply_to_id != null then del(.diff_hunk, .path, .line, .original_line, .start_line, .original_start_line, .side, .start_side, .subject_type) else . end'
$ GH_PAGER="" gh api /repos/{owner_login}/{repo_name}/pulls/{pr_number}/reviews --jq '.[] | {id, html_url, user_login: .user.login, body, created_at, updated_at} | select(.body != "" and .body != null)'
```
Adapt the instructions above for the user's operating system and shell. Then invoke the `insert_code_review_comments` tool.
6. After displaying comments, follow step 3 of the Procedure above: stop and ask the user how they want to proceed. Do NOT take any action on the comments without explicit user direction.
## Requirements
- `gh` CLI authenticated with repo access
- Current branch has an open pull request
resources/bundled/skills/pr-comments/scripts/fetch_github_review_comments.py
+234
View File
@@ -0,0 +1,234 @@
#!/usr/bin/env python3
"""Fetch GitHub PR review comments and output JSON for insert_code_review_comments.
Requires: gh CLI (authenticated), git.
Must be run from within a git repository whose current branch has an open PR.
Prints JSON to stdout matching the insert_code_review_comments tool schema.
"""
import json
import os
import subprocess
import sys
from trim_diff_hunk import trim_diff_hunk, line_in_hunk, last_reachable_line
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def run_command(args, error_msg="Command failed"):
"""Run a command and return stdout. Exits on failure."""
result = subprocess.run(
args,
capture_output=True,
text=True,
encoding="utf-8",
env={**os.environ, "GH_PAGER": ""},
)
if result.returncode != 0:
print(f"{error_msg}: {result.stderr.strip()}", file=sys.stderr)
sys.exit(1)
return result.stdout
def run_gh_api(endpoint):
"""Run ``gh api --paginate`` and return a list of JSON objects.
Handles the case where ``gh`` outputs multiple concatenated JSON arrays
(one per page).
"""
text = run_command(
["gh", "api", endpoint, "--paginate"],
error_msg=f"gh api {endpoint} failed",
).strip()
if not text:
return []
items = []
decoder = json.JSONDecoder()
pos = 0
try:
while pos < len(text):
while pos < len(text) and text[pos] in " \t\n\r":
pos += 1
if pos >= len(text):
break
obj, end = decoder.raw_decode(text, pos)
items.extend(obj if isinstance(obj, list) else [obj])
pos = end
except json.JSONDecodeError as exc:
print(f"Failed to parse API response: {exc}", file=sys.stderr)
sys.exit(1)
return items
# ---------------------------------------------------------------------------
# Comment building
# ---------------------------------------------------------------------------
def _comment(cid, author, ts, body, url, location=None, reply_to=None):
"""Build a dict matching the insert_code_review_comments comment schema."""
c = {
"comment_id": cid,
"author": author,
"last_modified_timestamp": ts,
"comment_body": body,
"html_url": url,
}
if reply_to:
c["reply_metadata"] = {"parent_comment_id": reply_to}
elif location:
c["location_metadata"] = location
return c
def _resolve_line(hunk, line, original_line, side):
"""Pick the first line number that is reachable in the hunk on *side*.
Tries *line* first (current diff position), then *original_line*
(position when the comment was placed). If neither is reachable,
falls back to the last reachable line in the hunk on *side*.
Returns the resolved line number, or ``None`` if nothing is reachable.
"""
if line and line_in_hunk(hunk, line, side):
return line
if original_line and line_in_hunk(hunk, original_line, side):
return original_line
return last_reachable_line(hunk, side)
def _resolve_comment_line(comment, hunk):
"""Resolve validated (end_line, start_line, side) for a diff comment.
Uses ``side`` from the GitHub API as the authoritative diff side.
Returns ``(end_line, start_line | None, side)`` or ``None`` when the
comment cannot be attached to any line in the hunk.
"""
side = comment.get("side") or "RIGHT"
end_line = _resolve_line(
hunk,
comment.get("line"),
comment.get("original_line"),
side,
)
if end_line is None:
return None
raw_start = comment.get("start_line")
raw_original_start = comment.get("original_start_line")
if raw_start or raw_original_start:
start_line = _resolve_line(hunk, raw_start, raw_original_start, side)
if start_line is not None and start_line > end_line:
start_line = None
else:
start_line = None
return (end_line, start_line, side)
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main():
repo_root = run_command(
["git", "rev-parse", "--show-toplevel"],
"Not a git repository",
).strip()
pr = json.loads(
run_command(
[
"gh", "pr", "view",
"--json", "number,headRepository,headRepositoryOwner,baseRefName",
],
"Failed to get PR info (is there an open PR on this branch?)",
)
)
number = pr["number"]
owner = pr["headRepositoryOwner"]["login"]
repo = pr["headRepository"]["name"]
base = pr["baseRefName"]
api = f"/repos/{owner}/{repo}"
issue_comments = run_gh_api(f"{api}/issues/{number}/comments")
diff_comments = run_gh_api(f"{api}/pulls/{number}/comments")
reviews = run_gh_api(f"{api}/pulls/{number}/reviews")
comments = []
# -- Issue comments (PR-level, no location or reply metadata) -----------
for c in issue_comments:
comments.append(
_comment(
str(c["id"]),
c["user"]["login"] if c.get("user") else "[deleted]",
c["updated_at"],
c["body"],
c["html_url"],
)
)
# -- Diff comments (line-level, with location or reply) -----------------
for c in diff_comments:
cid = str(c["id"])
author = c["user"]["login"] if c.get("user") else "[deleted]"
ts = c["updated_at"]
body = c["body"]
url = c["html_url"]
reply_to_id = c.get("in_reply_to_id")
if reply_to_id:
comments.append(
_comment(cid, author, ts, body, url, reply_to=str(reply_to_id))
)
continue
hunk = c.get("diff_hunk", "")
resolved = _resolve_comment_line(c, hunk)
loc = {"filepath": c["path"]}
if resolved:
end_line, start_line, side = resolved
if hunk:
loc["diff_hunk"] = trim_diff_hunk(
hunk, end_line, side=side, start_line=start_line
)
loc["end_line"] = end_line
if start_line:
loc["start_line"] = start_line
loc["side"] = side
comments.append(_comment(cid, author, ts, body, url, location=loc))
# -- Reviews (PR-level, no location) ------------------------------------
for r in reviews:
if not r.get("body"):
continue
comments.append(
_comment(
str(r["id"]),
r["user"]["login"] if r.get("user") else "[deleted]",
r.get("submitted_at", ""),
r["body"],
r["html_url"],
)
)
result = {
"local_repository_path": repo_root,
"base_branch": base,
"comments": comments,
}
json.dump(result, sys.stdout, indent=2)
if __name__ == "__main__":
main()
resources/bundled/skills/pr-comments/scripts/test_fetch_comments.py
+202
View File
@@ -0,0 +1,202 @@
"""Tests for _resolve_comment_line and _resolve_line (no network calls).
Uses synthetic GitHub API comment dicts and diff hunks to exercise the
fallback chain: line → original_line → last reachable line → None.
"""
import sys
import os
import unittest
sys.path.insert(0, os.path.dirname(__file__))
from fetch_github_review_comments import _resolve_comment_line, _resolve_line
# ---------------------------------------------------------------------------
# A small reusable hunk: new file lines 20-24, old file lines 10-14.
# ---------------------------------------------------------------------------
_CONTEXT_HUNK = "\n".join(
["@@ -10,5 +20,5 @@"] + [f" line {10 + i}" for i in range(5)]
)
def _gh_comment(**overrides):
"""Build a minimal GitHub-API-shaped comment dict."""
base = {
"side": "RIGHT",
"line": None,
"original_line": None,
"start_line": None,
"original_start_line": None,
}
base.update(overrides)
return base
# ---------------------------------------------------------------------------
# _resolve_line
# ---------------------------------------------------------------------------
class TestResolveLine(unittest.TestCase):
def test_line_matches(self):
assert _resolve_line(_CONTEXT_HUNK, 22, None, "RIGHT") == 22
def test_line_mismatches_original_matches(self):
assert _resolve_line(_CONTEXT_HUNK, 9999, 23, "RIGHT") == 23
def test_both_mismatch_falls_back_to_last_reachable(self):
# last new-file line in the hunk is 24
assert _resolve_line(_CONTEXT_HUNK, 9999, 8888, "RIGHT") == 24
def test_empty_hunk_returns_none(self):
assert _resolve_line("", 22, None, "RIGHT") is None
def test_none_candidates_falls_back_to_last_reachable(self):
assert _resolve_line(_CONTEXT_HUNK, None, None, "RIGHT") == 24
def test_left_side(self):
assert _resolve_line(_CONTEXT_HUNK, 12, None, "LEFT") == 12
# ---------------------------------------------------------------------------
# _resolve_comment_line
# ---------------------------------------------------------------------------
class TestResolveCommentLine(unittest.TestCase):
def test_line_matches_hunk(self):
c = _gh_comment(side="RIGHT", line=22)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result == (22, None, "RIGHT")
def test_line_mismatches_original_matches(self):
c = _gh_comment(side="RIGHT", line=9999, original_line=23)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result == (23, None, "RIGHT")
def test_both_mismatch_returns_last_reachable(self):
c = _gh_comment(side="RIGHT", line=9999, original_line=8888)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
# last reachable new-file line is 24
assert result == (24, None, "RIGHT")
def test_both_mismatch_no_reachable_lines_returns_none(self):
deletion_only = "@@ -5,3 +5,0 @@\n-del A\n-del B\n-del C"
c = _gh_comment(side="RIGHT", line=9999, original_line=8888)
result = _resolve_comment_line(c, deletion_only)
assert result is None
def test_empty_hunk_returns_none(self):
c = _gh_comment(side="RIGHT", line=22)
result = _resolve_comment_line(c, "")
assert result is None
def test_both_line_and_original_none_falls_back(self):
c = _gh_comment(side="RIGHT")
result = _resolve_comment_line(c, _CONTEXT_HUNK)
# No explicit line, but the hunk has reachable lines → last reachable
assert result == (24, None, "RIGHT")
def test_start_line_resolved(self):
c = _gh_comment(side="RIGHT", line=23, start_line=21)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result == (23, 21, "RIGHT")
def test_start_line_falls_back_to_original(self):
c = _gh_comment(
side="RIGHT", line=23,
start_line=9999, original_start_line=20,
)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result == (23, 20, "RIGHT")
def test_side_defaults_to_right(self):
c = _gh_comment(line=22)
c.pop("side")
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result[2] == "RIGHT"
# -- Inverted range: start_line > end_line after fallback ----------------
def test_start_line_cleared_when_inverted(self):
"""Both start_line and original_start_line are stale.
_resolve_line falls back to last_reachable_line (24) for the start,
but end_line resolved to an earlier line (21). The fix must clear
start_line to None instead of returning an inverted range.
"""
c = _gh_comment(
side="RIGHT",
line=21,
start_line=9999,
original_start_line=8888,
)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
end_line, start_line, side = result
assert end_line == 21
assert start_line is None
assert side == "RIGHT"
def test_start_line_kept_when_not_inverted(self):
"""start_line resolves to a line before end_line — should be kept."""
c = _gh_comment(
side="RIGHT",
line=23,
start_line=21,
)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result == (23, 21, "RIGHT")
def test_start_line_cleared_when_equal_to_end(self):
"""start_line == end_line is fine — only strictly greater is inverted."""
c = _gh_comment(
side="RIGHT",
line=22,
start_line=22,
)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
assert result == (22, 22, "RIGHT")
def test_start_fallback_exceeds_end_on_left_side(self):
"""Same inverted-range scenario but on the LEFT side."""
c = _gh_comment(
side="LEFT",
line=11,
start_line=9999,
original_start_line=8888,
)
result = _resolve_comment_line(c, _CONTEXT_HUNK)
end_line, start_line, side = result
assert end_line == 11
# last_reachable_line on LEFT = 14, which > 11 → cleared
assert start_line is None
assert side == "LEFT"
# -- Regression: PR #22932 outdated comment ----------------------------
def test_regression_outdated_comment_falls_back(self):
"""Comment 2898341466: line=2899 (repositioned), hunk +3066.
Neither line nor original_line is in the hunk, so we expect the
last reachable line (3071 in the original 6-body-line hunk).
"""
hunk = (
"@@ -2926,6 +3066,17 @@ fn render_response_footer\n"
" }\n"
" }\n"
" \n"
"+ // Bulk-import review comments button\n"
"+ // has any imported review comments.\n"
"+ if props.conversation_has_imported_comments {"
)
c = _gh_comment(side="RIGHT", line=2899, original_line=2899)
result = _resolve_comment_line(c, hunk)
assert result is not None
end_line, _start, side = result
assert side == "RIGHT"
# Should fall back to last reachable new-file line in the hunk body.
assert end_line == 3071
if __name__ == "__main__":
unittest.main()
resources/bundled/skills/pr-comments/scripts/test_trim_diff_hunk.py
+597
View File
@@ -0,0 +1,597 @@
"""Tests for trim_diff_hunk and _parse_hunk_header.
Reference: app/src/code_review/comments/diff_hunk_parser_tests.rs
"""
import sys
import os
import unittest
sys.path.insert(0, os.path.dirname(__file__))
from trim_diff_hunk import trim_diff_hunk, _parse_hunk_header, _prepare_lines, _annotate_hunk_body, line_in_hunk, last_reachable_line
# ---------------------------------------------------------------------------
# Helper to build large hunks for testing
# ---------------------------------------------------------------------------
def _make_context_hunk(old_start, new_start, count):
"""Build a pure-context hunk with ``count`` lines."""
header = f"@@ -{old_start},{count} +{new_start},{count} @@"
body = [f" line {old_start + i}" for i in range(count)]
return "\n".join([header] + body)
# ---------------------------------------------------------------------------
# _prepare_lines
# ---------------------------------------------------------------------------
class TestPrepareLines(unittest.TestCase):
def test_splits_and_strips_trailing_empties(self):
assert _prepare_lines("a\nb\nc\n\n") == ["a", "b", "c"]
def test_no_trailing_empties(self):
assert _prepare_lines("a\nb") == ["a", "b"]
def test_single_line(self):
assert _prepare_lines("@@ -1,1 +1,1 @@") == ["@@ -1,1 +1,1 @@"]
def test_empty_string(self):
assert _prepare_lines("") == []
# ---------------------------------------------------------------------------
# _parse_hunk_header
# ---------------------------------------------------------------------------
class TestParseHunkHeader(unittest.TestCase):
def test_standard_header(self):
assert _parse_hunk_header("@@ -10,5 +20,7 @@") == (10, 5, 20, 7, "")
def test_header_with_context_text(self):
result = _parse_hunk_header("@@ -10,5 +20,7 @@ fn main()")
assert result == (10, 5, 20, 7, " fn main()")
def test_omitted_counts_default_to_one(self):
assert _parse_hunk_header("@@ -10 +20 @@") == (10, 1, 20, 1, "")
def test_invalid_headers(self):
assert _parse_hunk_header("not a header") is None
assert _parse_hunk_header("@@ invalid @@") is None
assert _parse_hunk_header("") is None
def test_only_old_count_omitted(self):
assert _parse_hunk_header("@@ -10 +20,3 @@") == (10, 1, 20, 3, "")
def test_only_new_count_omitted(self):
assert _parse_hunk_header("@@ -10,3 +20 @@") == (10, 3, 20, 1, "")
# ---------------------------------------------------------------------------
# trim_diff_hunk – basic / passthrough cases
# ---------------------------------------------------------------------------
class TestTrimPassthrough(unittest.TestCase):
"""Cases where trim_diff_hunk should return the input unchanged."""
def test_empty_string(self):
assert trim_diff_hunk("", 1) == ""
def test_none_input(self):
assert trim_diff_hunk(None, 1) is None
def test_small_hunk_unchanged(self):
hunk = "@@ -1,2 +1,3 @@\n first line\n+added line\n last line"
assert trim_diff_hunk(hunk, 2, context_lines=3) == hunk
def test_invalid_header_unchanged(self):
hunk = "not a valid header\n+line1\n+line2\n+line3\n+line4\n+line5"
assert trim_diff_hunk(hunk, 1) == hunk
def test_target_not_found_unchanged(self):
hunk = _make_context_hunk(1, 1, 20)
assert trim_diff_hunk(hunk, 999) == hunk
# ---------------------------------------------------------------------------
# trim_diff_hunk – trimming behaviour
# ---------------------------------------------------------------------------
class TestTrimBehaviour(unittest.TestCase):
def test_trims_to_context_window(self):
"""Large hunk trimmed to ±3 lines around target."""
hunk = _make_context_hunk(1, 1, 20)
result = trim_diff_hunk(hunk, 10, context_lines=3)
result_lines = result.split("\n")
# Target ± 3 → lines 7-13 = 7 body lines + header
assert result_lines[0] == "@@ -7,7 +7,7 @@"
assert " line 7" in result
assert " line 10" in result
assert " line 13" in result
assert " line 6" not in result
assert " line 14" not in result
def test_target_at_beginning(self):
hunk = _make_context_hunk(1, 1, 20)
result = trim_diff_hunk(hunk, 1, context_lines=3)
result_lines = result.split("\n")
# Can't go before line 1 → lo=0, hi=3
assert result_lines[0] == "@@ -1,4 +1,4 @@"
assert " line 1" in result
assert " line 4" in result
assert " line 5" not in result
def test_target_at_end(self):
hunk = _make_context_hunk(1, 1, 20)
result = trim_diff_hunk(hunk, 20, context_lines=3)
result_lines = result.split("\n")
assert result_lines[0] == "@@ -17,4 +17,4 @@"
assert " line 17" in result
assert " line 20" in result
assert " line 16" not in result
def test_preserves_whitespace(self):
"""Mirrors Rust test_parse_preserves_whitespace."""
lines = ["@@ -1,15 +1,16 @@"]
for i in range(1, 8):
lines.append(f" line {i}")
lines.append("+ heavily indented") # new line 8
for i in range(8, 16):
lines.append(f" line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 8, side="RIGHT")
assert "+ heavily indented" in result
def test_preserves_header_context_text(self):
lines = ["@@ -1,20 +1,20 @@ fn example()"]
for i in range(1, 21):
lines.append(f" line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 10)
assert result.split("\n")[0].endswith(" fn example()")
def test_trailing_empty_lines_stripped(self):
hunk = _make_context_hunk(1, 1, 20) + "\n\n\n"
result = trim_diff_hunk(hunk, 10)
assert not result.endswith("\n")
# ---------------------------------------------------------------------------
# trim_diff_hunk – LEFT / RIGHT side targeting
# ---------------------------------------------------------------------------
class TestSideTargeting(unittest.TestCase):
def test_right_side_skips_deletions(self):
"""RIGHT side tracks new-file line numbers; deletions are invisible."""
lines = ["@@ -1,12 +1,14 @@"]
for i in range(1, 4):
lines.append(f" line {i}") # old 1-3, new 1-3
lines.append("-deleted A") # old 4
lines.append("-deleted B") # old 5
lines.append("+added A") # new 4
lines.append("+added B") # new 5
lines.append("+added C") # new 6
lines.append("+added D") # new 7
for i in range(6, 14):
lines.append(f" line {i}") # old 6-13, new 8-15
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 6, side="RIGHT")
assert "+added C" in result
def test_left_side_skips_additions(self):
"""LEFT side tracks old-file line numbers; additions are invisible."""
lines = ["@@ -10,15 +10,16 @@"]
for i in range(15):
if i == 7:
lines.append("+added line") # new-only, no old num
lines.append(f" context {i}")
hunk = "\n".join(lines)
# old line 15 = old_start(10) + 5 context lines → " context 5"
result = trim_diff_hunk(hunk, 15, side="LEFT")
assert " context 5" in result
def test_left_targets_deletion(self):
"""Targeting a deleted line by old-file number."""
lines = ["@@ -1,12 +1,10 @@"]
for i in range(1, 5):
lines.append(f" line {i}")
lines.append("-removed A") # old 5
lines.append("-removed B") # old 6
for i in range(5, 13):
lines.append(f" line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 5, side="LEFT")
assert "-removed A" in result
# ---------------------------------------------------------------------------
# trim_diff_hunk – multi-line comment ranges
# ---------------------------------------------------------------------------
class TestMultilineRange(unittest.TestCase):
def test_keeps_full_range_plus_context(self):
hunk = _make_context_hunk(1, 1, 20)
# start_line=8, target=12, context=3 → window is [8-3, 12+3] = [5, 15]
result = trim_diff_hunk(hunk, 12, start_line=8, context_lines=3)
result_lines = result.split("\n")
assert result_lines[0] == "@@ -5,11 +5,11 @@"
assert " line 5" in result
assert " line 8" in result
assert " line 12" in result
assert " line 15" in result
assert " line 4" not in result
assert " line 16" not in result
def test_range_start_at_hunk_boundary(self):
hunk = _make_context_hunk(1, 1, 20)
# start_line=1, context=3 → lo clamped to 0
result = trim_diff_hunk(hunk, 5, start_line=1, context_lines=3)
assert " line 1" in result
assert " line 8" in result
# ---------------------------------------------------------------------------
# trim_diff_hunk – pure additions / pure deletions
# ---------------------------------------------------------------------------
class TestPureAdditionsAndDeletions(unittest.TestCase):
def test_only_additions(self):
"""Old-file count should be 0 in trimmed header."""
lines = ["@@ -5,0 +5,15 @@"]
for i in range(5, 20):
lines.append(f"+new line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 10, side="RIGHT")
header = result.split("\n")[0]
# All trimmed lines are additions → old count = 0
assert ",0 +" in header
assert "+new line 10" in result
def test_only_deletions(self):
"""New-file count should be 0 in trimmed header."""
lines = ["@@ -5,15 +5,0 @@"]
for i in range(5, 20):
lines.append(f"-old line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 10, side="LEFT")
header = result.split("\n")[0]
assert "+5,0 @@" in header
assert "-old line 10" in result
# ---------------------------------------------------------------------------
# trim_diff_hunk – special markers
# ---------------------------------------------------------------------------
class TestSpecialMarkers(unittest.TestCase):
def test_no_newline_marker_does_not_shift_line_numbers(self):
r"""'\ No newline at end of file' must not affect line counting."""
lines = ["@@ -1,15 +1,16 @@"]
for i in range(1, 8):
lines.append(f" line {i}")
lines.append("+added line") # new line 8
lines.append("\\ No newline at end of file")
for i in range(8, 16):
lines.append(f" line {i}")
hunk = "\n".join(lines)
# Use context=3 so the marker is within the trim window
result = trim_diff_hunk(hunk, 8, side="RIGHT", context_lines=3)
assert "+added line" in result
def test_no_newline_marker_excluded_from_counts(self):
r"""The marker should not inflate old/new counts in the header."""
lines = ["@@ -1,15 +1,16 @@"]
for i in range(1, 8):
lines.append(f" line {i}")
lines.append("+added line") # new line 8
lines.append("\\ No newline at end of file")
for i in range(8, 16):
lines.append(f" line {i}")
hunk = "\n".join(lines)
# Use context=3 so the marker is within the trim window
result = trim_diff_hunk(hunk, 8, side="RIGHT", context_lines=3)
header = result.split("\n")[0]
parsed = _parse_hunk_header(header)
old_count, new_count = parsed[1], parsed[3]
# Count body lines manually to verify
body = result.split("\n")[1:]
expected_old = sum(
1 for l in body if l and l[0] not in ("+", "\\")
)
expected_new = sum(
1 for l in body if l and l[0] not in ("-", "\\")
)
assert old_count == expected_old
assert new_count == expected_new
# ---------------------------------------------------------------------------
# trim_diff_hunk – zero context (default)
# ---------------------------------------------------------------------------
class TestZeroContext(unittest.TestCase):
"""With context_lines=0, only the exact commented line(s) are kept."""
def test_isolates_single_context_line(self):
hunk = _make_context_hunk(1, 1, 20)
result = trim_diff_hunk(hunk, 10)
result_lines = result.split("\n")
assert result_lines[0] == "@@ -10,1 +10,1 @@"
assert len(result_lines) == 2
assert " line 10" in result
def test_isolates_addition(self):
lines = ["@@ -1,5 +1,6 @@"]
for i in range(1, 4):
lines.append(f" line {i}")
lines.append("+added line") # new line 4
for i in range(4, 6):
lines.append(f" line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 4, side="RIGHT")
result_lines = result.split("\n")
assert len(result_lines) == 2
assert "+added line" in result
header = _parse_hunk_header(result_lines[0])
assert header[1] == 0 # old count = 0
assert header[3] == 1 # new count = 1
def test_isolates_deletion(self):
lines = ["@@ -1,6 +1,5 @@"]
for i in range(1, 4):
lines.append(f" line {i}")
lines.append("-deleted line") # old line 4
for i in range(5, 7):
lines.append(f" line {i}")
hunk = "\n".join(lines)
result = trim_diff_hunk(hunk, 4, side="LEFT")
result_lines = result.split("\n")
assert len(result_lines) == 2
assert "-deleted line" in result
header = _parse_hunk_header(result_lines[0])
assert header[1] == 1 # old count = 1
assert header[3] == 0 # new count = 0
def test_multiline_range_no_context(self):
hunk = _make_context_hunk(1, 1, 20)
result = trim_diff_hunk(hunk, 12, start_line=8)
result_lines = result.split("\n")
assert result_lines[0] == "@@ -8,5 +8,5 @@"
assert " line 8" in result
assert " line 12" in result
assert " line 7" not in result
assert " line 13" not in result
# ---------------------------------------------------------------------------
# line_in_hunk
# ---------------------------------------------------------------------------
class TestLineInHunk(unittest.TestCase):
def test_target_within_range(self):
hunk = _make_context_hunk(10, 20, 5)
assert line_in_hunk(hunk, 22, side="RIGHT") is True
def test_target_within_range_left(self):
hunk = _make_context_hunk(10, 20, 5)
assert line_in_hunk(hunk, 12, side="LEFT") is True
def test_target_past_end(self):
hunk = _make_context_hunk(10, 20, 5) # new 20-24
assert line_in_hunk(hunk, 30, side="RIGHT") is False
def test_target_before_start(self):
hunk = _make_context_hunk(10, 20, 5) # new 20-24
assert line_in_hunk(hunk, 5, side="RIGHT") is False
def test_right_target_in_deletion_only_hunk(self):
"""A hunk with only deletions has no reachable RIGHT lines."""
hunk = "@@ -5,3 +5,0 @@\n-del A\n-del B\n-del C"
assert line_in_hunk(hunk, 5, side="RIGHT") is False
def test_empty_hunk(self):
assert line_in_hunk("", 1) is False
def test_none_hunk(self):
assert line_in_hunk(None, 1) is False
def test_target_past_hunk_end(self):
"""Comment 2892844016: target 11078, hunk new 10816-10822."""
hunk = (
"@@ -10808,9 +10816,7 @@ impl Workspace {\n"
" comment,\n"
" diff_mode,\n"
" } => {\n"
"- if !pane_group.as_ref(ctx).right_panel_open {\n"
"- self.open_code_review_panel_from_arg(open_code_review, pane_group.clone(), ctx);\n"
"- }\n"
"+ self.open_code_review_panel_from_arg(open_code_review, pane_group.clone(), ctx);"
)
assert line_in_hunk(hunk, 11078, side="RIGHT") is False
def test_target_before_hunk_start(self):
"""Comment 2898341466: target 2899, hunk new 3066-3082."""
hunk = (
"@@ -2926,6 +3066,17 @@ fn render_response_footer(props: Props, app: &AppContext) -> Option<Box<dyn Elem\n"
" }\n"
" }\n"
" \n"
"+ // Bulk-import review comments button, shown on the latest exchange when the conversation\n"
"+ // has any imported review comments.\n"
"+ if props.conversation_has_imported_comments && !props.shared_session_status.is_viewer() {"
)
assert line_in_hunk(hunk, 2899, side="RIGHT") is False
def test_truncated_hunk(self):
"""Comment 2954102612: header claims +4770,109 but body has only 5 lines.
Simulates a truncated hunk where the target (4918) is past the body.
"""
lines = ["@@ -4769,6 +4770,109 @@ impl AIBlock {"]
for i in range(4770, 4775):
lines.append(f"+ line {i}")
hunk = "\n".join(lines)
assert line_in_hunk(hunk, 4918, side="RIGHT") is False
# ---------------------------------------------------------------------------
# last_reachable_line
# ---------------------------------------------------------------------------
class TestLastReachableLine(unittest.TestCase):
def test_context_hunk(self):
hunk = _make_context_hunk(10, 20, 5) # new 20-24
assert last_reachable_line(hunk, side="RIGHT") == 24
def test_addition_hunk(self):
hunk = "@@ -5,0 +5,3 @@\n+new A\n+new B\n+new C"
assert last_reachable_line(hunk, side="RIGHT") == 7
def test_deletion_only_right(self):
hunk = "@@ -5,3 +5,0 @@\n-del A\n-del B\n-del C"
assert last_reachable_line(hunk, side="RIGHT") is None
def test_deletion_only_left(self):
hunk = "@@ -5,3 +5,0 @@\n-del A\n-del B\n-del C"
assert last_reachable_line(hunk, side="LEFT") == 7
def test_empty_hunk(self):
assert last_reachable_line("", side="RIGHT") is None
def test_none_hunk(self):
assert last_reachable_line(None, side="RIGHT") is None
# ---------------------------------------------------------------------------
# trim_diff_hunk – multi-hunk inputs
# ---------------------------------------------------------------------------
class TestMultiHunk(unittest.TestCase):
"""trim_diff_hunk should find and trim the correct sub-hunk."""
_MULTI = (
"@@ -1,3 +1,4 @@ fn foo()\n"
" ctx1\n"
"+added_early\n"
" ctx2\n"
" ctx3\n"
"@@ -100,3 +101,4 @@ fn bar()\n"
" ctx100\n"
"+added_late\n"
" ctx101\n"
" ctx102"
)
def test_target_in_second_hunk(self):
"""Target 102 (new-file line in second sub-hunk) is found and trimmed."""
result = trim_diff_hunk(self._MULTI, 102, side="RIGHT")
assert "+added_late" in result
# The result should NOT contain the first hunk's content.
assert "+added_early" not in result
header = _parse_hunk_header(result.split("\n")[0])
assert header is not None
# new_start should be from the second hunk's range (101+)
assert header[2] == 102 # trimmed to just line 102
def test_target_in_first_hunk(self):
result = trim_diff_hunk(self._MULTI, 2, side="RIGHT")
assert "+added_early" in result
assert "+added_late" not in result
def test_target_in_no_hunk(self):
result = trim_diff_hunk(self._MULTI, 9999, side="RIGHT")
assert result == self._MULTI
def test_multi_hunk_line_in_hunk(self):
"""line_in_hunk should search across all sub-hunks."""
assert line_in_hunk(self._MULTI, 102, side="RIGHT") is True
assert line_in_hunk(self._MULTI, 2, side="RIGHT") is True
assert line_in_hunk(self._MULTI, 9999, side="RIGHT") is False
# ---------------------------------------------------------------------------
# Edge case: markdown list items in pure-addition hunks
# ---------------------------------------------------------------------------
class TestMarkdownListItem(unittest.TestCase):
r"""Hunks from where commented lines are markdown list items.
The line `+- \`specs/<issue-number>/TECH.md\`` starts with `+-`. The `-`
is a markdown list marker, NOT a diff deletion prefix.
"""
_HUNK = (
"@@ -0,0 +1,116 @@\n"
"+---\n"
"+name: write-tech-spec\n"
"+description: desc\n"
"+---\n"
"+\n"
"+# write-tech-spec\n"
"+\n"
"+Write a spec.\n"
"+\n"
"+## Overview\n"
"+\n"
"+The tech spec overview.\n"
"+\n"
"+Write specs into source control under:\n"
"+\n"
"+- `specs/<issue-number>/TECH.md`"
)
def test_annotate_hunk_body_classifies_plus_dash_as_addition(self):
"""_annotate_hunk_body must treat `+-` lines as additions."""
body = _prepare_lines(self._HUNK)[1:] # skip header
annotated = _annotate_hunk_body(body, 0, 1)
# Line 16 (new-file) should be the markdown list item.
target_entry = annotated[15] # 0-indexed
old_num, new_num, text = target_entry
assert old_num is None, f"Expected no old-file line number, got {old_num}"
assert new_num == 16, f"Expected new-file line 16, got {new_num}"
assert text == "+- `specs/<issue-number>/TECH.md`"
def test_trim_preserves_plus_dash_line(self):
"""trim_diff_hunk must preserve the full `+-` line text."""
result = trim_diff_hunk(self._HUNK, 16, side="RIGHT")
assert "+- `specs/<issue-number>/TECH.md`" in result
def test_trim_header_counts_plus_dash_as_new(self):
"""Trimmed header must count the `+-` line as a new-file line."""
result = trim_diff_hunk(self._HUNK, 16, side="RIGHT")
header = _parse_hunk_header(result.split("\n")[0])
assert header is not None
old_count, new_count = header[1], header[3]
# A single addition line: old_count=0, new_count=1
assert old_count == 0, f"Expected old_count=0, got {old_count}"
assert new_count == 1, f"Expected new_count=1, got {new_count}"
def test_line_in_hunk_finds_plus_dash_line(self):
"""line_in_hunk must locate line 16 on the RIGHT side."""
assert line_in_hunk(self._HUNK, 16, side="RIGHT") is True
def test_last_reachable_line_includes_plus_dash(self):
"""last_reachable_line must reach line 16 on the RIGHT side."""
assert last_reachable_line(self._HUNK, side="RIGHT") == 16
if __name__ == "__main__":
unittest.main()
resources/bundled/skills/pr-comments/scripts/trim_diff_hunk.py
+212
View File
@@ -0,0 +1,212 @@
"""Diff-hunk trimming for unified diffs.
Mirrors the logic in app/src/code_review/comments/diff_hunk_parser.rs:
walk hunk lines tracking old/new file line numbers, locate the target line,
then trim unneeded lines from the start and end (never the middle) and
rewrite the hunk header to match the trimmed window.
"""
import re
_HUNK_HEADER_RE = re.compile(
r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@(.*)$"
)
def _parse_hunk_header(line):
m = _HUNK_HEADER_RE.match(line)
if not m:
return None
return (
int(m.group(1)),
int(m.group(2)) if m.group(2) is not None else 1,
int(m.group(3)),
int(m.group(4)) if m.group(4) is not None else 1,
m.group(5),
)
def _annotate_hunk_body(body_lines, old_start, new_start):
"""Annotate *body_lines* with ``(old_num | None, new_num | None, text)``."""
old_num, new_num = old_start, new_start
annotated = []
for text in body_lines:
ch = text[0] if text else " "
if ch == "+":
annotated.append((None, new_num, text))
new_num += 1
elif ch == "-":
annotated.append((old_num, None, text))
old_num += 1
elif ch == "\\":
annotated.append((None, None, text))
else:
annotated.append((old_num, new_num, text))
old_num += 1
new_num += 1
return annotated
def _split_hunks(lines):
"""Split *lines* (with trailing empties already stripped) into sub-hunks.
Returns a list of ``(header_tuple, header_text, body_lines)`` where
*header_tuple* is the parsed ``(old_start, old_count, new_start, new_count,
ctx)`` and *body_lines* are the raw diff body strings.
"""
hunks = []
current_header = None
current_header_text = None
current_body = []
for line in lines:
parsed = _parse_hunk_header(line)
if parsed is not None:
if current_header is not None:
hunks.append((current_header, current_header_text, current_body))
current_header = parsed
current_header_text = line
current_body = []
else:
current_body.append(line)
if current_header is not None:
hunks.append((current_header, current_header_text, current_body))
return hunks
def _find_target_idx(annotated, target_line, use_new):
"""Return the index into *annotated* where the target line lives, or None."""
for i, (o, n, _) in enumerate(annotated):
num = n if use_new else o
if num is not None and num == target_line:
return i
return None
# ---------------------------------------------------------------------------
# Public helpers used by fetch_github_review_comments.py
# ---------------------------------------------------------------------------
def _prepare_lines(diff_hunk):
"""Split *diff_hunk* into lines and strip trailing empty strings."""
lines = diff_hunk.split("\n")
while lines and lines[-1] == "":
lines.pop()
return lines
def line_in_hunk(diff_hunk, target_line, side="RIGHT"):
"""Return ``True`` if *target_line* is reachable on *side* of the hunk."""
if not diff_hunk:
return False
lines = _prepare_lines(diff_hunk)
use_new = side != "LEFT"
for header, _, body in _split_hunks(lines):
old_start, _, new_start, _, _ = header
annotated = _annotate_hunk_body(body, old_start, new_start)
if _find_target_idx(annotated, target_line, use_new) is not None:
return True
return False
def last_reachable_line(diff_hunk, side="RIGHT"):
"""Return the last line number reachable on *side*, or ``None``."""
if not diff_hunk:
return None
lines = _prepare_lines(diff_hunk)
use_new = side != "LEFT"
last = None
for header, _, body in _split_hunks(lines):
old_start, _, new_start, _, _ = header
for o, n, _ in _annotate_hunk_body(body, old_start, new_start):
num = n if use_new else o
if num is not None:
last = num
return last
# ---------------------------------------------------------------------------
# trim_diff_hunk
# ---------------------------------------------------------------------------
def trim_diff_hunk(diff_hunk, target_line, side="RIGHT", start_line=None, context_lines=0):
"""Return *diff_hunk* trimmed to ±*context_lines* around *target_line*.
The hunk header is rewritten so the line numbers stay correct.
If the hunk is already small enough, it is returned unchanged.
"""
if not diff_hunk:
return diff_hunk
lines = _prepare_lines(diff_hunk)
hunks = _split_hunks(lines)
if not hunks:
return diff_hunk
use_new = side != "LEFT"
# Find the sub-hunk that contains the target line.
for header, _header_text, body in hunks:
old_start, _, new_start, _, hdr_ctx = header
annotated = _annotate_hunk_body(body, old_start, new_start)
target_idx = _find_target_idx(annotated, target_line, use_new)
if target_idx is None:
continue
# Found the right sub-hunk — trim within it.
if len(annotated) <= context_lines * 2 + 1:
# Small enough already — return just this sub-hunk.
new_hdr = _rewrite_header(annotated, old_start, new_start, hdr_ctx)
return "\n".join([new_hdr] + [t for _, _, t in annotated])
range_start_idx = None
if start_line:
range_start_idx = _find_target_idx(annotated, start_line, use_new)
first = range_start_idx if range_start_idx is not None else target_idx
lo = max(0, first - context_lines)
hi = min(len(annotated) - 1, target_idx + context_lines)
trimmed = annotated[lo : hi + 1]
new_hdr = _rewrite_header(trimmed, old_start, new_start, hdr_ctx)
return "\n".join([new_hdr] + [t for _, _, t in trimmed])
# Target not found in any sub-hunk — return original unchanged.
return diff_hunk
def _rewrite_header(trimmed, old_start, new_start, hdr_ctx):
"""Build a unified diff header from a trimmed annotation window."""
t_os = t_ns = None
t_oc = t_nc = 0
for o, n, text in trimmed:
ch = text[0] if text else " "
if ch == "+":
t_nc += 1
if t_ns is None and n is not None:
t_ns = n
elif ch == "-":
t_oc += 1
if t_os is None and o is not None:
t_os = o
elif ch == "\\":
pass
else:
t_oc += 1
t_nc += 1
if t_os is None and o is not None:
t_os = o
if t_ns is None and n is not None:
t_ns = n
return f"@@ -{t_os or old_start},{t_oc} +{t_ns or new_start},{t_nc} @@{hdr_ctx}"
resources/bundled/skills/tab-configs/SKILL.md
+263
View File
@@ -0,0 +1,263 @@
---
name: tab-configs
description: Reference the Warp tab config schema, validation rules, and examples. Use when creating or updating Warp tab config TOML files or when another tab-config skill needs the canonical schema details.
---
# tab-configs
This skill is the canonical reference for Warp tab config TOML files.
Tab configs live under `~/.warp/tab_configs/` for standard Warp builds. Non-stable builds (e.g. Preview or pre-release channels) use a channel-specific variant such as `~/.warp-<channel>/tab_configs/`. Do NOT use `~/Library/Application Support/` — that is the wrong location.
To find the active data directory, inspect which `~/.warp*` directories exist on the user's machine. Each `.toml` file defines a single tab layout that appears in the `+` menu. When the user selects one, Warp opens a new tab with the configured pane layout and runs any specified commands.
Use this skill as shared context:
- `create-tab-config` should use it when authoring a new config.
- `update-tab-config` should use it when modifying an existing config.
- If this skill is invoked directly for a creation or editing task, follow the same guidance below.
## Guidance before writing or editing
- If the request is missing important details, use the `ask_user_question` tool instead of guessing. Clarify things like:
- whether the user wants a new config or wants to edit an existing one
- the desired pane layout
- directories, commands, and which pane should be focused
- whether parameters are needed
- Use descriptive pane IDs such as `"editor"`, `"server"`, or `"top_left"` instead of generic names like `"pane1"`.
- Use snake_case file names for new configs.
- When creating or locating tab configs, use the correct tab config directory for the user's current Warp build/channel rather than hardcoding a single base directory.
- If updating an existing config, read the file first and preserve user structure where possible.
## Schema Reference
### Top-level fields
- `name` (required, string): Display name shown in the `+` menu.
- `title` (optional, string): Custom tab title. Supports `{{param}}` template variables.
- `color` (optional, string): Tab color. One of: `"black"`, `"red"`, `"green"`, `"yellow"`, `"blue"`, `"magenta"`, `"cyan"`, `"white"`.
### Pane list
All panes are defined in a flat `[[panes]]` array of tables. The **first entry is the root**. Each entry is either a **split node** (branch) or a **leaf node**.
#### Leaf node fields
- `id` (required, string): Unique identifier.
- `type` (required, string): `"terminal"` (standard shell), `"agent"` (opens in Agent Mode), or `"cloud"` (cloud mode pane, no local shell).
- `directory` (optional, string): Initial working directory. Supports `~` expansion. Applies to `terminal` and `agent` types.
- `commands` (optional, array of strings): Commands to run in sequence on open. Applies to `terminal` and `agent` types.
- `is_focused` (optional, bool): Set `true` on at most one pane to give it initial focus.
- `shell` (optional, string): Shell executable to open this pane in (e.g. `"pwsh"`, `"zsh"`, `"bash"`, `"fish"`). Only applies to `terminal` and `agent` types. If omitted or the shell is not installed, the user's default shell is used.
#### Split node fields
- `id` (required, string): Unique identifier for this node.
- `split` (required, string): `"horizontal"` (children arranged left-to-right) or `"vertical"` (children arranged top-to-bottom).
- `children` (required, array of strings): Ordered list of child pane IDs. Must have 2+ entries. Order = visual order.
```toml
[[panes]]
id = "server"
type = "terminal"
directory = "~/code/backend"
commands = ["cargo watch -x run"]
```
### Sizing
All children within a split are equally sized. There are no flex or proportion values.
### Parameters
Parameters let users fill in values at open time via a modal UI. Declare them with `[params.<name>]` tables. Reference them in `directory`, `commands`, and `title` with `{{name}}` syntax.
`{{autogenerated_branch_name}}` is a special-cased reserved template variable: if a tab config references it, Warp generates a unique worktree branch name on each open instead of prompting the user for that value.
Each parameter has:
- `type` (optional, string): `"text"` (default), `"branch"` (git branch picker), or `"repo"` (repo picker).
- `description` (optional, string): Shown in the fill-in UI.
- `default` (optional, string): Default value.
```toml
[params.project_dir]
type = "repo"
description = "Project directory"
[params.branch]
type = "branch"
description = "Branch to check out"
```
### Validation rules
Ensure your output follows these rules:
- Every ID in a `children` array must reference an existing `[[panes]]` entry.
- No cycles in the pane tree.
- Leaf panes must NOT have `children` or `split`.
- Leaf panes must have a `type` field (`"terminal"`, `"agent"`, or `"cloud"`).
- Split panes (have `split`) must have `children` with at least 2 entries.
- At most one pane may have `is_focused = true`.
- All `id` values must be unique across the file.
- The first `[[panes]]` entry is the root of the tree.
## Examples
### Single pane
```toml
name = "Dev Server"
[[panes]]
id = "main"
type = "terminal"
directory = "~/code/my-app"
commands = ["npm run dev"]
```
### Two panes side by side
```toml
name = "Editor + Server"
color = "green"
[[panes]]
id = "root"
split = "horizontal"
children = ["editor", "server"]
[[panes]]
id = "editor"
type = "terminal"
directory = "~/code/my-app"
commands = ["nvim ."]
is_focused = true
[[panes]]
id = "server"
type = "terminal"
directory = "~/code/my-app"
commands = ["npm run dev"]
```
### 2x2 grid
```toml
name = "Full Stack"
[[panes]]
id = "root"
split = "horizontal"
children = ["left_col", "right_col"]
[[panes]]
id = "left_col"
split = "vertical"
children = ["frontend", "backend"]
[[panes]]
id = "frontend"
type = "terminal"
directory = "~/code/frontend"
commands = ["npm run dev"]
is_focused = true
[[panes]]
id = "backend"
type = "terminal"
directory = "~/code/backend"
commands = ["cargo watch -x run"]
[[panes]]
id = "right_col"
split = "vertical"
children = ["tests", "logs"]
[[panes]]
id = "tests"
type = "terminal"
directory = "~/code/backend"
commands = []
[[panes]]
id = "logs"
type = "terminal"
directory = "~/code/backend"
commands = ["tail -f logs/dev.log"]
```
### Three-pane layout (1 left, 2 stacked right)
```toml
name = "Main + Side Stack"
[[panes]]
id = "root"
split = "horizontal"
children = ["main", "side_stack"]
[[panes]]
id = "main"
type = "terminal"
directory = "~/code/project"
commands = []
is_focused = true
[[panes]]
id = "side_stack"
split = "vertical"
children = ["top_right", "bottom_right"]
[[panes]]
id = "top_right"
type = "terminal"
directory = "~/code/project"
commands = []
[[panes]]
id = "bottom_right"
type = "terminal"
directory = "~/code/project"
commands = []
```
### With parameters
Warp-generated worktree configs default to `~/.warp/worktrees/<repo-name>/<worktree-name>`. When writing a reusable config by hand, derive the repo-name segment at shell runtime with `$(basename {{repo}})`.
```toml
name = "New Worktree"
title = "{{branch_name}}"
[[panes]]
id = "main"
type = "terminal"
directory = "{{repo}}"
commands = [
"git worktree add -b {{branch_name}} $HOME/.warp/worktrees/$(basename {{repo}})/{{branch_name}} {{base_branch}}",
"cd $HOME/.warp/worktrees/$(basename {{repo}})/{{branch_name}}",
]
[params.repo]
type = "repo"
description = "Repository path"
[params.base_branch]
type = "branch"
description = "Base branch to branch from"
[params.branch_name]
type = "text"
description = "New branch name"
default = "my-feature"
```
## Common patterns
When the user says... | Generate...
"single pane" / "simple tab" | One `[[panes]]` leaf entry
"split" / "side by side" / "two panes" | Horizontal split with 2 terminal children
"top and bottom" / "stacked" | Vertical split with 2 terminal children
"2x2" / "grid" / "four panes" | Horizontal split → 2 vertical splits → 4 terminals
"run X" / "auto-run X" | Put `X` in the `commands` array
"with claude" / "running claude" | `commands = ["claude"]` in the relevant pane
"three panes" (no further detail) | Horizontal split: 1 terminal + 1 vertical split with 2 terminals
resources/bundled/skills/update-tab-config/SKILL.md
+25
View File
@@ -0,0 +1,25 @@
---
name: update-tab-config
description: Update existing Warp tab config TOML files from natural-language edit requests. Use when the user wants to modify a tab config that already exists or when editing a tab config file already open in Warp.
---
# update-tab-config
Update an existing Warp tab config in place.
## Required context
- Use the `tab-configs` skill as the canonical source of truth for:
- schema details
- validation rules
- examples
- common layout patterns
## Workflow
1. Read the existing tab config file before making changes.
2. Understand the requested edit.
3. If important details are missing or ambiguous, use the `ask_user_question` tool before editing. Do not guess about layout changes, command changes, parameters, or `on_close` behavior.
4. Make sure you are editing the tab config that belongs to the user's current Warp build/channel rather than assuming a single hardcoded base directory, then update it so it remains valid according to the `tab-configs` schema.
5. Preserve the user's existing structure and naming where possible unless the requested change requires restructuring.
6. Briefly explain what changed.