public/galaxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial public release of Warp.

Browse Source 
Repo-Sync-Origin: warpdotdev/warp-internal@12af1d983b
This commit is contained in:
David Stern
2026-04-28 08:43:33 -05:00
commit 0dbd3d567a
4982 changed files with 1431549 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
specs/kevinchevalier/APP-3857/PRODUCT.md
+103
View File
@@ -0,0 +1,103 @@
# APP-3857: Memory-safe animated image rendering in markdown
## Summary
Warp should continue to display animated image sources embedded in rendered markdown without causing large, unbounded memory spikes. In rendered markdown and notebook views, animated images should appear as inline static previews of their first frame, matching the current markdown UX while avoiding the multi-frame decode and resize work that currently drives excessive memory usage.
## Problem
Today, large animated images in markdown can drive Warp into multi-GB memory usage. That makes a document viewer path unreliable: opening or scrolling a markdown file can cause severe jank, memory pressure, or crashes even though the user only needs to read the document. Users need to be able to open and view markdown documents containing animated images without Warp becoming unstable.
## Goals
- Allow markdown documents containing animated images to open and render inline in Warp's markdown viewer.
- Preserve the existing markdown viewing experience for animated image sources by rendering them as inline first-frame previews.
- Bound memory use so a single image or document does not cause the multi-GB spike seen in the Petra repro.
- Degrade gracefully when even extracting a preview frame is expensive or fails.
- Preserve authored markdown, selection, copy, and export behavior.
## Non-goals
- Adding new playback controls, autoplay, zoom controls, or user preferences for markdown image animation.
- Reworking ordinary static image rendering behavior beyond what is needed to keep markdown image handling safe.
- Expanding this ticket to every image-rendering surface in Warp outside the markdown and notebook viewer.
- Rewriting the user's markdown or image assets on disk.
- Guaranteeing perfect animation fidelity for very large assets if doing so would violate the memory-safety goal.
## Figma / design references
Figma: none provided
This is a behavior-focused change. Existing markdown image visuals and layout should be reused unless the image falls back to a static preview.
## User experience
### In-scope surfaces
- Rendered markdown files and notebook markdown content that use Warp's existing markdown image blocks.
- Raw markdown and editable markdown views remain unchanged and continue to show the source markdown.
### Default behavior
- Animated image sources referenced from markdown render inline in document flow, using the same sizing and layout rules that markdown images already use.
- In rendered markdown, animated GIFs and animated WebPs are shown as a static preview of their first visible frame.
- This iteration adds no new controls, settings, badges, or toasts for normal animated-image preview rendering.
### Resource-safe rendering behavior
- Warp must not require multi-GB memory usage to display an animated image source in markdown.
- Warp should do only the work required to show a static first-frame preview rather than decode, cache, or resize the full animation.
- The document layout must remain stable. Rendering an animated image preview must not cause surrounding markdown blocks to overlap, jump unexpectedly, or reserve obviously incorrect space.
### Graceful handling for problematic animated assets
- If Warp can decode enough of an animated image to extract a first visible frame, it should show that static preview.
- The static preview should occupy the same rendered markdown block footprint that the image would otherwise use, so this behavior does not create a separate layout mode.
- If the image cannot be decoded even far enough to extract a preview frame, existing image-load failure behavior remains unchanged.
- Preview-only handling is silent in this iteration. Warp does not add a warning label or toast merely because the source asset is animated.
### Failure handling
- If an image is too expensive to animate safely but can still be decoded enough for a preview, the user sees the static preview rather than a broken or empty region.
- If the image itself cannot be loaded or decoded at all, existing image-load failure behavior remains unchanged.
- One problematic animated image must not prevent the rest of the markdown document from rendering and remaining usable.
### Editing, copy, and export behavior
- Warp does not rewrite the source markdown. The authored `![](...)` reference remains unchanged.
- Selection, copy, and export behavior continue to preserve the original markdown image reference rather than serializing a transformed static asset.
- This ticket does not introduce a new user-visible export format for rendered animated images.
### Scope boundaries for this iteration
- This spec applies to markdown document viewing, not to terminal image protocols, chat attachments, or other non-markdown image surfaces.
- This iteration optimizes for "can open and read the document safely" rather than for pixel-perfect playback of extremely large animations.
## Success criteria
1. Opening the Petra repro markdown associated with `APP-3857` no longer causes the multi-GB memory spike represented by `heap-profile-petra-1.pb`.
2. The affected markdown document remains readable instead of causing Warp to freeze, crash, or become unusably slow.
3. Animated GIFs and animated WebPs in rendered markdown display as static first-frame previews without decoding and resizing every frame into memory.
4. Large animated assets no longer cause runaway memory growth just to display a markdown preview.
5. Preview rendering preserves the same document layout footprint as the rendered image block.
6. A document containing one expensive or malformed animated image still renders the rest of its markdown content correctly.
7. Raw markdown, selection, copy, and export continue to preserve the original markdown image reference.
8. This change does not introduce a visible regression for ordinary static images or for modest animated image sources viewed as static previews in the same markdown viewer.
## Validation
- Open the Petra repro markdown document associated with `APP-3857` and verify that Warp remains usable while the document is opened and scrolled.
- Measure memory while opening and scrolling the repro document and confirm the previous multi-GB spike pattern is gone rather than merely delayed.
- Verify that a modest animated GIF or animated WebP in markdown renders as a stable first-frame preview in rendered view.
- Verify that a large animated image renders a stable first-frame preview instead of blank content, a broken-image state, or a crash.
- Verify that raw markdown and markdown export still preserve the original `![](...)` reference for animated images.
- Verify that markdown documents containing a mix of text, static images, and animated images still lay out correctly and remain selectable and copyable.
- Add automated coverage for:
- first-frame preview rendering for animated GIF and animated WebP markdown images
- graceful handling for an oversized or expensive animated asset
- a regression guard against the previous all-frames-at-once memory behavior at the viewer or image-cache layer
## Open questions
- Do we want a numeric memory budget for markdown animated-image preview rendering in a follow-up, or is "no multi-GB spike on the known repro" sufficient for this iteration?
- In a future iteration, should Warp expose opt-in playback for animated images in markdown once a memory-safe animation path exists?
specs/kevinchevalier/APP-3857/TECH.md
+99
View File
@@ -0,0 +1,99 @@
# APP-3857: Memory-safe animated image rendering in markdown
See `specs/kevinchevalier/APP-3857/PRODUCT.md` for the product spec.
## Problem
Markdown image blocks currently pay for full animated-image decoding even though the markdown renderer does not actually play animations. `RenderableImage` builds a plain `warpui::elements::Image` without `enable_animation_with_start_time`, so the markdown surface effectively shows only the first frame today (`crates/editor/src/render/element/image.rs:13-49`, `crates/warpui_core/src/elements/image.rs:107-172`). However, the shared image asset path still decodes animated GIF/WebP assets into `AnimatedImage { frames: Vec<...> }`, and the resize path then creates resized copies of every decoded frame (`crates/warpui_core/src/image_cache.rs (259-359)`, `crates/warpui_core/src/image_cache.rs (457-511)`). On large assets this produces the multi-GB memory spike seen in `APP-3857`.
## Relevant code
- `crates/editor/src/content/edit.rs:66-97` — resolves markdown image source strings into `AssetSource`.
- `crates/editor/src/content/edit.rs:677-759` — converts `BufferBlockItem::Image` into a laid-out `BlockItem::Image` with default markdown image sizing.
- `crates/editor/src/render/element/image.rs:13-49` — markdown render block that constructs `warpui::elements::Image` from the resolved asset source.
- `crates/warpui_core/src/elements/image.rs:107-172` — animation only occurs when callers explicitly provide a start time; otherwise the first frame is painted.
- `crates/warpui_core/src/image_cache.rs (259-359)` — `ImageType::try_from_bytes` currently uses `collect_frames()` for animated GIF and animated WebP inputs.
- `crates/warpui_core/src/image_cache.rs (457-511)` — animated resize path duplicates every frame at the target render size.
- `crates/warpui_core/src/image_cache.rs:666-772` — shared image-cache entry point; caches rendered images by asset source hash plus render properties and is where preview-vs-animation behavior can diverge without changing the underlying asset type.
- `crates/warpui_core/src/assets/asset_cache.rs:284-320` — asset cache keys include both `AssetSource` and `TypeId`; this implementation continues to use the existing `ImageType` asset path and keeps full animated assets there.
- `app/src/resource_center/section_views/changelog_section.rs:152` — an existing real app surface that explicitly enables animation and therefore must keep the current full-animation path.
- `crates/warpui_core/src/image_cache_tests.rs (1-261)` and `crates/editor/src/content/markdown_tests.rs (270-337)` — the closest existing test coverage for image decoding/cache behavior and markdown image handling.
## Current state
- Markdown image syntax is parsed as a block item and stored as `BufferBlockItem::Image { alt_text, source }`, then laid out as `BlockItem::Image { asset_source, config, .. }` using the shared editor/render pipeline (`crates/editor/src/content/text.rs:302-373`, `crates/editor/src/content/edit.rs:677-759`).
- The markdown render element constructs a generic `warpui::elements::Image` with `.contain()` and no animation start time (`crates/editor/src/render/element/image.rs:42-49`).
- The `Image` element will animate only if a caller explicitly opts in via `enable_animation_with_start_time`; otherwise it repeatedly paints frame 0 (`crates/warpui_core/src/elements/image.rs:107-172`).
- Despite that static markdown UX, the shared asset loader eagerly decodes animated GIF/WebP files into an in-memory `AnimatedImage` containing all RGBA frames (`crates/warpui_core/src/image_cache.rs (259-359)`).
- `ImageCache::image` then resizes animated assets by resizing every frame, which creates a second full set of frame buffers at markdown display size (`crates/warpui_core/src/image_cache.rs (457-511)`, `crates/warpui_core/src/image_cache.rs:666-772`).
- This is why the current surface is especially wasteful: markdown gets only a first-frame preview, but pays the memory cost of full animation decode plus resized-frame duplication.
## Proposed changes
- Add an explicit animated-image behavior to `warpui::elements::Image`, with two modes:
- `FullAnimation` — current behavior for surfaces that intentionally animate.
- `FirstFramePreview` — render animated sources as a static first-frame image.
- Keep `FullAnimation` as the default so existing animated callers continue to work unchanged. Markdown will opt into `FirstFramePreview` at the call site.
- Keep a single `ImageType` asset path in `crates/warpui_core/src/image_cache.rs`. Animated GIF/WebP assets will continue to decode into `AnimatedImage` in the asset cache, but `ImageCache::image` will materialize `FirstFramePreview` requests as a static first-frame render result instead of resizing every decoded animation frame.
- Thread the new behavior into `ImageCache::image` so the rendered-image cache chooses between a static preview and a full animation from the same underlying `ImageType` asset.
- Replace the current rendered-image cache key shape with a struct that includes, at minimum:
- asset source hash
- target bounds
- animated-image behavior
This avoids collisions where the same source and bounds are requested once as a full animation and once as a static preview.
- Update `crates/editor/src/render/element/image.rs` so markdown image blocks always construct `Image::new(asset_source).contain().first_frame_preview()` (or equivalent builder naming). No markdown storage, export, or copy behavior changes are needed.
- Leave explicit animated callers unchanged. `app/src/resource_center/section_views/changelog_section.rs:152` should continue to opt into animation, and the `warpui` animated-image example should still exercise the full-animation path.
- Do not implement a bounded frame-buffer animation system in this ticket. That is the browser / media-viewer style solution, but it is unnecessary for this markdown surface because markdown does not animate today. The simpler preview-only split matches the current product behavior and removes the blow-up at the source.
- This mirrors common approaches elsewhere:
- browser/media stacks like Chromium separate decode from caching and leave room for smarter bounded frame caches
- native image libraries often provide either a rolling frame buffer for true animation (for example Gifu) or a first-frame-only preview mode for list/thumbnail surfaces (for example Kingfisher)
## End-to-end flow
```mermaid
graph TD
A[Markdown ![](...)] --> B[BufferBlockItem::Image]
B --> C[LayoutTask::Image resolves AssetSource]
C --> D[BlockItem::Image]
D --> E[RenderableImage]
E --> F[warpui::elements::Image with FirstFramePreview]
F --> G[ImageCache preview path]
G --> H[AssetCache.load_asset<ImageType>]
H --> I[Decode AnimatedImage once]
I --> J[ImageCache extracts and resizes only frame 0]
J --> K[Paint as static markdown image]
L[Changelog / explicit animated surface] --> M[Image.enable_animation_with_start_time]
M --> N[ImageCache full-animation path]
N --> O[AssetCache.load_asset<ImageType>]
O --> P[AnimatedImage]
```
## Risks and mitigations
- **Risk: preview/full cache collisions.**
- Mitigation: include animated-image behavior in the rendered-image cache key rather than relying on `AssetSource + bounds` only.
- **Risk: the asset cache still holds full animated frame data.**
- Mitigation: acceptable for this version of the fix. The user-facing memory win comes from avoiding resized copies of every frame in `ImageCache`, and we add tests that preview requests still render static output while the asset cache keeps the full animated asset. If this is still too expensive in practice, a follow-up can teach `AssetCache` to cache raw bytes or accept decode parameters.
- **Risk: accidental regression for existing animated surfaces.**
- Mitigation: keep `FullAnimation` as the default, make preview behavior opt-in at the markdown call site, and add tests that the explicit animated path still produces `Image::Animated`.
- **Risk: very large single-frame assets remain expensive.**
- Mitigation: acceptable for this ticket because the current failure is multi-frame explosion. Follow-up work can add explicit decode limits or dimension guards if needed.
- **Risk: first-frame choice differs slightly from a browser’s composited “current frame” semantics for malformed files.**
- Mitigation: the product spec only requires a stable first visible frame preview, not browser-perfect animation fidelity.
## Testing and validation
- Add `warpui_core` unit tests with small animated GIF and animated WebP fixtures:
- preview policy returns `Image::Static`
- full-animation policy still returns `Image::Animated`
- preview policy does not regress static formats
- Add an image-cache test that preview and full-animation requests for the same source/bounds do not collide in the rendered-image cache.
- Add/extend markdown image tests only where needed to confirm markdown export and serialization still preserve the authored `![](...)` reference.
- Manually validate against the Petra repro markdown from `APP-3857`:
- open the document
- scroll through it
- confirm the old multi-GB spike is gone
- confirm animated image sources render as stable first-frame previews
- Run the usual repo validation before implementation is considered done:
- `cargo fmt`
- targeted Rust tests for the touched editor / warpui_core modules
- wasm build check for this repo, since image/rendering code is shared across desktop and wasm-adjacent targets
## Follow-ups
- If product later wants real markdown animation, add a bounded frame-buffer animation path instead of reusing the eager all-frames asset.
- Add explicit decode limits / pixel-budget guards for pathological single-frame assets if the Petra fix reveals a second-order issue there.
- Consider reusing the preview-only animated-image behavior in any other list, thumbnail, or markdown-adjacent surfaces that currently do not need full animation.