galaxy/specs/zachlloyd/wide-markdown-table-scrolling/TECH.md

Wide Markdown Table Scrolling — Tech Spec

Product spec: specs/zachlloyd/wide-markdown-table-scrolling/PRODUCT.md

Context

The branch implements the behavior described in the sibling PRODUCT.md: wide Markdown tables scroll horizontally inside their own viewport, and selection / caret / hit-testing / copy stay attached to content while the table or any clipped surface is scrolled.

Investigation against crates/markdown_parser/examples/table-demo/all_test_cases.md surfaced two classes of bugs that both needed to be addressed:

• Editor-backed tables did not yet own a horizontal viewport, so wide tables simply overflowed the surrounding editor.
• Once a cell contained inline Markdown (**Bold**, *Italic*, links), source offsets and rendered offsets were mixed in selection and copy paths, producing drift even before the wide-table work.

Separately, the shared clipped-scrollable path could re-use stale screen-space coordinates after a scroll, which showed up as wrong word selection and wrong copied text.

Relevant code

• crates/editor/src/render/model/mod.rs — LaidOutTable (horizontal scroll state, reveal logic, character bounds) and RenderState::autoscroll (table-aware reveal)
• crates/editor/src/render/element/table.rs — RenderableTable painting, clipping, and event dispatch for scroll/drag/hover
• crates/editor/src/render/model/location.rs — hit-testing that accounts for local table scroll
• crates/editor/src/render/model/table_offset_map.rs — TableOffsetMap (table-wide) and TableCellOffsetMap (per-cell source↔rendered)
• crates/editor/src/render/layout.rs — TextLayout::with_container_scrolls_horizontally carries the carve-out flag
• crates/editor/src/content/text.rs — BufferBlockStyle::Table variant, TableCache / TableBlockCache, and table_cell_offset_maps
• crates/editor/src/content/edit.rs — layout_table_block, measure_table_cells, per-cell size clamp
• crates/editor/src/content/buffer.rs — table-aware clipboard text extraction and partial-table HTML filtering
• crates/warpui_core/src/elements/shared_scrollbar.rs — shared ScrollbarAppearance / ScrollbarGeometry and scrollbar math
• crates/warpui_core/src/elements/clipped_scrollable.rs — selection anchor (anchor_and_adjust_selection_for_scroll)
• crates/warpui_core/src/elements/new_scrollable/mod.rs — dispatches selection APIs through the anchor helper
• crates/warpui_core/src/elements/formatted_text_element.rs — horizontal bounds check for smart selection
• app/src/notebooks/editor/mod.rs — notebook table appearance (scrollbar colors, etc.)

Proposed changes

Shared scrollbar primitives in warpui_core

crates/warpui_core/src/elements/shared_scrollbar.rs is the single source of truth for:

• ScrollbarAppearance / ScrollbarGeometry (overlay scrollbar geometry, thumb bounds, track bounds)
• Minimum thumb sizing (MIN_SCROLLBAR_THUMB_LENGTH)
• compute_scrollbar_geometry(...)
• project_scroll_delta_by_sensitivity(...) for axis projection of mixed-axis gestures
• scroll_delta_for_pointer_movement(...) for pointer-drag-to-scroll translation

Both the editor table renderer and the existing new_scrollable utilities call through these helpers. The goal is not to nest a NewScrollable inside tables but to share scrollbar math and mixed-axis resolution.

Editor tables own a local horizontal viewport

LaidOutTable in crates/editor/src/render/model/mod.rs gains local horizontal scroll state:

• scroll_left: Cell<Pixels>
• TableScrollbarInteractionState (drag state, hovered)
• viewport_width(), max_scroll_left(), set_scroll_left(), scroll_horizontally()
• reveal_offset() for keyboard/caret autoscroll
• horizontal_scroll_allowed flag (see carve-out below)

RenderableTable::paint in crates/editor/src/render/element/table.rs:

• Computes a clipped viewport rectangle for the visible region.
• Translates painted content by scroll_left.
• Paints inside a clipped layer.
• Derives the thumb via compute_scrollbar_geometry() and paints it using notebook theme colors from MarkdownTableAppearance / TableStyle.

RenderableTable::dispatch_event handles LeftMouseDown (thumb + gutter), LeftMouseDragged, LeftMouseUp, MouseMoved, and ScrollWheel. Two event-propagation rules worth calling out:

• The ScrollWheel handler returns the boolean result of scroll_horizontally() rather than unconditionally consuming the event. When the table is pinned at an edge, the event falls through to the surrounding vertical scroller (PRODUCT invariant 11).
• The MouseMoved handler returns false regardless of whether the pointer is over the thumb; scrollbar hover state is still updated via set_scrollbar_hovered() and ctx.notify(), but the event continues to propagate to downstream handlers (PRODUCT invariant 12).

Horizontally scrollable container carve-out

Table-local horizontal scrolling only applies when the surrounding surface doesn't own horizontal scroll. In code editors using WidthSetting::InfiniteWidth, wide tables render at full intrinsic width:

• TextLayout carries a container_scrolls_horizontally flag, set via TextLayout::with_container_scrolls_horizontally(...).
• RenderState::container_scrolls_horizontally() returns true when width_setting == WidthSetting::InfiniteWidth; RenderState::layout_context() and TextLayout::from_layout_context(...) propagate the flag.
• layout_table_block() reads the flag and sets horizontal_scroll_allowed on the resulting LaidOutTable to its inverse.
• When horizontal_scroll_allowed == false: viewport_width() returns the full table width, max_scroll_left() returns zero, and scroll_left() / scroll_horizontally() / reveal_offset() / set_scroll_left() become no-ops. Paint widens the clip layer to the full table, table_scrollbar() returns None, and scrollbar drag/hover handling short-circuits. Wheel events fall through to the surrounding editor.

AI block list contexts keep the default false, preserving existing behavior.

Per-cell source↔rendered offset mapping

TableCellOffsetMap in crates/editor/src/render/model/table_offset_map.rs tracks source↔rendered ranges per inline fragment. For each fragment it records rendered start/end, visible source start/end, and source end including Markdown markers. It supports rendered_length(), source_length(), rendered_to_source(), and source_to_rendered().

TableCellOffsetMap::from_inline_and_source(source, inline) derives spans by walking the raw cell source character-by-character alongside each fragment's rendered text, rather than reconstructing marker lengths from style flags. For each fragment:

• Advance through source until the fragment's first rendered character is found. Intervening source characters are attributed to this fragment as markers.
• Consume the remaining rendered characters from source, treating \<punct> as a single-character escape (two source chars → one rendered char).

This replaces an earlier hardcoded fragment_source_marker_lengths helper and therefore:

• Handles backslash-escaped punctuation correctly (previously silently drifted).
• Stays correct if the Markdown parser ever changes marker syntax, because we walk actual source rather than regenerating it.
• Handles nested styles where adjacent fragments share outer markers (e.g. **a *b* c**), attributing each marker once.

content::text::table_cell_offset_maps(table, source) receives the parsed FormattedTable and the raw tab/newline-separated source, splits the source into rows/cells, and passes each cell's source through the builder. Synthetic cells added by normalize_shape get an empty source string and produce empty maps.

An in-code TODO above impl TableCellOffsetMap notes that moving cell/row boundaries into the SumTree with new BufferText marker types would let editable tables derive per-cell offsets by seeking to boundaries rather than reparsing. That refactor is deferred to the editable-tables workstream.

Table layout is source-based, rendering stays rendered-text-based

layout_table_block() in crates/editor/src/content/edit.rs:

• Parses the table into formatted inline fragments (via the cache below when available).
• Builds TableCellOffsetMaps for every cell.
• Builds the table-wide TableOffsetMap from cell source lengths.
• Preserves content_length from the original text block source.

LaidOutTable stores the per-cell maps and uses them whenever a source offset touches rendered geometry (coordinate→offset conversion, link lookup, relative character bounds, selection highlight ranges, cursor placement).

The rule: editor-facing APIs stay source-based; text layout, frame widths, and painting stay rendered-text-based; conversions happen explicitly at the cell boundary.

Hit-testing, selection, and caret geometry

• crates/editor/src/render/model/location.rs adds scroll_left before calling coordinate_to_offset().
• LaidOutTable::character_bounds() subtracts scroll_left when returning screen-space bounds.
• RenderState::autoscroll() calls reveal_autoscroll_offsets_in_tables() so keyboard movement horizontally reveals the active caret/selection inside the table.
• softwrap_point_to_offset() for tables resolves to the first visible character in the row instead of using raw row starts.
• crates/editor/src/render/element/mod.rs hit-testing uses the editor element's own layer bounds rather than the clipped child layer bounds, so clicks inside a wide table are not rejected before block-level hit-testing runs.

Per-cell maximum content width

crates/editor/src/content/edit.rs introduces:

• MAX_TABLE_CELL_CONTENT_WIDTH_PX = 500.0 — maximum content width (exclusive of cell padding).
• maximum_table_cell_width(table_style) helper, mirroring minimum_table_cell_width(..).

In measure_table_cells, per-cell measured widths are clamped with both .max(minimum_table_cell_width(..)) and .min(maximum_table_cell_width(..)) before they fold into the shared column width. The second layout pass in layout_table_block lays each cell at cell_content_width = column_width - cell_padding * 2.0, and the existing text layout soft-wraps the clamped cells without further changes.

The cap is applied unconditionally — including when horizontal_scroll_allowed == false — so infinite-width containers still get readability benefits. Single very long unbreakable tokens (e.g. long URLs) may still render wider than the cap because soft-wrap can't break them; the column width stays clamped and the token visually overflows inside the clipped table region.

Clipboard behavior is table-aware

crates/editor/src/content/buffer.rs:

• Plain-text copy walks selected block segments and routes table segments through clipboard_table_text_in_range().
• Table copy rebuilds the formatted table (from cache when available), computes source-based cell ranges, converts the selected source span in each cell to rendered offsets, and slices visible cell text accordingly.
• Partial table selections return rendered plain text with tabs/newlines preserved.
• HTML export in selected_text_as_html filters only the ranges that contain a partial table selection, serializing the remaining clean ranges to HTML normally. Only when every range is a partial-table range does it return None.

Clipped scrollables keep selections anchored to content

crates/warpui_core/src/elements/clipped_scrollable.rs:

• ClippedScrollStateHandle stores a selection_scroll_anchor (original selection + scroll position at the time it was observed).
• anchor_and_adjust_selection_for_scroll(selection, axis) either records the anchor (first time) or shifts the selection by the delta between current scroll and the anchored scroll. Doc comment spells out the three branches (None clears anchor; unmatched Selection installs a new anchor; matched Selection returns a scroll-compensated copy).
• clear_selection_scroll_anchor() resets that state when a fresh mouse-down starts a new interaction.

NewScrollable routes selection APIs through the anchor helper during paint, in get_selection(), and in calculate_clickable_bounds(). New mouse-down clears the anchor.

Smart selection respects visible horizontal bounds

FormattedTextElement::smart_select() returns None when the click point is outside the visible horizontal bounds of the text frame (not just outside vertical bounds). This prevents double-click word selection from targeting text that is off-screen after horizontal scrolling.

Lazy cache on the BufferBlockStyle::Table marker

Both the clipboard copy path and the layout path previously parsed each table block on every invocation. That parse is deterministic from the block's plain text plus alignments, so it is lifted onto the marker:

• crates/editor/src/content/text.rs introduces TableBlockCache (owns parsed FormattedTable, Vec<Vec<TableCellOffsetMap>>, and TableOffsetMap) and a TableCache newtype wrapping Arc<OnceLock<TableBlockCache>>.
• BufferBlockStyle::Table gains a cache: TableCache field. A BufferBlockStyle::table(alignments) helper constructs the variant with an empty cache.
• TableCache implements PartialEq / Eq / Hash as no-ops so BufferBlockStyle equality and hashing remain a function of alignments. Clone stays cheap because the field is a shared Arc.
• TableCache::get_or_populate(text, alignments) runs the parse once and returns &TableBlockCache on subsequent calls. Marker clones share the same OnceLock.
• clipboard_table_text_in_range and layout_table_block both route through the cache. A defensive fallback in layout_table_block builds an owned cache on the stack when the style isn't Table.

The staleness trade-off is the same one that already applies to alignments: if a cell were edited in place without replacing the block marker, the cache could go stale. This matters only when editable tables land; for read-only tables, marker replacement on edit is sufficient.

Testing and validation

Each numbered invariant in PRODUCT.md maps to at least one test or verification step below.

Automated tests

• PRODUCT invariants 2–3 (overflow rule), 9 (scrollbar interaction): render::element::table::tests::table_scrollbar_uses_shared_overlay_geometry, table_scrollbar_drag_state_survives_renderable_recreation, table_scrollbar_pointer_movement_matches_drag_and_gutter_behavior.
• PRODUCT invariant 4 (container carve-out): exercised via horizontal_scroll_allowed == false code paths in model tests; manual pass in a code editor surface configured with WidthSetting::InfiniteWidth.
• PRODUCT invariants 7–8 (per-cell max width): content::edit::tests::test_layout_table_block_clamps_cell_width_to_max.
• PRODUCT invariant 11 (wheel edge propagation): covered by the ScrollWheel handler returning scroll_horizontally()'s boolean; manual trackpad pass confirms fall-through.
• PRODUCT invariant 12 (MouseMoved not consumed): covered by the handler returning false; manual hover-link check confirms downstream handlers still fire.
• PRODUCT invariants 13–15 (selection and copy while scrolled): content::buffer::tests::test_selected_table_copy_uses_visible_plain_text, test_partial_table_selection_does_not_export_html, test_partial_table_selection_still_exports_html_for_non_table_ranges, test_clipboard_table_copy_uses_source_offsets_for_later_formatted_cells.
• PRODUCT invariants 13–14 (selection anchor stability): warpui_core::elements::new_scrollable::scrollable_test regressions for viewport-coordinate selection APIs, re-anchoring existing selections across horizontal scroll, and clearing the anchor on new mouse-down interactions.
• PRODUCT invariant 13 (double-click word selection across columns): smart_select_returns_none_when_point_is_outside_horizontal_bounds (formatted_text_element_tests).
• PRODUCT invariant 18 (caret reveal during keyboard movement): render::model::location_tests (hit-testing after horizontal scroll) and render::model::mod_tests (reveal/autoscroll for offsets inside tables).
• PRODUCT invariants 6 and 13–15 as applied to formatted cells (inline Markdown correctness): render::model::table_offset_map::tests::test_table_cell_offset_map_handles_bold_and_links, test_table_cell_offset_map_handles_backslash_escaped_punctuation, test_table_cell_offset_map_handles_nested_styles.

Manual validation

• crates/markdown_parser/examples/table-demo/all_test_cases.md opened in a notebook for a visual pass covering PRODUCT invariants 2–3, 6, 7–8, 9–15, 19–21.
• AI block list responses containing a wide Markdown table — trackpad scroll, direct scrollbar interaction, double-click word selection across columns after scroll, partial-range text copy, and an existing selection that remains visually anchored across further horizontal scroll.
• Regression pass confirming vertical scrolling stays owned by the surrounding notebook or block list (PRODUCT invariant 5).
• Regression pass confirming narrow tables and non-table Markdown remain unchanged (PRODUCT invariant 22).

Pre-merge gates

• cargo fmt
• cargo clippy --workspace --all-targets --all-features --tests -- -D warnings
• cargo check -p warp_editor and cargo check -p warpui_core
• Targeted cargo nextest run --no-fail-fast --workspace ... for the regressions listed above.

Risks and mitigations

Two offset spaces drift again

If any table path assumes source and rendered offsets are interchangeable, formatted cells will reintroduce selection/copy drift. Mitigation: keep TableCellOffsetMap as the single translation layer; build the table-wide TableOffsetMap from source lengths only; route hit-testing, cursor, selection, and copy through explicit source↔rendered conversions.

Selection pinned to viewport coordinates

If clipped scrollables reuse raw selection rectangles after scroll changes, highlights and copied text target the wrong content. Mitigation: anchor selections via ClippedScrollStateHandle, clear the anchor on fresh mouse interactions, and route selection APIs through anchor_and_adjust_selection_for_scroll().

Scrollbar behavior diverges between surfaces

If the editor table path reintroduces bespoke thumb sizing or drag math, it will drift from the rest of WarpUI. Mitigation: keep geometry and pointer/scroll conversion in shared_scrollbar.rs and call the shared helpers from both the editor table renderer and the new_scrollable utilities.

Clipped child layers interfere with editor hit-testing

If editor bounds checks use the clipped child layer instead of the editor element bounds, clicks can be rejected before block-level hit-testing runs. Mitigation: use the editor element's own layer bounds in crates/editor/src/render/element/mod.rs.

Stale cache on in-place cell edits

TableCache is keyed by marker instance. If a future change edits a cell in place without replacing the block marker, the cache will be stale. Mitigation (current): marker replacement on edit is sufficient for read-only tables. Mitigation (longer-term): the SumTree follow-up below invalidates per-cell state naturally.

Follow-ups

• If additional inline Markdown styles are supported in table cells, add a focused TableCellOffsetMap::from_inline_and_source regression alongside the parser change — the source-walking algorithm should handle them automatically, but a test pins the behavior.
• Other clipped horizontal surfaces adopting the same selection APIs inherit the ClippedScrollStateHandle behavior automatically, but they should get a manual regression pass when adopting new selection UX.
• When editable tables or other complex table operations land, evaluate moving cell/row boundaries into the SumTree with new BufferText marker types so per-cell offsets can be derived by seeking instead of reparsing. See the in-code TODO above impl TableCellOffsetMap.