spreadsheet-handling Release Notes

The Functional Model was extended from requirement-inquiry material rather than from implementation details alone. New or updated use-case concepts cover Cell Codec, Compact Multiaxis, Sparse Defaults, derived frame views, validation use cases, header shape projection, hierarchical config projection, aggregate/pivot reconciliation, workbook-view boundary behavior, and XRef/discriminator composition.

The repository-wide June-2026 transformation/capability family inventory was archived in cold storage and linked from Project Memory as the canonical reference for implemented capability families and Functional Model coverage status. The inventory remains evidence material, not a backlog or second source of truth.

Future requirement inquiries were preserved for later work, including nested structures, typed cell values, conditional follow-up table views, and multi-row header presentation. These are recorded as future concepts, not as implemented 0.2.1 runtime capabilities.

ADR-CONTRACT-FIRST-DOMAIN-META-REWRITE accepts a contract-first domain/meta reset around a stabilized bridge DSL. The ADR explicitly frames the domain/IO seam as a shared Frames + Meta language whose writer, reader, lifecycle, conflict, and deletion rules should be derived from Functional Model use cases. It preserves adapter/parser/renderer knowledge, allows targeted deletion of obvious duplicate paths, and rejects recreating the current registry-scale surface under new names.

Project Memory now records the bridge-DSL reset as a high-weight Domain/Meta steering decision. This is intended to counterbalance overly conservative readings of the existing documentation: low external adoption makes bounded breaking changes acceptable when they reduce historical implementation sediment.

User-authored text orientation (vertical / rotated cells) is preserved through XLSX roundtrips under the text_orientations presentation metadata key. ODS roundtrips also preserve text orientation: style:rotation-angle is written on style:table-cell-properties, the carrier ODF defines for cell text rotation, so the rotation is honoured by external consumers such as LibreOffice Calc rather than only by the project’s own parser.

ODS column widths are preserved across render/parse via style:table-column-properties with an approximate Excel-character / cm conversion. The conversion is documented as approximate; widths authored by external ODS producers may not round-trip with byte-exact XLSX parity.

The ODS presentation-metadata extractors (column widths, text orientations) share the grid parser’s defense against sheet-wide LibreOffice formatting fillers: extraction is clipped to the parsed content extent so that a number-rows-repeated / number-columns-repeated filler carrying a rotation or column-width style cannot materialize one metadata entry per implied address, and the same ParserLimits are honoured on per-axis and per-cell bases as defense-in-depth.

User-authored horizontal cell alignment is preserved through XLSX and ODS roundtrips under the horizontal_alignments presentation metadata key. Canonical vocabulary is intentionally narrow in this slice — only left, center, and right are recognised; XLSX general / fill / justify / distributed / centerContinuous are dropped on read and never emitted on write. The ODS write path emits fo:text-align on style:paragraph-properties (the ODF-defined carrier for cell text alignment); the read path accepts both absolute (left / right) and locale-neutral (start / end) forms and normalises to the canonical absolute encoding so re-imports under different locales remain stable.

The XLSX renderer now merges alignment attributes when more than one render op touches cell.alignment (currently SetTextOrientation and SetHorizontalAlignment). Existing rotation behaviour is unchanged when no other alignment op is present; cells that combine rotation and horizontal alignment now retain both attributes regardless of op-dispatch order.

The ODS horizontal-alignment extractor inherits the same content-extent clipping and ParserLimits enforcement as the existing text-orientation and column-width extractors, so a sheet-wide "select all + left-align" filler cannot materialise one metadata entry per implied address.

The presentation-metadata families column_widths, text_orientations, and horizontal_alignments now share a single replace-or-clear policy on parse: parsed carrier state is authoritative for the sheet-local entry, and clearing the visible carrier removes any persisted entry for that sheet so the next roundtrip cannot silently reapply formatting the user has removed. Previously this behaviour was implemented only for horizontal_alignments; text_orientations and column_widths retained merge-only semantics and could surface stale embedded metadata after a cleared visible carrier. Behavioural note for column_widths: removing a custom column width from the visible carrier now removes the embedded entry as well, where previously the entry persisted. This is a deliberate alignment of behaviour across the three families and matches user intent ("I cleared this; do not reapply it").

User-authored vertical cell alignment is preserved through XLSX and ODS roundtrips under the vertical_alignments presentation metadata key. Canonical vocabulary is intentionally narrow in this slice — only top, center, and bottom are recognised; XLSX justify / distributed and ODS automatic are dropped on read and never emitted on write. The ODS write path emits style:vertical-align on style:table-cell-properties (the ODF-defined carrier for cell vertical alignment) and applies the intrinsic ODF ↔ OOXML vocabulary remap — canonical center serialises as ODF middle on write and ODF middle normalises to canonical center on read — so cross-backend roundtrips stay stable. The new family joins column_widths / text_orientations / horizontal_alignments under the same authoritative-carrier semantics.

The hidden XLSX / ODS _meta carrier now roundtrips the canonical workbook meta as one coherent mapping even when a sheet-level representation accidentally leaks into the writer or a malformed file carries a nested workbook_meta_blob wrapper. The carrier-local helper field workbook_meta_blob is recognised as a hidden-sheet convention only, so it can no longer shadow or structurally replace _meta.workbook_view during readback. This restores XLSX workbook-view readback parity with ODS for declarative project_by_role pipelines. See BUG-XLSX-WORKBOOK-VIEW-BLOB-READBACK-P4A.

As a follow-up hardening, canonicalize_workbook_meta now strips the sheet-level export marker keys (author, exported_at, version) from canonical workbook meta whenever they carry the MetaPass-default empty signature (None / empty string). They describe the hidden _meta sheet itself, not the canonical workbook meta contract. Without the strip, a corrupt or empty inner blob would leave the outer wrapper visible as if it were canonical content, hiding the absence of workbook_view behind wrapper noise (the post-e939623 residual symptom). Marker keys with caller-assigned non-empty values still round-trip unchanged. See BUG-XLSX-WORKBOOK-VIEW-CANONICAL-META-LOSS-P4A.

The persistence boundary now applies a narrow Intent-vs-Resolution rule for the cross-carrier _meta sidecar failure observed in the worldbuilding consumer. Persisted meta keeps declarative intent, but drops three reconstructible resolution facets from the captured ODS-produced sidecar: legend_blocks[].resolved, xref_crosstable[].dense_axes.resolved, and xref_crosstable[*].column_keys. This prevents backend-produced resolved layout / axis snapshots from poisoning later XLSX workbook-view readback while leaving broader lifecycle classification to the existing metadata follow-up FTRs. See BUG-CROSS-CARRIER-META-ROUNDTRIP-P4A.

New pipeline step project_by_role reorders / retains a projected frame’s columns by foundation column-role (row_identity, display_helper, matrix_value). direction: is required (outbound retains all three roles; inbound drops display_helper); there is no silent default. Optional role_order: permutes the per-direction default order; including display_helper in an inbound role_order is a validation error. Optional helper_columns: and key_columns: are interim per-call-site overrides that win over scattered upstream sources. See FTR-DYNAMIC-FRAME-PROJECTION-IMPL-P5.

Role classification is consumed from the foundation taxonomy defined in FTR-PROJECTED-FRAME-COLUMN-SEMANTICS-P5; the step does not introduce a parallel role vocabulary. A new shared resolver module (spreadsheet_handling.domain.column_roles) is the single source of truth for role detection in the current code base; the future targeting implementation slice (FTR-DYNAMIC-VIEW-COLUMN-TARGETING-IMPL-P5) will reuse it.

The worldbuilding consumer migrated its two projection plugins (build_story_cast_matrix_view, build_story_cast_matrix_payload) to project_by_role at all 10 current call sites (5 outbound
5 inbound). The plugin functions remain importable and behavior-compatible for one release cycle; per-surface rollback is a one-line YAML edit (outbound: restore the *_matrix_enriched join_frames output: name and re-introduce the plugin step; inbound: restore the *_matrix_payload consumer matrix: name and re-introduce the plugin step).

add_validations rule targets now support role-based column targeting for dynamic workbook views. A target can specify roles: [matrix_value] (or row_identity / display_helper) and the step expands the target to the matching concrete columns using the shared column-role resolver. This is intended to replace consumer-side plugin fan-out such as add_from_legend_constraints_for_matrix for dynamic matrix columns. The first slice covers the validation / constraint metadata path; broader presentation-target surfaces remain deferred.

FK Helper Slice 2 (v1 retirement): configure_fk_helpers now writes only the durable v2 relation model under _meta.helper_policies.fk.relations; the legacy v1 per-target dict (_meta.helper_policies.fk.<target_key>) is no longer written. Those v2 relations are now durable — the persistence boundary no longer prunes relations by produced_by.step, so they survive into the sidecar / workbook and drive reverse-pipeline cleanup without a v1 dict. produced_by is retained on each relation as provenance and as the cross-producer conflict key. The Dino-shaped replay failure that motivated pruning configure relations is now contained at the materialisation point: enrich_helpers preserves the source frame’s column flatness and skips relations whose target frame is absent, so a durable relation can no longer turn a flat frame into a non-flat one or crash on a missing target. Public YAML / CLI surface is unchanged; the persisted _meta shape shrinks (no v1 dict). Migration: an artifact produced by a pre-Slice-2 run (v1 dict kept, configure v2 pruned) that is reimported without a fresh forward run has no v2 cleanup anchor unless it also recorded sheets.<frame>.helper_columns; re-run the forward pipeline once to re-bootstrap the durable v2 relation. See audit/fk_helper_slice2_v1_retirement_review.adoc.

New CLI entry point sheets-schema-maintain performs explicit one-shot schema column maintenance over loaded Frames. The supported operations are add_column, drop_column, rename_column, and reorder_columns.

The command is dry-run by default. Dry-run output is routed to the new output-only saver kind discard, which accepts the final Frames payload but writes no artifact. User-selected output writes require --write; write mode uses the caller-provided --out-kind and --out-path.

Schema maintenance reports are emitted outside Frames and outside _meta. The report is printed as JSON to stdout by default, or written to the path supplied with --report.

Known limitations for this maintenance slice: there is no normal sheets-run YAML step and no public pipeline registry step; this is not a generic migration framework; workbook-header inference and dynamic projection schema handling are out of scope; FK relation drops remain blocked. (The earlier limitation about residual v1 _meta.helper_policies.fk per-target entries being blocked no longer applies: FK Helper Slice 2 retired the v1 producer write, so no such entry is produced; a v1 dict in a legacy sidecar is inert and passes through unchanged.)

Removed the placeholder example console scripts sheets-example-json-to-xlsx and sheets-example-xlsx-to-json. The maintained public CLI surface is now sheets-run plus the dedicated sheets-schema-maintain command.

Still true from 0.2.0: visible spreadsheet payload is a string substrate. Numeric, boolean, and date/time values in workbook cells are read and written as text by both the XLSX and ODS backends; type information present in the carrier is not preserved by default. A JSON value such as {"sequence_no": 1} may therefore round-trip through a workbook as {"sequence_no": "1"}. Consumers that need typed roundtrip for specific columns should declare a codec through the cell_codecs workbook metadata family. The decision and documented migration paths are recorded in ADR-VISIBLE-CELL-TYPING-STRING-SUBSTRATE (docs/technical_model/ch03_architecture_decisions/); the closing of FTR-VISIBLE-CELL-TYPING-STRATEGY-P5 is the explicit project-level position behind this behaviour.

Repeated export/reimport/promote cycles are stable: the orchestrator now projects the runtime _meta onto a persistable contract before any backend writes, so runtime-produced FK-helper v2 relations no longer leak into persisted sidecars and replay as stale policy on the next run. This closes the Frame 'groups' must have flat columns failure that the worldbuilding adoption surfaced. The boundary is intentionally narrow and post-pipeline: it prevents new contamination on every successful run, but a _meta.yaml already promoted from a pre-fix run may still need a one-time cleanup of its helper_policies.fk.relations entries before the next export can succeed. A complete _meta lifecycle inventory and load-time sanitizer are deferred to post-release follow-ups (FTR-FK-HELPER-POLICY-LIFECYCLE-P4A, FTR-META-LIFECYCLE-INVENTORY-P5).

ODS reimport is robust against sheet-wide formatting fillers. Workbooks edited with whole-sheet selections (e.g. Ctrl+A + format change in LibreOffice Calc) no longer trigger uncontrolled allocation in the ODS parser, and a typed SpreadsheetTooLargeError surfaces before unbounded growth when an input declares implausible dimensions.

FK-helper behavior has moved from convention-driven primitive inference to explicit relation policy. Pipelines should now configure or infer FK relation policy before running FK-helper primitives.

Public documentation entry points were clarified: the README, Pages root, release notes, user guide, and demo walkthrough paths now form an intent-based user journey rather than a collection of repository paths.

Release-readiness tooling was tightened with read-only checks for release branch state, README link versioning, Pages publication structure, and cross-repo release status.

The semantic model documentation now gives maintainers and agents a shared vocabulary for metadata meaning, lifecycle state, registry authority, and FK-helper contract review.

Ctrl+C during sheets-run now exits cleanly with Interrupted by user. on stderr and a non-zero exit code (130, the POSIX SIGINT convention), in place of a raw Python traceback. --debug or -vv still shows the full traceback for debugging. The installed entry point is now routed through the existing run_cli wrapper, so the previously silent Error: …​ short-message path also reaches real users; reinstall with make setup or pip install -e . to regenerate .venv/bin/sheets-run.

The core README now directs readers to the maintained demo, the latest user guide, and the version archive with clearer intent labels.

The Pages root now acts as a version archive with links to core release notes, the user guide, and the maintained demo walkthrough.

Release-bound README links are guarded so PyPI and other frozen release contexts use /versions/<tag>/…​ documentation links instead of drifting to /latest/…​.

The user-guide transformation catalog was updated for the current FK-helper model, including the infer_fk_relations configuration step and policy-driven FK-helper primitives.

Canonical workbook reimport pipelines that declare helper cleanup now remove visible FK helper columns again after the persistence boundary strips transient _meta.derived provenance.

Added infer_fk_relations as an explicit configuration step. It can inspect current frames using a bounded naming-convention mode and write resolved FK relation policy to _meta.helper_policies.fk.

configure_fk_helpers remains the explicit/manual FK policy path. targets: "auto" was removed from this step; use infer_fk_relations when heuristic relation discovery is wanted.

FK-helper primitives now consume v2 relation policy under _meta.helper_policies.fk (schema_version: 2) instead of inferring relation identity from frame, table, or column names.

add_fk_helpers writes derived helper provenance that includes target_key, so later validation and cleanup can use the declared target primary-key column.

remove_fk_helpers, reorder_fk_helpers, and FK-helper validation now use policy and/or derived provenance. Missing required policy or provenance is a clear error rather than a silent prefix-based fallback.

apply_derived_column_policy …​ policy: drop now falls back to durable workbook-view helper metadata (_meta.sheets.<frame>.helper_columns) when transient _meta.derived.sheets.<frame>.helper_columns is absent after a fresh workbook reimport.

validate_references gained a no_helper_columns assertion rule. Use it after cleanup in reimport pipelines to report helper columns that would otherwise reach canonical output.

The FK column wire format, such as id_(<target>), remains valid for spreadsheet roundtrips. The change is where relation meaning is resolved: configuration steps may use the convention, but primitive FK-helper steps do not infer relation identity from it.

No broad new spreadsheet backend capability is introduced in this release draft. XLSX remains the primary spreadsheet backend, and ODS/Calc remains the second backend used to prove the spreadsheet contract.

Roundtrip-sensitive helper behavior is now more explicit because helper cleanup and validation rely on declared policy/provenance rather than hidden name-pattern recovery.

The orchestrator now owns a narrow persistence boundary that projects runtime meta onto a carrier-neutral persistable contract before save. Top-level derived and _*-prefixed keys are dropped, and FK-helper v2 relation entries under helper_policies.fk.relations are removed when their produced_by.step is configure_fk_helpers. The boundary applies to every output kind (JSON/YAML/XML/CSV directories and XLSX/ODS); the projection is the same for all carriers, with spreadsheet backends layering rendering on top. The pipeline.runner.run_app entry point is a thin adapter over orchestrate and inherits the same boundary, so there is exactly one load/step/save path in the project. See docs/semantic_model/08_lifecycle_and_update_semantics.adoc for the boundary contract, the post-pipeline / write-side scope, and the explicitly deferred follow-ups (FTR-FK-HELPER-POLICY-LIFECYCLE-P4A, FTR-META-LIFECYCLE-INVENTORY-P5).

The ODS parser no longer materializes empty cells declared via number-columns-repeated / number-rows-repeated. This neutralizes the common LibreOffice "select all + apply style" carrier pattern, which previously expanded a single empty styled cell into millions of dict entries and could exhaust process memory on real-world inputs.

parse_workbook accepts a new keyword argument limits=ParserLimits(…​) for defense-in-depth caps on parsed rows, columns, and materialized cells. Exceeding a limit raises the typed SpreadsheetTooLargeError (re-exported from spreadsheet_handling.io_backends.parser_limits) instead of a raw MemoryError. Defaults align with the spreadsheet maxima (1,048,576 rows × 16,384 columns) with a 10,000,000 cell ceiling per sheet.

Public documentation entry points were realigned around the README, Pages root, user guide, release notes, and demo walkthroughs.

The maintained demo/tutorial path was realigned with explicit FK-helper policy semantics.

The semantic model documentation was added under docs/semantic_model/, including the problem statement, semantic sets and lifecycle model, registry role, adoption strategy, and FK-helper contract matrix.

The project evolution narrative was updated so future maintainers and agents can understand the release, FK-helper, registry, and semantic-model phases without reading every historical FTR.

Pipelines that relied on configure_fk_helpers with targets: "auto" must switch to an explicit infer_fk_relations step before FK-helper primitives, or provide explicit/manual relation policy through configure_fk_helpers.

Pipelines that previously expected add_fk_helpers to discover FK relations implicitly from column names must run a configuration step first.

Pipelines that relied on prefix fallback during remove_fk_helpers should preserve the derived helper provenance written by add_fk_helpers or provide explicit policy. Recovery from externally edited workbooks without provenance is not part of the normal primitive behavior.

FK relation policy now uses the v2 relation-oriented shape under the existing _meta.helper_policies.fk root. The relation shape includes schema_version: 2, relations, source_frame, source_column, target_frame, mandatory target_key, helper fields/columns, and produced_by metadata.

The old id_(<target>) FK column convention is not removed. It is still the cell-level wire format and may be used by infer_fk_relations; it is no longer the hidden authority inside primitive FK-helper execution.

Registry contract lifecycle guidance now distinguishes current state, target state, and transition bridges so registry entries can describe migrations without overstating runtime truth.

Registry artifacts were relocated to root-level registries/ paths while documentation keeps the reviewed registry contract context.

Release tooling now includes read-only helpers for release branch validation, Pages publication checks, cross-repo release status, and README link versioning.

GitHub Actions workflow permissions were tightened to minimal defaults with job-level overrides where required.

The FK-helper semantic contract matrix and guards reduce the chance that future refactors reintroduce primitive inference or stale registry wording.

0.2.0 remains pre-1.0. Public behavior is more coherent than the beta prototype line, but compatibility promises are intentionally limited.

Historical beta-by-beta release notes are not reconstructed here. This page focuses on changes since v0.1.0b6.

Some semantic-model adoption work is intentionally documentation-first. Future transformation families still need their own adoption slices.

REVIEW-002 residual maintainability items remain tracked outside these release notes. They are not treated as release blockers unless later review promotes them to 0.2.0 scope.

The FK-helper policy model does not yet claim to cover composite keys, generic relation modeling, or every possible recovery flow for externally edited workbooks.