spreadsheet-handling Release Notes

Repeated export/reimport/promote cycles are stable: the orchestrator now projects the runtime _meta onto a persistable contract before any backend writes, so runtime-produced FK-helper v2 relations no longer leak into persisted sidecars and replay as stale policy on the next run. This closes the Frame 'groups' must have flat columns failure that the worldbuilding adoption surfaced. The boundary is intentionally narrow and post-pipeline: it prevents new contamination on every successful run, but a _meta.yaml already promoted from a pre-fix run may still need a one-time cleanup of its helper_policies.fk.relations entries before the next export can succeed. A complete _meta lifecycle inventory and load-time sanitizer are deferred to post-release follow-ups (FTR-FK-HELPER-POLICY-LIFECYCLE-P4A, FTR-META-LIFECYCLE-INVENTORY-P5).

ODS reimport is robust against sheet-wide formatting fillers. Workbooks edited with whole-sheet selections (e.g. Ctrl+A + format change in LibreOffice Calc) no longer trigger uncontrolled allocation in the ODS parser, and a typed SpreadsheetTooLargeError surfaces before unbounded growth when an input declares implausible dimensions.

FK-helper behavior has moved from convention-driven primitive inference to explicit relation policy. Pipelines should now configure or infer FK relation policy before running FK-helper primitives.

Public documentation entry points were clarified: the README, Pages root, release notes, user guide, and demo walkthrough paths now form an intent-based user journey rather than a collection of repository paths.

Release-readiness tooling was tightened with read-only checks for release branch state, README link versioning, Pages publication structure, and cross-repo release status.

The semantic model documentation now gives maintainers and agents a shared vocabulary for metadata meaning, lifecycle state, registry authority, and FK-helper contract review.

Ctrl+C during sheets-run now exits cleanly with Interrupted by user. on stderr and a non-zero exit code (130, the POSIX SIGINT convention), in place of a raw Python traceback. --debug or -vv still shows the full traceback for debugging. The installed entry point is now routed through the existing run_cli wrapper, so the previously silent Error: …​ short-message path also reaches real users; reinstall with make setup or pip install -e . to regenerate .venv/bin/sheets-run.

The core README now directs readers to the maintained demo, the latest user guide, and the version archive with clearer intent labels.

The Pages root now acts as a version archive with links to core release notes, the user guide, and the maintained demo walkthrough.

Release-bound README links are guarded so PyPI and other frozen release contexts use /versions/<tag>/…​ documentation links instead of drifting to /latest/…​.

The user-guide transformation catalog was updated for the current FK-helper model, including the infer_fk_relations configuration step and policy-driven FK-helper primitives.

Canonical workbook reimport pipelines that declare helper cleanup now remove visible FK helper columns again after the persistence boundary strips transient _meta.derived provenance.

Added infer_fk_relations as an explicit configuration step. It can inspect current frames using a bounded naming-convention mode and write resolved FK relation policy to _meta.helper_policies.fk.

configure_fk_helpers remains the explicit/manual FK policy path. targets: "auto" was removed from this step; use infer_fk_relations when heuristic relation discovery is wanted.

FK-helper primitives now consume v2 relation policy under _meta.helper_policies.fk (schema_version: 2) instead of inferring relation identity from frame, table, or column names.

add_fk_helpers writes derived helper provenance that includes target_key, so later validation and cleanup can use the declared target primary-key column.

remove_fk_helpers, reorder_fk_helpers, and FK-helper validation now use policy and/or derived provenance. Missing required policy or provenance is a clear error rather than a silent prefix-based fallback.

apply_derived_column_policy …​ policy: drop now falls back to durable workbook-view helper metadata (_meta.sheets.<frame>.helper_columns) when transient _meta.derived.sheets.<frame>.helper_columns is absent after a fresh workbook reimport.

validate_references gained a no_helper_columns assertion rule. Use it after cleanup in reimport pipelines to report helper columns that would otherwise reach canonical output.

The FK column wire format, such as id_(<target>), remains valid for spreadsheet roundtrips. The change is where relation meaning is resolved: configuration steps may use the convention, but primitive FK-helper steps do not infer relation identity from it.

No broad new spreadsheet backend capability is introduced in this release draft. XLSX remains the primary spreadsheet backend, and ODS/Calc remains the second backend used to prove the spreadsheet contract.

Roundtrip-sensitive helper behavior is now more explicit because helper cleanup and validation rely on declared policy/provenance rather than hidden name-pattern recovery.

The orchestrator now owns a narrow persistence boundary that projects runtime meta onto a carrier-neutral persistable contract before save. Top-level derived and _*-prefixed keys are dropped, and FK-helper v2 relation entries under helper_policies.fk.relations are removed when their produced_by.step is configure_fk_helpers. The boundary applies to every output kind (JSON/YAML/XML/CSV directories and XLSX/ODS); the projection is the same for all carriers, with spreadsheet backends layering rendering on top. The pipeline.runner.run_app entry point is a thin adapter over orchestrate and inherits the same boundary, so there is exactly one load/step/save path in the project. See docs/semantic_model/08_lifecycle_and_update_semantics.adoc for the boundary contract, the post-pipeline / write-side scope, and the explicitly deferred follow-ups (FTR-FK-HELPER-POLICY-LIFECYCLE-P4A, FTR-META-LIFECYCLE-INVENTORY-P5).

The ODS parser no longer materializes empty cells declared via number-columns-repeated / number-rows-repeated. This neutralizes the common LibreOffice "select all + apply style" carrier pattern, which previously expanded a single empty styled cell into millions of dict entries and could exhaust process memory on real-world inputs.

parse_workbook accepts a new keyword argument limits=ParserLimits(…​) for defense-in-depth caps on parsed rows, columns, and materialized cells. Exceeding a limit raises the typed SpreadsheetTooLargeError (re-exported from spreadsheet_handling.io_backends.parser_limits) instead of a raw MemoryError. Defaults align with the spreadsheet maxima (1,048,576 rows × 16,384 columns) with a 10,000,000 cell ceiling per sheet.

Public documentation entry points were realigned around the README, Pages root, user guide, release notes, and demo walkthroughs.

The maintained demo/tutorial path was realigned with explicit FK-helper policy semantics.

The semantic model documentation was added under docs/semantic_model/, including the problem statement, semantic sets and lifecycle model, registry role, adoption strategy, and FK-helper contract matrix.

The project evolution narrative was updated so future maintainers and agents can understand the release, FK-helper, registry, and semantic-model phases without reading every historical FTR.

Pipelines that relied on configure_fk_helpers with targets: "auto" must switch to an explicit infer_fk_relations step before FK-helper primitives, or provide explicit/manual relation policy through configure_fk_helpers.

Pipelines that previously expected add_fk_helpers to discover FK relations implicitly from column names must run a configuration step first.

Pipelines that relied on prefix fallback during remove_fk_helpers should preserve the derived helper provenance written by add_fk_helpers or provide explicit policy. Recovery from externally edited workbooks without provenance is not part of the normal primitive behavior.

FK relation policy now uses the v2 relation-oriented shape under the existing _meta.helper_policies.fk root. The relation shape includes schema_version: 2, relations, source_frame, source_column, target_frame, mandatory target_key, helper fields/columns, and produced_by metadata.

The old id_(<target>) FK column convention is not removed. It is still the cell-level wire format and may be used by infer_fk_relations; it is no longer the hidden authority inside primitive FK-helper execution.

Registry contract lifecycle guidance now distinguishes current state, target state, and transition bridges so registry entries can describe migrations without overstating runtime truth.

Registry artifacts were relocated to root-level registries/ paths while documentation keeps the reviewed registry contract context.

Release tooling now includes read-only helpers for release branch validation, Pages publication checks, cross-repo release status, and README link versioning.

GitHub Actions workflow permissions were tightened to minimal defaults with job-level overrides where required.

The FK-helper semantic contract matrix and guards reduce the chance that future refactors reintroduce primitive inference or stale registry wording.

0.2.0 remains pre-1.0. Public behavior is more coherent than the beta prototype line, but compatibility promises are intentionally limited.

Historical beta-by-beta release notes are not reconstructed here. This page focuses on changes since v0.1.0b6.

Some semantic-model adoption work is intentionally documentation-first. Future transformation families still need their own adoption slices.

REVIEW-002 residual maintainability items remain tracked outside these release notes. They are not treated as release blockers unless later review promotes them to 0.2.0 scope.

The FK-helper policy model does not yet claim to cover composite keys, generic relation modeling, or every possible recovery flow for externally edited workbooks.