art/ismap
1
0
Fork 0
Code Issues 56 Pull requests 1 Projects Releases Packages Wiki Activity Actions

Closes #120 #131

Merged
art merged 2 commits from issue-120-is-cover-validity into main 2026-08-15 19:16:26 +00:00
Conversation 0 Commits 2 Files changed 3 +115 -11
art commented 2026-08-15 17:21:26 +00:00
Owner
Copy link

Closes #120

What and why

Iceland's map_cover_13_is had exactly one invalid geometry: a CLC 522 estuary
(ring self-intersection, 64.31°N / 14.93°W) found by #115's first acceptance-test
pass and deliberately left unguarded by #119 because it failed against live data.

The source is broken, not the importer. Before repairing anything I checked the
delivered file: data/is_lcv_lmi_epsg_4326.gpkg row id 14109 (lcv_class 522) is
Ring Self-intersection at -14.915589 / 64.327391 — LMÍ ships the polygon invalid.
So the fix belongs in the ismap cover query, exactly as the issue proposed.

The mechanism. The cover layer's landcover source read the GeoPackage directly,
which executes the source query in SQLite — there is no ST_MakeValid there. The
source now reads the new is_lcv_lmi_landcoverunit staging table (a byte-identical
load of the same rows and codes; only the repair moved into SQL) through PostGIS,
and the layer query applies R139's pattern: ST_MakeValid per source polygon +
ST_CollectionExtract(..., 3) so a self-intersecting ring cannot leak stray
linework into the typed Polygon column, then the usual ST_Dump/ST_Transform.
The CLC-code → cover_class mapping the Go-side enum map used to apply moved into
the query as a CASE (the canonical vocabulary from conf/common.yaml
enums.landcover; the CASE lists exactly the 32 codes that occur). The unusable
523 sea stays excluded, and the computed-sea source is untouched.

Verification (live DB, after importer/importer -layer cover conf/ismap.yaml, ~2 min)

  • 0 invalid geometries at every zoom z3–z13 (was 1 at z13).
  • Row counts per zoom unchanged: z13 17 633, z12 11 031, z5 3 441, z3 1 665
    (identical to the pre-fix snapshot); 0 NULL cover_class; part_key all NULL
    as designed.
  • Area-conserving repair: per-class area at z13 matches the source to the m²
    for all 31 land classes (0 mismatches); the repaired estuary keeps its
    10 301 668 m².
  • Decoded tile: zxy_map_cover(13, 3756, 2167) over the estuary decodes
    cleanly — 11 cover features including the repaired estuary and the sea.
  • Real-browser render (scripts/cdp-map-check.mjs, CDP, no virtual time):
    ?map=is#13/64.3274/-14.9156 loads, tiles load, 209 rendered cover features
    including 4 estuary features at the exact location.
  • Acceptance tally (make acceptance, ismap config):
    acceptance: 12 test(s): 12 passed, 0 failed, 0 skipped, 0 not run
    (the new test itself: PASS 2.5s, invalid=0). Repo-wide fast acceptance: every
    config green; Sweden's R35 (the 5.4M-row validity check, 100s bucket) still
    passes via -acceptance-only -acceptance all conf/semap.yaml.

The acceptance test

The test #119 deliberately did not ship now lands on Iceland's cover layer:

  • R139: Iceland's maxz cover has 0 invalid geometries (issue #120) — 10 s bucket
    (2.5 s live), requires: [map_cover_13_<cc>], count(*) = 0 over
    map_cover_13_is WHERE NOT ST_IsValid(geom). It fails on the pre-fix data and
    passes now.

Notes

  • The new is_lcv_lmi_landcoverunit staging table persists in the DB like
    strandlina/mork/fi_clc — that is the designed behavior for load-kind
    staging.
  • The -layer cover reimport also rebuilt strandlina (a staging dep of the
    sea source) — a deterministic reload of the same gpkg, no content change.
Closes #120 --- ## What and why Iceland's `map_cover_13_is` had exactly one invalid geometry: a CLC 522 estuary (ring self-intersection, 64.31°N / 14.93°W) found by #115's first acceptance-test pass and deliberately left unguarded by #119 because it failed against live data. **The source is broken, not the importer.** Before repairing anything I checked the delivered file: `data/is_lcv_lmi_epsg_4326.gpkg` row id 14109 (lcv_class 522) is `Ring Self-intersection` at -14.915589 / 64.327391 — LMÍ ships the polygon invalid. So the fix belongs in the ismap cover query, exactly as the issue proposed. **The mechanism.** The cover layer's landcover source read the GeoPackage directly, which executes the source query in SQLite — there is no `ST_MakeValid` there. The source now reads the new `is_lcv_lmi_landcoverunit` staging table (a byte-identical load of the same rows and codes; only the repair moved into SQL) through PostGIS, and the layer query applies R139's pattern: `ST_MakeValid` per source polygon + `ST_CollectionExtract(..., 3)` so a self-intersecting ring cannot leak stray linework into the typed Polygon column, then the usual `ST_Dump`/`ST_Transform`. The CLC-code → `cover_class` mapping the Go-side enum map used to apply moved into the query as a CASE (the canonical vocabulary from `conf/common.yaml` `enums.landcover`; the CASE lists exactly the 32 codes that occur). The unusable 523 sea stays excluded, and the computed-sea source is untouched. ## Verification (live DB, after `importer/importer -layer cover conf/ismap.yaml`, ~2 min) - **0 invalid geometries at every zoom z3–z13** (was 1 at z13). - **Row counts per zoom unchanged**: z13 17 633, z12 11 031, z5 3 441, z3 1 665 (identical to the pre-fix snapshot); 0 NULL `cover_class`; `part_key` all NULL as designed. - **Area-conserving repair**: per-class area at z13 matches the source to the m² for all 31 land classes (0 mismatches); the repaired estuary keeps its 10 301 668 m². - **Decoded tile**: `zxy_map_cover(13, 3756, 2167)` over the estuary decodes cleanly — 11 cover features including the repaired `estuary` and the sea. - **Real-browser render** (`scripts/cdp-map-check.mjs`, CDP, no virtual time): `?map=is#13/64.3274/-14.9156` loads, tiles load, 209 rendered cover features including 4 `estuary` features at the exact location. - **Acceptance tally (make acceptance, ismap config)**: `acceptance: 12 test(s): 12 passed, 0 failed, 0 skipped, 0 not run` (the new test itself: PASS 2.5s, invalid=0). Repo-wide fast acceptance: every config green; Sweden's R35 (the 5.4M-row validity check, 100s bucket) still passes via `-acceptance-only -acceptance all conf/semap.yaml`. ## The acceptance test The test #119 deliberately did not ship now lands on Iceland's `cover` layer: - `R139: Iceland's maxz cover has 0 invalid geometries (issue #120)` — 10 s bucket (2.5 s live), `requires: [map_cover_13_<cc>]`, `count(*) = 0` over `map_cover_13_is WHERE NOT ST_IsValid(geom)`. It fails on the pre-fix data and passes now. ## Notes - The new `is_lcv_lmi_landcoverunit` staging table persists in the DB like `strandlina`/`mork`/`fi_clc` — that is the designed behavior for load-kind staging. - The `-layer cover` reimport also rebuilt `strandlina` (a staging dep of the sea source) — a deterministic reload of the same gpkg, no content change.
Iceland's map_cover_13_is carried exactly one invalid geometry: a CLC 522
estuary (1.94 km2, off the west coast at 64.31N / 14.93W) with a ring
self-intersection, found while writing the first acceptance tests (#115) and
deliberately left unguarded by #119 because it failed against live data.

Verified first that LMÍ ships it broken: the polygon is invalid in the
GeoPackage itself (ST_IsValidReason: Ring Self-intersection at
-14.915589 / 64.327391), so the importer did not introduce it and the fix
belongs in the ismap cover query, not in a loader.

The cover layer's landcover source read the GeoPackage directly, which runs
the source query in SQLite — no ST_MakeValid there. It now reads the new
is_lcv_lmi_landcoverunit staging table (a byte-identical load of the same
rows, codes intact) through PostGIS, and the layer query applies R139's
pattern: ST_MakeValid per source polygon, ST_CollectionExtract(..., 3) so
the repair cannot leak stray linework into the typed Polygon column, then
ST_Dump/ST_Transform as before. The CLC code -> cover_class mapping that the
Go-side enum map used to do moved into the query as a CASE (the canonical
vocabulary, conf/common.yaml enums.landcover); the 523 sea stays excluded.

Verification against the live DB after a 2-minute single-layer reimport
(importer/importer -layer cover conf/ismap.yaml):
  - 0 invalid geometries at every zoom z3-z13 (was 1 at z13)
  - row counts per zoom unchanged (z13: 17633, z12: 11031, z5: 3441,
    z3: 1665); no NULL cover_class; part_key all NULL as designed
  - area-conserving: sum per class at z13 matches the source to the m2 for
    all 31 land classes; the repaired estuary keeps its 10,301,668 m2
  - decoded z13 tile over the estuary (zxy_map_cover(13, 3756, 2167)) and a
    real-browser CDP render at #13/64.3274/-14.9156 both show the estuary
    drawn; 12/12 Iceland acceptance tests pass, repo-wide fast acceptance
    all green, Sweden's R35 (the 5.4M-row validity check) still passes

The acceptance test #119 could not ship now does: "R139: Iceland's maxz
cover has 0 invalid geometries (issue #120)" on the cover layer
(10s bucket; 2.5s live), requires: [map_cover_13_<cc>].
Moving the CLC → cover_class mapping out of the Go enum map and into a SQL CASE
traded a loud failure for a silent one: an unmapped code used to abort the insert
(the enum map yields '' for a miss and ''::landcover is rejected), and now falls
out as a NULL cover_class. R34's land-area sum filters `cover_class <> 'sea'`,
which drops NULL rows, so the silence would compound. A second acceptance test on
the layer restores the loudness.

Also: the CASE lists 31 classes, not the 32 the comment claimed — the 32nd is the
523 sea the WHERE excludes. R139 had no *Check:* line at all in
docs/country-requirements.md; it has one now, per that document's own rule that a
check automated in a commit is recorded there in the same commit. And
docs/country-iceland.md still described the layer as a direct GeoPackage read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
art merged commit d7c922f624 into main 2026-08-15 19:16:26 +00:00
art deleted branch issue-120-is-cover-validity 2026-08-15 19:16:26 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
blocked
issue is stuck: missing decision, credential, or data (work-issue blocked path)

  
claude-wip
night-shift

  
map-report
map data feedback (mapsrv)

  
map:is
map data feedback (mapsrv)

  
map:se
map data feedback (mapsrv)

  
needs-review
night-shift

No labels
blocked
claude-wip
map-report
map:is
map:se
needs-review
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
art/ismap!131
No description provided.