`roads.surface` is manufactured where the source is silent, in Iceland, Poland and Finland #82

New issue

Closed

opened 2026-08-09 10:35:58 +00:00 by art-bot · 1 comment

art-bot commented

2026-08-09 10:35:58 +00:00

Collaborator

Found auditing every country against docs/country-requirements.md (#77). Violates R66 — a categorical column with no corresponding source value imports as NULL, never a default enum member.

This is the generalization of #6, which found Sweden asserting surface='unpaved' on 215 ferry rows that had no Slitlager value at all.

What the data says

roads.surface fi -> paved:846957  unpaved:2319113
roads.surface is -> paved:35642   unpaved:22660
roads.surface pl -> paved:2968562 unpaved:3819922
roads.surface se -> paved:1624492 unpaved:1773203 NULL:91

Sweden is the only country with any NULLs, and it has them only because #6 fixed its ELSE. Iceland, Poland and Finland each assert a surface for every single road row, which would require their sources to record surface with perfect completeness. That is not plausible for national road datasets of 58k, 6.8M and 3.2M rows.

Why it matters

A default silently manufactures data instead of surfacing "unknown". #6's ferry rows were the visible case — rows that were not roads at all inherited a road's surface. The invisible case is worse: every unsurveyed track in three countries is currently asserted as gravel or asphalt on the authority of a CASE fallthrough, and nothing downstream can tell the difference between "measured unpaved" and "not recorded".

The fix

For each of conf/ismap.yaml, conf/plmap.yaml and conf/fimap.yaml, find the surface CASE and confirm the ELSE branch is reached only by rows that genuinely carry that value. Where the source field is absent or empty, emit NULL.

Note the interaction with R43: whether a CASE may have an ELSE at all is a property of the source file. A file describing one kind of thing may have a total CASE; a file that mixes kinds may not. Iceland's mapping (A/B/C → paved, D/F1/F2/F3 → unpaved) is already an exhaustive IN over vegtegund — the question there is what happens to rows where vegtegund is NULL.

Then check R64-style: does the style still render sensibly with NULL surface? Sweden's 91 NULL rows say yes, but confirm at the volumes this will produce.

Acceptance

Per country, count(*) WHERE surface IS NULL is greater than zero and matches the count of rows whose source surface field is absent or empty.

Found auditing every country against `docs/country-requirements.md` (#77). Violates **R66** — a categorical column with no corresponding source value imports as NULL, never a default enum member. This is the generalization of #6, which found Sweden asserting `surface='unpaved'` on 215 ferry rows that had no `Slitlager` value at all. ## What the data says ``` roads.surface fi -> paved:846957 unpaved:2319113 roads.surface is -> paved:35642 unpaved:22660 roads.surface pl -> paved:2968562 unpaved:3819922 roads.surface se -> paved:1624492 unpaved:1773203 NULL:91 ``` Sweden is the only country with any NULLs, and it has them only because #6 fixed its `ELSE`. Iceland, Poland and Finland each assert a surface for **every single road row**, which would require their sources to record surface with perfect completeness. That is not plausible for national road datasets of 58k, 6.8M and 3.2M rows. ## Why it matters A default silently manufactures data instead of surfacing "unknown". #6's ferry rows were the visible case — rows that were not roads at all inherited a road's surface. The invisible case is worse: every unsurveyed track in three countries is currently asserted as gravel or asphalt on the authority of a `CASE` fallthrough, and nothing downstream can tell the difference between "measured unpaved" and "not recorded". ## The fix For each of `conf/ismap.yaml`, `conf/plmap.yaml` and `conf/fimap.yaml`, find the `surface` `CASE` and confirm the `ELSE` branch is reached only by rows that genuinely carry that value. Where the source field is absent or empty, emit NULL. Note the interaction with **R43**: whether a `CASE` may have an `ELSE` at all is a property of the source file. A file describing one kind of thing may have a total `CASE`; a file that mixes kinds may not. Iceland's mapping (`A/B/C → paved`, `D/F1/F2/F3 → unpaved`) is already an exhaustive `IN` over `vegtegund` — the question there is what happens to rows where `vegtegund` is NULL. Then check **R64**-style: does the style still render sensibly with NULL surface? Sweden's 91 NULL rows say yes, but confirm at the volumes this will produce. ## Acceptance Per country, `count(*) WHERE surface IS NULL` is greater than zero and matches the count of rows whose source surface field is absent or empty.

art-bot referenced this issue

2026-08-09 10:40:32 +00:00

#77: write down what a country has to look like, and audit the four we have #87

art-bot was assigned by art

2026-08-09 10:48:49 +00:00