# #82: roads.surface is manufactured where the source is silent #89

Merged

art-bot merged 1 commit from issue-82-surface-null into main

2026-08-09 15:56:26 +00:00

art commented

2026-08-09 13:53:10 +00:00

Owner

#82: roads.surface is manufactured where the source is silent

Closes #82

What changed

Three roads surface CASEs audited per R66. Two needed a fix, one is verified already total.

Poland — `conf/plmap.yaml` (the real manufacturing)

The surface CASE was CASE WHEN materialnawierzchni IN (6 paved values) THEN 'paved' ELSE 'unpaved' END. The ELSE branch was reached by:

grunt naturalny / tłuczeń / żwir — genuinely unpaved (natural ground, crushed stone, gravel) — fine;
inny ("other") — 1108 rows with no corresponding surface class;
NULL — 1389 rows where the source records no material at all.

Both of the latter were being asserted as unpaved, i.e. a default silently manufactured where the source is silent. The ELSE is now an explicit list of the three genuinely-unpaved values, and anything else (including inny and NULL) imports as NULL surface.

Before: pl had 0 NULL surfaces. After: 2497 NULLs = 1389 rows with no MATERIALNAWIERZCHNI + 1108 rows whose value is inny ("other"), a value with no surface class (R66: "a categorical column with no corresponding source value imports as NULL").

Iceland — `conf/ismap.yaml` (defensive, no data change)

The mapping A/B/C → paved, D/F1/F2/F3 → unpaved was already an exhaustive IN over vegtegund (R43), and vegtegund is complete in the delivery (0 NULL/empty). But the old CASE's ELSE would have manufactured unpaved for a NULL vegtegund if one ever appeared. The CASE now enumerates both classes explicitly and emits NULL for a NULL/empty vegtegund. 0 NULLs result — correct, because the source genuinely records a class on every row.

Finland — `conf/fimap.yaml` (verified total, no change)

CASE WHEN p.LINK_ID IS NOT NULL THEN 'paved' ELSE 'unpaved' END reads presence in Digiroad's own paved-roads overlay (DR_PAALLYSTETTY_TIE_K). Digiroad's model is that gravel roads get no row at all, so absence is the value "unpaved" — a total CASE by the source's own definition (R43), not a manufactured default. Verified against the data: every motorway (LINKKITYYP 1), semi-motorway (2) and main road (TOIMINN 1–3) is present in the overlay; the absent rows are predominantly private roads and vehicle tracks, which genuinely are gravel. 0 NULLs correct.

Verification (live DB)

Reimported the roads layer for is and pl (-layer roads, atomic swap).
map_roads_14 surface distribution per country:

cc	paved	unpaved	NULL
is	35642	22660	0
pl	2968712	3817422	2497
fi	846957	2319113	0
se	1624492	1773203	91 (unchanged)

Poland's NULL count matches the source exactly: pl_skjz has 1389 rows with NULL/empty MATERIALNAWIERZCHNI + 1108 with inny = 2497.
Partitions present and cascading correctly at every zoom (z5–z14) for both reimported countries.
Decoded tile (zxy_map(14, 9062, 5558), a Poland trunk with NULL surface): roads features carry paved/unpaved/absent surface as expected.
Real-browser render (CDP, ?map=pl): a trunk road with NULL surface draws as a normal trunk and is excluded from the brown dashed roads_unpaved overlay — the same behaviour Sweden's 91 NULL rows already exercise. Style has no NULL-surface predicate, so nothing else needed changing.
make build, go test ./... in importer/ and mapsrv/: pass.

Notes / things left

The inny ("other") value decision: BDOT10k defines MATERIALNAWIERZCHNI with ten values; the config comment previously said "nine" and never accounted for inny or NULL. inny has no corresponding surface class, so it imports as NULL rather than being guessed.
Config comments and docs/country-iceland.md / docs/workstream-a.md updated to record the R66 reasoning next to the queries.

# #82: roads.surface is manufactured where the source is silent Closes #82 --- ## What changed Three roads surface CASEs audited per R66. Two needed a fix, one is verified already total. ### Poland — `conf/plmap.yaml` (the real manufacturing) The surface CASE was `CASE WHEN materialnawierzchni IN (6 paved values) THEN 'paved' ELSE 'unpaved' END`. The ELSE branch was reached by: - `grunt naturalny` / `tłuczeń` / `żwir` — genuinely unpaved (natural ground, crushed stone, gravel) — fine; - **`inny` ("other") — 1108 rows** with no corresponding surface class; - **NULL — 1389 rows** where the source records no material at all. Both of the latter were being asserted as `unpaved`, i.e. a default silently manufactured where the source is silent. The ELSE is now an explicit list of the three genuinely-unpaved values, and anything else (including `inny` and NULL) imports as NULL surface. Before: `pl` had 0 NULL surfaces. After: **2497 NULLs** = 1389 rows with no `MATERIALNAWIERZCHNI` + 1108 rows whose value is `inny` ("other"), a value with no surface class (R66: "a categorical column with no corresponding source value imports as NULL"). ### Iceland — `conf/ismap.yaml` (defensive, no data change) The mapping `A/B/C → paved, D/F1/F2/F3 → unpaved` was already an exhaustive IN over `vegtegund` (R43), and `vegtegund` is complete in the delivery (0 NULL/empty). But the old CASE's ELSE would have manufactured `unpaved` for a NULL `vegtegund` if one ever appeared. The CASE now enumerates both classes explicitly and emits NULL for a NULL/empty `vegtegund`. **0 NULLs result** — correct, because the source genuinely records a class on every row. ### Finland — `conf/fimap.yaml` (verified total, no change) `CASE WHEN p.LINK_ID IS NOT NULL THEN 'paved' ELSE 'unpaved' END` reads presence in Digiroad's own paved-roads overlay (`DR_PAALLYSTETTY_TIE_K`). Digiroad's model is that gravel roads get no row at all, so absence *is* the value "unpaved" — a total CASE by the source's own definition (R43), not a manufactured default. Verified against the data: every motorway (LINKKITYYP 1), semi-motorway (2) and main road (TOIMINN 1–3) is present in the overlay; the absent rows are predominantly private roads and vehicle tracks, which genuinely are gravel. **0 NULLs correct.** ## Verification (live DB) - Reimported the `roads` layer for `is` and `pl` (`-layer roads`, atomic swap). - `map_roads_14` surface distribution per country: | cc | paved | unpaved | NULL | |----|-------|---------|------| | is | 35642 | 22660 | 0 | | pl | 2968712 | 3817422 | **2497** | | fi | 846957 | 2319113 | 0 | | se | 1624492 | 1773203 | 91 (unchanged) | - Poland's NULL count matches the source exactly: `pl_skjz` has 1389 rows with NULL/empty `MATERIALNAWIERZCHNI` + 1108 with `inny` = 2497. - Partitions present and cascading correctly at every zoom (z5–z14) for both reimported countries. - Decoded tile (`zxy_map(14, 9062, 5558)`, a Poland trunk with NULL surface): roads features carry `paved`/`unpaved`/absent surface as expected. - Real-browser render (CDP, `?map=pl`): a trunk road with NULL surface draws as a normal trunk and is **excluded** from the brown dashed `roads_unpaved` overlay — the same behaviour Sweden's 91 NULL rows already exercise. Style has no NULL-surface predicate, so nothing else needed changing. - `make build`, `go test ./...` in `importer/` and `mapsrv/`: pass. ## Notes / things left - The `inny` ("other") value decision: BDOT10k defines `MATERIALNAWIERZCHNI` with ten values; the config comment previously said "nine" and never accounted for `inny` or NULL. `inny` has no corresponding surface class, so it imports as NULL rather than being guessed. - Config comments and `docs/country-iceland.md` / `docs/workstream-a.md` updated to record the R66 reasoning next to the queries.

art added 1 commit

2026-08-09 13:53:10 +00:00

#82 : roads.surface imports NULL where the source is silent (pl, is) 6dc7412352

R66 audit of the three roads surface CASEs. Poland was the real
manufacturing: the ELSE ('unpaved') was reached by 1389 rows whose
MATERIALNAWIERZCHNI is NULL and 1108 rows whose value is `inny`
("other") — neither genuinely carries 'unpaved'. The CASE now lists the
three genuinely-unpaved values explicitly and ELSE NULL, so pl gains
2497 NULL surfaces (1389 absent + 1108 other), matching the source.

Iceland's mapping was already an exhaustive IN over vegtegund (R43),
but its ELSE would have manufactured 'unpaved' for a NULL vegtegund;
the CASE now enumerates both classes and emits NULL for NULL/empty
vegtegund. vegtegund is complete in the delivery, so 0 NULLs is correct
— the ELSE is never reached by a silent row.

Finland was verified already-total, no change: presence in Digiroad's
own paved-roads overlay means paved and absence means unpaved by the
source's own definition (all motorways/trunks present; absent rows are
private roads and tracks, which genuinely are gravel).

Verified against the live DB: reimported roads for is and pl, surface
NULL counts match the source exactly (pl 2497), partitions at every
zoom, decoded tile, real-browser render showing a NULL-surface trunk
drawn as normal trunk and excluded from the roads_unpaved overlay.

art referenced this pull request

2026-08-09 13:53:14 +00:00

roads.surface is manufactured where the source is silent, in Iceland, Poland and Finland #82