#93: places dedupe picks the cluster's highest register tier before font size #126

Merged

art merged 3 commits from issue-93-places-dedupe-tier into main

2026-08-15 10:19:48 +00:00

art commented

2026-08-15 09:47:42 +00:00

Owner

Closes #93

#93 — Finland's places dedupe tiebreaks on the register tier before font size

What changed

One query, conf/fimap.yaml's places layer, and its acceptance tests.

The (name, cl) cluster pick ordered on (kielikoodi = 'fin') then
kirjasinkoko, and importance was read off whichever row won. The tiers'
font ranges overlap (48120 reaches 200 where 48130 starts at 160 and goes
to 220), so a hamlet row could outrank the village in its own cluster and
import the cluster one tier lower than the register says — 36 clusters, all
2 → 1 (reproduced: the old pick left 36 clusters whose chosen row's tier sat
below the cluster max, every one a village demoted to hamlet).

Tier before font in the cluster pick. The (name, cl) ORDER BY is now
(kielikoodi = 'fin') DESC, importance DESC, kirjasinkoko DESC NULLS LAST —
the cluster is represented by its highest tier, and font size only breaks
ties within a tier. Language preference stays ahead of the tier (R92): a
Swedish top-tier row must not beat the Finnish name for the same place.
The importance CASE is hoisted into the innermost subquery and emitted
as a column, so the ordering and the emitted column are the same expression
and cannot drift apart (it was written twice — once in the ORDER BY, once
in the select list).

Verification (live DB, `places` layer reimported)

0 clusters with the chosen row below the cluster's max tier (was 36).
This is now a permanent acceptance test on the layer — "R85/R92: no (name,
cl) cluster is represented below its highest register tier" — which rebuilds
the pick exactly as the query does. On the old order it reports the 36
demoted clusters; on the new order it passes.
Row count 267,811, one less than the issue's stated 267,812. The 36
fixed clusters picked their village rows, and one of those rows landed on an
occupied point: Oitbacka (Swedish) and Oitmäki (Finnish) are the two
names of the same Kirkkonummi village, both now at tier 2 with font 200 at
the same coordinate. The coordinate dedupe keeps the Finnish name (R92), so
the map shows one correct label instead of a correct tier-2 label plus a
demoted tier-1 twin at a second point. The issue's "unchanged count"
assumption missed this same-place pair; the drop is the fix working.
R85 unchanged: top five by rank are still Helsinki, Turku, Vantaa,
Tampere, Oulu (the existing R85 test passes).
Per-zoom rank budgets unchanged: z4=1, z5=4, z6=17, z7=70, z8=282,
z9=1129, z10=4519, z11=18076, z12=72305 — each exactly the configured cap.
Decoded z13 tile around Oitmäki: the feature is present with
importance: 2, tile keys are name/importance/country — rank stays
excluded (R83's manual half, still no test — #125).
Sample clusters from the issue (Åkö, Annala, Fröjdö, Haikari, Hietaranta,
Isomäki, Kurkimäki, Liuksiala) now appear at importance 2.

Acceptance tallies (`make acceptance-all`, after the reimport)

11 test(s): 11 passed, 0 failed, 0 skipped, 0 not run   (ismap)
13 test(s): 13 passed, 0 failed, 0 skipped, 0 not run   (semap)
13 test(s): 13 passed, 0 failed, 0 skipped, 0 not run   (plmap)
12 test(s): 12 passed, 0 failed, 0 skipped, 0 not run   (fimap — includes the new test)
9 test(s):  9 passed, 0 failed, 0 skipped, 0 not run    (world)

Left undone

Nothing. The count-shift story above is the only place the issue's
expectations differed from what the data does; the durable property — no
cluster represented below its highest tier — is what the new test pins.

Closes #93 --- ## #93 — Finland's places dedupe tiebreaks on the register tier before font size ### What changed One query, `conf/fimap.yaml`'s `places` layer, and its acceptance tests. The `(name, cl)` cluster pick ordered on `(kielikoodi = 'fin')` then `kirjasinkoko`, and `importance` was read off whichever row won. The tiers' font ranges overlap (48120 reaches 200 where 48130 starts at 160 and goes to 220), so a hamlet row could outrank the village in its own cluster and import the cluster one tier lower than the register says — 36 clusters, all 2 → 1 (reproduced: the old pick left 36 clusters whose chosen row's tier sat below the cluster max, every one a village demoted to hamlet). 1. **Tier before font in the cluster pick.** The `(name, cl)` ORDER BY is now `(kielikoodi = 'fin') DESC, importance DESC, kirjasinkoko DESC NULLS LAST` — the cluster is represented by its highest tier, and font size only breaks ties within a tier. Language preference stays ahead of the tier (R92): a Swedish top-tier row must not beat the Finnish name for the same place. 2. **The importance CASE is hoisted** into the innermost subquery and emitted as a column, so the ordering and the emitted column are the same expression and cannot drift apart (it was written twice — once in the ORDER BY, once in the select list). ### Verification (live DB, `places` layer reimported) - **0 clusters with the chosen row below the cluster's max tier** (was 36). This is now a permanent acceptance test on the layer — "R85/R92: no (name, cl) cluster is represented below its highest register tier" — which rebuilds the pick exactly as the query does. On the old order it reports the 36 demoted clusters; on the new order it passes. - **Row count 267,811, one less than the issue's stated 267,812.** The 36 fixed clusters picked their village rows, and one of those rows landed on an occupied point: **Oitbacka** (Swedish) and **Oitmäki** (Finnish) are the two names of the same Kirkkonummi village, both now at tier 2 with font 200 at the same coordinate. The coordinate dedupe keeps the Finnish name (R92), so the map shows one correct label instead of a correct tier-2 label plus a demoted tier-1 twin at a second point. The issue's "unchanged count" assumption missed this same-place pair; the drop is the fix working. - **R85 unchanged**: top five by rank are still Helsinki, Turku, Vantaa, Tampere, Oulu (the existing R85 test passes). - **Per-zoom rank budgets unchanged**: z4=1, z5=4, z6=17, z7=70, z8=282, z9=1129, z10=4519, z11=18076, z12=72305 — each exactly the configured cap. - **Decoded z13 tile** around Oitmäki: the feature is present with `importance: 2`, tile keys are `name`/`importance`/`country` — `rank` stays excluded (R83's manual half, still no test — #125). - Sample clusters from the issue (Åkö, Annala, Fröjdö, Haikari, Hietaranta, Isomäki, Kurkimäki, Liuksiala) now appear at importance 2. ### Acceptance tallies (`make acceptance-all`, after the reimport) ``` 11 test(s): 11 passed, 0 failed, 0 skipped, 0 not run (ismap) 13 test(s): 13 passed, 0 failed, 0 skipped, 0 not run (semap) 13 test(s): 13 passed, 0 failed, 0 skipped, 0 not run (plmap) 12 test(s): 12 passed, 0 failed, 0 skipped, 0 not run (fimap — includes the new test) 9 test(s): 9 passed, 0 failed, 0 skipped, 0 not run (world) ``` ### Left undone Nothing. The count-shift story above is the only place the issue's expectations differed from what the data does; the durable property — no cluster represented below its highest tier — is what the new test pins.

art added 1 commit

2026-08-15 09:47:42 +00:00

#93 : places dedupe picks the cluster's highest register tier before font size 44abd6455f

The (name, cl) cluster pick ordered on (kielikoodi = 'fin') then
kirjasinkoko, and importance was read off whichever row won. The tiers'
font ranges overlap (48120 reaches 200 where 48130 starts at 160 and
goes to 220), so a hamlet row could outrank the village in its own
cluster and import the cluster one tier lower than the register says —
36 clusters, all 2 -> 1.

Order the tiebreak on the importance tier before the font size, and
hoist the importance CASE into the innermost subquery so the ordering
and the emitted column cannot drift apart. Language preference stays
ahead of the tier (R92): demoting it would let a Swedish top-tier row
beat the Finnish name for the same place.

Verification against the live DB (places layer reimported):
- 0 clusters with the chosen row below the cluster's max tier (was 36).
- Row count 267,811 (was 267,812): the Oitbacka/Oitmäki pair — Swedish
  and Finnish names of the same Kirkkonummi village — now both pick
  their tier-2 rows, which sit at the same point, and the coordinate
  dedupe keeps the Finnish name. One label for the place instead of a
  correct tier-2 label plus a demoted tier-1 twin.
- R85 top five unchanged (Helsinki, Turku, Vantaa, Tampere, Oulu);
  per-zoom rank budgets unchanged (z5=4 … z12=72305).
- Decoded z13 tile: Oitmäki importance 2, rank absent from tile keys.
- make acceptance-all: 58/58 across all configs, incl. the new test
  "R85/R92: no (name, cl) cluster is represented below its highest
  register tier" (fails on the old order: 36 demoted clusters).

art referenced this pull request

2026-08-15 09:47:44 +00:00

Finland's places dedupe tiebreaks on font size, demoting 36 clusters a tier #93

art-bot added 1 commit

2026-08-15 10:11:57 +00:00

#93 review: say which way the font ranges overlap, and what a language/tier collision would mean 3df6050e82

The new test's comment had 48120 "reaching" 200 and 48130 "going to" 220,
which reads as two ceilings. 200 is 48120's floor: the tiers span 200–375
and 160–220, and the overlap is the 200–220 band both live in — that band
is why a hamlet row at 220 could outrank a village row at 200.

Also record why the test's assertion and R92 do not fight: no cluster in
the register mixes languages and tiers at once, so the Finnish-first pick
is never the demoted row. Should a vintage ever produce one, the test
fires on a rule conflict, not on a regression, and the reader now knows
to weigh the two rather than reach for the previous ordering.

art-bot added 1 commit

2026-08-15 10:16:15 +00:00

#93 review: drop the kohdeluokka passthrough the importance hoist orphaned 422ca65e97

Hoisting the importance CASE down into the innermost subquery left
g.kohdeluokka selected there with nothing above it reading the column —
the tier is now carried as `importance` instead. A plain subquery with an
unreferenced select-list column is a no-op to trim, and the layer query
run both ways agrees: 267811 rows either way, same md5 over
(name, importance, rank) ordered by rank, 5820560abff41f87ce2ae1400e38aeb1.

art merged commit 422ca65e97 into main

2026-08-15 10:19:48 +00:00

art deleted branch issue-93-places-dedupe-tier

2026-08-15 10:19:49 +00:00