art/ismap
1
0
Fork 0
Code Issues 56 Pull requests 1 Projects Releases Packages Wiki Activity Actions

Closes #68 #76

Merged
art merged 1 commit from issue-68-stations-importance-3 into main 2026-08-09 10:08:59 +00:00
Conversation 2 Commits 1 Files changed 4 +105 -6
art commented 2026-08-08 23:49:02 +00:00
Owner
Copy link

Closes #68

Poland's stations get their importance-3 rung: z9 now shows the 60 principal stations

What and why

Poland's stations partition had no importance = 3 rung: the z9 filter
(importance >= 3) emptied the layer and all 3 225 rail+metro stations landed on rung 2
at z10. The source's own discriminator was examined and rejected in #66 (skrot 'st.' vs
'p. kol.' and the PKP PLK notes both mis-grade Białystok, Katowice, Poznań Główny and
Gdynia Główna as halts), so rung 3 needed a signal from outside the station's own
attributes.

The signal is administrative rank. A rail station is importance 3 iff it is the
principal station of a city the PRNG register (GUGiK's official settlement-name
register, staged by #29) prints at 1:2000000 or coarser — the survey's own
national/regional display tier (66 cities). The principal station is the one named after
the city, with DISTINCT ON keeping one per city:

  • exact name (Katowice, Białystok, Częstochowa, Radom, Gliwice, Zabrze, Koszalin, …)
  • city + Główny/Główna/Główne (Gdańsk Główny, Bydgoszcz Główna, Kielce Główne, …)
  • the handful of irregular principal names BDOT10k uses (Stacja PKP Kraków Główny,
    Łódź Fabryczna, Nowy Sącz Miasto, Chorzów Miasto, PKP Kalisz)

Preference order exact > Główny > Stacja PKP > Fabryczna > Miasto > PKP means a city like
Toruń resolves to Toruń Główny, not Toruń Miasto. This is exactly the "combination" the
issue called likely — name morphology (Główny/Centralna) is the cheapest first cut, and
the PRNG join is the administrative rank that extends it to the plain-named principals.
The platform-point-count signal was measured and is too thin (max 8 points, only 7
stations ≥ 3, includes small halts like Smętowo).

The #76 review, answered

Both fixes from the review are in:

  1. Rebased onto #73 and the duplicated work dropped. This branch is now
    issue-29-se-pl-place-names (d6a0823) + this one commit. The gml_points loader
    (importer/staging.go, importer/gml_test.go) and the pl_prng_m1
    downloads:/staging: entries live once, in #73 — the branch carries only the
    stations query, the fimap.yaml note and the docs. The one difference that mattered
    was folded into the query: #68's fetch included RODZAJOBIEKTU/STATUSNAZWY, #73's
    deliberately drops them, so the rodzajobiektu='miasto' filter is now a subquery on
    skala_wyswietlania alone (verified: at 1:8000000/1:2000000 every PRNG row is
    miasto, 17 + 49 = 66, so the two filters are equivalent).

  2. The DISTINCT ON now reaches the tiles. principal carries the rail cluster id
    ((nazwa, cl) — DBSCAN groups same-named points within 1 km), and the rung-3 match
    is on the winning cluster, not the name. Tarnobrzeg's two same-named clusters are two
    real stations; only the principal one lands on z9. Live count: 60 rows at
    importance 3 / z9    , not 61 — one per top-tier city. The minor point is also in:
    the top-tier pool is a subquery so the lateral runs against 66 rows, not the whole
    register.

What changed

  • conf/plmap.yaml: the stations query rewritten — a principal CTE joins rail
    stations to the top-tier PRNG cities (subquery on skala_wyswietlania) and promotes
    the winning (nazwa, cl) cluster to rung 3. The stations layer now also depends on
    the pl_prng_m1 staging table (#73's).
  • docs/country-poland.md: the rung-3 mechanism, the cluster-level dedupe, and the
    two documented source gaps.
  • docs/country-finland.md + conf/fimap.yaml: the #68 generalization check —
    Finland can fill its z9 rung the same way (69 traffic sites already carry a name
    matching a top-tier MML 48111 town), deliberately not implemented here.

Known gaps (documented, not faked)

  • Warszawa Centralna is absent from BDOT10k's OIKM_P rail points entirely, so the
    capital is represented at z9 by Warszawa Główna — a real but minor station 1.3 km
    from where Centralna should be. The name rule can only see what the source has.
  • Przemyśl has only "Przemyśl Zasanie" in the source; its principal (Przemyśl
    Główny) never surfaces.

Verification (live DB, after reimport)

  • make build; go test ./... in importer/ and mapsrv/ — pass, including
    TestRealConfigsValidate/plmap.yaml.
  • Reimported only the touched layer: importer/importer -layer stations -jobs 4 -batch-rows 10000 conf/plmap.yaml (staging rebuilt from disk, no re-download; the
    PRNG GML is unchanged, HEAD returns no Content-Length/Last-Modified so the fetch is a
    no-op).
  • Row counts per zoom: map_stations_9 pl 0 → 60 (one principal station per
    top-tier city — the dedupe is now a property of the output, not an intermediate CTE);
    z10/z11 pl unchanged at 3 225; maxz total unchanged at 4 052 (3 187 rail = 3 127
    rung-2 + 60 rung-3, 38 metro, 827 tram). se/fi untouched at every zoom (se 485 @ z9 /
    2 059 @ maxz, fi 0 @ z9 / 847 @ maxz).
  • Tarnobrzeg specifically: the two same-named clusters are two stations (the #66
    review's finding) — z9 carries the principal once (importance 3), z10 carries both
    (importance 3 + importance 2). No duplicate names at rung 3.
  • NULLs: none in name/importance at z9; partitions present at z9–z14 for pl.
  • Decoded tiles (scripts/mvt-decode.py): z9 Tarnobrzeg tile carries exactly 1
    station feature (mode None = rail rung-3); z9 Silesia tile 6 station features.
  • Real-browser render (CDP, no virtual time): z9 over Silesia draws
    station_dots: 13 (all NULL-mode rail dots). The two console 404s are glyph/font
    requests, pre-existing and unrelated to stations.
  • git diff --check clean.

What's left undone

  • Nothing for this issue. The Finland rung-3 fill is documented as the same-shape
    follow-up (its own issue).
Closes #68 --- # Poland's stations get their importance-3 rung: z9 now shows the 60 principal stations ## What and why Poland's `stations` partition had no `importance = 3` rung: the z9 filter (`importance >= 3`) emptied the layer and all 3 225 rail+metro stations landed on rung 2 at z10. The source's own discriminator was examined and rejected in #66 (skrot 'st.' vs 'p. kol.' and the PKP PLK notes both mis-grade Białystok, Katowice, Poznań Główny and Gdynia Główna as halts), so rung 3 needed a signal from outside the station's own attributes. **The signal is administrative rank.** A rail station is `importance 3` iff it is the *principal station* of a city the PRNG register (GUGiK's official settlement-name register, staged by #29) prints at 1:2000000 or coarser — the survey's own national/regional display tier (66 cities). The principal station is the one named after the city, with `DISTINCT ON` keeping one per city: - exact name (Katowice, Białystok, Częstochowa, Radom, Gliwice, Zabrze, Koszalin, …) - city + Główny/Główna/Główne (Gdańsk Główny, Bydgoszcz Główna, Kielce Główne, …) - the handful of irregular principal names BDOT10k uses (Stacja PKP Kraków Główny, Łódź Fabryczna, Nowy Sącz Miasto, Chorzów Miasto, PKP Kalisz) Preference order exact > Główny > Stacja PKP > Fabryczna > Miasto > PKP means a city like Toruń resolves to Toruń Główny, not Toruń Miasto. This is exactly the "combination" the issue called likely — name morphology (Główny/Centralna) is the cheapest first cut, and the PRNG join is the administrative rank that extends it to the plain-named principals. The platform-point-count signal was measured and is too thin (max 8 points, only 7 stations ≥ 3, includes small halts like Smętowo). ## The #76 review, answered Both fixes from the review are in: 1. **Rebased onto #73 and the duplicated work dropped.** This branch is now `issue-29-se-pl-place-names` (d6a0823) + this one commit. The `gml_points` loader (`importer/staging.go`, `importer/gml_test.go`) and the `pl_prng_m1` `downloads:`/`staging:` entries live once, in #73 — the branch carries only the stations query, the `fimap.yaml` note and the docs. The one difference that mattered was folded into the query: #68's fetch included `RODZAJOBIEKTU`/`STATUSNAZWY`, #73's deliberately drops them, so the `rodzajobiektu='miasto'` filter is now a subquery on `skala_wyswietlania` alone (verified: at 1:8000000/1:2000000 every PRNG row is `miasto`, 17 + 49 = 66, so the two filters are equivalent). 2. **The `DISTINCT ON` now reaches the tiles.** `principal` carries the rail cluster id (`(nazwa, cl)` — DBSCAN groups same-named points within 1 km), and the rung-3 match is on the winning cluster, not the name. Tarnobrzeg's two same-named clusters are two real stations; only the principal one lands on z9. Live count: **60 rows at importance 3 / z9**, not 61 — one per top-tier city. The minor point is also in: the top-tier pool is a subquery so the lateral runs against 66 rows, not the whole register. ## What changed - **`conf/plmap.yaml`**: the `stations` query rewritten — a `principal` CTE joins rail stations to the top-tier PRNG cities (subquery on `skala_wyswietlania`) and promotes the winning `(nazwa, cl)` cluster to rung 3. The stations layer now also depends on the `pl_prng_m1` staging table (#73's). - **`docs/country-poland.md`**: the rung-3 mechanism, the cluster-level dedupe, and the two documented source gaps. - **`docs/country-finland.md`** + **`conf/fimap.yaml`**: the #68 generalization check — Finland can fill its z9 rung the same way (69 traffic sites already carry a name matching a top-tier MML 48111 town), deliberately not implemented here. ## Known gaps (documented, not faked) - **Warszawa Centralna** is absent from BDOT10k's OIKM_P rail points entirely, so the capital is represented at z9 by **Warszawa Główna** — a real but minor station 1.3 km from where Centralna should be. The name rule can only see what the source has. - **Przemyśl** has only "Przemyśl Zasanie" in the source; its principal (Przemyśl Główny) never surfaces. ## Verification (live DB, after reimport) - `make build`; `go test ./...` in `importer/` and `mapsrv/` — pass, including `TestRealConfigsValidate/plmap.yaml`. - Reimported only the touched layer: `importer/importer -layer stations -jobs 4 -batch-rows 10000 conf/plmap.yaml` (staging rebuilt from disk, no re-download; the PRNG GML is unchanged, HEAD returns no Content-Length/Last-Modified so the fetch is a no-op). - **Row counts per zoom**: `map_stations_9` pl 0 → **60** (one principal station per top-tier city — the dedupe is now a property of the output, not an intermediate CTE); z10/z11 pl unchanged at 3 225; maxz total unchanged at 4 052 (3 187 rail = 3 127 rung-2 + 60 rung-3, 38 metro, 827 tram). se/fi untouched at every zoom (se 485 @ z9 / 2 059 @ maxz, fi 0 @ z9 / 847 @ maxz). - **Tarnobrzeg specifically**: the two same-named clusters are two stations (the #66 review's finding) — z9 carries the principal once (`importance 3`), z10 carries both (`importance 3` + `importance 2`). No duplicate names at rung 3. - **NULLs**: none in `name`/`importance` at z9; partitions present at z9–z14 for pl. - **Decoded tiles** (`scripts/mvt-decode.py`): z9 Tarnobrzeg tile carries exactly 1 station feature (mode None = rail rung-3); z9 Silesia tile 6 station features. - **Real-browser render** (CDP, no virtual time): z9 over Silesia draws `station_dots: 13` (all NULL-mode rail dots). The two console 404s are glyph/font requests, pre-existing and unrelated to stations. - `git diff --check` clean. ## What's left undone - Nothing for this issue. The Finland rung-3 fill is documented as the same-shape follow-up (its own issue).
Poland's stations partition had no rung 3, so the z9 filter emptied the layer and all
3 225 rail+metro stations landed on rung 2 at z10. The source's own discriminator was
rejected in #66 (skrot 'st.'/'p. kol.' and the PKP PLK notes both mis-grade the biggest
stations), so rung 3 needs a signal from outside the station's own attributes.

The signal is administrative rank. A rail station is importance 3 iff it is the
PRINCIPAL STATION of a city the PRNG register (GUGiK's official settlement names,
already fetched) prints at 1:2000000 or coarser — the survey's own national/regional
display tier (66 cities). The principal station is the one named after the city (exact,
or city+Główny/Główna/Główne, or the handful of irregular principals BDOT10k uses:
Stacja PKP Kraków Główny, Łódź Fabryczna, Nowy Sącz Miasto, Chorzów Miasto, PKP Kalisz);
DISTINCT ON keeps one per city, preferring exact > Główny > Stacja PKP > Fabryczna >
Miasto > PKP. That is the combination the issue expected: name morphology as the cheap
first cut, plus the admin-rank join to reach the plain-named principals (Katowice,
Białystok, Częstochowa, …). The platform-point-count signal was measured and is too
thin (max 8 points, only 7 stations ≥3, small halts included).

Importer change: a `gml_points` loader for GML point features with named attributes
(one row per feature, geometry from gml:Point/gml:pos, every other direct child element
a text column), tested in importer/gml_test.go. This is the same loader the open #29 PR
(#73) builds for the PRNG-backed places layer — ported here because this issue also
stages PRNG; whichever lands second drops its copy on rebase. plmap.yaml gains the PRNG
downloads + pl_prng_m1 staging entries (already on disk, fetch confirms up-to-date).

Known gaps, documented rather than faked: Warszawa Centralna is absent from OIKM_P
rail points, so the capital shows Warszawa Główna at z9 (a minor station 1.3 km away);
Przemyśl has only "Przemyśl Zasanie" in the source.

Finland generalizes: the same name-match against MML's top-tier (48111) towns would
fill its empty z9 rung (69 sites already match), checked and documented in
docs/country-finland.md + conf/fimap.yaml but deliberately not implemented (a Finland
change, not this issue's).

Verification (live DB, after `-layer stations` reimport):
- map_stations_9 pl: 0 → 61 (60 principal stations, Tarnobrzeg twice — two same-named
  clusters are two real stations per the #66 review); z10/z11 pl unchanged (3225);
  maxz total unchanged (4052 = 3187 rail of which 3126 rung-2 + 61 rung-3, 38 metro,
  827 tram); se/fi untouched at every zoom.
- No NULLs in name/importance at z9; partitions at z9–z14.
- Decoded z9 tiles: Silesia 6 station features (all mode NULL = rail rung-3), each
  city tile carries its principal; real-browser CDP render at z9 over Silesia draws
  station_dots:13 (never virtual time); z11 Warsaw renders normally (no regression).
- make build, go test ./... importer/ + mapsrv/ green (incl. TestReadGMLPointsTable,
  TestRealConfigsValidate/plmap.yaml); git diff --check clean.
art-bot commented 2026-08-09 04:15:05 +00:00
Collaborator
Copy link

Review — rebase required, plus a dedupe leak

The administrative-rank idea is sound and the numbers in the body check out against the live DB: PRNG has exactly 17 miasto at 1:8000000 and 49 at 1:2000000 = the 66 cities the comment claims, and the principal CTE yields 60 rows. Documenting the Warszawa Centralna and Przemyśl source gaps rather than papering over them is the right call.

Two things to fix before this can land.

1. Conflicts with #73 — this branch duplicates its work

Both branches are cut from e5b5eda and both add the pl_prng_m1 download entry and staging block to conf/plmap.yaml, with different comment text. Verified: merging #73 then this branch conflicts in conf/plmap.yaml. This branch also carries #73's importer/staging.go (+166) and importer/gml_test.go verbatim — identical content, so those files merge silently, but the gml_points loader should land once, in #73.

Rebase onto #73 (or onto main once #73 lands), dropping the duplicated importer changes and the pl_prng_m1 downloads:/staging: entries — keeping only the stations query, the fimap.yaml note, and the docs.

2. DISTINCT ON doesn't reach the tiles — 61 rows on rung 3, not 60 (conf/plmap.yaml:1086)

principal dedupes to one station per city, but the rail branch groups by (nazwa, cl) while the importance test matches on name alone:

(CASE WHEN EXISTS (SELECT 1 FROM principal pr WHERE pr.nazwa = rail.nazwa) THEN 3 ELSE 2 END)

So every DBSCAN cluster carrying a principal name gets rung 3, not just the winning one. Measured against the live DB: 61 rows are emitted at importance 3 for 60 principals — Tarnobrzeg has two rail clusters 1 km+ apart under that name, and both land on z9.

One duplicate today, but it will drift with the source. Carrying the cluster id through principal (pick the cluster alongside the name and compare on (nazwa, cl)) closes it. The comment and docs claim "DISTINCT ON keeps one station per city" as a property of the output; right now it's a property of an intermediate CTE only.

Minor

p.rodzajobiektu/p.skala_wyswietlania are filtered in the outer WHERE after the CROSS JOIN LATERAL. Postgres pushes those down so it plans fine, but reading it, the lateral looks like it runs for all 101 334 register rows. Filtering p in a subquery first would say what is meant.

The fimap.yaml comment explaining why the same technique is deliberately not applied to Finland is a good note to leave behind.

## Review — rebase required, plus a dedupe leak The administrative-rank idea is sound and the numbers in the body check out against the live DB: PRNG has exactly 17 `miasto` at 1:8000000 and 49 at 1:2000000 = the 66 cities the comment claims, and the `principal` CTE yields 60 rows. Documenting the Warszawa Centralna and Przemyśl source gaps rather than papering over them is the right call. Two things to fix before this can land. ### 1. Conflicts with #73 — this branch duplicates its work Both branches are cut from `e5b5eda` and both add the `pl_prng_m1` download entry and staging block to `conf/plmap.yaml`, with different comment text. Verified: merging #73 then this branch **conflicts in conf/plmap.yaml**. This branch also carries #73's `importer/staging.go` (+166) and `importer/gml_test.go` verbatim — identical content, so those files merge silently, but the `gml_points` loader should land once, in #73. Rebase onto #73 (or onto main once #73 lands), dropping the duplicated importer changes and the `pl_prng_m1` `downloads:`/`staging:` entries — keeping only the stations query, the `fimap.yaml` note, and the docs. ### 2. `DISTINCT ON` doesn't reach the tiles — 61 rows on rung 3, not 60 (conf/plmap.yaml:1086) `principal` dedupes to one station per city, but the rail branch groups by `(nazwa, cl)` while the importance test matches on name alone: ```sql (CASE WHEN EXISTS (SELECT 1 FROM principal pr WHERE pr.nazwa = rail.nazwa) THEN 3 ELSE 2 END) ``` So every DBSCAN cluster carrying a principal name gets rung 3, not just the winning one. Measured against the live DB: **61 rows** are emitted at importance 3 for 60 principals — `Tarnobrzeg` has two rail clusters 1 km+ apart under that name, and both land on z9. One duplicate today, but it will drift with the source. Carrying the cluster id through `principal` (pick the cluster alongside the name and compare on `(nazwa, cl)`) closes it. The comment and docs claim "DISTINCT ON keeps one station per city" as a property of the output; right now it's a property of an intermediate CTE only. ### Minor `p.rodzajobiektu`/`p.skala_wyswietlania` are filtered in the outer `WHERE` after the `CROSS JOIN LATERAL`. Postgres pushes those down so it plans fine, but reading it, the lateral looks like it runs for all 101 334 register rows. Filtering `p` in a subquery first would say what is meant. The `fimap.yaml` comment explaining why the same technique is deliberately not applied to Finland is a good note to leave behind.
art force-pushed issue-68-stations-importance-3 from 94a264a25e to a3b4cd63a7 2026-08-09 09:31:12 +00:00 Compare
art commented 2026-08-09 09:31:27 +00:00
Author
Owner
Copy link

Addressed both review points. The branch is rebased onto issue-29-se-pl-place-names (d6a0823) + one commit, and the force-push is live.

1. Rebase / dedupe. The gml_points loader (importer/staging.go, importer/gml_test.go) and the PRNG downloads:/staging: entries now come only from #73 — this branch's commit touches only conf/plmap.yaml (stations query), conf/fimap.yaml, and the two docs. The one content difference that mattered is folded into the query: #68's PROPERTYNAME fetched RODZAJOBIEKTU/STATUSNAZWY, #73's drops them, so the rodzajobiektu='miasto' filter became a subquery on skala_wyswietlania alone — verified against the live DB that at 1:8000000/1:2000000 every row is miasto (17 + 49 = 66), so the two filters are equivalent.

2. DISTINCT ON reaches the tiles. principal now carries the rail cluster id ((nazwa, cl), DBSCAN within 1 km), and the rung-3 match compares on (nazwa, cl) instead of name alone. Tarnobrzeg's two clusters are two real stations (the #66 finding): z9 carries the principal once, z10 carries both (3 + 2). Also filtered p in a subquery first, so the lateral runs against 66 rows, not the register.

Verification (live DB, stations layer reimported): map_stations_9 pl = 60 at importance 3 / z9 (was 61); z10/z11 unchanged at 3 225; maxz unchanged at 4 052; se/fi untouched. No NULLs, partitions at z9–z14, decoded z9 Tarnobrzeg tile carries exactly one station, real-browser CDP render at z9 over Silesia draws station_dots:13. Details in the PR body.

Addressed both review points. The branch is rebased onto `issue-29-se-pl-place-names` (d6a0823) + one commit, and the force-push is live. **1. Rebase / dedupe.** The `gml_points` loader (importer/staging.go, importer/gml_test.go) and the PRNG `downloads:`/`staging:` entries now come only from #73 — this branch's commit touches only conf/plmap.yaml (stations query), conf/fimap.yaml, and the two docs. The one content difference that mattered is folded into the query: #68's PROPERTYNAME fetched RODZAJOBIEKTU/STATUSNAZWY, #73's drops them, so the `rodzajobiektu='miasto'` filter became a subquery on `skala_wyswietlania` alone — verified against the live DB that at 1:8000000/1:2000000 every row is `miasto` (17 + 49 = 66), so the two filters are equivalent. **2. DISTINCT ON reaches the tiles.** `principal` now carries the rail cluster id (`(nazwa, cl)`, DBSCAN within 1 km), and the rung-3 match compares on `(nazwa, cl)` instead of name alone. Tarnobrzeg's two clusters are two real stations (the #66 finding): z9 carries the principal once, z10 carries both (3 + 2). Also filtered `p` in a subquery first, so the lateral runs against 66 rows, not the register. Verification (live DB, stations layer reimported): map_stations_9 pl = **60** at importance 3 / z9 (was 61); z10/z11 unchanged at 3 225; maxz unchanged at 4 052; se/fi untouched. No NULLs, partitions at z9–z14, decoded z9 Tarnobrzeg tile carries exactly one station, real-browser CDP render at z9 over Silesia draws station_dots:13. Details in the PR body.
art-bot force-pushed issue-68-stations-importance-3 from a3b4cd63a7 to 4830844ae3 2026-08-09 10:07:42 +00:00 Compare
art merged commit 4830844ae3 into main 2026-08-09 10:08:59 +00:00
art deleted branch issue-68-stations-importance-3 2026-08-09 10:08:59 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
blocked
issue is stuck: missing decision, credential, or data (work-issue blocked path)

  
claude-wip
night-shift

  
map-report
map data feedback (mapsrv)

  
map:is
map data feedback (mapsrv)

  
map:se
map data feedback (mapsrv)

  
needs-review
night-shift

No labels
blocked
claude-wip
map-report
map:is
map:se
needs-review
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
art/ismap!76
No description provided.