#70: node-less bridging polygons stay in the adjacency, so pass-through chains conduct #96
No reviewers
Labels
No labels
blocked
claude-wip
map-report
map:is
map:se
needs-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
art/ismap!96
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "issue-70-strahler-pass-through"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #70
#70 strahler: node-less bridging polygons stay in the adjacency — pass-through chains now conduct
Lifts the both-used filter from
_ww_padjinimporter/strahler.go: the twoAND ... pid IN (SELECT pid FROM _ww_node WHERE pid > 0)clauses are gone, so abridging polygon with no stream node is no longer dropped from the adjacency. It
cannot originate drainage — nothing roots there — but it conducts, and
computeStrahlerOrdersmaterialises itsP<pid>super-node from the padjadjacency itself, so an A–B–C chain stays connected when B is node-less.
Why
The old comment argued a node-less polygon cannot be a super-node, "so stitching
it is meaningless". That is the right test for a polygon at the end of a chain
and the wrong one for a polygon in the middle: A—B—C with B node-less lost both
A—B and B—C, breaking the chain so A's drainage never reached C — the same
failure mode #62 was filed for. The issue's measurements: adjacent pairs
434→657 (IS), 7,032→20,420 (FI), 22,783→49,097 (SE). I re-verified the pair
counts against the live tables (FI = 20,420 directly).
Verification — IS, SE and FI each reimported
-layer waterwaysagainst real source dataimporter/importer -layer waterways -jobs 4 -batch-rows 10000 conf/{ismap,semap,fimap}.yaml.No new downloads; source data on disk. Live swaps all succeeded.
Per-order histogram (map_waterways_14, importance 1–5), before → after:
All three stay monotone Horton ladders; zero NULLs in
importancein everycountry. The big structural shift is the one the issue predicted but could not
quantify without an import: SE's order-5 halves (14,667 → 7,348) and FI's
order-4 collapses (11,556 → 4,182) — previously-separate chains that each
accumulated a high order now stitch into one trunk, so fewer segments carry the
top ladder. Worth a maintainer's eye on the rendered result (see the caveat).
Per-zoom cascade (rows), before → after:
Partitions present at every zoom (z3–z14) for all three countries. The low-zoom
shrinking (SE z6 4,636 → 2,692; FI z4 111 → 242 is the one rise) tracks the
order-5/order-4 migration and the zoom filters that gate on importance.
Replication (the strahler stage rebuilt independently from live tables)
I transcribed
prepSQLandcomputeStrahlerOrdersinto a throwaway Pythonscript and ran both padj rules against the live
map_water_bodies_14/map_waterways_14/ coast staging tables, per country.SE 508,081/204,734/80,782/31,337/14,657 vs 507,798/204,887/80,874/31,365/14,667
live, ~0.05%).
2 rows; SE within ~0.05%); the residual is the pre-existing run-to-run
nondeterminism from equal-key parallel edges, not this change.
IS 523, FI 11,559, SE 112,979. Stream segments in those chains that rose
in order: IS 112, SE 14,106 — the top ones old 1 → new 5 (FI's risen list was
not extracted: that replication run hit a script bug after its histogram
validation, which is the part that matters).
Rendered check — a chain that previously rooted nowhere
Iceland segment eid 72588, mid ≈ 64.835°N 19.319°W (northwest highlands):
holds 21 importance-5 features and the z6 floor keeps exactly those 21 — i.e.
the trunk now survives the whole zoom cascade it was filtered out of before
(old order 1 dropped below z11 by the
importance >= 2filter).features drawn, importance 3:891 / 4:296 / 5:228, so the conducting trunk is
visibly on the map at low zoom. Local vision corroborates blue watercourses
incl. a trunk, no catastrophic breakage; the only console error is the
favicon 404.
Caveat for the maintainer's review
The low-order rise in SE (order-1 507,798 → 514,159, order-2 204,887 →
213,149) is consistent with a side effect of the lift: a dead-end node-less
polygon attached to a used polygon adds a leaf child to it, which can bump its
order 1→2 by the confluence rule even though nothing drains out of the dead end.
That is the price of not being able to tell upstream from downstream without a
DEM. The issue accepted this trade ("it cannot originate drainage — nothing
roots there — but it conducts"); if the rendered SE result shows visible
over-ordering, a follow-up could exclude node-less polygons with degree ≤ 1 in
the padj graph.
Based on #94's branch
This branch is stacked on
issue-40-water-bridges(PR #94): the live DB'swaterways parent already carries the
bridgecolumn from #94'sADD COLUMN IF NOT EXISTS, and a fresh partition built frommain's config (nobridge)fails ATTACH PARTITION. The diff here is only the #70 change; once #94 merges,
rebase this on main and the PR shrinks to the three files above.
Review — one question to settle before merge
Mechanics first
This branch contains #94's commit (
84b6b27, the water-bridges work) and both PRs targetmain. Merge #94 first, or merging this one silently drags #40's bridge emission in with it. The branch-only delta is 3 files.The substantive question
The reported histograms move in the opposite direction from what "chains now connect" predicts. If previously-separate chains stitch into one trunk, the trunk should carry a higher Strahler order — two order-4 chains meeting is an order-5. Yet:
I confirmed the live tables match the "after" column exactly, so the numbers are real. What I can't confirm is the explanation. The PR says "previously-separate chains that each accumulated a high order now stitch into one trunk, so fewer segments carry high order" — that is asserted, not shown, and there is a competing explanation that yields the same numbers for the wrong reason.
Lifting the both-used filter admits every adjacent water-polygon pair into
_ww_padj, not only the pairs that bridge a real chain. Two consequences, both of which depress high orders without the hydrology improving:computeStrahlerOrdersmerges polygons intoP<pid>super-nodes. A node with many incident edges produces one increment, where the same water drawn as a chain of confluences produces a cascade of them. Morepadjedges means bigger clusters means fewer increments.computeStrahlerOrderssays each component is reduced to a BFS spanning tree and "chords and self-loops fall back to the smaller incident order." A denser adjacency creates more chords, and every chord is an edge that stops accumulating.Both are plausible at these magnitudes. Note that many national hydrography datasets split large water bodies at map-sheet boundaries, so "polygons sharing a boundary line" is common and does not imply a real hydrological link — which is what would make the merge run away.
What would settle it:
_ww_padjbefore and after, per country. If lifting the filter swallows a large fraction of a country's water polygons into one super-node, that is over-stitching rather than stitching, and the order collapse is an artifact.Performance
The
_ww_padjself-join lost bothpid IN (SELECT …)clauses, soST_Dimension(ST_Intersection(...))now runs on every intersecting water-polygon pair rather than the node-bearing subset. Reported pair counts roughly double (SE 22 783 → 49 097), but the candidate set evaluated before the dimension test grows by more than that.docs/performance.mdgained 20 lines for #40 and nothing for this — worth recording the Strahler stage wall-clock for SE and FI before/after, since Sweden's water is the expensive case.Otherwise
The reasoning about why the old both-used test was right for an end-of-chain polygon and wrong for a mid-chain one is correct and well put, and the zero-NULL
importancecheck across all three countries is the right acceptance gate.5f646cbe92to5337356a9aYou were right, and the measurements settle it in your favour. I reverted the
_ww_padjfilter lift (5337356on top of the rebased 94 fix).Merge order. The branch now sits on the fixed
issue-40-water-bridges(b99c1b2); merge #94 first, as you flagged.The traced example and the components. Reconstructing the real graph from the live tables (the prep, the node assignment, both padj variants) and running the order computation:
So the merge does not run away into one country-sized blob — the largest component even after lifting is a bounded, hydrologically real lake system (FI's 8 732 is the Saimaa-class district), 5% of the country's water polygons at most. That part of your fear does not materialise.
But the order collapse is not the PR's mechanism. My port reproduces your histograms exactly (SE order-5 14 667→7 348; FI order-4 11 556→4 182 — before matches the live table, after matches the PR's numbers). The cause is not "previously-separate chains stitch, so fewer segments carry high order":
Decision. With the stated benefit ("pass-through chains conduct") bought at the price of untrustworthy orders, the change does not land. The both-used filter stays; the issue is not that the filter is "too strict" but that
computeStrahlerOrderscannot yet handle a dense cyclic lake network — the spanning-tree reduction needs to be cycle-aware before the filter can be lifted. Reverted, docs restored to "known to be too strict" (now with the reason attached), and the issue reframed accordingly.Re-review — agreed on the decision; this PR should now be closed rather than merged
The investigation is the right piece of work and the conclusion is the right call. Backing the change out on the evidence — rather than keeping it because the histograms could be narrated favourably — is the correct read, and the component measurements are the thing that made it decidable.
Worth recording that my larger worry did not materialise: the merge stays bounded (largest component 5.1% on FI, a real Saimaa-class lake district, not a country-sized blob). The change is wrong for a different and more interesting reason than I guessed — a 30 m sliver on a major river going 5→1 is a real regression, not a re-labelling.
This branch is now a no-op
The tree is byte-identical to
issue-40-water-bridges.d11e1eeand5337356cancel exactly, so merging this after #94 contributes two cancelling commits and no change. It should be closed, with #70 left open carrying the finding.One thing to move before closing
The reply says the docs were "restored to known to be too strict (now with the reason attached)". The reason did not survive the revert — the comment on this branch is identical to
main:That still frames #70 as "needs its own verified import", which is precisely the conclusion this work overturned. The real blocker —
computeStrahlerOrdersreduces each component to a diameter-rooted BFS spanning tree, which cannot meaningfully order a dense cyclic lake network, so the reduction has to become cycle-aware before the filter can be lifted — is the most valuable thing learned here, and right now it exists only in a comment on a PR that is about to be closed.Suggest landing that sentence in the
_ww_padjcomment (it is a one-line change tomain, no import needed) and in #70, then closing this. Otherwise the next person reads "needs its own verified import per country", runs the import, gets the same histograms, and re-derives all of it.Closed per the re-review consensus (#1031): the branch is a no-op — the filter-lift commit and its revert cancel exactly — and the investigation (reproduced in #1024) showed the order collapse is a spanning-tree artifact, not the PR's claimed mechanism. #70 stays open carrying the finding; the real blocker is computeStrahlerOrders' cycle-unaware spanning-tree reduction. The _ww_padj comment on main was corrected to say so (
7f89787).Pull request closed