- Go 90.6%
- HTML 3.5%
- Makefile 3.1%
- Python 1.1%
- JavaScript 0.9%
- Other 0.8%
The add-country gate (R5/R6/R2) for the four countries in issues #210-#213. Nothing is imported; these are catalogue-and-gate documents in the docs/country-gr.md shape, with the sources verified live rather than taken from a research report. What the four verdicts are: - **Czechia** is a GO on every shared layer from a single source. ZABAGED polohopis covers roads (with class *and* number), rail, stations, the Prague metro, trams in seven cities, 34 river ferries, airports, water, 35 224 springs, 386 waterfalls, 3.9 M building polygons and settlements carrying name, population and settlement type in one layer. It is anonymous WFS plus a bulk GeoPackage, and its licence is a machine-readable per-dataset CC BY 4.0 assertion, which is exactly what Greece's YPEN records lack. No new importer provider needed. - **Norway** is a GO on every layer with no credential anywhere. Kartverket's N50 was verified by actually ordering and unpacking two municipalities, so the feature-type names in the doc are from delivered GML rather than a product spec: Bygning is gml:Surface (polygons), Stasjon carries navn, and ElvMidtlinje/InnsjoMidtlinje give the throughflow centrelines a Strahler network needs. Geonorge's order API is Geotorget's shape without the OAuth - a POST with no account returned a ReadyForDownload file - so Norway needs one new provider (#219) and no secrets. NVDB API v3 is decommissioned; v4 is pinned. - **Denmark** is a GO split across two access routes, and the split is the trap: the anonymous INSPIRE route keeps its attributes in every theme *except* transport, where roadlink is 2 506 759 features with no class, no name and no ref. Classes, road numbers, station names and building footprints are behind a token (#220). Denmark is a building_points country - its 5 994 941 buildings are MultiPoint. Transit is unresolved rather than absent. - **Slovakia** is a GO whose problem is delivery, not data. Only two ZBGIS themes are WFS; everything else is behind an ArcGIS MapServer whose /query endpoint answered 503 on every attempt. That is recorded as a re-test (#226), not a verdict - the host resolves and its metadata returns 200, so this is load-shedding, not an unreachable source. The pilot is unblocked anyway: the mask WFS works, and SAZP serves CORINE 2018 with a code_18 column over an anonymous CC BY 4.0 WFS, so Slovakia needs neither the clms: provider nor a credential. Three absences are deliberately left as UNRESOLVED rather than written down (Denmark's water_points and ferries, Slovakia's ferries): R6 says an absence is a claim about the national dataset, and the provider class lists those claims would rest on have not been read. An absent verdict is never tested by anything, which is why it gets the higher bar. R7 is green for all four, checked against the embedded Go font's cmap: Latin Extended-A is 128/128, so Czech e-caron/r-caron/u-ring, Slovak l-caron/l-acute/r-acute and North Sami d-stroke/eng/t-stroke all render. Recorded in R7's example; the invariant test is #214. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzoRW8cpUdAiQyKcXp1rjL |
||
|---|---|---|
| .bob/context | ||
| .claude | ||
| bench | ||
| conf | ||
| docs | ||
| html | ||
| importer | ||
| mapsrv | ||
| scripts | ||
| stylegen | ||
| .gitignore | ||
| AGENTS.md | ||
| CLAUDE.md | ||
| Makefile | ||
| opencode.jsonc | ||
| README.md | ||
ismap
How far can we build a map of European countries from official public data, without relying on OpenStreetMap?
That is the question this project exists to answer. It is a PostGIS-based importer and tile server for official national map data, currently covering Iceland (LMÍ), Sweden (Lantmäteriet + Trafikverket), Poland (GUGiK) and Finland (MML + Väylävirasto + SYKE). All four are first-class — none is a test case for the others — and the list is expected to grow, though there is no ambition to cover all of Europe.
Sweden and Finland share a land border, which is how the central claim — that a road class
means the same thing in every country — stopped being an argument and became a test: at
Tornio–Haparanda the two national road networks meet with a 0 m gap and the same class on
both sides. See docs/country-finland.md.
Every layer comes from a national mapping agency or an equivalent official source, plus Natural Earth for the global background. When data is missing or awkward the answer is to find the official source that has it, not to reach for OSM or an OSM derivative.
The project name
ismapis obsolete. It stood for Iceland map, which stopped being what this is. A rename is wanted; no replacement has been chosen yet, so read the repository name as a historical accident rather than as a description. It also survives as the defaultismap_table prefix, which shared layers replace withmap_. (conf/isand theismap key are not obsolete — thereisis just Iceland's country code, same asseandpl. Thezxy_istile-source function is gone as of issue #152:zxy_mapserves every country now.)
The end product is a PostgreSQL/PostGIS database populated with correctly imported and generalized map data, served as Mapbox Vector Tiles (MVT). The database schema is the deliverable; the server is a development convenience for looking at the result.
What each country currently imports reflects where importer development went rather than a
considered scope — Poland in particular is thin. Feature parity is the goal, as far as
each country's data allows. The three maps used to be three parallel stacks sharing an
importer; converging them on one MVT source, one style and one set of shared table
definitions is done, and docs/unification.md is the record of it.
Architecture
national map data ──► importer ──► PostGIS (maps db) ──► mapsrv ──► browser
(GeoPackage, the deliverable :8080
shapefile, GML,
GTFS)
Two Go binaries and a database:
- importer — the focus of development. Reads the layer config, fetches source data, creates the schema, imports, generalizes across zoom levels, and installs the stored MVT functions.
- mapsrv — one process serving the whole map from a single origin: vector tiles from
the stored functions, MapLibre glyphs and sprites generated on the fly, the static map
pages, and the feedback API that files map-data reports as Forgejo issues. It speaks
HTTP/1.1 and HTTP/2; a browser only takes the latter over TLS, so set
tls_cert/tls_keyinconf/mapsrv.yamlif you want it (it is worth less than it sounds — seedocs/performance.md). - PostGIS — shared
map_<layer>_<z>tables partitioned by country, one partition per country per zoom level, holding generalized geometry. Land cover, water, roads, rail and the world background all live there. A handful of Iceland-only layers still use the older per-country<prefix>_<layer>_<z>naming.
Prerequisites
- Go (1.25+) — the only build dependency; both binaries are pure Go, no cgo
- PostgreSQL with PostGIS — see below
- make, curl, unzip
- Disk: source data is ~900 MB for Iceland, ~25 GB for Sweden, ~11 GB for Poland (from a ~15 GB download) and ~29 GB for Finland, plus the database itself — Poland's land cover alone is ~11 GB imported
First setup
1. PostgreSQL + PostGIS
On Debian/Ubuntu the distribution packages are enough, and the server starts on install:
sudo apt install postgresql postgis
If your distribution has no usable PostGIS package, run one in Docker instead — this is the only thing Docker is needed for, and it is not part of the everyday development path:
docker run -d --name ismap-postgres -p 127.0.0.1:5432:5432 \
-e POSTGRES_PASSWORD=postgres -v ismap-pgdata:/var/lib/postgresql/data \
postgis/postgis:18-3.6
2. Tune it for the import
The import is a bulk workload and the stock configuration is sized for something else.
In particular the partitioned generalization cascade (Sweden and Poland land cover) will
fail on the default 8 MB temp_buffers with "no empty local buffer available".
sudo cp conf/postgresql-tuning.conf /etc/postgresql/18/main/conf.d/
sudo systemctl restart postgresql
The values assume ~30 GB RAM and 8 cores; see the comments in the file to scale them. Iceland alone imports fine without this, just more slowly.
Sweden's and Poland's land cover also need the importer told how much memory it may use:
the cascade's peak is roughly -jobs × -batch-rows polygons held at once, and the
defaults (8 × 50000) are too generous for them on a machine this size. Import those two
with -jobs 4 -batch-rows 10000 (make import-se / make import-pl do this for you). If
a cascade step dies rather than errors — the importer says conn is dead — that is a
Postgres backend being OOM-killed, and lowering -batch-rows further is the fix.
3. Create the role, database and extension
make setup-db
This is the one step needing postgres superuser rights, so it shells out to
sudo -u postgres psql. It creates the maps role, the maps database it owns, and
the PostGIS and SFCGAL extensions inside that database — putting PostGIS in the
postgres maintenance database instead is the easy mistake here. It is idempotent,
so re-running it is safe and cheap. Against the Docker fallback:
SUPER_PSQL="docker exec -i ismap-postgres psql -U postgres" ./scripts/setup-db.sh
4. Import and serve
make import # fetches any missing source data, then imports Iceland (~10 min)
# plus the shared world background, which takes about a second
make start # mapsrv in the background on :8080
make smoke-test # verify a tile comes back
make acceptance # run the data's own acceptance tests against the imported database
Then open http://localhost:8080. Iceland is the quickest country to bring up and needs
no credentials, which is why it is the one in the first-run instructions; make import-se,
make import-pl and make import-fi are the others (see below for what they need —
Finland needs no credentials either, just patience and disk).
Everyday use
make build # build both binaries
make test # run the importer and mapsrv unit tests
make acceptance # run the acceptance tests against the live database (fast buckets)
make acceptance-all # ... including the 100s and 1000s ones (the full review pass)
make serve # mapsrv in the foreground (Ctrl-C to stop)
make start/stop/restart # mapsrv in the background (run/mapsrv.pid, run/mapsrv.log)
make psql # psql shell on the maps database
make import # (re)import Iceland
make import-se # (re)import Sweden
make import-pl # (re)import Poland — needs make fetch-bdot10k first, ~15 GB
make import-fi # (re)import Finland — no credentials, first run pulls ~29 GB
make import-world # (re)build the shared, country-independent layers (~1 s)
make reset-db # drop every imported table and tile function
make fresh-start # reset-db, import Iceland + Sweden + world, restart, smoke-test
make graph # refresh the graphify code index (optional, needs graphify)
make test covers the Go code; make acceptance covers the data. The acceptance tests
are declared in conf/*.yaml next to the layers they check — "Sweden's five biggest cities
top the place ladder", "no cover partition is empty at any zoom", "both sides of the
Tornio–Haparanda bridge carry the same trunk E4" — and the fast ones also run automatically
after every layer import. Anything you would otherwise verify by hand in psql belongs there;
docs/importer-cli.md has the contract.
Each country import chains import-world itself, so you rarely need to run it directly.
It exists because the world background is Natural Earth land minus the union of every
country's outline, which changes whenever a country is imported — and the importer's
dependency graph is per config, so it can't see that across configs. Rebuilding the world
needs no credentials and no country: given an empty set of country outlines it just builds
the whole world with no holes.
To iterate on a single layer without re-running the whole cascade, call the importer directly — it resolves the layer's dependencies from the config's import DAG:
importer/importer import --layer waterways conf/is
To iterate on generalization — min_area_pixels, min_width_pixels, tolerances,
rollups, zoom filters —
add --regeneralize <lo>-<hi> to rebuild only that inclusive zoom window, generalizing
down from whatever is already live above it. It skips fetching, staging and the maxz
import, none of which a generalization parameter can change:
importer/importer import --layer cover --regeneralize 5-11 -jobs 6 -batch-rows 10000 conf/se
importer/importer import --layer cover --regeneralize 5 conf/se # just the bottom step
The difference is large for a big country: Sweden's cover is most of an hour before the
cascade even starts, while its coarsest step alone re-runs in under a minute.
The maps
One page, html/index.html, serves all of them, selected by ?map=<key>:
| Key | Map | Config | Sources |
|---|---|---|---|
is |
Iceland | conf/is |
LMÍ IS 50V + land cover (open download) |
se |
Sweden | conf/se |
Lantmäteriet Topo 10/50, Trafikverket NVDB/NJDB, Trafiklab GTFS |
pl |
Poland | conf/pl |
GUGiK BDOT10k + PRG |
Every map also draws the layers that have been unified across countries from a shared
map source, declared by conf/world.yaml: the low-zoom world background, land cover,
and water (lakes and rivers, watercourses, and waterfalls/rapids/springs). All but the
background are built by each country's own config into its own partition of one table.
Poland is already served entirely from there and has no source of its own.
Iceland needs no credentials. Sweden requires API credentials for Lantmäteriet
Geotorget, Trafikverket Lastkajen and Trafiklab, and the map feedback API needs a
Forgejo bot token; all of them live in conf/secrets.yaml, which is gitignored. See
conf/secrets.example.yaml for the shape. Poland needs no credentials but does need the
make fetch-bdot10k bulk download (~15 GB pulled, ~11 GB kept).
Data sources
None of the data files are committed. The importer fetches what a given run needs (see
downloads: in each config), re-downloading only when the server's Content-Length or
Last-Modified differs from the last fetch, which is recorded in a .fetchmeta sidecar
next to each file. The make fetch-* targets below predate that and remain for fetching
individual Iceland datasets by hand.
make fetch-data # all Iceland base datasets (~200 MB, excludes elevation)
make fetch-elevation # elevation contours (~350 MB, separate due to size)
make fetch-bdot10k # Poland land cover + water + roads/rail, 380 per-county downloads
# (~15 GB pulled; the county zips are deleted after extraction, so
# widening BDOT10K_CLASSES needs the whole set pulled again — the
# marker file is named after the class set to force exactly that)
Iceland datasets
Only Iceland's sources are tabulated here, because they are individually fetchable by hand
from open URLs. Sweden's and Poland's are declared in conf/se and
conf/pl (downloads:, plus the geotorget:/lastkajen:/trafiklab: provider
blocks) and documented in docs/country-sweden.md and
docs/country-poland.md.
| Makefile target | File | Source | Last updated | Description |
|---|---|---|---|---|
fetch-lcv |
is_lcv_lmi_epsg_4326.gpkg |
LMÍ / natt.is | 2023-12-30 | Land cover polygons, CORINE/CLC classification |
fetch-roads |
is_50v_samgongur_epsg_4326.gpkg |
LMÍ / natt.is | 2025-07-18 | Roads and transportation network |
fetch-water |
is_50v_vatnafar_epsg_4326.gpkg |
LMÍ / natt.is | 2025-02-15 | Rivers, lakes, water bodies (~120 MB) |
fetch-coastline |
is_50v_strandlina_epsg_4326.gpkg |
LMÍ / natt.is | 2024-05-15 | Coastline |
fetch-admin |
is_50v_mork_epsg_4326.gpkg |
LMÍ / natt.is | 2025-02-27 | Municipal and administrative boundaries |
fetch-buildings |
is_50v_mannvirki_epsg_4326.gpkg |
LMÍ / natt.is | 2024-05-19 | Buildings and structures |
fetch-names |
is_50v_ornefni_epsg_4326.gpkg |
LMÍ / natt.is | 2026-04-22 | Place names (örnefni) |
fetch-surface |
is_50v_yfirbord_epsg_4326.gpkg |
LMÍ / natt.is | 2024-01-12 | Land surface classes per ÍST120 |
fetch-elevation |
is_50v_haedargogn_epsg_4326.gpkg |
LMÍ / natt.is | 2024-01-02 | Elevation contours and spot heights (~350 MB) |
fetch-clc |
clc.csv |
EEA EIONET | — | CORINE Land Cover code table |
All LMÍ datasets are downloaded from https://atlas.lmi.is/heikir/downloadData/. LMÍ
merged into Náttúrufræðistofnun (natt.is) in July 2024; the atlas.lmi.is URLs still work.
All data is CC BY 4.0.
Database
host: 127.0.0.1:5432
database: maps
user: maps
password: maps
Connection string: postgresql://maps:maps@127.0.0.1/maps
These are development credentials for a local database and are checked in deliberately. Do not expose this port beyond localhost.
Developer notes
New to maps and GIS? Start with docs/geo-primer.md — "So you
were sucked into a map project", a one-sitting introduction to projections, tiles, zoom
levels, generalization, MVT and the ways real-world map data is messy, annotated with where
each idea shows up in this codebase. It assumes you can code and knows nothing about maps.
CLAUDE.md is the short orientation and the index to everything else: it says what the
project is, the conventions to work by, and which docs/ file holds which mechanism (the
import DAG, the generalization strategies, the schema, the shared-layer machinery, the
importer's flags, the server, the style).
The rest is under docs/:
architecture.md— the data flow end to end: the import dependency DAG, the per-layer pipeline and atomic swap, staging tables, and a map of every source file.schema.md— tables and columns, the shared country-partitioned layer machinery, MVT zoom-range behaviour, derived columns.generalization.md— every cascade strategy and the config keys that tune it.importer-cli.md— running an import:-layer,-regeneralize, the memory budget, postgres tuning, the fetch policy.mapsrv.md— the server: HTTP/2, caching and the data-version ETag, generated glyphs and sprites, the map-report → Forgejo API.style.md—html/style.jsonand the invariants that break invisibly..bob/context/— the operating texts of the bob-agent review loop that works the backlog: the implementer, the two reviewers and the orchestrator.adding-a-country.md— the repeatable shape for adding a country, from the go/no-go catalogue to a verified import.naming.md— why the project is calledismapand what is obsolete about it, plus the table and function naming conventions.country-iceland.md,country-sweden.md,country-poland.md,country-finland.md,country-gr.md— each country's sources, providers, credentials and class mappings. Read the relevant one before touching a country's layers. Finland's is the fullest: it also carries the complete available→imported→worth-importing catalogue for its providers, and the Sweden–Finland border test.unification.md— how the three per-country stacks became one set of shared partitioned tables, one MVT source and one style, and the design decisions behind it. Worth reading before starting work on a shared layer.performance.md— what tiles actually cost and why, with the measurements behind the generalization settings.
The work backlog is Forgejo issues on art/ismap, not a TODO list in a file.