Data directory from the environment: `$ISMAP_DATA`, so a checkout or worktree needs no hand-made `data` symlink #240

New issue

Open

opened 2026-10-09 06:48:22 +00:00 by art-bot · 0 comments

art-bot commented

2026-10-09 06:48:22 +00:00

Collaborator

Problem

data/ is a gitignored symlink to /stuff/ismap-data/data, the download cache shared between the accounts on this machine (AGENTS.md, data/ conventions). Every human checkout here has the link. A bob-agent session worktree does not, so every import there fails.

The link is a property of checkouts on this machine, not of the repository. Committing it would force one machine's layout on anyone else who clones the repo, so the location should come from the environment instead.

How configurable it is today

Barely:

Makefile: DATA_DIR = data (Makefile:1), but only 4 places use it: the BDOT10k paths and one download rule.
Configs: 190 literal data/... paths across 11 files under conf/, most of them in */downloads.yaml.
Importer: there is no data-root concept. Every config path is opened relative to the working directory, across about eight path fields (one per data provider: fetch.go, arcgis.go, clms.go, geotorget.go, lastkajen.go, trafiklab.go, staging.go's file, databases:).
mapsrv does not read data/.

Making every one of those paths configurable is a large change for no gain. Keeping ./data as the one name everything uses, and creating it from a machine setting, is a small one.

Proposal

When ./data does not exist (as a directory or as a link), and the environment names a location, create ./data as a link to it:

$ISMAP_DATA, if set.
Open question: $HOME/ismap-data, if it exists, as a fallback that needs no environment set up at all.
Otherwise nothing changes: data/ is an ordinary directory, created on first fetch, exactly as today.

Do it in two places:

The importer, early in main, so direct runs (importer/importer acceptance conf/<cc>/, which the review loop's texts use) are covered.
One Makefile rule that the targets touching $(DATA_DIR) depend on, so make and its curl-based download targets are covered.

Rules:

Never replace an existing ./data. If it is a link to somewhere other than $ISMAP_DATA, print a warning and leave it alone.
The link is already gitignored (/data), so it never makes a checkout or a session worktree dirty.

Docs: the data/ paragraph in AGENTS.md describes the variable instead of the symlink; README's setup section gets one line; the comment at the top of importer/sharedtree.go names the variable.

How it is used here

Humans (art, claude): set ISMAP_DATA=/stuff/ismap-data/data in the shell profile, or keep the existing links, which keep working.
bob-agent sessions: the projects set ISMAP_DATA through project environment variables (art/bob-agent#884), and the importer creates the link on first use in each worktree. Once art/bob-agent#885 (worktree setup command) lands, the link can instead be made when the worktree is created, so the first command a session runs already sees it.

Test plan

Importer unit test, in a temp dir:
- no data + ISMAP_DATA set → link created to it;
- data is a directory → untouched;
- data is a link elsewhere → untouched, with a warning;
- neither variable set → nothing created.
End to end: in a fresh git worktree with ISMAP_DATA set, importer/importer acceptance conf/gr/ runs without a hand-made link.

🤖 Generated with Claude Code

## Problem `data/` is a gitignored symlink to `/stuff/ismap-data/data`, the download cache shared between the accounts on this machine (AGENTS.md, *data/ conventions*). Every human checkout here has the link. A bob-agent session worktree does not, so every import there fails. The link is a property of **checkouts on this machine, not of the repository**. Committing it would force one machine's layout on anyone else who clones the repo, so the location should come from the environment instead. ## How configurable it is today Barely: - **Makefile:** `DATA_DIR = data` (`Makefile:1`), but only 4 places use it: the BDOT10k paths and one download rule. - **Configs:** 190 literal `data/...` paths across 11 files under `conf/`, most of them in `*/downloads.yaml`. - **Importer:** there is no data-root concept. Every config path is opened relative to the working directory, across about eight path fields (one per data provider: `fetch.go`, `arcgis.go`, `clms.go`, `geotorget.go`, `lastkajen.go`, `trafiklab.go`, `staging.go`'s `file`, `databases:`). - **mapsrv** does not read `data/`. Making every one of those paths configurable is a large change for no gain. Keeping `./data` as the one name everything uses, and creating it from a machine setting, is a small one. ## Proposal When `./data` does not exist (as a directory or as a link), and the environment names a location, **create `./data` as a link to it**: 1. `$ISMAP_DATA`, if set. 2. *Open question:* `$HOME/ismap-data`, if it exists, as a fallback that needs no environment set up at all. 3. Otherwise nothing changes: `data/` is an ordinary directory, created on first fetch, exactly as today. Do it in two places: - **The importer**, early in `main`, so direct runs (`importer/importer acceptance conf/<cc>/`, which the review loop's texts use) are covered. - **One Makefile rule** that the targets touching `$(DATA_DIR)` depend on, so `make` and its `curl`-based download targets are covered. Rules: - **Never replace an existing `./data`.** If it is a link to somewhere other than `$ISMAP_DATA`, print a warning and leave it alone. - The link is already gitignored (`/data`), so it never makes a checkout or a session worktree dirty. Docs: the `data/` paragraph in AGENTS.md describes the variable instead of the symlink; README's setup section gets one line; the comment at the top of `importer/sharedtree.go` names the variable. ## How it is used here - **Humans** (`art`, `claude`): set `ISMAP_DATA=/stuff/ismap-data/data` in the shell profile, or keep the existing links, which keep working. - **bob-agent sessions:** the projects set `ISMAP_DATA` through project environment variables (art/bob-agent#884), and the importer creates the link on first use in each worktree. Once art/bob-agent#885 (worktree setup command) lands, the link can instead be made when the worktree is created, so the first command a session runs already sees it. ## Test plan - **Importer unit test, in a temp dir:** - no `data` + `ISMAP_DATA` set → link created to it; - `data` is a directory → untouched; - `data` is a link elsewhere → untouched, with a warning; - neither variable set → nothing created. - **End to end:** in a fresh `git worktree` with `ISMAP_DATA` set, `importer/importer acceptance conf/gr/` runs without a hand-made link. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

art-bot referenced this issue

2026-10-09 06:55:30 +00:00

review loop: retire the old skills, add-country as a doc, operating texts fitted to ismap #241