diff --git a/CHANGELOG.md b/CHANGELOG.md index 81b5d7c..a6ec2da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,37 +26,48 @@ the last committed value. GPS noise is suppressed without losing real climbs. suppression, barometric vs GPS threshold difference, real climb approximation, unknown-treated-as-gps invariant -### New feature — DEM-based elevation recalculation from the edit drawer +### New feature — On-demand elevation recalculation from the edit drawer -A new **"Recalculate from terrain map (DEM)"** button in the activity edit -drawer replaces noisy GPS altitude with SRTM terrain data, then re-applies -hysteresis accumulation to compute corrected gain/loss. +Two new buttons in the activity edit drawer fix inaccurate elevation stats +without re-uploading the file: -This is the recommended fix for activities that still show inaccurate -elevation after the hysteresis improvement (e.g. activities recorded before -re-extracting from sources, or uploads where the source file had severe GPS -altitude noise). +**📐 Recalculate (hysteresis)** — re-applies source-aware hysteresis +accumulation to the original recorded elevation. Fast, offline, no network +required. Best for barometric altimeters (Karoo 2, Garmin with +`enhanced_altitude`, Wahoo) that were extracted before the noise-filtering +improvement. -How it works: -1. The server subsamples the activity's 1 Hz GPS track (one point every 10 s) -2. Queries an Open-Elevation-compatible API for terrain elevation -3. Linearly interpolates DEM elevation back to every GPS-valid second -4. Applies 5 m hysteresis to compute the corrected gain and loss -5. Writes the updated `elevation_m` array to the timeseries (chart updates) -6. Patches `elevation_gain_m` / `elevation_loss_m` in the activity JSON and - `index.json` summary +**⛰ Recalculate (DEM)** — replaces GPS altitude with SRTM terrain data, then +re-applies hysteresis. Best for GPS-only devices where the recorded altitude +is noisy. -- **`bincio/extract/dem.py`** (new) — `lookup_elevations()` (batched HTTP POST, - Open-Elevation wire format) + `recalculate_elevation()` (full pipeline above) -- **`POST /api/activity/{id}/recalculate-elevation`** — on both `bincio serve` - (auth-gated, triggers `merge_one` + rebuild) and `bincio edit` (no auth) +DEM pipeline (revised after discovering that a naive 5 m threshold produced +results worse than no correction on some activities): +1. Subsample GPS track to one point per 10 s +2. Query Open-Elevation API in batches of 512 +3. Linearly interpolate back to the full 1 Hz series +4. Apply a **45 s sliding median filter** to suppress SRTM tile-boundary + steps (occur every ~7 s at cycling speed; were accumulating through 5 m + threshold and inflating gain by 50 %+) +5. Apply **10 m hysteresis** to the smoothed series +6. Back up original `elevation_m` as `elevation_m_original` in the timeseries + on the first DEM run (never overwrites an existing backup) + +- **`bincio/extract/dem.py`** (new) — `lookup_elevations()`, + `recalculate_elevation()` (DEM + median + 10 m hysteresis), + `recalculate_elevation_hysteresis()` (offline, reads `elevation_m_original` + if available, uses 5 m/10 m source-aware threshold) +- **`POST /api/activity/{id}/recalculate-elevation/dem`** and + **`POST /api/activity/{id}/recalculate-elevation/hysteresis`** — on both + `bincio serve` (auth-gated, triggers `merge_one` + rebuild) and + `bincio edit` (no auth) - **`bincio serve --dem-url URL`** / **`bincio edit --dem-url URL`** — override the default DEM endpoint (also read from `DEM_URL` env var) -- Default DEM endpoint: **`https://api.open-elevation.com`** — works out of the - box with no configuration +- Default DEM endpoint: **`https://api.open-elevation.com`** — works out of + the box with no configuration - **`GET /api/me`** response gains `dem_configured: bool` -- **`EditDrawer.svelte`** — button with spinner, shows `↑ Xm ↓ Ym` on success - or an inline error (e.g. if the DEM API is unreachable) +- **`EditDrawer.svelte`** — two side-by-side buttons with individual spinners, + shows `↑ Xm ↓ Ym` on success or inline error --- diff --git a/docs/elevation.md b/docs/elevation.md index 8dfb2d2..c53fa61 100644 --- a/docs/elevation.md +++ b/docs/elevation.md @@ -277,23 +277,55 @@ require re-extraction from source files. ### Medium term — ✅ Implemented (2026-04-20) -**On-demand DEM correction** via the edit drawer, using the Open-Elevation API -(SRTM30 data): +**Two on-demand recalculation options** in the activity edit drawer: + +#### Option 1 — Hysteresis (fast, offline) + +Re-applies the same source-aware hysteresis accumulation as the extract +pipeline directly to the recorded elevation, with no network calls. + +- Uses `elevation_m_original` from the timeseries (the backup saved on the + first DEM run) if present; otherwise uses the current `elevation_m`. +- Threshold: **5 m** for barometric sources, **10 m** for GPS. +- Does not modify the elevation array in the timeseries — only patches + `elevation_gain_m` / `elevation_loss_m`. +- Best for: devices with a barometric altimeter (e.g. Karoo 2, Garmin with + `enhanced_altitude`) where the recorded elevation is already accurate but + was extracted before the hysteresis fix was deployed. + +#### Option 2 — DEM terrain correction (SRTM30, requires network) + +Replaces the recorded GPS altitude with terrain data from an +Open-Elevation-compatible API (SRTM30, ~30 m resolution): 1. GPS track subsampled to one point per 10 s to minimise API calls. 2. Terrain elevation fetched via `POST https://api.open-elevation.com/api/v1/lookup` in batches of 512. 3. DEM elevation linearly interpolated back to the full 1 Hz series. -4. 5 m hysteresis applied to the interpolated series. -5. Timeseries and activity JSON patched in place; chart and stats update immediately. +4. **45 s sliding median filter** applied to suppress SRTM tile-boundary + steps (these occur every ~7 s at cycling speed and accumulate as phantom + gain through a naive threshold). +5. **10 m hysteresis** applied to the smoothed series. +6. Original elevation backed up as `elevation_m_original` in the timeseries + (only on the first DEM run — never overwrites an existing backup). +7. Timeseries and activity JSON patched in place; chart and stats update. -Implementation: `bincio/extract/dem.py` + `POST /api/activity/{id}/recalculate-elevation` -on both servers. Default endpoint: `https://api.open-elevation.com`; override with +Best for: GPS-only devices (no barometric sensor) where the recorded +altitude is noisy and the DEM terrain is a better ground truth. + +> **Why median + 10 m, not 5 m?** SRTM30 at 1 Hz produces step changes at +> tile boundaries of 1–3 m every few seconds. A 5 m threshold lets most of +> these through; they accumulate and can inflate the result by 50 %+. The +> 45 s median smooths the steps before the dead-band sees them; 10 m catches +> any residual outliers. + +Implementation: `bincio/extract/dem.py` — `lookup_elevations()`, +`recalculate_elevation()`, `recalculate_elevation_hysteresis()`. +API endpoints: `POST /api/activity/{id}/recalculate-elevation/dem` and +`POST /api/activity/{id}/recalculate-elevation/hysteresis` on both servers. +Default DEM endpoint: `https://api.open-elevation.com`; override with `--dem-url` or `DEM_URL` env var. -This is the recommended fix for activities uploaded before the hysteresis improvement, -or any activity where GPS noise is severe. - --- ## Implementation status @@ -305,8 +337,8 @@ or any activity where GPS noise is severe. | `bincio/extract/parsers/gpx.py` | ✅ sets `altitude_source = "gps"` | | `bincio/extract/parsers/tcx.py` | ✅ sets `altitude_source = "gps"` | | `bincio/extract/metrics.py` | ✅ hysteresis `_elevation()` with source-aware threshold | -| `bincio/extract/dem.py` | ✅ `lookup_elevations()` + `recalculate_elevation()` | -| `bincio/serve/server.py` | ✅ `POST /api/activity/{id}/recalculate-elevation` | -| `bincio/edit/server.py` | ✅ same endpoint (single-user) | -| `site/src/components/EditDrawer.svelte` | ✅ "Recalculate from terrain map" button | +| `bincio/extract/dem.py` | ✅ `lookup_elevations()` + `recalculate_elevation()` (median+10m) + `recalculate_elevation_hysteresis()` | +| `bincio/serve/server.py` | ✅ `POST /api/activity/{id}/recalculate-elevation/{dem\|hysteresis}` | +| `bincio/edit/server.py` | ✅ same endpoints (single-user) | +| `site/src/components/EditDrawer.svelte` | ✅ two buttons: "Recalculate (hysteresis)" + "Recalculate (DEM)" | | `tests/test_metrics.py` | ✅ 5 parametric tests | diff --git a/docs/user-guide.md b/docs/user-guide.md index 66b3faf..5c59210 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -73,20 +73,23 @@ Click **Edit** on any activity to: Changes save instantly. The site rebuilds in the background. -### Recalculating elevation from terrain data +### Recalculating elevation -If an activity shows an unrealistic elevation gain (common with GPS-only devices on flat -routes, or with older Garmin/Wahoo files), the edit drawer has a -**"Recalculate from terrain map (DEM)"** button. +If an activity shows an unrealistic elevation gain, the edit drawer has two buttons: -Clicking it replaces the recorded GPS altitude with SRTM terrain data from the -[Open-Elevation API](https://open-elevation.com) and recomputes the gain and loss. The -elevation chart and the summary stats both update. This usually brings the numbers in -line with what Strava or your device's app reports. +**📐 Recalculate (hysteresis)** — recomputes gain and loss from the original recorded +elevation using a noise-filtering dead-band algorithm. Fast and offline — no network +call. Best for devices with a barometric altimeter (Garmin, Karoo, Wahoo) whose +elevation data is accurate but was extracted before the noise-filtering was improved. -> **Note:** The correction requires a GPS track (activities marked *No GPS* cannot be -> corrected). The DEM has ~30 m horizontal resolution, so very short or indoor activities -> are not meaningfully improved. +**⛰ Recalculate (DEM)** — replaces the recorded GPS altitude with SRTM terrain data +from the [Open-Elevation API](https://open-elevation.com) and recomputes gain and +loss. The elevation chart and summary stats both update. Best for GPS-only devices +(no barometric sensor) where the recorded altitude is noisy. + +> **Note:** Both corrections require a GPS track (activities marked *No GPS* cannot be +> corrected). The DEM option uses ~30 m resolution terrain data; very short or indoor +> activities see little improvement from DEM correction. ### Photo gallery