Each sync run now writes _strava_sync_status.json per user (status,
imported count, error message). New admin endpoints expose this data
and allow triggering an on-demand sync. The admin page gains a Strava
Sync section showing per-user token/credentials state, total imported,
last sync time, and last-run status with inline error messages.
- Add bincio sync-strava command: headless multi-user Strava sync
designed for systemd timer. Discovers users via strava_token.json,
skips users without their own strava_credentials.json, respects
Strava visibility (only_me → unlisted). Treats 404 stream errors as
no-GPS activities rather than retrying every run.
- Add deploy/systemd/bincio-sync.{service,timer}: runs every 3 hours,
Persistent=true to catch up after downtime.
- Add POST /api/internal/rebuild: webhook for sync timer to trigger
site rebuild, authenticated via X-Sync-Secret header.
- Add suspended column to users table with auto-migration on open_db.
Suspended users are blocked at login and session lookup (covers both
activity site and wiki, which share instance.db).
- Add POST /api/admin/users/{handle}/suspend|unsuspend and
DELETE /api/admin/users/{handle}/account endpoints.
- Admin panel: Suspend/Unsuspend toggle, Del account button, suspended
badge on user row.
Replace per-upload Astro build threads with a single background worker
(_site_rebuild_worker) that waits on an event, sleeps 60 s to let upload
bursts settle, then runs one full build + rsync. 271 concurrent uploads now
produce one build instead of 271 serialised builds, eliminating the OOM kill.
--webroot is re-enabled; merge-only path still runs immediately per upload.
Also: date filter row added to ActivityFeed.svelte (sport + date presets
with dynamic year pills); deploy/vps gitignored for VPS config backups.
Four related issues made uploading 271+ activities unreliable:
1. merge_all/write_combined_feed were inside the extraction try/except —
any merge race returned 422 even though the file was on disk, causing
the mobile app to permanently mark the upload as failed. Fixed by
moving them to a separate best-effort try/except after the extraction
block. Switch to merge_one (single-activity symlink) instead of
merge_all (full rebuild) so each upload is O(1) FS ops, not O(N).
2. The dev watcher fired merge_all for every activity .json write AND the
upload endpoint also ran merge_all — O(N²) symlink operations during
bulk uploads. Watcher now skips activities/*.json changes (upload
endpoint handles those directly).
3. Vite/Chokidar followed the public/data symlink and opened a handle per
activity file; constant merge rebuilds exhausted file descriptors and
crashed the Astro dev server. Fixed with watch.ignored on public/data.
4. _write_year_shards and write_combined_feed used f.unlink() without
missing_ok=True — concurrent callers racing the same file threw
FileNotFoundError which propagated as a false extraction failure.
Inline script in Base.astro sets --accent / --accent-dim CSS variables
before first paint based on the current date. Switches to pink (Giro),
yellow (Tour), or red (Vuelta) during each Grand Tour window; falls back
to the default blue. Also aligns default --accent with the mobile app
(#60a5fa instead of #00c8ff).
Android release builds block cleartext HTTP by default (debug builds override
this via the debug manifest overlay). Add usesCleartextTraffic=true to app.json
so expo prebuild includes it in the generated manifest — required to reach local
Bincio instances over HTTP.
In bincio dev (Astro dev server), /activity/<id>/ routes 404 because
getStaticPaths() returns [] and there is no nginx try_files fallback. Add a Vite
middleware plugin to astro.config.mjs that rewrites /activity/<id>/ to /activity/
in dev, matching what nginx does in production.
- dem.py: apply 45s median filter before hysteresis to suppress SRTM
tile-boundary steps that were accumulating through the 5m threshold;
raise DEM hysteresis threshold from 5m to 10m
- dem.py: back up elevation_m as elevation_m_original in timeseries
before the first DEM overwrite, so original sensor data is preserved
- dem.py: add recalculate_elevation_hysteresis() — recomputes gain/loss
from original recorded elevation (reads elevation_m_original if a DEM
run already replaced elevation_m) using source-aware thresholds
(5m barometric, 10m GPS/unknown); does not touch the elevation array
- edit/server.py, serve/server.py: split /recalculate-elevation into
two endpoints: /recalculate-elevation/dem and
/recalculate-elevation/hysteresis
- EditDrawer.svelte: replace single DEM button with two side-by-side
buttons — "Recalculate (hysteresis)" (fast, offline) and
"Recalculate (DEM)" (SRTM lookup)
Adds a "Recalculate from terrain map (DEM)" button to the activity edit
drawer. On click it queries an Open-Elevation-compatible API to replace
GPS altitude with SRTM terrain data, applies 5m hysteresis, and updates
the activity's elevation stats and timeseries chart in place.
- bincio/extract/dem.py: lookup_elevations() (batched HTTP POST) +
recalculate_elevation() (subsample → DEM → interpolate → hysteresis →
patch activity JSON, timeseries JSON, index.json)
- POST /api/activity/{id}/recalculate-elevation on both serve and edit
servers; serve endpoint is auth-gated and triggers merge + rebuild
- --dem-url flag (also DEM_URL env var) on bincio serve and bincio edit;
logged at startup; missing URL returns a clear 503 with setup instructions
- /api/me response gains dem_configured bool
- EditDrawer: button with loading state, shows new ↑/↓ values on success
The counter now shows "50 of 16398 activities" using the total from
feed.json, matching the previous behaviour where all activities were
loaded upfront.
Three bugs in the time↔distance x-axis toggle:
1. GPS speed glitches (e.g. a 1-second spike at 222 km/h) were accumulated
into dist_km, pushing all subsequent points ~60 m too far right on the
distance axis and compressing the rest of the chart. Cap speed at 150 km/h
during dist_km integration; values above that are treated as 0 movement.
2. Observable Plot auto-infers the y domain from plottable points only.
When x-mode changes, which points are "plottable" changes too, so the
y axis range silently shifted between time and distance views. Fix by
computing lineDomainMin/Max once from the full dataset and passing an
explicit domain to Plot.
3. monotone-x curve requires strictly increasing x. In distance mode,
stopped segments produce consecutive points with identical dist_km,
causing NaN Bézier control points and visual artifacts. Use linear
curve for distance mode (data is dense enough that it looks smooth).
Instead of the browser resolving 20+ user shards recursively (~27 MB),
generate a pre-sorted feed.json at merge time with 50 activities per
page. The global feed loads one ~30 KB file on first paint; "Load more"
fetches subsequent pages (feed-2.json, feed-3.json, etc.).
Per-user profile pages still use year-sharded loadIndexPaged as before.
- serve/server.py GET adds private:bool to the response (true when
privacy is "unlisted" or legacy "private") so EditDrawer can read it
- edit/server.py GET: same fix for the single-user edit server
- EditDrawer: fall back to d.privacy if d.private is absent; rename
"Private" toggle label to "Unlisted"
rewriteActivityUrls now skips URLs that are already absolute (start with
/ or http). Before this fix, the new user→year two-level nesting caused
year-shard URLs (/data/brut/_merged/activities/X.json) to be prepended
again at the user-shard level, producing broken doubled paths and making
every activity show "Activity not found".
delete_activity now updates data_dir/index.json so merge_all no longer
re-adds the summary for a deleted activity, preventing the broken
"Activity not found" state after deletion.
ActivityDetailLoader switches from loadIndex (all year shards) to
loadIndexPaged (first year shard only) + direct file fallback, so
opening an activity detail page no longer downloads the entire history.
merge_all/_merged/index.json is now a shard manifest; activities are
split into index-{year}.json files. The feed loads only the most-recent
year on first paint (~200 activities instead of all of them). Older
years are fetched lazily when the user clicks "Load older activities".
Also strips best_efforts / best_climb_m / source from shard files —
these fields are aggregation inputs only, never read by the feed UI.
Add application/gzip, application/x-gzip, and .gz to the accept
attribute so browsers show .gpx.gz / .fit.gz / .tcx.gz in the picker.
Browsers often ignore double-extension filters (.gpx.gz) without the
matching MIME type.
- EditDrawer: stop auto-inserting  into description on
upload — images are tracked via custom.images; the refs only cluttered
the textarea. Strip any pre-existing refs on load so old sidecars are
also cleaned up when the drawer is opened.
- ActivityDetail: imageBase now treats detail_url that starts with '/'
as already-absolute (same fix pattern as track_url / detail_url);
was prepending ${base}data/ on top of /data/... → double path.
Move "Delete original files" out of the Storage card and into the
Danger zone as a first, less-destructive step (simple confirm, no
password needed). "Delete all activity data" and "Delete account"
follow below it, both still password-gated. Descriptions clarify
exactly what each action does and does not remove.
- user_prefs table in db.py with get/set helpers
- GET/PUT /api/me/prefs endpoints for bulk pref management
- GET/PUT/DELETE /api/me/strava-credentials; PUT preserves existing
secret when client_secret field is left blank
- _strava_creds() helper resolves per-user → instance fallback across
all five Strava endpoints
- Settings page: Navigation card (hide Feed/Community/About toggles)
and Strava credentials card
- Base.astro: ids on feed/community/about nav links; applies
nav_hide_* prefs after login
API endpoints (all auth-gated to the logged-in user):
- GET /api/me/storage — per-category disk breakdown
- DELETE /api/me/originals — free originals/ dir (post-extraction cleanup)
- DELETE /api/me/activities — wipe all activity data (password confirm)
- DELETE /api/me — delete account + all data (password confirm)
- PUT /api/me/display-name — update display name
- PUT /api/me/password — change password (requires current password)
Page at /settings/:
- Storage card: activities / originals / Strava originals / photos / total
with one-click 'Delete original files' when originals exist
- Profile card: display name field with inline save
- Password card: change password form
- Danger zone: delete all activities or delete account (both require
password confirmation in a modal before proceeding)
Nav: 'Settings' link appears in the top bar after login (same as Admin).
When 'Overwrite existing activities' is checked, duplicate activities are
re-extracted and replaced instead of silently skipped:
- Deletes {id}.json, .geojson, .timeseries.json from activities/ and _merged/
- Removes the stale index summary and dedup cache entry
- Ingests the new file fresh via ingest_parsed
- Reports 'overwritten' (↺) status in the SSE stream vs 'imported' (↓)
- done event includes 'overwritten' count; UI shows it alongside 'added'
- /api/admin/disk now includes in_db flag per user (true if account exists in DB)
- Ghost users (directory exists, no DB account) show amber 'ghost' badge and only
Diag + Delete dir buttons (no Re-extract, Rebuild, Reset pwd, Reset data)
- DELETE /api/admin/users/{handle}/directory wipes the entire directory and updates
the root manifest; refuses if the account still exists in the DB
- Wires up rmdir-btn with a window.confirm before calling the new endpoint
resolveShards also rewrites track_url to absolute paths (/data/…).
The trackUrl reactive statement only handled http:// prefixes,
producing double /data//data/… for the GeoJSON fetch → map had no track.
resolveShards rewrites detail_url to absolute paths (e.g. /data/brut/_merged/activities/{id}.json)
when fetching from a user shard. loadActivity and loadTimeseries only checked for http:// prefixes
and treated /data/... paths as relative, producing double /data//data/... in the fetch URL → 404.
Fix: treat URLs starting with / as already absolute, same as http:// URLs.
- Generator now yields a 'status' event immediately so the client can
distinguish 'working' from 'failed silently before first event'
- Batch mode: call write_activity per file but write index.json and
athlete.json only once at the end (was O(n²) — 2015 rewrites)
- JS: check r.ok before reading the body stream; show HTTP error detail
instead of staying stuck at 'Starting…'
- Handle 'status' event type in the progress log
- POST /api/admin/users/{handle}/reextract-originals: reads stored
originals/strava/*.json and re-runs strava_to_parsed + ingest_parsed
without hitting the Strava API; streams SSE progress; calls merge_all
and rebuild on completion
- GET /api/admin/users/{handle}/diag: now shows _merged/activities/
file counts, a sample of filenames in activities/ (with symlink flag),
and lists pending_files by name
- Admin page: Re-extract button per user with live SSE progress modal
- bincio.serve logger wired into uvicorn output: rebuild steps, upload
errors, strava-zip progress all now appear in the server log
- _trigger_rebuild: capture stdout/stderr, log errors instead of silently
discarding; exceptions logged with traceback instead of swallowed
- upload handler: log per-file errors with traceback; include error detail
in the SSE event sent back to the browser
- strava-zip handler: log imported/error counts on completion
- GET /api/admin/users/{handle}/diag: snapshot of a user's data dir
(file counts, sizes, index activity counts, pending uploads)
- POST /api/admin/users/{handle}/rebuild-sync: blocking rebuild that
returns full stdout/stderr — for debugging without SSH log access
- Admin page: Diag button per user opens a modal showing the diag JSON
getStaticPaths now returns [] — all /activity/{id}/ URLs are served by
the activity/index.html shell via nginx try_files and hydrated by
ActivityDetailLoader. Pre-rendering thousands of pages was exhausting
server RAM and killing the build. The dynamic loader already handles
public, unlisted, and local activities identically.
db.py: reset_codes table (code, handle, created_by, created_at,
expires_at, used_at); create_reset_code() invalidates any prior unused
code for the same handle; use_reset_code() validates handle match,
expiry (24 h), and single-use; change_password() updates the hash.
server.py: POST /api/admin/users/{handle}/reset-password-code (admin)
returns a code; POST /api/auth/reset-password (public) validates the
code + handle and sets the new password.
Admin page: "Reset pwd" button per user — shows the code inline on
click (monospace, click-to-copy).
/reset-password/ page: handle + code + new password form.
Login page: "Forgot password?" link.
The old DELETE /api/admin/users/{handle}/activities only removed *.json
files and _merged/, leaving originals/ (Strava FIT files) and edits/
untouched — causing the 968 MB disk usage after a delete.
_wipe_user_activities() now removes activities/, edits/, originals/,
_merged/, index.json, athlete.json, and .bincio_cache.json. Admin page
button renamed to "Reset data" with updated confirmation text.
Server endpoint removes the activity JSON, GeoJSON, timeseries, sidecar
edit, and images directory. Also purges the dedup cache entry so the
file can be re-uploaded if needed. Runs merge_all + rebuild afterwards.
EditDrawer: two-click delete button (click once → "Confirm delete?",
click again → deletes). On success, dispatches 'deleted' event.
ActivityDetail navigates back to the feed on delete.
FIT parser: try enhanced_altitude before altitude. Barometric altimeters
on modern Garmins (Edge 540, 840, etc.) write enhanced_altitude in
record messages and total_ascent in lap messages. The old code read only
altitude, producing null elevation_m per point → null elevation_gain_m
at the activity root while laps had correct values from total_ascent.
ActivityMap: use preview_coords (passed from ActivitySummary) to
initialise the map at the activity's location on mount, eliminating the
flash of world-view before the async detail JSON / bbox arrives.
- "unlisted" = not shown in the public feed, but GPS track, timeseries
and detail JSON are all accessible by direct URL (security by obscurity)
- "private" accepted as legacy alias everywhere (backward compat with
existing data on disk)
- New writes from Strava sync / ZIP upload / sidecar use "unlisted"
- Only "no_gps" now suppresses the GPS track
- isUnlisted() helper in format.ts used by all Svelte/Astro components
- SCHEMA.md and CLAUDE.md document the privacy model and the distinction
between "unlisted" and "no_gps"
If the index-based lookup fails (shard fetch silently failed, stale
index state, etc.), try fetching the activity detail file directly from
each user shard's _merged/activities/ directory. This makes private
activities and newly-synced activities accessible even when the index
resolution fails.
Also add console.error logging when shards fail in resolveShards to
help diagnose root causes.
added 3256 more.
- danilo: _merged/ is 8 KB — basically empty. merge_all likely ran concurrently (multiple file uploads trigger multiple rebuilds without a lock in --no-build mode),
causing a race where shutil.rmtree(merged_acts) from one run wiped what another run was writing.
Two fixes: serialize --no-build rebuilds with the same lock, and add a "Rebuild" button to the admin page.
Root causes fixed:
1. merge_all race condition — --no-build rebuilds now hold _rebuild_lock, same as full builds
2. The SSE rebuild-trigger bug (already fixed earlier) was brut's original cause
next server restart.
Admin page now shows:
- Overall disk bar (used/free/%)
- Per-user table: total, activities (with file count), originals (with Strava breakdown), merged, images
- A mini bar per user showing relative size
- Red ⚠ warning if orphaned temp ZIPs are still present for a user
- Delete activities button (reloads sizes after)
Davide Scaini