When a user saves new Strava credentials with a different client_id,
auto-delete the existing token (it belongs to a different OAuth app
and will always fail on refresh). Add POST /api/strava/disconnect
endpoint and a "Disconnect from Strava" button in settings, visible
only when connected.
Immediate: deleted diego_p's stale token so he can reconnect.
Apple Watch and similar devices record exactly 0.0 for elevation while
waiting for barometric/GPS lock, then jump to the real altitude. The
hysteresis accumulator was seeding from 0.0, counting the full jump as
ascent. Fix: detect a leading near-zero run followed by a large jump
and seed the accumulator from the first real value instead.
Applied in both _elevation() (fresh extractions) and
recalculate_elevation_hysteresis() (recompute path). Added a bulk
admin endpoint POST /api/admin/users/{handle}/recompute-elevation and
corresponding button to fix existing stored activities.
Each sync run now writes _strava_sync_status.json per user (status,
imported count, error message). New admin endpoints expose this data
and allow triggering an on-demand sync. The admin page gains a Strava
Sync section showing per-user token/credentials state, total imported,
last sync time, and last-run status with inline error messages.
- Add bincio sync-strava command: headless multi-user Strava sync
designed for systemd timer. Discovers users via strava_token.json,
skips users without their own strava_credentials.json, respects
Strava visibility (only_me → unlisted). Treats 404 stream errors as
no-GPS activities rather than retrying every run.
- Add deploy/systemd/bincio-sync.{service,timer}: runs every 3 hours,
Persistent=true to catch up after downtime.
- Add POST /api/internal/rebuild: webhook for sync timer to trigger
site rebuild, authenticated via X-Sync-Secret header.
- Add suspended column to users table with auto-migration on open_db.
Suspended users are blocked at login and session lookup (covers both
activity site and wiki, which share instance.db).
- Add POST /api/admin/users/{handle}/suspend|unsuspend and
DELETE /api/admin/users/{handle}/account endpoints.
- Admin panel: Suspend/Unsuspend toggle, Del account button, suspended
badge on user row.
Replace per-upload Astro build threads with a single background worker
(_site_rebuild_worker) that waits on an event, sleeps 60 s to let upload
bursts settle, then runs one full build + rsync. 271 concurrent uploads now
produce one build instead of 271 serialised builds, eliminating the OOM kill.
--webroot is re-enabled; merge-only path still runs immediately per upload.
Also: date filter row added to ActivityFeed.svelte (sport + date presets
with dynamic year pills); deploy/vps gitignored for VPS config backups.
Adds edits_json column (migration v3) to store user overrides separately
from detail_json so Option A server re-extraction never clobbers them.
- Tap the title in the detail screen to edit (local activities only, shown
with a ✎ hint). Saves on keyboard dismiss via onEndEditing.
- Cards and search display user_title ?? title.
- Raw upload: user_title sent to server -> sidecar written so web UI shows
the correct title (server re-extracts from FIT, which has Karoo's title).
- BAS upload: detail.title overridden before sending, no sidecar needed.
Four related issues made uploading 271+ activities unreliable:
1. merge_all/write_combined_feed were inside the extraction try/except —
any merge race returned 422 even though the file was on disk, causing
the mobile app to permanently mark the upload as failed. Fixed by
moving them to a separate best-effort try/except after the extraction
block. Switch to merge_one (single-activity symlink) instead of
merge_all (full rebuild) so each upload is O(1) FS ops, not O(N).
2. The dev watcher fired merge_all for every activity .json write AND the
upload endpoint also ran merge_all — O(N²) symlink operations during
bulk uploads. Watcher now skips activities/*.json changes (upload
endpoint handles those directly).
3. Vite/Chokidar followed the public/data symlink and opened a handle per
activity file; constant merge rebuilds exhausted file descriptors and
crashed the Astro dev server. Fixed with watch.ignored on public/data.
4. _write_year_shards and write_combined_feed used f.unlink() without
missing_ok=True — concurrent callers racing the same file threw
FileNotFoundError which propagated as a false extraction failure.
Three bugs fixed:
- /api/upload/bas and /api/upload/raw never updated user_dir/index.json, so
merge_all couldn't include uploaded activities in year shards — they existed
on disk but were invisible to the browser feed. Fixed by _upsert_index_summary()
called before merge_all().
- Silent catch {} in uploadLocalActivities swallowed all per-activity errors;
replaced with console.warn so failures are visible in Expo logs.
- After a server wipe, synced_at flags on the device caused "Nothing to upload"
forever. uploadFeed() now reconciles against GET /api/feed at the start of each
upload: local activities not found on the server get synced_at cleared.
Also: live upload progress ("Uploading N / M…"), failed count in result message,
onProgress callback on uploadFeed(), countPendingUploads() helper.
merge_all(user_dir) updates the per-user _merged/ shard but the home page
loads feed.json first via loadCombinedFeed. write_combined_feed was only
called by the CLI render command, not by the API upload endpoints or the
dev watcher, leaving feed.json permanently stale after any runtime upload.
Add write_combined_feed(_get_data_dir()) after every merge_all call in
/api/upload/bas, /api/upload/raw, the dev.py file watcher, and dev startup.
- Skip MapLibre on Android <29 (Karoo): SELinux denies kgsl-3d0 access
from untrusted_app context, crashing the GPU driver on any OpenGL
surface. Replace with SvgRouteView — equirectangular SVG route trace
using react-native-svg, no native GL surface needed.
- Add +/- zoom buttons to full-screen MapLibre map on modern devices
via Camera ref and onRegionDidChange.
- Skip PyodideWebView on Android <29: same GPU driver conflict; set
_engineUnavailable at module init via API level gate (< 29).
- Add engine_unavailable fast path in PyodideWebView: post message
immediately if WebAssembly.Global is absent (Chrome <69) instead of
attempting 30 MB Pyodide download.
- Add server-side extraction fallback (extractServer.ts): when engine
unavailable, POST raw file as base64 to /api/upload/raw; server runs
full Python pipeline and returns extracted data.
- Add /api/upload/raw endpoint in server.py.
- Add pre-flight auth check (checkServerAuth) before batch import so
an expired token errors immediately rather than after N files.
- Fix uploadLocalActivities in sync.ts: was reading original_path as
JSON (binary FIT file, always threw), silently skipping every upload.
Now reads detail_json from DB directly.
- Redesign Feed header: replace single Sync button with Upload /
Download / Refresh. Pull-to-refresh and Refresh button are local-only.
Auto-refresh on tab focus via useFocusEffect.
- Replace ActivityIndicator with plain Text everywhere (native animation
also crashes Karoo GPU driver).
- Raise macOS open-file limit in dev_test.py to prevent EMFILE errors
from Astro file watcher.
- Document all Karoo hardware constraints in docs/mobile-app.md.
- Server: POST /api/upload/bas accepts pre-extracted BAS JSON (activity + optional timeseries/geojson), writes files and triggers merge_all
- sync.ts: uploadLocalActivities reads unsynced local activities by original_path, POSTs to /api/upload/bas, marks synced_at on success
- Settings: Upload toggle (Off / Upload local activities) in Sync section with subLabel dividers for Download / Upload groups
- Feed: sync message includes uploaded count when activities are pushed
- Add /api/activity/{id}/geojson and /api/activity/{id}/timeseries endpoints
(bearer-token-gated, falls back from _merged to raw activities dir)
- Rewrite activity detail screen with MapLibreGL v11 API (Map, Camera,
GeoJSONSource, Layer) and react-native-svg area chart with gradient fill
- On-demand fetch for remote activities that have no local geojson/timeseries
- Add react-native-svg dependency; requires dev build (npx expo run:android)
_merged/index.json is a shard manifest with activities:[] when the user
has >FEED_PAGE_SIZE activities. The endpoint now collects from all
index-{year}.json shard files before returning.
SyncResult gains a `total` field (activities received from server) so the
feed screen can distinguish "server returned nothing" from "all already
stored locally". Messages: "No activities on instance" / "Up to date (N)"
/ "X of N activities synced".
Server (bincio/serve/server.py):
- Add _require_auth: accepts session cookie OR Authorization: Bearer token
- POST /api/auth/token: same as /api/auth/login but returns token in body
(password used once, not stored; mobile stores only the session token)
- GET /api/feed: auth-gated; reads _merged/index.json for the user and
returns the activities array as JSON
Mobile:
- db/sync.ts: syncFeed(db) fetches /api/feed, upserts each summary into
local SQLite as origin='remote'; skips locally-imported activities
- db/queries.ts: add upsertRemoteActivity (INSERT ... ON CONFLICT DO UPDATE
WHERE origin='remote' — never overwrites local imports); fix feed sort
order to started_at DESC instead of insertion order
- settings.tsx: Connect section — password field (not persisted) + Connect
button calls POST /api/auth/token and stores token; Disconnect clears it
- index.tsx: ↓ Sync button + pull-to-refresh both trigger syncFeed; cloud
badge on remote activities; empty state updated
- Add GET /api/wheel/download to serve/server.py and edit/server.py:
serves dist/bincio-*.whl via FileResponse; in production nginx takes
the request before FastAPI, so this is a no-op there but works locally
- wheel_version response now includes api_url: "/api/wheel/download"
alongside the nginx-served url field
- Bundle mobile/assets/bincio.whl (built from dist/) as an offline
fallback for Pyodide testing before the first instance sync
- docs/mobile-app.md: document dev setup — bundled asset, local server
endpoint, and how to refresh the bundle with uv build + cp
Same fix as cea1dbc (post-receive hook) but missed in server.py: Astro resolves
the public/data symlink and copies all activity JSON into dist/; without pruning,
every Strava sync / upload / edit that triggers a full build + rsync duplicates
GBs of data into the nginx webroot.
Both rsync callsites now rm -rf dist/data + pass --exclude=data/ to rsync.
- dem.py: apply 45s median filter before hysteresis to suppress SRTM
tile-boundary steps that were accumulating through the 5m threshold;
raise DEM hysteresis threshold from 5m to 10m
- dem.py: back up elevation_m as elevation_m_original in timeseries
before the first DEM overwrite, so original sensor data is preserved
- dem.py: add recalculate_elevation_hysteresis() — recomputes gain/loss
from original recorded elevation (reads elevation_m_original if a DEM
run already replaced elevation_m) using source-aware thresholds
(5m barometric, 10m GPS/unknown); does not touch the elevation array
- edit/server.py, serve/server.py: split /recalculate-elevation into
two endpoints: /recalculate-elevation/dem and
/recalculate-elevation/hysteresis
- EditDrawer.svelte: replace single DEM button with two side-by-side
buttons — "Recalculate (hysteresis)" (fast, offline) and
"Recalculate (DEM)" (SRTM lookup)
Adds a "Recalculate from terrain map (DEM)" button to the activity edit
drawer. On click it queries an Open-Elevation-compatible API to replace
GPS altitude with SRTM terrain data, applies 5m hysteresis, and updates
the activity's elevation stats and timeseries chart in place.
- bincio/extract/dem.py: lookup_elevations() (batched HTTP POST) +
recalculate_elevation() (subsample → DEM → interpolate → hysteresis →
patch activity JSON, timeseries JSON, index.json)
- POST /api/activity/{id}/recalculate-elevation on both serve and edit
servers; serve endpoint is auth-gated and triggers merge + rebuild
- --dem-url flag (also DEM_URL env var) on bincio serve and bincio edit;
logged at startup; missing URL returns a clear 503 with setup instructions
- /api/me response gains dem_configured bool
- EditDrawer: button with loading state, shows new ↑/↓ values on success
astro build resolves the public/data symlink and copies all activity JSON
into dist/; rsync then copied that to the webroot — but nginx already serves
/data/ directly from /var/bincio/data/ via alias, so both copies were dead
weight. Freed 36 GB → 14 GB on the live server.
- post-receive hook: prune dist/data/ before rsync, add --exclude=data/
- docs: update manual rebuild command and nginx comment to match
- serve/server.py: _mb() now uses lstat() to count symlinks at face value
rather than following them to targets, so admin storage panel no longer
double-counts _merged/ (which is mostly symlinks into activities/)
- serve/server.py GET adds private:bool to the response (true when
privacy is "unlisted" or legacy "private") so EditDrawer can read it
- edit/server.py GET: same fix for the single-user edit server
- EditDrawer: fall back to d.privacy if d.private is absent; rename
"Private" toggle label to "Unlisted"
delete_activity now updates data_dir/index.json so merge_all no longer
re-adds the summary for a deleted activity, preventing the broken
"Activity not found" state after deletion.
ActivityDetailLoader switches from loadIndex (all year shards) to
loadIndexPaged (first year shard only) + direct file fallback, so
opening an activity detail page no longer downloads the entire history.
- user_prefs table in db.py with get/set helpers
- GET/PUT /api/me/prefs endpoints for bulk pref management
- GET/PUT/DELETE /api/me/strava-credentials; PUT preserves existing
secret when client_secret field is left blank
- _strava_creds() helper resolves per-user → instance fallback across
all five Strava endpoints
- Settings page: Navigation card (hide Feed/Community/About toggles)
and Strava credentials card
- Base.astro: ids on feed/community/about nav links; applies
nav_hide_* prefs after login
API endpoints (all auth-gated to the logged-in user):
- GET /api/me/storage — per-category disk breakdown
- DELETE /api/me/originals — free originals/ dir (post-extraction cleanup)
- DELETE /api/me/activities — wipe all activity data (password confirm)
- DELETE /api/me — delete account + all data (password confirm)
- PUT /api/me/display-name — update display name
- PUT /api/me/password — change password (requires current password)
Page at /settings/:
- Storage card: activities / originals / Strava originals / photos / total
with one-click 'Delete original files' when originals exist
- Profile card: display name field with inline save
- Password card: change password form
- Danger zone: delete all activities or delete account (both require
password confirmation in a modal before proceeding)
Nav: 'Settings' link appears in the top bar after login (same as Admin).
When 'Overwrite existing activities' is checked, duplicate activities are
re-extracted and replaced instead of silently skipped:
- Deletes {id}.json, .geojson, .timeseries.json from activities/ and _merged/
- Removes the stale index summary and dedup cache entry
- Ingests the new file fresh via ingest_parsed
- Reports 'overwritten' (↺) status in the SSE stream vs 'imported' (↓)
- done event includes 'overwritten' count; UI shows it alongside 'added'
- /api/admin/disk now includes in_db flag per user (true if account exists in DB)
- Ghost users (directory exists, no DB account) show amber 'ghost' badge and only
Diag + Delete dir buttons (no Re-extract, Rebuild, Reset pwd, Reset data)
- DELETE /api/admin/users/{handle}/directory wipes the entire directory and updates
the root manifest; refuses if the account still exists in the DB
- Wires up rmdir-btn with a window.confirm before calling the new endpoint
One Python process for 2015 activities exhausts all RAM + swap on a
cheap VPS. Split into sequential batches of 100: each subprocess handles
100 activities and exits, returning all memory to the OS before the
next batch starts. The server chains batches in the SSE event_stream
and triggers a single rebuild when all batches complete.
uv is unreliable in systemd environments where PATH omits ~/.local/bin.
Use sys.executable's parent directory to find the venv's bincio script
directly — this always works since the server itself runs from the venv.
The in-process approach loaded all 2015 Strava originals into the server
process memory, causing OOM kills. Now spawns `bincio reextract-originals`
as a child process; heavy work runs in an isolated Python interpreter that
exits when done, freeing all memory.
Also adds `bincio reextract-originals` as a standalone CLI command that
prints JSON-lines progress to stdout — useful for running directly on the
VPS via SSH for large backlogs.
The per-call run_in_executor pattern caused network errors.
New approach: one thread runs the entire extraction loop and puts
SSE strings into an asyncio.Queue via call_soon_threadsafe; the
async generator drains the queue. This is the correct pattern for
background-thread + SSE streaming in FastAPI.
The sync generator was failing with a network error because Starlette's
iterate_in_threadpool doesn't properly propagate exceptions from sync
generators — the connection resets with no body.
Fix: convert event_stream to an async generator (Starlette handles these
natively without thread wrapping), move imports to the endpoint function
scope so failures raise HTTPException before the stream starts, and run
CPU-intensive work (parse + write) via loop.run_in_executor so the
async generator can actually yield between activities.
- Generator now yields a 'status' event immediately so the client can
distinguish 'working' from 'failed silently before first event'
- Batch mode: call write_activity per file but write index.json and
athlete.json only once at the end (was O(n²) — 2015 rewrites)
- JS: check r.ok before reading the body stream; show HTTP error detail
instead of staying stuck at 'Starting…'
- Handle 'status' event type in the progress log
- POST /api/admin/users/{handle}/reextract-originals: reads stored
originals/strava/*.json and re-runs strava_to_parsed + ingest_parsed
without hitting the Strava API; streams SSE progress; calls merge_all
and rebuild on completion
- GET /api/admin/users/{handle}/diag: now shows _merged/activities/
file counts, a sample of filenames in activities/ (with symlink flag),
and lists pending_files by name
- Admin page: Re-extract button per user with live SSE progress modal
- bincio.serve logger wired into uvicorn output: rebuild steps, upload
errors, strava-zip progress all now appear in the server log
- _trigger_rebuild: capture stdout/stderr, log errors instead of silently
discarding; exceptions logged with traceback instead of swallowed
- upload handler: log per-file errors with traceback; include error detail
in the SSE event sent back to the browser
- strava-zip handler: log imported/error counts on completion
- GET /api/admin/users/{handle}/diag: snapshot of a user's data dir
(file counts, sizes, index activity counts, pending uploads)
- POST /api/admin/users/{handle}/rebuild-sync: blocking rebuild that
returns full stdout/stderr — for debugging without SSH log access
- Admin page: Diag button per user opens a modal showing the diag JSON
db.py: reset_codes table (code, handle, created_by, created_at,
expires_at, used_at); create_reset_code() invalidates any prior unused
code for the same handle; use_reset_code() validates handle match,
expiry (24 h), and single-use; change_password() updates the hash.
server.py: POST /api/admin/users/{handle}/reset-password-code (admin)
returns a code; POST /api/auth/reset-password (public) validates the
code + handle and sets the new password.
Admin page: "Reset pwd" button per user — shows the code inline on
click (monospace, click-to-copy).
/reset-password/ page: handle + code + new password form.
Login page: "Forgot password?" link.
The old DELETE /api/admin/users/{handle}/activities only removed *.json
files and _merged/, leaving originals/ (Strava FIT files) and edits/
untouched — causing the 968 MB disk usage after a delete.
_wipe_user_activities() now removes activities/, edits/, originals/,
_merged/, index.json, athlete.json, and .bincio_cache.json. Admin page
button renamed to "Reset data" with updated confirmation text.
Server endpoint removes the activity JSON, GeoJSON, timeseries, sidecar
edit, and images directory. Also purges the dedup cache entry so the
file can be re-uploaded if needed. Runs merge_all + rebuild afterwards.
EditDrawer: two-click delete button (click once → "Confirm delete?",
click again → deletes). On success, dispatches 'deleted' event.
ActivityDetail navigates back to the feed on delete.
Single-activity writes now trigger a fast merge_one instead of a full
user rebuild. post_activity was fixed earlier; this completes the fix
for upload_image and delete_image endpoints.
- "unlisted" = not shown in the public feed, but GPS track, timeseries
and detail JSON are all accessible by direct URL (security by obscurity)
- "private" accepted as legacy alias everywhere (backward compat with
existing data on disk)
- New writes from Strava sync / ZIP upload / sidecar use "unlisted"
- Only "no_gps" now suppresses the GPS track
- isUnlisted() helper in format.ts used by all Svelte/Astro components
- SCHEMA.md and CLAUDE.md document the privacy model and the distinction
between "unlisted" and "no_gps"
added 3256 more.
- danilo: _merged/ is 8 KB — basically empty. merge_all likely ran concurrently (multiple file uploads trigger multiple rebuilds without a lock in --no-build mode),
causing a race where shutil.rmtree(merged_acts) from one run wiped what another run was writing.
Two fixes: serialize --no-build rebuilds with the same lock, and add a "Rebuild" button to the admin page.
Root causes fixed:
1. merge_all race condition — --no-build rebuilds now hold _rebuild_lock, same as full builds
2. The SSE rebuild-trigger bug (already fixed earlier) was brut's original cause
next server restart.
Admin page now shows:
- Overall disk bar (used/free/%)
- Per-user table: total, activities (with file count), originals (with Strava breakdown), merged, images
- A mini bar per user showing relative size
- Red ⚠ warning if orphaned temp ZIPs are still present for a user
- Delete activities button (reloads sizes after)
index.json, then triggers a rebuild. Admin-only.
- /admin/ page — lists all users, each with a "Delete activities" button. Clicking asks for
confirmation in a <dialog> before firing the request. Button shows "Deleted (N)" or an error inline.
- "Admin" nav link — appears in the top-right for admins only, hidden for everyone else.
Davide Scaini