The per-call run_in_executor pattern caused network errors.
New approach: one thread runs the entire extraction loop and puts
SSE strings into an asyncio.Queue via call_soon_threadsafe; the
async generator drains the queue. This is the correct pattern for
background-thread + SSE streaming in FastAPI.
The sync generator was failing with a network error because Starlette's
iterate_in_threadpool doesn't properly propagate exceptions from sync
generators — the connection resets with no body.
Fix: convert event_stream to an async generator (Starlette handles these
natively without thread wrapping), move imports to the endpoint function
scope so failures raise HTTPException before the stream starts, and run
CPU-intensive work (parse + write) via loop.run_in_executor so the
async generator can actually yield between activities.
- Generator now yields a 'status' event immediately so the client can
distinguish 'working' from 'failed silently before first event'
- Batch mode: call write_activity per file but write index.json and
athlete.json only once at the end (was O(n²) — 2015 rewrites)
- JS: check r.ok before reading the body stream; show HTTP error detail
instead of staying stuck at 'Starting…'
- Handle 'status' event type in the progress log
- POST /api/admin/users/{handle}/reextract-originals: reads stored
originals/strava/*.json and re-runs strava_to_parsed + ingest_parsed
without hitting the Strava API; streams SSE progress; calls merge_all
and rebuild on completion
- GET /api/admin/users/{handle}/diag: now shows _merged/activities/
file counts, a sample of filenames in activities/ (with symlink flag),
and lists pending_files by name
- Admin page: Re-extract button per user with live SSE progress modal
- bincio.serve logger wired into uvicorn output: rebuild steps, upload
errors, strava-zip progress all now appear in the server log
- _trigger_rebuild: capture stdout/stderr, log errors instead of silently
discarding; exceptions logged with traceback instead of swallowed
- upload handler: log per-file errors with traceback; include error detail
in the SSE event sent back to the browser
- strava-zip handler: log imported/error counts on completion
- GET /api/admin/users/{handle}/diag: snapshot of a user's data dir
(file counts, sizes, index activity counts, pending uploads)
- POST /api/admin/users/{handle}/rebuild-sync: blocking rebuild that
returns full stdout/stderr — for debugging without SSH log access
- Admin page: Diag button per user opens a modal showing the diag JSON
db.py: reset_codes table (code, handle, created_by, created_at,
expires_at, used_at); create_reset_code() invalidates any prior unused
code for the same handle; use_reset_code() validates handle match,
expiry (24 h), and single-use; change_password() updates the hash.
server.py: POST /api/admin/users/{handle}/reset-password-code (admin)
returns a code; POST /api/auth/reset-password (public) validates the
code + handle and sets the new password.
Admin page: "Reset pwd" button per user — shows the code inline on
click (monospace, click-to-copy).
/reset-password/ page: handle + code + new password form.
Login page: "Forgot password?" link.
The old DELETE /api/admin/users/{handle}/activities only removed *.json
files and _merged/, leaving originals/ (Strava FIT files) and edits/
untouched — causing the 968 MB disk usage after a delete.
_wipe_user_activities() now removes activities/, edits/, originals/,
_merged/, index.json, athlete.json, and .bincio_cache.json. Admin page
button renamed to "Reset data" with updated confirmation text.
Server endpoint removes the activity JSON, GeoJSON, timeseries, sidecar
edit, and images directory. Also purges the dedup cache entry so the
file can be re-uploaded if needed. Runs merge_all + rebuild afterwards.
EditDrawer: two-click delete button (click once → "Confirm delete?",
click again → deletes). On success, dispatches 'deleted' event.
ActivityDetail navigates back to the feed on delete.
FIT parser: try enhanced_altitude before altitude. Barometric altimeters
on modern Garmins (Edge 540, 840, etc.) write enhanced_altitude in
record messages and total_ascent in lap messages. The old code read only
altitude, producing null elevation_m per point → null elevation_gain_m
at the activity root while laps had correct values from total_ascent.
ActivityMap: use preview_coords (passed from ActivitySummary) to
initialise the map at the activity's location on mount, eliminating the
flash of world-view before the async detail JSON / bbox arrives.
Single-activity writes now trigger a fast merge_one instead of a full
user rebuild. post_activity was fixed earlier; this completes the fix
for upload_image and delete_image endpoints.
- "unlisted" = not shown in the public feed, but GPS track, timeseries
and detail JSON are all accessible by direct URL (security by obscurity)
- "private" accepted as legacy alias everywhere (backward compat with
existing data on disk)
- New writes from Strava sync / ZIP upload / sidecar use "unlisted"
- Only "no_gps" now suppresses the GPS track
- isUnlisted() helper in format.ts used by all Svelte/Astro components
- SCHEMA.md and CLAUDE.md document the privacy model and the distinction
between "unlisted" and "no_gps"
added 3256 more.
- danilo: _merged/ is 8 KB — basically empty. merge_all likely ran concurrently (multiple file uploads trigger multiple rebuilds without a lock in --no-build mode),
causing a race where shutil.rmtree(merged_acts) from one run wiped what another run was writing.
Two fixes: serialize --no-build rebuilds with the same lock, and add a "Rebuild" button to the admin page.
Root causes fixed:
1. merge_all race condition — --no-build rebuilds now hold _rebuild_lock, same as full builds
2. The SSE rebuild-trigger bug (already fixed earlier) was brut's original cause
next server restart.
Admin page now shows:
- Overall disk bar (used/free/%)
- Per-user table: total, activities (with file count), originals (with Strava breakdown), merged, images
- A mini bar per user showing relative size
- Red ⚠ warning if orphaned temp ZIPs are still present for a user
- Delete activities button (reloads sizes after)
index.json, then triggers a rebuild. Admin-only.
- /admin/ page — lists all users, each with a "Delete activities" button. Clicking asks for
confirmation in a <dialog> before firing the request. Button shows "Deleted (N)" or an error inline.
- "Admin" nav link — appears in the top-right for admins only, hidden for everyone else.
Key at data_dir.parent/.garmin_key — nginx serves location /data/ { alias /var/bincio/data/; } so
anything inside that dir is reachable. The key lives one level up at /var/bincio/.garmin_key,
outside nginx's reach.
Two-layer storage — garmin_creds.json holds the encrypted email+password (needed for re-login when
tokens expire); garmin_session/ holds the garth OAuth tokens in plain JSON (short-lived, not the
user's actual password).
test_login() — called by the connect endpoint before saving anything, so credentials are only
persisted if they actually work.
get_client() — tries the session first (fast, no network), falls back to full re-login
transparently. The caller never needs to think about whether the session is fresh.
- POST /api/upload now returns text/event-stream instead of JSON
- Per-file progress events stream back as each file is processed: ↓ 3/47 (6%) — morning_ride.fit
- Final done event shows the summary: "12 added, 35 duplicates"
- The Vite proxy is configured to stream this properly (no buffering)
For the admin:
- New GET /api/admin/jobs endpoint (admin-only) returns the list of active upload jobs, each with
user, started_at, total, done, current (filename being processed)
- A pulsing amber badge appears in the nav bar for admins when any user has an active upload running
— it shows e.g. "2 uploads running" with a tooltip listing each user's progress (@alice: 12/50
files)
- Polls every 5 seconds, disappears automatically when all jobs finish
stripping them; privacy filtering is now done client-side
- ActivityFeed: detect logged-in user via bincio:me event; show private
activities only when viewing your own profile; private cards get a lock
badge
last_sync_at timestamp was never written. After deploying, do a soft reset — it'll set last_sync_at to your most recent activity's timestamp so the next sync only fetches newer ones.
- Reset 404: Added POST /api/strava/reset to serve/server.py. The soft reset now looks in _merged/index.json first (multi-user path), falling back to index.json.
1. The build took minutes → 404 during that window
2. Even after the build, the output lands in site/dist/ — nginx serves from /var/www/bincio/ which is only updated by the rsync in the post-receive hook, not by the server process
Fixes applied:
1. bincio/render/cli.py: Added --no-build flag — merges sidecars and updates manifests but skips astro build. This is fast (~1 second).
2. bincio/serve/server.py _trigger_rebuild: Now passes --no-build. After an upload, _merged/ and root index.json are updated immediately, so the feed reflects the new activity. The static Astro pages are
only rebuilt on git push.
3. site/src/components/ActivityDetailLoader.svelte (new): Svelte component that reads the activity ID from the URL, calls loadIndex to resolve the shard tree, then renders ActivityDetail dynamically — no
pre-built page needed.
4. site/src/pages/activity/index.astro (new): Generic Astro shell page that renders ActivityDetailLoader. Gets compiled to dist/activity/index.html.
5. docs/deployment/vps.md: Added location /activity/ { try_files $uri $uri/ /activity/index.html; } to the nginx config. When a request arrives for /activity/2026-04-06T153345Z/ and no pre-built file
exists, nginx serves the shell, which loads the data dynamically from /data/ (which nginx already serves live from disk).
2. Image MIME type whitelist — _ALLOWED_IMAGE_TYPES blocks SVG XSS in both servers
3. Filename collision safety — _unique_image_name() helper in both servers
4. OAuth CSRF — state token generated in edit/server.py auth-url, stored in _oauth_states, validated and discarded in callback; strava_api.auth_url() accepts optional state param
5. Error message leak — upload processing errors now return generic "Processing failed" instead of exception type/message
6. Handle injection in subprocess — _trigger_rebuild now asserts handle matches _VALID_HANDLE before passing to subprocess
After registration creates the user's directories, it now calls _write_root_manifest(dd). This rewrites index.json to include the new handle's shard immediately. Since Astro dev re-evaluates getStaticPaths() on every request (reading that file), /u/pres/, /u/pres/stats/, and /u/pres/athlete/ will resolve correctly as soon as the new user navigates there.
Fix 2 — invites link (athlete/index.astro:33):
Added an Invites button (top-right, same style as "Edit profile") that starts hidden. When bincio:me fires and me === handle (you're on your own page), the subnav tabs are removed as before AND the invites button is revealed. Other visitors see neither.
- writer.py: timeseries is now written to {id}.timeseries.json as a separate file. The detail JSON gets a timeseries_url field instead. finalize_pending and cleanup_pending handle the extra file.
- merge.py (merge_one): symlinks the .timeseries.json file alongside the detail JSON. merge_all already handles it transparently (the .timeseries.json stem doesn't match any activity
ID in to_merge, so it falls through to the symlink branch).
- types.ts: timeseries is now timeseries?: Timeseries | null, and timeseries_url?: string | null added.
- dataloader.ts: new loadTimeseries(url, detailUrl, base) function that resolves paths correctly in both single- and multi-user modes (uses the fetched detail URL's directory as the base).
- ActivityDetail.svelte: loads timeseries separately after detail loads; uses detail.timeseries for IDB activities (embedded) or fetches via detail.timeseries_url for server activities. Charts show a pulse placeholder while loading.
2 — GZip
- GZipMiddleware (min 1 KB) added to both bincio/serve/server.py and bincio/edit/server.py — all API JSON responses are now gzip-compressed.
- For static files (the big timeseries JSONs), nginx should be configured with gzip on; gzip_types application/json application/geo+json; — no code change needed on the server side.
Net effect: opening an activity page now fetches ~1.4 KB (detail) instead of ~586 KB. The timeseries fetches ~60–150 KB gzip-compressed shortly after (it loads concurrently with the map rendering).
Edit/Upload UI — split into two concepts:
- PUBLIC_EDIT_URL — the server base URL (empty = use proxy at /api/)
- PUBLIC_EDIT_ENABLED=true — whether to show the edit/upload buttons at all
bincio dev now sets PUBLIC_EDIT_ENABLED=true when instance.db exists (multi-user mode), so the upload button, edit button, and edit drawer all appear. The fetch calls already produce correct relative URLs (${''}/api/upload = /api/upload) which the Vite proxy forwards to bincio serve.
- Add bincio/extract/ingest.py as a facade over the extract internals (ingest_parsed, strava_sync), reducing coupling from 6+ imports to one
- Add merge_one() to merge.py — fast single-activity path for interactive edits (rewrites one file + index, skips full directory rebuild)
- Rewrite edit/ops.py to delegate to the new facade; fix broken run_strava_sync return (was referencing undefined locals)
- Remove duplicated SPORTS, STAT_PANELS, VALID_ACTIVITY_ID from edit/server.py — now imported from ops.py
Davide Scaini