Both trigger_detect and me_segment_rescan were appending-only, so false
efforts recorded before the geometric speed check fix remained after
rescan. Now each rescan path clears the effort file first, making the
result authoritative.
Long circuit rides were matching a segment START early and finding the
segment END hours later on a second pass, producing effort times of
~17000s on a 4.7km segment. The conformance check passed because the
full-circuit track covers all interior points within 50m over 5 hours.
Add a per-sport minimum geometric speed (segment_distance / elapsed_s):
cycling ≥ 1.0 m/s, running ≥ 0.5 m/s, default ≥ 0.2 m/s. When the
check fails, advance past the current start candidate and retry, so a
legitimate later match (e.g. a second lap done at real speed) is still
detected.
- detect.py: truncate started_at to seconds so dedup key survives JSON round-trip
- store.py: dedup by (activity_id, iso-started_at) string key, not object equality
- server.py: extract _scan_segment_for_user helper; trigger background scan
for the creating user's activities when a new segment is saved
- best_activity_id now included in segment_summary API response
- Best time is a direct link to the activity that produced it
- Clicking a row expands an inline effort list (lazy-loaded from
/api/segments/{id}/efforts): date linked to activity, time, Δ vs PR
- Clicking again collapses; ▲/▼ chevron shows state
New API endpoints:
- GET /api/segments/{id} — single segment metadata
- GET /api/activities/{id}/segment_efforts — efforts for an activity (auth)
- GET /api/users/{handle}/segment_summary — public best time + count per segment
New components:
- SegmentDetail.svelte — map + metadata + effort table (with PR/Δ) + rescan button
- SegmentsPage.svelte — URL router: shows detail when /segments/{id}/, list otherwise
Updated:
- segments/index.astro — now uses SegmentsPage router
- nginx-activity.conf — add /segments/ try_files rule for client-side routing
- ActivityDetail.svelte — segment efforts block below laps
- AthleteView.svelte — Segments tab with best time + effort count per segment
- format.ts — add formatElapsed() for compact m:ss display
When a user saves new Strava credentials with a different client_id,
auto-delete the existing token (it belongs to a different OAuth app
and will always fail on refresh). Add POST /api/strava/disconnect
endpoint and a "Disconnect from Strava" button in settings, visible
only when connected.
Immediate: deleted diego_p's stale token so he can reconnect.
Apple Watch and similar devices record exactly 0.0 for elevation while
waiting for barometric/GPS lock, then jump to the real altitude. The
hysteresis accumulator was seeding from 0.0, counting the full jump as
ascent. Fix: detect a leading near-zero run followed by a large jump
and seed the accumulator from the first real value instead.
Applied in both _elevation() (fresh extractions) and
recalculate_elevation_hysteresis() (recompute path). Added a bulk
admin endpoint POST /api/admin/users/{handle}/recompute-elevation and
corresponding button to fix existing stored activities.
BINCIO_DATA_DIR is already set in the build env, so manifest.ts reads
the data root directly at build time without needing public/data.
Moving _link_data() into the serve-only branch prevents Astro from
following the symlink and copying the full data dir into dist/. Any
leftover symlink from a previous dev session is removed before build.
Dev mode is unchanged.
Each sync run now writes _strava_sync_status.json per user (status,
imported count, error message). New admin endpoints expose this data
and allow triggering an on-demand sync. The admin page gains a Strava
Sync section showing per-user token/credentials state, total imported,
last sync time, and last-run status with inline error messages.
- Add bincio sync-strava command: headless multi-user Strava sync
designed for systemd timer. Discovers users via strava_token.json,
skips users without their own strava_credentials.json, respects
Strava visibility (only_me → unlisted). Treats 404 stream errors as
no-GPS activities rather than retrying every run.
- Add deploy/systemd/bincio-sync.{service,timer}: runs every 3 hours,
Persistent=true to catch up after downtime.
- Add POST /api/internal/rebuild: webhook for sync timer to trigger
site rebuild, authenticated via X-Sync-Secret header.
- Add suspended column to users table with auto-migration on open_db.
Suspended users are blocked at login and session lookup (covers both
activity site and wiki, which share instance.db).
- Add POST /api/admin/users/{handle}/suspend|unsuspend and
DELETE /api/admin/users/{handle}/account endpoints.
- Admin panel: Suspend/Unsuspend toggle, Del account button, suspended
badge on user row.
Replace per-upload Astro build threads with a single background worker
(_site_rebuild_worker) that waits on an event, sleeps 60 s to let upload
bursts settle, then runs one full build + rsync. 271 concurrent uploads now
produce one build instead of 271 serialised builds, eliminating the OOM kill.
--webroot is re-enabled; merge-only path still runs immediately per upload.
Also: date filter row added to ActivityFeed.svelte (sport + date presets
with dynamic year pills); deploy/vps gitignored for VPS config backups.
Adds edits_json column (migration v3) to store user overrides separately
from detail_json so Option A server re-extraction never clobbers them.
- Tap the title in the detail screen to edit (local activities only, shown
with a ✎ hint). Saves on keyboard dismiss via onEndEditing.
- Cards and search display user_title ?? title.
- Raw upload: user_title sent to server -> sidecar written so web UI shows
the correct title (server re-extracts from FIT, which has Karoo's title).
- BAS upload: detail.title overridden before sending, no sidecar needed.
Four related issues made uploading 271+ activities unreliable:
1. merge_all/write_combined_feed were inside the extraction try/except —
any merge race returned 422 even though the file was on disk, causing
the mobile app to permanently mark the upload as failed. Fixed by
moving them to a separate best-effort try/except after the extraction
block. Switch to merge_one (single-activity symlink) instead of
merge_all (full rebuild) so each upload is O(1) FS ops, not O(N).
2. The dev watcher fired merge_all for every activity .json write AND the
upload endpoint also ran merge_all — O(N²) symlink operations during
bulk uploads. Watcher now skips activities/*.json changes (upload
endpoint handles those directly).
3. Vite/Chokidar followed the public/data symlink and opened a handle per
activity file; constant merge rebuilds exhausted file descriptors and
crashed the Astro dev server. Fixed with watch.ignored on public/data.
4. _write_year_shards and write_combined_feed used f.unlink() without
missing_ok=True — concurrent callers racing the same file threw
FileNotFoundError which propagated as a false extraction failure.
When the dev file-watcher and an upload endpoint both trigger merge_all on the
same user directory at the same time, the shutil.rmtree/_merged/activities/
sequence collides: one thread deletes the directory while the other is mid-walk,
causing ENOTEMPTY or ENOENT crashes.
Added a per-user-dir threading.Lock (_merge_lock) at the top of merge.py so
all callers (upload endpoints, watcher, admin rebuild) are serialized without
changing any call site. Also made the rmtree ignore_errors=True and the
symlink_to conditional to survive any stale leftover from a prior crash.
Three bugs fixed:
- /api/upload/bas and /api/upload/raw never updated user_dir/index.json, so
merge_all couldn't include uploaded activities in year shards — they existed
on disk but were invisible to the browser feed. Fixed by _upsert_index_summary()
called before merge_all().
- Silent catch {} in uploadLocalActivities swallowed all per-activity errors;
replaced with console.warn so failures are visible in Expo logs.
- After a server wipe, synced_at flags on the device caused "Nothing to upload"
forever. uploadFeed() now reconciles against GET /api/feed at the start of each
upload: local activities not found on the server get synced_at cleared.
Also: live upload progress ("Uploading N / M…"), failed count in result message,
onProgress callback on uploadFeed(), countPendingUploads() helper.
merge_all(user_dir) updates the per-user _merged/ shard but the home page
loads feed.json first via loadCombinedFeed. write_combined_feed was only
called by the CLI render command, not by the API upload endpoints or the
dev watcher, leaving feed.json permanently stale after any runtime upload.
Add write_combined_feed(_get_data_dir()) after every merge_all call in
/api/upload/bas, /api/upload/raw, the dev.py file watcher, and dev startup.
- Skip MapLibre on Android <29 (Karoo): SELinux denies kgsl-3d0 access
from untrusted_app context, crashing the GPU driver on any OpenGL
surface. Replace with SvgRouteView — equirectangular SVG route trace
using react-native-svg, no native GL surface needed.
- Add +/- zoom buttons to full-screen MapLibre map on modern devices
via Camera ref and onRegionDidChange.
- Skip PyodideWebView on Android <29: same GPU driver conflict; set
_engineUnavailable at module init via API level gate (< 29).
- Add engine_unavailable fast path in PyodideWebView: post message
immediately if WebAssembly.Global is absent (Chrome <69) instead of
attempting 30 MB Pyodide download.
- Add server-side extraction fallback (extractServer.ts): when engine
unavailable, POST raw file as base64 to /api/upload/raw; server runs
full Python pipeline and returns extracted data.
- Add /api/upload/raw endpoint in server.py.
- Add pre-flight auth check (checkServerAuth) before batch import so
an expired token errors immediately rather than after N files.
- Fix uploadLocalActivities in sync.ts: was reading original_path as
JSON (binary FIT file, always threw), silently skipping every upload.
Now reads detail_json from DB directly.
- Redesign Feed header: replace single Sync button with Upload /
Download / Refresh. Pull-to-refresh and Refresh button are local-only.
Auto-refresh on tab focus via useFocusEffect.
- Replace ActivityIndicator with plain Text everywhere (native animation
also crashes Karoo GPU driver).
- Raise macOS open-file limit in dev_test.py to prevent EMFILE errors
from Astro file watcher.
- Document all Karoo hardware constraints in docs/mobile-app.md.
The dedup cache (.bincio_cache.json) persists source hashes across runs.
If the activities/ directory is wiped (--fresh in another session, or macOS
clearing /tmp after a restart) while the cache file survives at the user dir
level, re-running extract skips all files as "already extracted" and leaves
activities/ empty. Now only hashes whose corresponding .json file is present
on disk are treated as known, so missing files are always re-extracted.
- Server: POST /api/upload/bas accepts pre-extracted BAS JSON (activity + optional timeseries/geojson), writes files and triggers merge_all
- sync.ts: uploadLocalActivities reads unsynced local activities by original_path, POSTs to /api/upload/bas, marks synced_at on success
- Settings: Upload toggle (Off / Upload local activities) in Sync section with subLabel dividers for Download / Upload groups
- Feed: sync message includes uploaded count when activities are pushed
- Add /api/activity/{id}/geojson and /api/activity/{id}/timeseries endpoints
(bearer-token-gated, falls back from _merged to raw activities dir)
- Rewrite activity detail screen with MapLibreGL v11 API (Map, Camera,
GeoJSONSource, Layer) and react-native-svg area chart with gradient fill
- On-demand fetch for remote activities that have no local geojson/timeseries
- Add react-native-svg dependency; requires dev build (npx expo run:android)
_merged/index.json is a shard manifest with activities:[] when the user
has >FEED_PAGE_SIZE activities. The endpoint now collects from all
index-{year}.json shard files before returning.
SyncResult gains a `total` field (activities received from server) so the
feed screen can distinguish "server returned nothing" from "all already
stored locally". Messages: "No activities on instance" / "Up to date (N)"
/ "X of N activities synced".
Server (bincio/serve/server.py):
- Add _require_auth: accepts session cookie OR Authorization: Bearer token
- POST /api/auth/token: same as /api/auth/login but returns token in body
(password used once, not stored; mobile stores only the session token)
- GET /api/feed: auth-gated; reads _merged/index.json for the user and
returns the activities array as JSON
Mobile:
- db/sync.ts: syncFeed(db) fetches /api/feed, upserts each summary into
local SQLite as origin='remote'; skips locally-imported activities
- db/queries.ts: add upsertRemoteActivity (INSERT ... ON CONFLICT DO UPDATE
WHERE origin='remote' — never overwrites local imports); fix feed sort
order to started_at DESC instead of insertion order
- settings.tsx: Connect section — password field (not persisted) + Connect
button calls POST /api/auth/token and stores token; Disconnect clears it
- index.tsx: ↓ Sync button + pull-to-refresh both trigger syncFeed; cloud
badge on remote activities; empty state updated
- Add GET /api/wheel/download to serve/server.py and edit/server.py:
serves dist/bincio-*.whl via FileResponse; in production nginx takes
the request before FastAPI, so this is a no-op there but works locally
- wheel_version response now includes api_url: "/api/wheel/download"
alongside the nginx-served url field
- Bundle mobile/assets/bincio.whl (built from dist/) as an offline
fallback for Pyodide testing before the first instance sync
- docs/mobile-app.md: document dev setup — bundled asset, local server
endpoint, and how to refresh the bundle with uv build + cp
Same fix as cea1dbc (post-receive hook) but missed in server.py: Astro resolves
the public/data symlink and copies all activity JSON into dist/; without pruning,
every Strava sync / upload / edit that triggers a full build + rsync duplicates
GBs of data into the nginx webroot.
Both rsync callsites now rm -rf dist/data + pass --exclude=data/ to rsync.
- dem.py: pre-smooth elevation with 30s moving average before hysteresis
in recalculate_elevation_hysteresis(); thresholds drop from 5m/10m to
1m (barometric) / 3m (GPS) — accurate after noise is smoothed out
- dem.py: widen DEM median-filter window 45s → 60s
- dem.py: rename response key source → altitude_source for consistency
- writer.py: write altitude_source into detail JSON at extract time
- tests/test_dem.py: 21 unit tests for pure functions and file-level hysteresis
- tests/test_edit_server.py: 11 TestClient API tests for both recalculate endpoints
- add httpx as dev dependency (required by FastAPI TestClient)
- dem.py: apply 45s median filter before hysteresis to suppress SRTM
tile-boundary steps that were accumulating through the 5m threshold;
raise DEM hysteresis threshold from 5m to 10m
- dem.py: back up elevation_m as elevation_m_original in timeseries
before the first DEM overwrite, so original sensor data is preserved
- dem.py: add recalculate_elevation_hysteresis() — recomputes gain/loss
from original recorded elevation (reads elevation_m_original if a DEM
run already replaced elevation_m) using source-aware thresholds
(5m barometric, 10m GPS/unknown); does not touch the elevation array
- edit/server.py, serve/server.py: split /recalculate-elevation into
two endpoints: /recalculate-elevation/dem and
/recalculate-elevation/hysteresis
- EditDrawer.svelte: replace single DEM button with two side-by-side
buttons — "Recalculate (hysteresis)" (fast, offline) and
"Recalculate (DEM)" (SRTM lookup)
Adds a "Recalculate from terrain map (DEM)" button to the activity edit
drawer. On click it queries an Open-Elevation-compatible API to replace
GPS altitude with SRTM terrain data, applies 5m hysteresis, and updates
the activity's elevation stats and timeseries chart in place.
- bincio/extract/dem.py: lookup_elevations() (batched HTTP POST) +
recalculate_elevation() (subsample → DEM → interpolate → hysteresis →
patch activity JSON, timeseries JSON, index.json)
- POST /api/activity/{id}/recalculate-elevation on both serve and edit
servers; serve endpoint is auth-gated and triggers merge + rebuild
- --dem-url flag (also DEM_URL env var) on bincio serve and bincio edit;
logged at startup; missing URL returns a clear 503 with setup instructions
- /api/me response gains dem_configured bool
- EditDrawer: button with loading state, shows new ↑/↓ values on success
GPS jitter and barometric quantization noise caused systematic overestimation
of elevation gain — in extreme cases 100% of reported gain was sub-1m noise.
Implements source-aware hysteresis: elevation is only committed when it
deviates from the last committed value by ≥5m (barometric) or ≥10m (GPS/GPX/TCX).
- ParsedActivity gains `altitude_source` field ("barometric"/"gps"/"unknown")
- FIT parser sets "barometric" when enhanced_altitude is present, else "gps"
- GPX and TCX parsers always set "gps"
- metrics._elevation() uses the threshold matching the source
- 5 new parametric tests covering flat GPS noise, threshold differences, and real climbs
Instead of the browser resolving 20+ user shards recursively (~27 MB),
generate a pre-sorted feed.json at merge time with 50 activities per
page. The global feed loads one ~30 KB file on first paint; "Load more"
fetches subsequent pages (feed-2.json, feed-3.json, etc.).
Per-user profile pages still use year-sharded loadIndexPaged as before.
astro build resolves the public/data symlink and copies all activity JSON
into dist/; rsync then copied that to the webroot — but nginx already serves
/data/ directly from /var/bincio/data/ via alias, so both copies were dead
weight. Freed 36 GB → 14 GB on the live server.
- post-receive hook: prune dist/data/ before rsync, add --exclude=data/
- docs: update manual rebuild command and nginx comment to match
- serve/server.py: _mb() now uses lstat() to count symlinks at face value
rather than following them to targets, so admin storage panel no longer
double-counts _merged/ (which is mostly symlinks into activities/)
- serve/server.py GET adds private:bool to the response (true when
privacy is "unlisted" or legacy "private") so EditDrawer can read it
- edit/server.py GET: same fix for the single-user edit server
- EditDrawer: fall back to d.privacy if d.private is absent; rename
"Private" toggle label to "Unlisted"
delete_activity now updates data_dir/index.json so merge_all no longer
re-adds the summary for a deleted activity, preventing the broken
"Activity not found" state after deletion.
ActivityDetailLoader switches from loadIndex (all year shards) to
loadIndexPaged (first year shard only) + direct file fallback, so
opening an activity detail page no longer downloads the entire history.
merge_all/_merged/index.json is now a shard manifest; activities are
split into index-{year}.json files. The feed loads only the most-recent
year on first paint (~200 activities instead of all of them). Older
years are fetched lazily when the user clicks "Load older activities".
Also strips best_efforts / best_climb_m / source from shard files —
these fields are aggregation inputs only, never read by the feed UI.
Davide Scaini