Commit Graph

170 Commits

Author SHA1 Message Date
Davide Scaini de07d8d4cf activities: trigger rebuild after edit so records update immediately 2026-05-15 00:09:51 +02:00
Davide Scaini 1ce94b8536 records: exclude indoor/treadmill/virtual sub_sport; rebuild athlete.json on bake
- fit.py: map FIT sub_sport 'treadmill' and 'virtual' to 'indoor'
- writer.py: broaden _is_outdoor to catch all indoor sub_sport variants
- render/cli.py: rebuild athlete.json from index.json on every bake so
  records never go stale when the exclusion logic changes
2026-05-14 23:37:54 +02:00
Davide Scaini 5167f2a988 explore: shard tracks into per-year files for progressive loading
bake_tracks now writes tracks_YYYY.json shards + tracks_index.json manifest
instead of a single monolithic tracks.json. API /api/me/tracks returns the
manifest; /api/me/tracks/{year} serves individual shards. Explore.svelte
fetches the two most recent years eagerly then streams the rest in the
background so the map renders immediately with recent data.
2026-05-14 18:34:53 +02:00
Davide Scaini c36b95e041 segments detect: add --fresh flag to clear efforts before re-detecting 2026-05-14 17:11:11 +02:00
Davide Scaini 862226305a Fix segment avg_speed: derive from distance/time; tighten speed bounds to reject false matches 2026-05-14 17:09:41 +02:00
Davide Scaini 8ff781661e Fix feedback JSON encoding: use ensure_ascii=False to preserve accented characters 2026-05-14 17:04:44 +02:00
Davide Scaini 5593764fdb explore: skip legacy bare-timestamp geojsons; type pill colors visible when inactive 2026-05-14 15:55:10 +02:00
Davide Scaini 537d1bb712 explore: exclude indoor/virtual activities from tracks.json 2026-05-14 14:34:44 +02:00
Davide Scaini 5307ae287c Explore: personal GPS heatmap tab under Athlete page
- bincio/explore.py: bake_tracks() simplifies GPS coords (RDP ε=0.0001),
  strips to [lng,lat], groups by sport type, writes per-handle tracks.json
- bake-tracks CLI command; render CLI calls _bake_tracks() after each build;
  strava_zip runs it once at end of batch
- /api/me/tracks endpoint serves the baked file; wipe_user cleans it up
- Explore.svelte: MapLibre full-screen map with sidebar — type pills,
  year/month date filter, Lines / Heatmap (global or by-type) view modes
- AthleteView: Explore tab visible only to profile owner (checks __bincioMe)
- Base.astro: fullscreen prop + Planner nav link
2026-05-14 14:31:21 +02:00
Davide Scaini 1a7d1dc8c3 serve: complete CurrentUserResponse model (add wiki_access, activity_access, dem_configured) 2026-05-14 11:06:35 +02:00
Davide Scaini a10164b932 metrics: fall back to GPS-derived speed in compute_best_efforts when device speed absent
FIT files from older devices (and GPX/TCX files) often omit the speed field.
The sliding-window best-effort algorithm was treating all such points as speed=0,
so no records were ever produced for these activities.

Fix: when p.speed_kmh is None but consecutive lat/lon are available, compute
haversine segment speed and spread it evenly across the 1Hz interval slots.
This mirrors what _gps_stats already does for avg/max speed computation.
2026-05-14 09:16:01 +02:00
Davide Scaini c85d2edf39 dev: fix _start_serve to set deps.data_dir/site_dir (not srv.*) after Step 3 split 2026-05-14 00:09:41 +02:00
Davide Scaini 27f6d141f7 Refactor step 4: narrow broad except Exception catches
Replaced 28 bare `except Exception` catches across 8 files with specific
exception types reflecting the actual failure modes:

- JSON file reads → (OSError, json.JSONDecodeError)
- datetime parsing → ValueError
- base64 decoding → ValueError
- YAML parsing → (OSError, yaml.YAMLError); import moved above try
- GeoJSON coord extraction → (TypeError, IndexError, AttributeError)
- Startup temp-file cleanup → OSError
- Single JSON line parsing (SSE batch) → json.JSONDecodeError

Kept broad catches only where intentional:
- Background thread top-level guards (tasks.py, admin.py) with log.exception
- SSE stream generator tops (strava.py, garmin.py, uploads.py)
- Per-item batch loops that must not abort the whole operation
- Explicitly non-fatal post-upload merge steps with log.warning
2026-05-13 23:58:14 +02:00
Davide Scaini 8380b1d2cc Refactor: split serve/server.py (3220 lines) into focused modules
serve/server.py is now 69 lines — app factory, middleware, and router
registration only.

New modules:
  deps.py    (168 lines) — module-level globals + auth dependency functions
  models.py   (85 lines) — all Pydantic request/response models
  tasks.py   (136 lines) — background workers and job tracker
  routers/               — one file per domain (10 routers, ~2750 lines total)
    auth.py, me.py, admin.py, activities.py, uploads.py,
    segments.py, strava.py, garmin.py, ideas.py, feed.py

cli.py updated to set globals on deps instead of server.

88 new regression tests in tests/serve/ cover auth guards and key
behaviours for every router; 294 total passing after the split.
2026-05-13 23:47:19 +02:00
Davide Scaini 2ec4d9157c Refactor: extract edit UI HTML into bincio/edit/templates/edit.html
The 285-line _HTML string literal in edit/server.py is replaced by a
template file loaded at request time. The route handler is unchanged in
behaviour — it still substitutes __SITE_URL__, __SPORT_OPTIONS__, and
__STAT_CHECKBOXES__ before returning the response.

Five new tests cover: 200 response, form presence, site_url injection,
no unresolved placeholders, and template file existence on disk.
2026-05-13 23:19:19 +02:00
Davide Scaini 9dd533825f Fix pre-existing test failures in test_writer and test_metrics
test_writer: _dummy_metrics() and test_build_summary_required_fields were
missing np_power_w=None after the field was added to ComputedMetrics.

test_metrics: the leading-zero elevation heuristic fired on a single 0.0
start value, incorrectly skipping the first legitimate elevation step.
Guard now requires at least 2 consecutive near-zero leading values before
activating the Apple Watch lock-acquisition workaround.
2026-05-13 23:15:26 +02:00
Davide Scaini e61d05fc41 Refactor: extract shared image upload utilities into bincio/shared/images.py
ALLOWED_IMAGE_TYPES, MAX_IMAGE_BYTES, and unique_image_name() were
duplicated identically in both the edit and serve servers. Centralising
them means a single change point for any future extension (e.g. adding
image/avif support).

Tests added in tests/test_shared_images.py cover no-collision, single
and chained collisions, no-suffix filenames, and constant values.
2026-05-13 23:13:08 +02:00
Davide Scaini cd97e4cc87 CORS: allow all *.bincio.org origins (for planner.bincio.org) 2026-05-13 22:51:00 +02:00
Davide Scaini 58a5d5b450 Strava sync: skip import when a FIT-file upload already covers the same start time
Before importing each Strava activity, build a set of existing timestamp
prefixes (YYYY-MM-DDTHHMMSSZ) from the activities directory. If the incoming
Strava activity matches an existing prefix, record its Strava ID as done and
skip — preventing duplicate entries when a FIT file and a Strava sync both
cover the same ride.

Also reports skipped-existing count in the summary line.
2026-05-13 21:52:07 +02:00
Davide Scaini b9a21e8bcc ideas: add inline edit for own ideas (author + admin) 2026-05-13 19:52:25 +02:00
Davide Scaini c30a15d295 ideas: add done/reopen status toggle for admins
Admin-only POST /api/ideas/{id}/status toggles status between open and
done. Done ideas are greyed out (opacity 0.55), show a green checkmark,
and sink to the bottom of the list. Admins see done/reopen buttons on
each card.
2026-05-13 19:32:30 +02:00
Davide Scaini 9553ca5ce7 ideas: add JSON-file-backed ideas API (list, create, vote, delete)
Ideas and votes are stored as flat JSON files in /var/bincio/_ideas/,
following the same filesystem-first philosophy as segments and efforts.
Vote toggling uses fcntl exclusive locking to prevent concurrent writes.
2026-05-13 19:27:54 +02:00
Davide Scaini cf9817e853 segments: clear stored efforts before rescan so stale entries are removed
Both trigger_detect and me_segment_rescan were appending-only, so false
efforts recorded before the geometric speed check fix remained after
rescan. Now each rescan path clears the effort file first, making the
result authoritative.
2026-05-13 16:35:44 +02:00
Davide Scaini 6e92ea4fce segments: reject false efforts via geometric speed check
Long circuit rides were matching a segment START early and finding the
segment END hours later on a second pass, producing effort times of
~17000s on a 4.7km segment. The conformance check passed because the
full-circuit track covers all interior points within 50m over 5 hours.

Add a per-sport minimum geometric speed (segment_distance / elapsed_s):
cycling ≥ 1.0 m/s, running ≥ 0.5 m/s, default ≥ 0.2 m/s. When the
check fails, advance past the current start candidate and retry, so a
legitimate later match (e.g. a second lap done at real speed) is still
detected.
2026-05-13 16:31:00 +02:00
Davide Scaini 994f4287ef Dedup segment efforts by started_at to handle same activity stored under two IDs 2026-05-13 16:20:10 +02:00
Davide Scaini ad9e428b1e Fix UnicodeEncodeError: sanitize surrogate pairs before JSON writes in merge.py 2026-05-13 16:16:58 +02:00
Davide Scaini c837464a28 Exclude indoor/virtual activities from records and power curve 2026-05-13 16:05:26 +02:00
Davide Scaini 2395a6e566 Fix segment effort duplicates; auto-scan on segment creation
- detect.py: truncate started_at to seconds so dedup key survives JSON round-trip
- store.py: dedup by (activity_id, iso-started_at) string key, not object equality
- server.py: extract _scan_segment_for_user helper; trigger background scan
  for the creating user's activities when a new segment is saved
2026-05-13 15:58:57 +02:00
Davide Scaini 0ff5473dfd Athlete segments tab: link best time to activity; expandable effort list
- best_activity_id now included in segment_summary API response
- Best time is a direct link to the activity that produced it
- Clicking a row expands an inline effort list (lazy-loaded from
  /api/segments/{id}/efforts): date linked to activity, time, Δ vs PR
- Clicking again collapses; ▲/▼ chevron shows state
2026-05-13 08:40:39 +02:00
Davide Scaini d7fd585e77 Add global segment rescan: POST /api/me/segment-rescan + Rescan all button 2026-05-13 08:17:18 +02:00
Davide Scaini f2075e29d2 Segments Phase 4: detail page, activity efforts, athlete tab, new APIs
New API endpoints:
- GET /api/segments/{id} — single segment metadata
- GET /api/activities/{id}/segment_efforts — efforts for an activity (auth)
- GET /api/users/{handle}/segment_summary — public best time + count per segment

New components:
- SegmentDetail.svelte — map + metadata + effort table (with PR/Δ) + rescan button
- SegmentsPage.svelte — URL router: shows detail when /segments/{id}/, list otherwise

Updated:
- segments/index.astro — now uses SegmentsPage router
- nginx-activity.conf — add /segments/ try_files rule for client-side routing
- ActivityDetail.svelte — segment efforts block below laps
- AthleteView.svelte — Segments tab with best time + effort count per segment
- format.ts — add formatElapsed() for compact m:ss display
2026-05-13 08:09:24 +02:00
Davide Scaini 6c9de35426 Enforce 500 m minimum segment length in UI and API 2026-05-13 00:56:04 +02:00
Davide Scaini 4d2df860ce Segments Phase 3: detection algorithm, CLI, ingest hook, and efforts API
- detect.py: ActivityTrack + detect_one/detect_all (bbox pre-filter →
  start/end proximity 25m → path conformance 50m/30% → effort extraction
  with avg speed/HR/power and Coggan NP)
- cli.py: `bincio segments detect` for retroactive detection over stored
  timeseries JSONs, with optional --activity-id / --segment-id filters
- ingest.py: non-fatal hook at end of ingest_parsed runs detect_all
- server.py: GET /api/segments/{id}/efforts and POST /api/segments/{id}/detect
2026-05-13 00:50:39 +02:00
Davide Scaini 79cad29ff1 Segments phase 1: models, store, and API endpoints (GET/POST/DELETE /api/segments) 2026-05-13 00:19:15 +02:00
Davide Scaini bd0595ee79 Add avg power and NP to activity summary; NP uses Coggan 30s rolling-average method 2026-05-12 23:47:06 +02:00
Davide Scaini 0b266d208c Strip pre-2000 leading points to prevent epoch-zero start time and absurd duration 2026-05-12 23:11:33 +02:00
Davide Scaini 867da767eb Add sub_sport editing to activity edit drawer 2026-05-12 23:01:12 +02:00
Davide Scaini 695dc9fdce Fix Strava re-auth when credentials change; add disconnect button
When a user saves new Strava credentials with a different client_id,
auto-delete the existing token (it belongs to a different OAuth app
and will always fail on refresh). Add POST /api/strava/disconnect
endpoint and a "Disconnect from Strava" button in settings, visible
only when connected.

Immediate: deleted diego_p's stale token so he can reconnect.
2026-05-10 16:33:52 +02:00
Davide Scaini 8f028101c7 Fix elevation gain inflation from device no-fix leading zeros
Apple Watch and similar devices record exactly 0.0 for elevation while
waiting for barometric/GPS lock, then jump to the real altitude. The
hysteresis accumulator was seeding from 0.0, counting the full jump as
ascent. Fix: detect a leading near-zero run followed by a large jump
and seed the accumulator from the first real value instead.

Applied in both _elevation() (fresh extractions) and
recalculate_elevation_hysteresis() (recompute path). Added a bulk
admin endpoint POST /api/admin/users/{handle}/recompute-elevation and
corresponding button to fix existing stored activities.
2026-05-10 16:21:24 +02:00
Davide Scaini 55d59112ad Fix: don't copy 9 GB data dir into dist/ during production builds
BINCIO_DATA_DIR is already set in the build env, so manifest.ts reads
the data root directly at build time without needing public/data.
Moving _link_data() into the serve-only branch prevents Astro from
following the symlink and copying the full data dir into dist/. Any
leftover symlink from a previous dev session is removed before build.
Dev mode is unchanged.
2026-05-08 13:56:31 +02:00
Davide Scaini 2287d6e2ee Add Strava sync status report and manual trigger to admin panel
Each sync run now writes _strava_sync_status.json per user (status,
imported count, error message). New admin endpoints expose this data
and allow triggering an on-demand sync. The admin page gains a Strava
Sync section showing per-user token/credentials state, total imported,
last sync time, and last-run status with inline error messages.
2026-05-08 13:44:23 +02:00
Davide Scaini 12693dbd60 feat: scheduled Strava sync + admin suspend/delete account
- Add bincio sync-strava command: headless multi-user Strava sync
  designed for systemd timer. Discovers users via strava_token.json,
  skips users without their own strava_credentials.json, respects
  Strava visibility (only_me → unlisted). Treats 404 stream errors as
  no-GPS activities rather than retrying every run.
- Add deploy/systemd/bincio-sync.{service,timer}: runs every 3 hours,
  Persistent=true to catch up after downtime.
- Add POST /api/internal/rebuild: webhook for sync timer to trigger
  site rebuild, authenticated via X-Sync-Secret header.
- Add suspended column to users table with auto-migration on open_db.
  Suspended users are blocked at login and session lookup (covers both
  activity site and wiki, which share instance.db).
- Add POST /api/admin/users/{handle}/suspend|unsuspend and
  DELETE /api/admin/users/{handle}/account endpoints.
- Admin panel: Suspend/Unsuspend toggle, Del account button, suspended
  badge on user row.
2026-05-08 10:36:21 +02:00
Davide Scaini e0b5a55ccc fix(dev): pass --host 0.0.0.0 to astro dev when --mobile, print LAN URL 2026-05-01 22:20:19 +02:00
Davide Scaini 82288a35ea feat(auth): wiki/activity access flags, SESSION_DOMAIN, wiki nav link 2026-05-01 21:56:02 +02:00
Davide Scaini f6e9fe8198 feat(serve): debounced site rebuild — burst uploads trigger one build, not N
Replace per-upload Astro build threads with a single background worker
(_site_rebuild_worker) that waits on an event, sleeps 60 s to let upload
bursts settle, then runs one full build + rsync. 271 concurrent uploads now
produce one build instead of 271 serialised builds, eliminating the OOM kill.
--webroot is re-enabled; merge-only path still runs immediately per upload.

Also: date filter row added to ActivityFeed.svelte (sport + date presets
with dynamic year pills); deploy/vps gitignored for VPS config backups.
2026-04-30 21:23:29 +02:00
Davide Scaini 946da685e5 feat(mobile): editable activity title for local activities
Adds edits_json column (migration v3) to store user overrides separately
from detail_json so Option A server re-extraction never clobbers them.

- Tap the title in the detail screen to edit (local activities only, shown
  with a ✎ hint). Saves on keyboard dismiss via onEndEditing.
- Cards and search display user_title ?? title.
- Raw upload: user_title sent to server -> sidecar written so web UI shows
  the correct title (server re-extracts from FIT, which has Karoo's title).
- BAS upload: detail.title overridden before sending, no sidecar needed.
2026-04-27 15:20:19 +02:00
Davide Scaini be772bd3df fix(upload): prevent false 422s and EMFILE crash during bulk uploads
Four related issues made uploading 271+ activities unreliable:

1. merge_all/write_combined_feed were inside the extraction try/except —
   any merge race returned 422 even though the file was on disk, causing
   the mobile app to permanently mark the upload as failed.  Fixed by
   moving them to a separate best-effort try/except after the extraction
   block.  Switch to merge_one (single-activity symlink) instead of
   merge_all (full rebuild) so each upload is O(1) FS ops, not O(N).

2. The dev watcher fired merge_all for every activity .json write AND the
   upload endpoint also ran merge_all — O(N²) symlink operations during
   bulk uploads.  Watcher now skips activities/*.json changes (upload
   endpoint handles those directly).

3. Vite/Chokidar followed the public/data symlink and opened a handle per
   activity file; constant merge rebuilds exhausted file descriptors and
   crashed the Astro dev server.  Fixed with watch.ignored on public/data.

4. _write_year_shards and write_combined_feed used f.unlink() without
   missing_ok=True — concurrent callers racing the same file threw
   FileNotFoundError which propagated as a false extraction failure.
2026-04-27 14:33:05 +02:00
Davide Scaini cfb3ba5871 fix(merge): serialize merge_all per user-dir to prevent concurrent rmtree race
When the dev file-watcher and an upload endpoint both trigger merge_all on the
same user directory at the same time, the shutil.rmtree/_merged/activities/
sequence collides: one thread deletes the directory while the other is mid-walk,
causing ENOTEMPTY or ENOENT crashes.

Added a per-user-dir threading.Lock (_merge_lock) at the top of merge.py so
all callers (upload endpoints, watcher, admin rebuild) are serialized without
changing any call site.  Also made the rmtree ignore_errors=True and the
symlink_to conditional to survive any stale leftover from a prior crash.
2026-04-27 11:11:05 +02:00
Davide Scaini 220efb0d05 fix(mobile/upload): activities now appear in browser after upload; reconcile synced_at on fresh server
Three bugs fixed:
- /api/upload/bas and /api/upload/raw never updated user_dir/index.json, so
  merge_all couldn't include uploaded activities in year shards — they existed
  on disk but were invisible to the browser feed. Fixed by _upsert_index_summary()
  called before merge_all().
- Silent catch {} in uploadLocalActivities swallowed all per-activity errors;
  replaced with console.warn so failures are visible in Expo logs.
- After a server wipe, synced_at flags on the device caused "Nothing to upload"
  forever. uploadFeed() now reconciles against GET /api/feed at the start of each
  upload: local activities not found on the server get synced_at cleared.

Also: live upload progress ("Uploading N / M…"), failed count in result message,
onProgress callback on uploadFeed(), countPendingUploads() helper.
2026-04-27 11:03:00 +02:00
Davide Scaini b1cf18a2f0 fix(feed): update feed.json after every upload so browser sees new activities
merge_all(user_dir) updates the per-user _merged/ shard but the home page
loads feed.json first via loadCombinedFeed. write_combined_feed was only
called by the CLI render command, not by the API upload endpoints or the
dev watcher, leaving feed.json permanently stale after any runtime upload.

Add write_combined_feed(_get_data_dir()) after every merge_all call in
/api/upload/bas, /api/upload/raw, the dev.py file watcher, and dev startup.
2026-04-26 21:37:19 +02:00