Commit Graph

180 Commits

Author SHA1 Message Date
Davide Scaini 307f1fbbc1 download bas: embed timeseries into the JSON so the file is self-contained 2026-05-15 18:38:53 +02:00
Davide Scaini c465e518e5 Add activity file downloads with per-activity download_disabled flag
New endpoint: GET /api/activity/{id}/download/{bas|original|gpx}
- bas: streams the BAS detail JSON as an attachment
- original: streams the original FIT or GPX file from originals/
- gpx: generates a GPX from the timeseries (always available when GPS exists)

download_disabled flag stored in sidecar (edits/{id}.md), propagated to
the merged BAS detail JSON. When set, only the owner can download.

Backend: ops.py writes flag to sidecar; merge.py propagates it to detail
JSON; download.py implements the endpoint; server.py registers the router.
Frontend: EditDrawer gets a "No download" toggle button; ActivityDetail
shows a Download section (hidden when disabled and viewer is not the owner).
2026-05-15 18:35:40 +02:00
Davide Scaini fe437626e6 Global feed: switch from sequential pages to month-based BAS shards
feed.json is now a BAS shard index pointing to feed-YYYY-MM.json files
(~150 activities / ~25 KB gzip each) instead of 400+ sequential feed-N.json
pages. The frontend can now jump directly to a specific month when filtering
by year or date range, without loading all newer data first.

- merge.py: write_combined_feed groups by YYYY-MM and emits a shard index
- dataloader.ts: isYearShardUrl matches feed-YYYY-MM.json; loadCombinedFeed
  returns pendingShards; FeedPage interface and loadCombinedFeedPage removed
- ActivityFeed.svelte: _yearFromShard handles both index-YYYY and feed-YYYY-MM;
  feedNextPage/feedTotalPages/loadingAllFeedPages removed; infinite-loop bug
  fixed (toLoad.length guard before setting loadingAllShards); onMount uses
  pendingShards from loadCombinedFeed
2026-05-15 10:25:01 +02:00
Davide Scaini d2151a4acf Ideas: add reopen button when awaiting; add /reopen endpoint 2026-05-15 09:07:49 +02:00
Davide Scaini 15e9969ca2 Ideas: add 'won't implement' status with decline/reopen button 2026-05-15 08:36:31 +02:00
Davide Scaini ed6a7ed39c Ideas: add 'awaiting feedback' status with amber section + admin comment
Status cycles open → awaiting → done → reopen.
Awaiting ideas float to the top in a 'Waiting for your feedback' section
with an amber border (#f59e0b).

Admin can attach an implementation note to any awaiting idea via
POST /api/ideas/{id}/comment. The note appears inside the same card
in a distinct sub-box with a subtle amber tint border, editable inline.
The sub-box is visible to all users once a note exists.
2026-05-15 08:18:44 +02:00
Davide Scaini 3b675a68b0 Elevation: skip near-zero dropout values mid-recording
Devices (Apple Watch, some GPS units) record 0.0 when they lose barometric/GPS
lock mid-activity. The old accumulation committed these as real sea-level points,
inflating both gain and loss by the current elevation (e.g. 792m dropout on the
Cosmo Walk added ~1584m of phantom gain+loss).

Fix: skip any elevation value < 1.0m when the current committed elevation is
significantly above zero (> threshold). Gradual legitimate descents to sea level
are unaffected because intermediate values are committed along the way.

Add --recompute-elevation flag to bincio render to backfill existing activities.
2026-05-15 01:21:34 +02:00
Davide Scaini 4ea2292e2b Indoor detection: title-based inference in merge layer + fix _merge_all_locked
- Add _INDOOR_TITLE_RE / _infer_indoor_title() to writer.py (matches zwift,
  ftp-builder, turbo-trainer, rodillo); replaces the narrower zwift-only regex
  that was local to write_athlete_json
- _is_outdoor now delegates to _infer_indoor_title so all four keywords are
  excluded from records and MMP aggregation
- apply_sidecar and _apply_sidecar_summary both set sub_sport=indoor when the
  title matches and no explicit sub_sport is already present
- _merge_one_locked: detect title-inferred activities as needs_merge and call
  apply_sidecar({},{}) so the _merged copy gets sub_sport=indoor written
- _merge_all_locked: read index upfront to populate to_merge with title-inferred
  IDs; call apply_sidecar({},{}) for activities in to_merge without sidecars;
  apply _apply_sidecar_summary to ALL summary entries (not only sidecar ones)
2026-05-15 01:03:17 +02:00
Davide Scaini a863cdd663 Records: exclude Zwift activities by title; show Saved confirmation before closing drawer
- _is_outdoor now also excludes activities whose title matches /\bzwift\b/i,
  covering the ~50 Strava-imported Zwift rides that lack sub_sport metadata.

- EditDrawer waits 900ms after a successful save before dispatching 'saved'
  (which closes the drawer), so the green "Saved" confirmation is visible.
2026-05-15 00:49:46 +02:00
Davide Scaini 9f1e9e4d3b Records: apply sidecars before computing; fix best_climb_m for long mountain climbs
- _rebuild_athlete_json now applies sidecar edits (sub_sport, sport, etc.)
  in-memory before passing summaries to write_athlete_json, so activities
  marked indoor via sidecar are correctly excluded from records.

- _best_climb now runs Kadane's over cumulative distance (not 1Hz dense
  time) so recording pauses don't create None gaps that falsely reset the
  climbing window. Grappa: 811m→1603m; Nivolet: 311m→2009m.

- Add bincio render --recompute-climbs to backfill existing activities
  from their stored timeseries.
2026-05-15 00:30:58 +02:00
Davide Scaini de07d8d4cf activities: trigger rebuild after edit so records update immediately 2026-05-15 00:09:51 +02:00
Davide Scaini 1ce94b8536 records: exclude indoor/treadmill/virtual sub_sport; rebuild athlete.json on bake
- fit.py: map FIT sub_sport 'treadmill' and 'virtual' to 'indoor'
- writer.py: broaden _is_outdoor to catch all indoor sub_sport variants
- render/cli.py: rebuild athlete.json from index.json on every bake so
  records never go stale when the exclusion logic changes
2026-05-14 23:37:54 +02:00
Davide Scaini 5167f2a988 explore: shard tracks into per-year files for progressive loading
bake_tracks now writes tracks_YYYY.json shards + tracks_index.json manifest
instead of a single monolithic tracks.json. API /api/me/tracks returns the
manifest; /api/me/tracks/{year} serves individual shards. Explore.svelte
fetches the two most recent years eagerly then streams the rest in the
background so the map renders immediately with recent data.
2026-05-14 18:34:53 +02:00
Davide Scaini c36b95e041 segments detect: add --fresh flag to clear efforts before re-detecting 2026-05-14 17:11:11 +02:00
Davide Scaini 862226305a Fix segment avg_speed: derive from distance/time; tighten speed bounds to reject false matches 2026-05-14 17:09:41 +02:00
Davide Scaini 8ff781661e Fix feedback JSON encoding: use ensure_ascii=False to preserve accented characters 2026-05-14 17:04:44 +02:00
Davide Scaini 5593764fdb explore: skip legacy bare-timestamp geojsons; type pill colors visible when inactive 2026-05-14 15:55:10 +02:00
Davide Scaini 537d1bb712 explore: exclude indoor/virtual activities from tracks.json 2026-05-14 14:34:44 +02:00
Davide Scaini 5307ae287c Explore: personal GPS heatmap tab under Athlete page
- bincio/explore.py: bake_tracks() simplifies GPS coords (RDP ε=0.0001),
  strips to [lng,lat], groups by sport type, writes per-handle tracks.json
- bake-tracks CLI command; render CLI calls _bake_tracks() after each build;
  strava_zip runs it once at end of batch
- /api/me/tracks endpoint serves the baked file; wipe_user cleans it up
- Explore.svelte: MapLibre full-screen map with sidebar — type pills,
  year/month date filter, Lines / Heatmap (global or by-type) view modes
- AthleteView: Explore tab visible only to profile owner (checks __bincioMe)
- Base.astro: fullscreen prop + Planner nav link
2026-05-14 14:31:21 +02:00
Davide Scaini 1a7d1dc8c3 serve: complete CurrentUserResponse model (add wiki_access, activity_access, dem_configured) 2026-05-14 11:06:35 +02:00
Davide Scaini a10164b932 metrics: fall back to GPS-derived speed in compute_best_efforts when device speed absent
FIT files from older devices (and GPX/TCX files) often omit the speed field.
The sliding-window best-effort algorithm was treating all such points as speed=0,
so no records were ever produced for these activities.

Fix: when p.speed_kmh is None but consecutive lat/lon are available, compute
haversine segment speed and spread it evenly across the 1Hz interval slots.
This mirrors what _gps_stats already does for avg/max speed computation.
2026-05-14 09:16:01 +02:00
Davide Scaini c85d2edf39 dev: fix _start_serve to set deps.data_dir/site_dir (not srv.*) after Step 3 split 2026-05-14 00:09:41 +02:00
Davide Scaini 27f6d141f7 Refactor step 4: narrow broad except Exception catches
Replaced 28 bare `except Exception` catches across 8 files with specific
exception types reflecting the actual failure modes:

- JSON file reads → (OSError, json.JSONDecodeError)
- datetime parsing → ValueError
- base64 decoding → ValueError
- YAML parsing → (OSError, yaml.YAMLError); import moved above try
- GeoJSON coord extraction → (TypeError, IndexError, AttributeError)
- Startup temp-file cleanup → OSError
- Single JSON line parsing (SSE batch) → json.JSONDecodeError

Kept broad catches only where intentional:
- Background thread top-level guards (tasks.py, admin.py) with log.exception
- SSE stream generator tops (strava.py, garmin.py, uploads.py)
- Per-item batch loops that must not abort the whole operation
- Explicitly non-fatal post-upload merge steps with log.warning
2026-05-13 23:58:14 +02:00
Davide Scaini 8380b1d2cc Refactor: split serve/server.py (3220 lines) into focused modules
serve/server.py is now 69 lines — app factory, middleware, and router
registration only.

New modules:
  deps.py    (168 lines) — module-level globals + auth dependency functions
  models.py   (85 lines) — all Pydantic request/response models
  tasks.py   (136 lines) — background workers and job tracker
  routers/               — one file per domain (10 routers, ~2750 lines total)
    auth.py, me.py, admin.py, activities.py, uploads.py,
    segments.py, strava.py, garmin.py, ideas.py, feed.py

cli.py updated to set globals on deps instead of server.

88 new regression tests in tests/serve/ cover auth guards and key
behaviours for every router; 294 total passing after the split.
2026-05-13 23:47:19 +02:00
Davide Scaini 2ec4d9157c Refactor: extract edit UI HTML into bincio/edit/templates/edit.html
The 285-line _HTML string literal in edit/server.py is replaced by a
template file loaded at request time. The route handler is unchanged in
behaviour — it still substitutes __SITE_URL__, __SPORT_OPTIONS__, and
__STAT_CHECKBOXES__ before returning the response.

Five new tests cover: 200 response, form presence, site_url injection,
no unresolved placeholders, and template file existence on disk.
2026-05-13 23:19:19 +02:00
Davide Scaini 9dd533825f Fix pre-existing test failures in test_writer and test_metrics
test_writer: _dummy_metrics() and test_build_summary_required_fields were
missing np_power_w=None after the field was added to ComputedMetrics.

test_metrics: the leading-zero elevation heuristic fired on a single 0.0
start value, incorrectly skipping the first legitimate elevation step.
Guard now requires at least 2 consecutive near-zero leading values before
activating the Apple Watch lock-acquisition workaround.
2026-05-13 23:15:26 +02:00
Davide Scaini e61d05fc41 Refactor: extract shared image upload utilities into bincio/shared/images.py
ALLOWED_IMAGE_TYPES, MAX_IMAGE_BYTES, and unique_image_name() were
duplicated identically in both the edit and serve servers. Centralising
them means a single change point for any future extension (e.g. adding
image/avif support).

Tests added in tests/test_shared_images.py cover no-collision, single
and chained collisions, no-suffix filenames, and constant values.
2026-05-13 23:13:08 +02:00
Davide Scaini cd97e4cc87 CORS: allow all *.bincio.org origins (for planner.bincio.org) 2026-05-13 22:51:00 +02:00
Davide Scaini 58a5d5b450 Strava sync: skip import when a FIT-file upload already covers the same start time
Before importing each Strava activity, build a set of existing timestamp
prefixes (YYYY-MM-DDTHHMMSSZ) from the activities directory. If the incoming
Strava activity matches an existing prefix, record its Strava ID as done and
skip — preventing duplicate entries when a FIT file and a Strava sync both
cover the same ride.

Also reports skipped-existing count in the summary line.
2026-05-13 21:52:07 +02:00
Davide Scaini b9a21e8bcc ideas: add inline edit for own ideas (author + admin) 2026-05-13 19:52:25 +02:00
Davide Scaini c30a15d295 ideas: add done/reopen status toggle for admins
Admin-only POST /api/ideas/{id}/status toggles status between open and
done. Done ideas are greyed out (opacity 0.55), show a green checkmark,
and sink to the bottom of the list. Admins see done/reopen buttons on
each card.
2026-05-13 19:32:30 +02:00
Davide Scaini 9553ca5ce7 ideas: add JSON-file-backed ideas API (list, create, vote, delete)
Ideas and votes are stored as flat JSON files in /var/bincio/_ideas/,
following the same filesystem-first philosophy as segments and efforts.
Vote toggling uses fcntl exclusive locking to prevent concurrent writes.
2026-05-13 19:27:54 +02:00
Davide Scaini cf9817e853 segments: clear stored efforts before rescan so stale entries are removed
Both trigger_detect and me_segment_rescan were appending-only, so false
efforts recorded before the geometric speed check fix remained after
rescan. Now each rescan path clears the effort file first, making the
result authoritative.
2026-05-13 16:35:44 +02:00
Davide Scaini 6e92ea4fce segments: reject false efforts via geometric speed check
Long circuit rides were matching a segment START early and finding the
segment END hours later on a second pass, producing effort times of
~17000s on a 4.7km segment. The conformance check passed because the
full-circuit track covers all interior points within 50m over 5 hours.

Add a per-sport minimum geometric speed (segment_distance / elapsed_s):
cycling ≥ 1.0 m/s, running ≥ 0.5 m/s, default ≥ 0.2 m/s. When the
check fails, advance past the current start candidate and retry, so a
legitimate later match (e.g. a second lap done at real speed) is still
detected.
2026-05-13 16:31:00 +02:00
Davide Scaini 994f4287ef Dedup segment efforts by started_at to handle same activity stored under two IDs 2026-05-13 16:20:10 +02:00
Davide Scaini ad9e428b1e Fix UnicodeEncodeError: sanitize surrogate pairs before JSON writes in merge.py 2026-05-13 16:16:58 +02:00
Davide Scaini c837464a28 Exclude indoor/virtual activities from records and power curve 2026-05-13 16:05:26 +02:00
Davide Scaini 2395a6e566 Fix segment effort duplicates; auto-scan on segment creation
- detect.py: truncate started_at to seconds so dedup key survives JSON round-trip
- store.py: dedup by (activity_id, iso-started_at) string key, not object equality
- server.py: extract _scan_segment_for_user helper; trigger background scan
  for the creating user's activities when a new segment is saved
2026-05-13 15:58:57 +02:00
Davide Scaini 0ff5473dfd Athlete segments tab: link best time to activity; expandable effort list
- best_activity_id now included in segment_summary API response
- Best time is a direct link to the activity that produced it
- Clicking a row expands an inline effort list (lazy-loaded from
  /api/segments/{id}/efforts): date linked to activity, time, Δ vs PR
- Clicking again collapses; ▲/▼ chevron shows state
2026-05-13 08:40:39 +02:00
Davide Scaini d7fd585e77 Add global segment rescan: POST /api/me/segment-rescan + Rescan all button 2026-05-13 08:17:18 +02:00
Davide Scaini f2075e29d2 Segments Phase 4: detail page, activity efforts, athlete tab, new APIs
New API endpoints:
- GET /api/segments/{id} — single segment metadata
- GET /api/activities/{id}/segment_efforts — efforts for an activity (auth)
- GET /api/users/{handle}/segment_summary — public best time + count per segment

New components:
- SegmentDetail.svelte — map + metadata + effort table (with PR/Δ) + rescan button
- SegmentsPage.svelte — URL router: shows detail when /segments/{id}/, list otherwise

Updated:
- segments/index.astro — now uses SegmentsPage router
- nginx-activity.conf — add /segments/ try_files rule for client-side routing
- ActivityDetail.svelte — segment efforts block below laps
- AthleteView.svelte — Segments tab with best time + effort count per segment
- format.ts — add formatElapsed() for compact m:ss display
2026-05-13 08:09:24 +02:00
Davide Scaini 6c9de35426 Enforce 500 m minimum segment length in UI and API 2026-05-13 00:56:04 +02:00
Davide Scaini 4d2df860ce Segments Phase 3: detection algorithm, CLI, ingest hook, and efforts API
- detect.py: ActivityTrack + detect_one/detect_all (bbox pre-filter →
  start/end proximity 25m → path conformance 50m/30% → effort extraction
  with avg speed/HR/power and Coggan NP)
- cli.py: `bincio segments detect` for retroactive detection over stored
  timeseries JSONs, with optional --activity-id / --segment-id filters
- ingest.py: non-fatal hook at end of ingest_parsed runs detect_all
- server.py: GET /api/segments/{id}/efforts and POST /api/segments/{id}/detect
2026-05-13 00:50:39 +02:00
Davide Scaini 79cad29ff1 Segments phase 1: models, store, and API endpoints (GET/POST/DELETE /api/segments) 2026-05-13 00:19:15 +02:00
Davide Scaini bd0595ee79 Add avg power and NP to activity summary; NP uses Coggan 30s rolling-average method 2026-05-12 23:47:06 +02:00
Davide Scaini 0b266d208c Strip pre-2000 leading points to prevent epoch-zero start time and absurd duration 2026-05-12 23:11:33 +02:00
Davide Scaini 867da767eb Add sub_sport editing to activity edit drawer 2026-05-12 23:01:12 +02:00
Davide Scaini 695dc9fdce Fix Strava re-auth when credentials change; add disconnect button
When a user saves new Strava credentials with a different client_id,
auto-delete the existing token (it belongs to a different OAuth app
and will always fail on refresh). Add POST /api/strava/disconnect
endpoint and a "Disconnect from Strava" button in settings, visible
only when connected.

Immediate: deleted diego_p's stale token so he can reconnect.
2026-05-10 16:33:52 +02:00
Davide Scaini 8f028101c7 Fix elevation gain inflation from device no-fix leading zeros
Apple Watch and similar devices record exactly 0.0 for elevation while
waiting for barometric/GPS lock, then jump to the real altitude. The
hysteresis accumulator was seeding from 0.0, counting the full jump as
ascent. Fix: detect a leading near-zero run followed by a large jump
and seed the accumulator from the first real value instead.

Applied in both _elevation() (fresh extractions) and
recalculate_elevation_hysteresis() (recompute path). Added a bulk
admin endpoint POST /api/admin/users/{handle}/recompute-elevation and
corresponding button to fix existing stored activities.
2026-05-10 16:21:24 +02:00
Davide Scaini 55d59112ad Fix: don't copy 9 GB data dir into dist/ during production builds
BINCIO_DATA_DIR is already set in the build env, so manifest.ts reads
the data root directly at build time without needing public/data.
Moving _link_data() into the serve-only branch prevents Astro from
following the symlink and copying the full data dir into dist/. Any
leftover symlink from a previous dev session is removed before build.
Dev mode is unchanged.
2026-05-08 13:56:31 +02:00