brut/bincio-activity
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

fix: remove sensitive files, sync manifest

Browse Source
This commit is contained in:
Davide Scaini
2026-03-30 20:38:00 +02:00
parent 2ad18883f8
commit 5bbc3d07d2
7 changed files with 51 additions and 999 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+48 -539
View File
@@ -24,16 +24,15 @@ Anyone can publish their data as BAS JSON and others can include it.
- **TCX files** from Garmin use both `http://` and `https://` namespace URIs —
parser handles both
## User's data
## Your data
- Source: `~/src/cycling_data_davide/`
- Source: `~/your-activity-data/`
- `activities/` — Strava export (GPX, FIT, TCX, all with .gz variants)
- `Karoo_2026/` — recent Karoo device FIT files
- `Karoo/` — older Karoo FIT files
- Any subdirectories with FIT files from Garmin/Karoo devices
- `activities.csv` — Strava metadata (names, descriptions, gear)
- Extracted output: `~/bincio_data/` (or `/tmp/bincio_test/` for testing)
- ~3,200 input files → ~2,082 unique activities after dedup
- Date range: 2014–2026
Configure input paths in `extract_config.yaml`.
## Project structure
@@ -51,32 +50,29 @@ bincio/ Python package
writer.py BAS JSON + GeoJSON writer
config.py extract_config.yaml loader
cli.py `bincio extract` CLI
import_/
strava.py Strava API importer (OAuth2, streams → BAS JSON)
cli.py `bincio import strava` CLI
render/
cli.py `bincio render` CLI (symlinks data, runs astro build/dev)
merge.py sidecar edit overlay (produces _merged/)
edit/
server.py FastAPI write API (activity edits, image upload, file upload)
cli.py `bincio edit` CLI
server.py FastAPI write API for the edit drawer
schema/
bas-v1.schema.json JSON Schema for BAS
SCHEMA.md Human-readable BAS spec
site/ Astro project
src/
layouts/Base.astro Nav (upload button + theme toggle), theme CSS vars
layouts/Base.astro
pages/
index.astro Activity feed (loads index.json client-side)
activity/[id].astro Single activity (SSG, loads detail JSON client-side)
stats/index.astro Heatmap + year totals
athlete/index.astro MMP curve + athlete profile (planned)
components/
ActivityFeed.svelte Card grid, sport filter, pagination
ActivityDetail.svelte Map + stats + charts wrapper
ActivityDetail.svelte Map + stats + charts + photo gallery
ActivityMap.svelte MapLibre GL (gradient track, linked hover dot)
ActivityCharts.svelte Observable Plot (elevation/speed/HR/cadence/power tabs)
StatsView.svelte Yearly heatmap + click-to-pin tooltip
EditDrawer.svelte Slide-in activity editor
ActivityCharts.svelte Observable Plot (elevation/speed/HR/cadence tabs)
StatsView.svelte Yearly heatmap + totals
EditDrawer.svelte Slide-in edit panel (visible when PUBLIC_EDIT_URL set)
lib/
types.ts BAS TypeScript types
format.ts formatDistance, formatDuration, sportIcon, etc.
@@ -85,26 +81,19 @@ site/ Astro project
## How to run
```bash
# Fast dev loop (50-file sample → /tmp/bincio_dev/, no real data touched)
uv run bincio extract --dev 50
uv run bincio import strava --dev 50 # 50 most recent Strava activities
uv run bincio render --serve --data-dir /tmp/bincio_dev
# Full extract from local files
# Extract
cd ~/src/bincio_activity
uv run bincio extract # uses extract_config.yaml
uv run bincio extract --input ~/your-activity-data/activities --output /tmp/bincio_test
# Import from Strava (credentials in extract_config.yaml under import.strava)
uv sync --extra strava
uv run bincio import strava # first run opens browser for OAuth
uv run bincio import strava # subsequent runs are incremental
# Site dev server
cd site
ln -sf /tmp/bincio_test/_merged public/data # point at merged output
cp .env.example .env && $EDITOR .env # set BINCIO_DATA_DIR
npm run dev
# Site dev server (render handles symlink + merge automatically)
uv run bincio render --serve
# Edit server (enables drawer + file upload in the site)
uv run bincio edit --data-dir ~/bincio_data # port 4041
# Set PUBLIC_EDIT_URL=http://localhost:4041 in site/.env
# Edit server (optional — enables edit drawer in the site)
uv run bincio edit --data-dir /tmp/bincio_test
# set PUBLIC_EDIT_URL=http://localhost:4041 in site/.env
# Tests
uv run pytest
@@ -164,551 +153,71 @@ vite: {
},
```
## StatsView heatmap — colour intensity scaling
Two approaches have been tried. The **active one is percentile-based** (preferred for now).
### Option A — Linear / max-relative (simpler, currently inactive)
```ts
$: maxDailyKm = Math.max(...[...byDate.values()].map(v => v / 1000), 1);
// inside cellColors loop:
const km = total / 1000;
const intensity = Math.min(0.12 + (km / maxDailyKm) * 0.88, 1.0);
```
- Busiest day = full brightness; all others scale linearly against it.
- Intuitive: you can visually read "this day was ~50% of my biggest day".
- Downside: one outlier (e.g. a 250 km day) compresses everything else into
near-darkness. Cross-sport comparison is unfair (10 km run vs 10 km cycling
look very different even when filtered to a single sport).
- Legend shows actual max km: `More (X km max)`.
### Option B — Percentile rank (active)
```ts
$: sortedDaily = [...byDate.values()].sort((a, b) => a - b);
function pctRank(value: number, sorted: number[]): number {
if (!sorted.length) return 0;
let lo = 0, hi = sorted.length;
while (lo < hi) { const mid = (lo + hi) >> 1; if (sorted[mid] <= value) lo = mid + 1; else hi = mid; }
return lo / sorted.length;
}
// inside cellColors loop:
const intensity = 0.12 + pctRank(total, sortedDaily) * 0.88;
```
- Each day is ranked against all other active days; the laziest active day =
intensity 0.12, the busiest = 1.0. The colour scale spreads evenly regardless
of km gaps.
- GitHub-contribution-graph style: easy to see "busy vs quiet" relative to
your own habits.
- Downside: absolute effort is not visible. A 5 km walk and a 200 km ride can
look the same if they're both 95th-percentile days for their respective sports.
- Legend says `More (percentile · max X km)` to hint at both dimensions.
### Shared infrastructure
- Blended colours: in "All" sport view, each cell's RGB is a weighted average
of sport colours by distance that day.
- `applyIntensity(hex, t)`: lerps from zinc-800 (#27272a = 39,39,42) to the
target colour, so dim cells fade into the background rather than going black.
- `$: cellColors = Map<string, string>` — precomputed reactively so Svelte
detects the dependency change when the sport filter or scale method changes
(plain function calls with static args don't trigger Svelte re-renders).
## ActivityCharts — controls and athlete zones
`ActivityCharts.svelte` renders Observable Plot charts for the activity detail page.
### Chart controls
- **Metric tabs**: Elevation · Speed · Heart Rate · Cadence · Power
- **Chart type toggle** (right-aligned): `↗ Line` | `▭ Hist`
- **X-axis toggle** (line mode only, shown when speed data present): `Time` | `Dist`
- Distance is integrated from `speed_kmh` at 1 Hz — no extra data needed.
- **Histogram controls** (visible only in histogram mode):
- **Dual range slider** — trims the x domain; two overlapping `<input type="range">` with CSS track highlight.
- **Bins slider** — exact bin count using explicit evenly-spaced thresholds (not d3's "nice" count, which ignores narrow ranges).
### Athlete zones
Zones are configured in `extract_config.yaml` under `athlete:` and written into
`index.json` at extract time (`owner.athlete`). The Astro activity page reads them
from the index and passes them down: `[id].astro` → `ActivityDetail` → `ActivityCharts`.
When viewing HR or Power in histogram mode, zone boundaries are drawn as dashed
vertical rules with Z1–Z5/Z7 labels at the top of the chart.
Labels and rules are clipped to the current trim range automatically.
Zone color palettes:
- HR (5 zones): `#60a5fa #4ade80 #facc15 #fb923c #f87171`
- Power (7 zones): `#60a5fa #34d399 #facc15 #fb923c #f87171 #c084fc #f43f5e`
### Zone calculation reference (Coggan)
| Zone | HR (% max HR) | Power (% FTP) |
|------|--------------|---------------|
| Z1 | < 55% | < 55% |
| Z2 | 55–75% | 55–75% |
| Z3 | 75–87% | 75–90% |
| Z4 | 87–93% | 90–105% |
| Z5 | > 93% | 105–120% |
| Z6 | — | 120–150% |
| Z7 | — | > 150% |
## Activity sidecar edits — design spec
Users edit activities via **sidecar markdown files** that live alongside BAS JSON in the data dir.
Users edit activities via **sidecar markdown files** in the data dir.
No database, no server — consistent with the project's static-files-only philosophy.
### File naming
```
~/bincio_data/
2024-05-15T10:30:00Z_cycling.json ← immutable extract output (never touched)
2024-05-15T10:30:00Z_cycling.md ← user edits (sidecar)
activities/{id}.json ← immutable extract output
edits/{id}.md ← user edits (sidecar)
edits/images/{id}/ ← uploaded photos
_merged/ ← render-time merge output (gitignored-style)
```
Same stem as the JSON, `.md` extension. `bincio extract` never writes `.md` files,
so re-running extract is always safe and will never clobber user edits.
### Sidecar format
YAML frontmatter + optional Markdown body:
```markdown
---
title: "Epic climb up Monte Grappa"
sport: cycling # override detected sport
hide_stats: [cadence] # suppress specific stat panels in detail view
highlight: true # pin/feature in feed (shown first, maybe badged)
private: false # exclude from public feed
gear: "Trek Domane" # freeform gear note
sport: cycling
hide_stats: [cadence]
highlight: true
private: false
gear: "Trek Domane"
---
Rode with Marco and Giulia. Legs felt great after the rest week...
Rode with friends. Legs felt great after the rest week...
```
- All frontmatter keys are optional; omit means "keep extracted value"
- The Markdown body becomes the activity's `description`, rendered as HTML in the detail page
- `hide_stats` takes stat panel names: `elevation`, `speed`, `heart_rate`, `cadence`, `power`
### Where overrides are applied: the render stage
The **render stage** (`bincio render`) is the right place — not extract, not the browser.
- Extract → clean BAS JSON (immutable)
- Render → merges sidecars → Astro build consumes enriched data
A `bincio.render.merge` module walks the data dir, finds `*.md` sidecars,
and produces either enriched JSON files or a separate `overrides/index.json`
that Astro reads at build time. The site never needs to fetch a `.md` file
at runtime — all merging is build-time, keeping the static-first guarantee.
### Federation angle
Sidecars work for *remote* activities too: if you include someone else's BAS feed,
you can write local `.md` sidecars for their activity IDs. Your render stage applies
your overrides on top of their data. This is a natural extension of the local case.
### Editing UX: drawer in Astro + `bincio edit` write API
The edit UI is a **slide-in drawer** (`EditDrawer.svelte`) in the Astro site.
The drawer fetches from and POSTs to the `bincio edit` FastAPI server (write API only —
the server no longer serves its own HTML UI).
- `bincio edit --data-dir ~/bincio_data` starts a FastAPI server on port 4041
- Set `PUBLIC_EDIT_URL=http://localhost:4041` in `site/.env` to enable the edit button
- Clicking Edit on any activity detail page opens a slide-in drawer
- Saving writes the sidecar and triggers `merge_all()` automatically
- `bincio render` always runs `merge_all()` before build/serve and symlinks `public/data` → `_merged/`
**How it works:**
### `PUBLIC_EDIT_URL` as feature flag
```
bincio render --serve # Astro dev server, port 4321
bincio edit --data-dir ~/… # write API only, port 4041
```
- Edit button appears on the activity detail page **only when `PUBLIC_EDIT_URL` is set** in `site/.env`
- Clicking Edit opens the drawer in the same page — no navigation, no copy-pasting IDs
- Drawer fetches `GET /api/activity/{id}` to pre-fill, `POST /api/activity/{id}` to save
- After save: server runs `merge_all()` automatically → Astro serves updated data immediately on refresh
- Closing the drawer applies `title` + `description` changes optimistically to the local page state
(no full reload required to see the text change)
**`PUBLIC_EDIT_URL` as feature flag:**
- **Unset** → no Edit button, no drawer. Works as a normal static site. Safe for public hosting.
- **Set** (e.g. `http://localhost:4041`) → editing enabled. Lives in `site/.env` (gitignored).
Each deployment opts in explicitly.
**Edit server API (`bincio edit --data-dir <dir>`):**
- `GET /api/activity/{id}` — current values (sidecar overrides layered on BAS JSON)
- `POST /api/activity/{id}` — write sidecar `.md`, trigger `merge_all()`
- `POST /api/activity/{id}/images` — multipart upload → `edits/images/{id}/{filename}`
- `DELETE /api/activity/{id}/images/{filename}` — remove uploaded image
**Edit drawer features:**
- Title, sport dropdown, gear
- Markdown textarea for description (images inserted as `![name](filename)` references)
- Image drag-and-drop zone with chip list + delete
- Hide stat panels (elevation, speed, heart_rate, cadence, power) — toggle buttons
- Highlight flag (★ — sorts to top of feed, visual badge)
- Private flag (⊘ — suppressed from index at render time)
### Image storage and serving
```
~/bincio_data/
edits/
2024-05-15T10:30:00Z_cycling.md
images/
2024-05-15T10:30:00Z_cycling/
col-summit.jpg
group-photo.jpg
```
Images are referenced in the markdown body with relative paths: `![Summit](col-summit.jpg)`.
`merge_all()` symlinks `edits/images/{id}/` → `_merged/activities/images/{id}/` so images
are served at `data/activities/images/{id}/{filename}` by the Astro dev server.
`ActivityDetail.svelte` rewrites relative image paths to this URL when rendering markdown.
**Note:** browsers cannot display `.HEIC` files. Convert to JPEG/PNG first:
`sips -s format jpeg photo.HEIC --out photo.jpg` (macOS).
### Decided
- **Sidecar location**: `edits/` subdirectory — cleaner, easier to backup/sync independently
- **Merge output**: `data/_merged/` — extracted data stays pristine; `public/data` → `_merged/`
- **`private: true`**: suppressed from `index.json` at render time (not client-side hide)
- **`highlight`**: sorts to top of feed; visual badge TBD
- **Edit UI**: drawer in Astro site, `bincio edit` is a pure write API (no HTML serving)
## Athlete page — design plan
### Goal
A `/athlete` page (and `/athlete/edit` drawer) giving the user:
1. **Performance analytics** — power curve (MMP), best efforts, optionally fitness/freshness
2. **Profile editing** — zones, gear (bikes/shoes), personal data — no YAML editing required
### Mean Maximal Power (MMP) curve
For every duration D, the MMP is the highest average power sustained over any contiguous
D-second window across all activities. Plotted on a log-scale x-axis.
**Key features:**
- **Time range filter**: all-time, last 30/90/365 days, or user-defined seasons
- **Season overlay**: multiple seasons plotted on the same chart for comparison
(e.g. "2023 vs 2024 vs 2025" — this is the primary use case)
- **Durations**: a fixed log-scale set, e.g.:
`1, 2, 5, 10, 15, 20, 30, 60, 120, 180, 300, 600, 1200, 1800, 3600` seconds
- **Null handling**: if an activity is shorter than duration D, it contributes nothing
to that point. No interpolation. The curve simply ends where data runs out.
- **Modelled curve overlay** (future): 2-parameter Critical Power model fitted to
the data; shows predicted W for any duration, even beyond recorded efforts.
**Where to compute:**
At **extract time**, each activity gets an `mmp` array:
```json
"mmp": [[1, 850], [5, 720], [30, 580], [300, 340], [3600, 210]]
```
Each pair is `[duration_s, avg_watts]`. Only activities with power data get this field.
The site then takes the **element-wise max** across all activities (filtered by date range).
This keeps the site fully static — no server needed to render the curve.
Computing MMP per activity is O(n × D) where n = timeseries length, D = number of
duration points (~15). At 1 Hz, a 2-hour ride is 7200 points × 15 durations = trivial.
Use a sliding window approach: for each duration d, maintain a running sum and advance
the window one sample at a time.
**Season definition** (user-configurable):
```yaml
athlete:
seasons:
- name: "2025"
start: "2025-01-01"
end: "2025-12-31"
- name: "2024"
start: "2024-01-01"
end: "2024-12-31"
```
If no seasons defined, the UI offers fixed presets (last 30d / 90d / 365d / all-time).
### Athlete profile editing — reusing edit infrastructure
Same pattern as activity editing:
```
bincio edit --data-dir ~/bincio_data # same server, new endpoints
```
New API endpoints:
- `GET /api/athlete` — current athlete config (zones, gear, display name)
- `POST /api/athlete` — write `edits/athlete.yaml`, trigger `merge_all()`
`edits/athlete.yaml` format:
```yaml
display_name: "Davide"
handle: "brutsalvadi"
max_hr: 190
ftp_w: 210
hr_zones:
- [0, 104]
- [104, 142]
- [142, 165]
- [165, 176]
- [176, 999]
power_zones:
- [0, 115]
# ...
gear:
bikes:
- name: "Trek Domane"
type: cycling
notes: "Road endurance"
shoes:
- name: "Asics GT-2000"
type: running
seasons:
- name: "2025"
start: "2025-01-01"
end: "2025-12-31"
```
The server reads `extract_config.yaml` as base defaults, applies `edits/athlete.yaml`
overrides on top, and writes back to `edits/athlete.yaml` on POST. The `extract_config.yaml`
is never written by the server — it stays as the authoritative static config.
`merge_all()` also writes athlete data into `_merged/athlete.json` which the site reads.
### AthleteDrawer.svelte (profile editing)
Reuses the same drawer pattern as `EditDrawer.svelte`:
- Number inputs for `max_hr`, `ftp_w`
- Zone editor: table of rows `[lo, hi]` with + / − buttons; auto-fills `lo` from previous `hi`
- Gear list: add/remove bikes and shoes; name + type + notes fields
- Season list: add/remove date ranges with names
### Site page: `/athlete`
Two tabs or sections:
1. **Performance** — MMP curve chart (Observable Plot, log x-axis), date range selector
2. **Profile** — display of current zones, gear list; Edit button opens AthleteDrawer
The MMP chart uses `index.json`'s `activities` array (already loaded by the feed) — filter
to power-having activities, pull their `mmp` arrays, take element-wise max per season.
### Implementation order
1. Add `mmp` computation to `metrics.py` and writer
2. Add `mmp` field to BAS schema and `types.ts`
3. Add `/api/athlete` GET+POST to the edit server
4. `merge_all()` writes `_merged/athlete.json`
5. Astro page `site/src/pages/athlete/index.astro`
6. `MmpChart.svelte` — Observable Plot line, log-scale x, multi-season overlay
7. `AthleteDrawer.svelte` — zones + gear editing form
8. Season config in `extract_config.yaml` / `edits/athlete.yaml`
## Data ingestion
How activity data gets into BincioActivity. Three orthogonal vectors.
### Vector 1 — Web file upload ✓
Drag a FIT/GPX/TCX onto the site while the edit server is running — activity
appears immediately.
**Backend** (`bincio/edit/server.py`):
```
POST /api/upload multipart FIT/GPX/TCX (also .gz variants)
→ stages file to data_dir/_uploads/<name>
→ parse_file() → compute() → write_activity() → build_summary()
→ 409 if activity already exists (same timestamp = same ID)
→ updates index.json + merge_all()
→ returns { ok: true, id: "2024-05-15T103000Z" }
→ cleans up staged file in finally block
```
**Frontend** (`site/src/layouts/Base.astro`):
- `↑` button in nav right cluster, only rendered when `PUBLIC_EDIT_URL` is set
- Modal with drag-and-drop zone + click-to-browse
- Auto-redirects to `/activity/{id}/` on success
- Escape / backdrop click closes modal
### Vector 2 — `bincio import strava` ✓
**`bincio/import_/strava.py`** + **`bincio/import_/cli.py`**
Install: `uv sync --extra strava`
```bash
# First run (full sync — opens browser):
bincio import strava --client-id 12345 --client-secret abc --output ~/bincio_data
# Subsequent runs (incremental — picks up from last sync automatically):
bincio import strava --client-id 12345 --client-secret abc
# Explicit date range:
bincio import strava --client-id 12345 --client-secret abc --since 2025-01-01
# Force re-auth (rotate credentials or re-authorize):
bincio import strava --client-id 12345 --client-secret abc --reauth
# Credentials via env vars (good for scripts):
export STRAVA_CLIENT_ID=12345
export STRAVA_CLIENT_SECRET=abc
bincio import strava --output ~/bincio_data
```
**Getting Strava API credentials (~2 minutes, no approval needed):**
1. Go to `strava.com/settings/api`
2. Create an application — name and website can be anything; set
**Authorization Callback Domain** to `localhost`
3. Paste Client ID and Client Secret into `extract_config.yaml`:
```yaml
import:
strava:
client_id: 12345
client_secret: your_secret_here
```
`extract_config.yaml` is gitignored — safe to store credentials there.
Strava's "developer" label is misleading: formal review is only required for
commercial apps that authenticate *other users*. For a personal self-hosted tool
you authenticate your own account — no review, no fees.
Rate limits: **100 req / 15 min, 1000 / day** (generous for personal use).
**How the importer works:**
*OAuth dance (first run):*
- Starts a one-shot local HTTP server on port 8976
- Opens `strava.com/oauth/authorize?scope=activity:read_all` in the browser
- Receives the authorization code at `/callback`
- Exchanges code for access + refresh tokens
- Saves to `~/.config/bincio/strava.json` (keyed by client_id)
- Subsequent runs load saved tokens and refresh silently when expired (6h TTL)
*Sync loop:*
- Reads `data_dir/_strava_sync.json` for set of already-imported Strava IDs
and timestamp of last sync
- Uses Strava `after=<unix_ts>` parameter for server-side filtering (efficient —
no need to scan all pages on incremental runs; 1-hour overlap to catch late saves)
- Per activity: `GET /activities/{id}/streams` → `_strava_to_parsed()` →
`compute()` → `_patch_from_summary()` → `write_activity()` → `build_summary()`
- Writes updated `index.json` + `_strava_sync.json`
- Calls `merge_all()` if `edits/` directory exists
*Conversion details:*
- `sport_type` (or `type`) → `normalise_sport()` — same mapping as FIT/GPX
- Streams: `time` (s since start) + `latlng` + `altitude` + `heartrate` +
`cadence` + `watts` + `velocity_smooth` (m/s → km/h) → `DataPoint` list
- `source_hash`: `sha256("strava:{id}")` — stable, not file-content-based
- `_patch_from_summary()`: fills `None` metric fields (distance, duration,
elevation, HR, power) from the Strava activity summary for activities with
missing/sparse sensors or manual entries
- Rate limit: warns at 85% of 15-min window; auto-retries with 60s sleep on 429
### Vector 3 — Platform watch mode (planned)
```bash
bincio extract --watch ~/Dropbox/Garmin/Activities --output ~/bincio_data
```
Directory watcher (`watchfiles` / `inotify`) for ongoing FIT sync from Karoo,
Garmin, Wahoo. New file → auto-extract → merge. Not yet implemented.
### Vector 4 — Federation (planned)
```yaml
# extract_config.yaml
sources:
- url: https://alice.example.com/data/index.json
handle: alice
```
`bincio render` fetches remote BAS index files at build time. No API keys.
Local `.md` sidecars can annotate remote activities.
---
- **Unset** → no Edit button, normal static site
- **Set** → edit drawer enabled; lives in `site/.env` (gitignored)
## Known issues / next steps
- `bincio render` Python CLI is a stub — site is built via `npm run build` directly
- Activity IDs in existing test data still use `+0000` format (pre-fix); re-run extract to get `Z` format
- `bincio render` Python CLI is functional but `--watch` mode not yet implemented
- Activity IDs in older test data may use `+0000` format (pre-fix); re-run extract to get `Z` format
- Some activities appear with both untitled and titled IDs (near-dedup timing race)
- Stats page heatmap month labels are embedded in the week-column flex grid (fixed March 2026); `getWeeks` uses `localISO()` not `toISOString()` to avoid UTC/local date mismatch
- Federation (remote data sources) not yet implemented in site
- Friends pages (`/friends/{handle}/`) not yet implemented
- `bincio render` should automate: symlink data → `astro build`
- The `site/.env` file is gitignored — document the setup for new users
- Add `--workers` benchmark: on 8 cores, ~7 min for 3,200 activities first run
## URL state — design pattern
All pages with filter or tab state persist it in the URL query string so the browser
back button restores the exact view. The pattern used in every component:
```ts
let mounted = false;
$: if (mounted) {
const params = new URLSearchParams(window.location.search);
if (value === DEFAULT) params.delete('key'); else params.set('key', value);
const qs = params.toString();
history.replaceState(null, '', qs ? `?${qs}` : window.location.pathname);
}
onMount(() => {
value = new URLSearchParams(window.location.search).get('key') ?? DEFAULT;
mounted = true;
});
```
Key decisions:
- `replaceState` not `pushState` — clicking filters doesn't create history entries; back always goes to the previous *page*, not previous filter state
- `mounted` flag — prevents the reactive block from firing before `onMount` reads the URL (which would overwrite the URL with the default before reading it)
- Default values omitted — `?sport=all` and `?tab=power` are never written; clean URLs
Current URL params:
| Page | Param | Values |
|------|-------|--------|
| `/` (feed) | `sport` | `cycling`, `running`, etc. |
| `/stats/` | `sport` | same |
| `/athlete/` | `tab` | `records`, `profile` (power is default) |
| `/athlete/?tab=records` | `sport` | `cycling`, `swimming`, etc. (running is default) |
If you add a new filter to any page, follow this pattern. Never use `pushState` for filter changes.
- The `site/.env` file is gitignored — copy from `site/.env.example`
## What "good" looks like (not yet done)
- [ ] `bincio render` Python CLI wraps `astro build` properly
- [ ] Friends/federation pages in site
- [ ] Athlete page: MMP power curve with season overlay
- [ ] Athlete page: profile editor (zones, gear, seasons) via AthleteDrawer
- [ ] MMP computation at extract time → `mmp` field in BAS JSON
- [ ] Personal records page (best efforts: 5km, 10km, etc.)
- [ ] Personal records page
- [ ] Activity search / full-text filter in feed
- [ ] Map thumbnail in activity cards (SVG path from GeoJSON)
- [ ] GitHub Actions template for auto-publish
- [x] **Ingestion: web file upload** — `POST /api/upload` in edit server, drag-and-drop in nav
- [x] **Ingestion: `bincio import strava`** — OAuth2 + streams API, idempotent incremental sync
- [ ] **Ingestion: `bincio extract --watch`** — directory watcher for ongoing FIT sync
- [ ] **Ingestion: `bincio import garmin`** — garminconnect library or FIT folder sync
- [ ] **Ingestion: federation** — `sources:` in config, remote BAS index pull at render time
- [ ] Karoo/Garmin Connect importers beyond Strava
- [x] `bincio.render.merge` — sidecar parser, `_merged/` output, private filter, highlight sort
- [x] `bincio edit` FastAPI write API (GET/POST activity, image upload/delete, triggers merge)
- [x] `EditDrawer.svelte` — slide-in edit UI in the Astro site (no separate HTML from server)
- [x] `PUBLIC_EDIT_URL` feature flag — unset = no edit UI, set = drawer enabled
- [x] `EditDrawer.svelte` — slide-in edit UI in the Astro site
- [x] `PUBLIC_EDIT_URL` feature flag
- [x] Markdown rendering in activity description with image path rewriting
- [x] `hide_stats` support in activity detail stats panel
- [x] ActivityCharts power tab (elevation/speed/HR/cadence/power)
- [x] Chart type toggle: line ↔ histogram
- [x] X-axis toggle: time ↔ distance (integrated from speed)
- [x] Histogram dual range slider + bins slider (exact thresholds)
- [x] Athlete zones in `extract_config.yaml` → `index.json` → chart overlays
- [x] StatsView heatmap click-to-pin tooltip (Esc / click-outside to dismiss)
- [x] Photo gallery with lightbox on activity detail page
- [ ] `bincio render --watch` incremental rebuild on sidecar/data changes
- [ ] Highlight badge in activity feed cards
- [ ] Image format warning (HEIC → JPEG conversion hint in the upload UI)
- [ ] HR / power zone defaults from `max_hr` / `ftp_w` when explicit zones not set
extract_config.example.yaml
+3 -32
View File
@@ -4,10 +4,9 @@ owner:
input:
dirs:
- ~/Activities # add as many dirs as needed; scanned recursively
# Optional — only needed for Strava bulk exports.
# Provides activity titles, descriptions, and authoritative sport type.
# If you only have device files (Garmin, Wahoo, Karoo, etc.) you can delete this line.
- ~/Activities/gpx
- ~/Activities/fit
# Strava bulk export metadata — provides names, descriptions, gear
# metadata_csv: ~/strava_export/activities.csv
output:
@@ -30,31 +29,3 @@ classifier:
enabled: false # ML activity type classifier (requires scikit-learn extra)
incremental: true # skip files whose hash hasn't changed since last run
# ── Platform importers ─────────────────────────────────────────────────────────
# Credentials for `bincio import strava`.
# Get them from strava.com/settings/api (2 minutes, no approval needed).
# Authorization Callback Domain must be set to: localhost
# import:
# strava:
# client_id: 12345
# client_secret: your_client_secret_here
# ── Athlete zones ───────────────────────────────────────────────────────────────
# athlete:
# max_hr: 182 # used to derive default HR zone display
# ftp_w: 280 # functional threshold power in watts
# hr_zones: # explicit bpm boundaries [[lo, hi], ...]
# - [0, 115] # Z1 recovery
# - [115, 137] # Z2 endurance
# - [137, 155] # Z3 tempo
# - [155, 169] # Z4 threshold
# - [169, 999] # Z5 VO2max
# power_zones: # explicit watt boundaries
# - [0, 168] # Z1 active recovery (< 55% FTP)
# - [168, 224] # Z2 endurance (55–78%)
# - [224, 266] # Z3 tempo (78–95%)
# - [266, 308] # Z4 threshold (95–109%)
# - [308, 364] # Z5 VO2max (109–130%)
# - [364, 420] # Z6 anaerobic (130–150%)
# - [420, 9999] # Z7 neuromuscular (> 150%)
extract_config.yaml
-55
View File
@@ -1,55 +0,0 @@
owner:
handle: brutsalvadi
display_name: Bru
input:
dirs:
- ~/src/cycling_data_davide/activities
- ~/src/cycling_data_davide/Karoo_2026
- ~/src/cycling_data_davide/Karoo
# Strava bulk export metadata — provides names, descriptions, gear
metadata_csv: ~/src/cycling_data_davide/activities.csv
output:
dir: ~/src/bincio_data
default_privacy: public
sensors:
heart_rate: true
cadence: true
temperature: true
power: true
track:
simplify: rdp
rdp_epsilon: 0.0001 # ~11m at equator
timeseries_hz: 1 # 1 sample/second max
classifier:
enabled: false # ML activity type classifier (requires scikit-learn extra)
incremental: true # skip files whose hash hasn't changed since last run
import:
strava:
client_id: 218194 # paste your Client ID from strava.com/settings/api
client_secret: e787e62357c80c754e4f6129bfdfb849c4c03490 # paste your Client Secret
athlete:
max_hr: 190
ftp_w: 210
hr_zones: # 5-zone Coggan, % of max HR 190 bpm
- [0, 104] # Z1 recovery < 55%
- [104, 142] # Z2 endurance 55–75%
- [142, 165] # Z3 tempo 75–87%
- [165, 176] # Z4 threshold 87–93%
- [176, 999] # Z5 VO2max > 93%
power_zones: # 7-zone Coggan, % of FTP 210 W
- [0, 115] # Z1 active recovery < 55%
- [115, 157] # Z2 endurance 55–75%
- [157, 189] # Z3 tempo 75–90%
- [189, 220] # Z4 threshold 90–105%
- [220, 252] # Z5 VO2max 105–120%
- [252, 315] # Z6 anaerobic 120–150%
- [315, 9999] # Z7 neuromuscular > 150%
scripts/__init__.py
View File
scripts/backfill.py
-225
View File
@@ -1,225 +0,0 @@
"""Backfill MMP and best-effort records into existing BAS activity JSONs.
Reads 1Hz timeseries (power_w, speed_kmh, elevation_m) from already-extracted
detail JSONs — no need to re-parse source FIT/GPX/TCX files.
Run once after upgrading to the MMP + records extract pipeline, or whenever
the computation logic changes and you want to refresh all activities.
Usage:
uv run python scripts/backfill.py [--data-dir ~/src/bincio_data]
"""
import json
import sys
from pathlib import Path
import click
from rich.console import Console
from rich.progress import BarColumn, MofNCompleteColumn, Progress, TextColumn, TimeElapsedColumn
console = Console()
# ── MMP ───────────────────────────────────────────────────────────────────────
MMP_DURATIONS_S = [1, 2, 5, 10, 15, 20, 30, 60, 120, 180, 300, 600, 1200, 1800, 3600]
def _compute_mmp(power_w: list) -> list[list[int]] | None:
samples = [w for w in power_w if w is not None]
if len(samples) < 2:
return None
n = len(samples)
results = []
for d in MMP_DURATIONS_S:
if d > n:
break
window_sum = sum(samples[:d])
best = window_sum
for i in range(1, n - d + 1):
window_sum += samples[i + d - 1] - samples[i - 1]
if window_sum > best:
best = window_sum
results.append([d, round(best / d)])
return results if results else None
# ── Best efforts ──────────────────────────────────────────────────────────────
BEST_EFFORT_DISTANCES: dict[str, list[float]] = {
"running": [0.4, 1.0, 1.609, 5.0, 10.0, 21.097, 42.195],
"cycling": [5.0, 10.0, 20.0, 50.0, 100.0],
"swimming": [0.1, 0.2, 0.5, 1.0, 2.0],
}
def _fastest_time(speed_kmh: list, target_km: float) -> int | None:
left = 0
window_dist = 0.0
best_s = None
for right, spd in enumerate(speed_kmh):
window_dist += (spd or 0.0) / 3600.0
while window_dist >= target_km and left <= right:
window_s = right - left + 1
if best_s is None or window_s < best_s:
best_s = window_s
window_dist -= (speed_kmh[left] or 0.0) / 3600.0
left += 1
return best_s
def _compute_best_efforts(speed_kmh: list, sport: str) -> list[list[float]] | None:
targets = BEST_EFFORT_DISTANCES.get(sport, [])
if not targets or not speed_kmh:
return None
results = []
for d_km in targets:
t_s = _fastest_time(speed_kmh, d_km)
if t_s is not None:
results.append([d_km, t_s])
return results if results else None
def _compute_best_climb(elevation_m: list) -> float | None:
valid = [e for e in elevation_m if e is not None]
if len(valid) < 2:
return None
max_gain = current = 0.0
for a, b in zip(valid, valid[1:]):
current = max(0.0, current + (b - a))
if current > max_gain:
max_gain = current
return round(max_gain, 1) if max_gain > 0 else None
# ── Main ──────────────────────────────────────────────────────────────────────
@click.command()
@click.option("--data-dir", default="~/src/bincio_data", show_default=True)
@click.option("--dry-run", is_flag=True)
@click.option("--force", is_flag=True, help="Recompute even if fields already present.")
def main(data_dir: str, dry_run: bool, force: bool) -> None:
"""Backfill mmp, best_efforts, and best_climb_m into existing activity JSONs."""
data = Path(data_dir).expanduser()
acts_dir = data / "activities"
if not acts_dir.exists():
console.print(f"[red]Activities dir not found: {acts_dir}[/red]")
sys.exit(1)
jsons = sorted(acts_dir.glob("*.json"))
console.print(f"Found [bold]{len(jsons)}[/bold] activity JSONs in {acts_dir}")
updated = skipped = 0
with Progress(
TextColumn("[progress.description]{task.description}"),
BarColumn(), MofNCompleteColumn(), TimeElapsedColumn(),
console=console,
) as progress:
task = progress.add_task("Backfilling…", total=len(jsons))
for path in jsons:
progress.advance(task)
try:
detail = json.loads(path.read_text())
except Exception:
skipped += 1
continue
already_done = (
detail.get("mmp") is not None
and detail.get("best_efforts") is not None
or detail.get("best_efforts") == [] # explicitly empty = computed, no results
)
if already_done and not force:
skipped += 1
continue
sport = detail.get("sport", "other")
ts = detail.get("timeseries") or {}
power_w = ts.get("power_w") or []
speed_kmh = ts.get("speed_kmh") or []
ele_m = ts.get("elevation_m") or []
changed = False
if detail.get("mmp") is None or force:
mmp = _compute_mmp(power_w)
if mmp is not None:
detail["mmp"] = mmp
changed = True
if detail.get("best_efforts") is None or force:
be = _compute_best_efforts(speed_kmh, sport)
detail["best_efforts"] = be # store None or list (None = sport has no targets)
changed = True
if (detail.get("best_climb_m") is None or force) and sport == "cycling":
bc = _compute_best_climb(ele_m)
if bc is not None:
detail["best_climb_m"] = bc
changed = True
if changed:
if not dry_run:
path.write_text(json.dumps(detail, indent=2, ensure_ascii=False))
updated += 1
console.print(
f"\n[green]Done.[/green] "
f"Updated [bold]{updated}[/bold], skipped [bold]{skipped}[/bold]."
)
if dry_run:
console.print("[yellow]Dry run — nothing written.[/yellow]")
return
# Patch index.json summaries
console.print("Patching index.json summaries…")
index_path = data / "index.json"
index = json.loads(index_path.read_text())
lookup: dict[str, dict] = {}
for path in acts_dir.glob("*.json"):
try:
d = json.loads(path.read_text())
lookup[d["id"]] = {
"mmp": d.get("mmp"),
"best_efforts": d.get("best_efforts"),
"best_climb_m": d.get("best_climb_m"),
}
except Exception:
pass
patched = 0
for s in index.get("activities", []):
row = lookup.get(s["id"])
if not row:
continue
if row.get("mmp") and not s.get("mmp"):
s["mmp"] = row["mmp"]; patched += 1
if row.get("best_efforts") is not None and s.get("best_efforts") is None:
s["best_efforts"] = row["best_efforts"]; patched += 1
if row.get("best_climb_m") and not s.get("best_climb_m"):
s["best_climb_m"] = row["best_climb_m"]; patched += 1
index_path.write_text(json.dumps(index, indent=2, ensure_ascii=False))
console.print(f" {patched} fields patched in index.json.")
# Rebuild athlete.json
console.print("Rebuilding athlete.json…")
from bincio.extract.writer import write_athlete_json
owner = index.get("owner", {})
athlete_cfg = {k: v for k, v in (owner.get("athlete") or {}).items() if v is not None}
write_athlete_json(index.get("activities", []), data, athlete_cfg)
console.print(" athlete.json written.")
# Re-merge
console.print("Running merge_all…")
from bincio.render.merge import merge_all
n = merge_all(data)
console.print(f" merge_all done ({n} sidecars).")
if __name__ == "__main__":
main()
scripts/backfill_mmp.py
-146
View File
@@ -1,146 +0,0 @@
"""Backfill MMP curve into existing BAS activity JSONs and index.json.
Reads power_w from the already-extracted 1Hz timeseries — no need to re-parse
source FIT files. Run once after upgrading to the MMP-enabled extract pipeline.
Usage:
uv run python scripts/backfill_mmp.py [--data-dir ~/src/bincio_data]
"""
import json
import sys
from pathlib import Path
import click
from rich.console import Console
from rich.progress import BarColumn, MofNCompleteColumn, Progress, TextColumn, TimeElapsedColumn
console = Console()
MMP_DURATIONS_S = [1, 2, 5, 10, 15, 20, 30, 60, 120, 180, 300, 600, 1200, 1800, 3600]
def compute_mmp_from_timeseries(power_w: list) -> list[list[int]] | None:
"""Compute MMP from an already-extracted 1Hz power array (nulls dropped)."""
samples = [w for w in power_w if w is not None]
if len(samples) < 2:
return None
n = len(samples)
results = []
for d in MMP_DURATIONS_S:
if d > n:
break
window_sum = sum(samples[:d])
best = window_sum
for i in range(1, n - d + 1):
window_sum += samples[i + d - 1] - samples[i - 1]
if window_sum > best:
best = window_sum
results.append([d, round(best / d)])
return results if results else None
@click.command()
@click.option("--data-dir", default="~/src/bincio_data", show_default=True,
help="Path to the BAS data directory.")
@click.option("--dry-run", is_flag=True, help="Compute but don't write anything.")
def main(data_dir: str, dry_run: bool) -> None:
"""Backfill mmp field into existing activity JSONs and index.json."""
data = Path(data_dir).expanduser()
acts_dir = data / "activities"
if not acts_dir.exists():
console.print(f"[red]Activities dir not found: {acts_dir}[/red]")
sys.exit(1)
jsons = sorted(acts_dir.glob("*.json"))
console.print(f"Found [bold]{len(jsons)}[/bold] activity JSONs in {acts_dir}")
updated = skipped = no_power = 0
with Progress(
TextColumn("[progress.description]{task.description}"),
BarColumn(), MofNCompleteColumn(), TimeElapsedColumn(),
console=console,
) as progress:
task = progress.add_task("Backfilling MMP…", total=len(jsons))
for path in jsons:
progress.advance(task)
try:
detail = json.loads(path.read_text())
except Exception:
skipped += 1
continue
# Skip if already has mmp
if detail.get("mmp") is not None:
skipped += 1
continue
ts = detail.get("timeseries") or {}
power_w = ts.get("power_w") or []
mmp = compute_mmp_from_timeseries(power_w)
if not mmp:
no_power += 1
continue
detail["mmp"] = mmp
if not dry_run:
path.write_text(json.dumps(detail, indent=2, ensure_ascii=False))
updated += 1
console.print(
f"\n[green]Done.[/green] "
f"Updated [bold]{updated}[/bold], "
f"already had mmp [bold]{skipped}[/bold], "
f"no power data [bold]{no_power}[/bold]."
)
if dry_run:
console.print("[yellow]Dry run — nothing written.[/yellow]")
return
# Rebuild index.json summaries with mmp
console.print("Updating index.json summaries…")
index_path = data / "index.json"
if index_path.exists():
index = json.loads(index_path.read_text())
# Build a lookup from the now-updated detail JSONs
mmp_by_id: dict[str, list] = {}
for path in acts_dir.glob("*.json"):
try:
d = json.loads(path.read_text())
if d.get("mmp"):
mmp_by_id[d["id"]] = d["mmp"]
except Exception:
pass
for s in index.get("activities", []):
if s["id"] in mmp_by_id and not s.get("mmp"):
s["mmp"] = mmp_by_id[s["id"]]
index_path.write_text(json.dumps(index, indent=2, ensure_ascii=False))
console.print(f" Patched {len(mmp_by_id)} summaries in index.json.")
# Rebuild athlete.json
console.print("Rebuilding athlete.json…")
from bincio.extract.writer import write_athlete_json
index = json.loads(index_path.read_text())
owner = index.get("owner", {})
athlete_cfg = {k: v for k, v in (owner.get("athlete") or {}).items() if v is not None}
write_athlete_json(index.get("activities", []), data, athlete_cfg)
console.print(" athlete.json written.")
# Re-run merge_all so _merged picks up the changes
console.print("Running merge_all…")
from bincio.render.merge import merge_all
n = merge_all(data)
console.print(f" merge_all done ({n} sidecars).")
if __name__ == "__main__":
main()
site/.env
-2
View File
@@ -1,2 +0,0 @@
BINCIO_DATA_DIR=/tmp/bincio_test
PUBLIC_EDIT_URL=http://localhost:4041