docs: expand mobile app design — hybrid extraction, Karoo integration, platform independence vision
This commit is contained in:
Davide Scaini
+214
-84
@@ -1,15 +1,44 @@
|
||||
# Bincio Mobile App — Design Document
|
||||
|
||||
## Vision
|
||||
|
||||
The long-term goal is full independence from Garmin Connect, Strava, and similar
|
||||
platforms. Today those platforms act as mandatory intermediaries: your device
|
||||
syncs to their cloud, you authorise third parties to pull from their API, and your
|
||||
data effectively lives on their servers.
|
||||
|
||||
The Bincio mobile app removes that dependency:
|
||||
|
||||
- Your FIT/GPX/TCX files live on your device.
|
||||
- The app reads them directly — no platform sync required.
|
||||
- A Bincio instance (bincio.org or self-hosted) is an optional upgrade for backup,
|
||||
sharing, and web access — not a prerequisite.
|
||||
- Devices like the **Karoo 2** (Android-based) are a first-class target: activities
|
||||
are already saved locally as FIT files, so the app can pick them up directly from
|
||||
the filesystem without any export step.
|
||||
|
||||
---
|
||||
|
||||
## Philosophy
|
||||
|
||||
The Bincio mobile app follows a **local-first** model:
|
||||
**Local-first.** All activity data lives on the device. The app works fully offline
|
||||
— no account, no internet connection, no platform authorisation required.
|
||||
|
||||
- All activity data lives on the device. The app works fully offline — no account or internet connection required.
|
||||
- An online instance (bincio.org or a self-hosted server) is an optional upgrade, not a prerequisite.
|
||||
- Activities are stored in the open **BAS format** (the same JSON schema the server uses), so data is always portable and readable without the app.
|
||||
- Sync is explicit and user-initiated — the app never silently overwrites local data.
|
||||
**Original files as source of truth.** The raw FIT/GPX/TCX file is always stored on
|
||||
device alongside the extracted BAS JSON. This means:
|
||||
|
||||
The goal is a personal activity log that you own. The cloud is a backup and a sharing mechanism, not the source of truth.
|
||||
- You can re-extract at any time (e.g. when the algorithm improves, or to apply DEM
|
||||
correction after connecting to an instance).
|
||||
- Sync to a remote instance is just pushing the original file — the server
|
||||
re-extracts with the full Python pipeline.
|
||||
- No data is ever locked into a proprietary representation.
|
||||
|
||||
**Sync is optional and explicit.** Connecting to a Bincio instance (bincio.org or
|
||||
self-hosted) adds cloud backup, the web feed, and the ability to share activities.
|
||||
The app never silently overwrites local data. Sync is user-initiated.
|
||||
|
||||
**Open format.** Activities are stored in the BAS schema (the same JSON format the
|
||||
server uses). Any tool — in any language — can read them.
|
||||
|
||||
---
|
||||
|
||||
@@ -20,12 +49,12 @@ Several pieces of the mobile app are already implemented or proven:
|
||||
| Piece | Where | Notes |
|
||||
|---|---|---|
|
||||
| BAS schema | `docs/schema.md` | The on-device data format — identical to the server format |
|
||||
| In-browser FIT/GPX/TCX parsing | `site/src/pages/convert/` | Pyodide + the Python extractor running in a browser tab. Proves the extraction works without a server. Not suitable for mobile (Pyodide is 30 MB, browser-only). |
|
||||
| Local activity storage | `site/src/pages/convert/` | IndexedDB + service worker serving local activities in the feed. Proves the concept; needs a native equivalent. |
|
||||
| Content-addressed dedup | `bincio/extract/dedup.py` | `source_hash` (SHA-256 of raw file) already prevents duplicates on upload |
|
||||
| In-browser FIT/GPX/TCX parsing | `site/src/pages/convert/` | Pyodide + the Python extractor running in a browser tab. Proves local extraction works. Not portable to mobile (Pyodide is 30 MB, browser-only). |
|
||||
| Local activity storage | `site/src/pages/convert/` | IndexedDB + service worker in the web app. Proves the concept; the mobile app uses SQLite instead. |
|
||||
| Content-addressed dedup | `bincio/extract/dedup.py` | `source_hash` (SHA-256 of raw file) prevents duplicates on upload |
|
||||
| Sync-ready REST API | `bincio/serve/server.py` | Login, upload, activity detail, index.json — the sync primitives are already there |
|
||||
| Settings persistence | `bincio/serve/db.py` | `settings` table (key/value) in the multi-user SQLite DB; the mobile_app branch added this |
|
||||
| Elevation algorithms | `bincio/extract/metrics.py`, `bincio/extract/dem.py` | Hysteresis accumulation and DEM correction — need a TypeScript port |
|
||||
| Settings persistence | `bincio/serve/db.py` | `settings` table (key/value) for instance URL, auth token, sync preferences |
|
||||
| Elevation algorithms | `bincio/extract/metrics.py`, `bincio/extract/dem.py` | Hysteresis and DEM correction — need a TypeScript port for offline use |
|
||||
|
||||
---
|
||||
|
||||
@@ -36,33 +65,70 @@ Several pieces of the mobile app are already implemented or proven:
|
||||
**Expo** (React Native + the Expo SDK) is the recommended platform:
|
||||
|
||||
- TypeScript-first, large ecosystem
|
||||
- `expo-sqlite` (v2+) provides a fast on-device SQLite database with WAL mode — good fit for BAS-style JSON blobs
|
||||
- File picking and sharing: `expo-document-picker`, `expo-sharing`
|
||||
- Maps: MapLibre React Native (`@maplibre/maplibre-react-native`) — same tile standard as the web app, self-hostable
|
||||
- `expo-sqlite` (v2+) — fast on-device SQLite with WAL mode
|
||||
- File picking from device storage: `expo-document-picker`
|
||||
- Direct filesystem access (important for Karoo): `expo-file-system`
|
||||
- Maps: MapLibre React Native (`@maplibre/maplibre-react-native`) — same tile
|
||||
standard as the web app, self-hostable
|
||||
- Background tasks: `expo-background-fetch` / `expo-task-manager`
|
||||
- OTA updates via Expo's update service
|
||||
- Expo Go app for rapid development without a build step
|
||||
- Builds for iOS and Android from a single codebase via EAS Build
|
||||
- Expo Go for rapid development without a build step
|
||||
- EAS Build for iOS and Android distribution
|
||||
|
||||
**Why not alternatives:**
|
||||
|
||||
| Option | Reason for skipping |
|
||||
|---|---|
|
||||
| Capacitor + Svelte | Reuses web components but WebView performance is poor for map-heavy activities; Pyodide can't run on mobile |
|
||||
| Flutter | Dart is unfamiliar, requires rewriting all logic; no advantage over RN for this use case |
|
||||
| PWA | iOS severely limits background sync, local storage quotas, and file access — not viable for an activity logger |
|
||||
| Capacitor + Svelte | WebView performance is poor for map-heavy activity detail; Pyodide can't run on mobile |
|
||||
| Flutter | Dart is a new language to learn; no practical advantage over RN for this use case |
|
||||
| PWA | iOS limits background sync, local storage quotas, and filesystem access — not viable for an activity logger |
|
||||
|
||||
### Extraction engine: TypeScript port
|
||||
---
|
||||
|
||||
The Python extractor (`bincio/extract/`) cannot run on mobile. It must be re-implemented in TypeScript. The math is straightforward:
|
||||
## Extraction: hybrid model
|
||||
|
||||
- **FIT parsing**: `@garmin/fitsdk` or `fit-file-parser` (existing JS libraries)
|
||||
- **GPX/TCX parsing**: standard XML parsing (`DOMParser` or `fast-xml-parser`)
|
||||
- **Metrics**: distance (Haversine), speed, HR/power averages, lap splits — all simple loops
|
||||
- **Elevation**: direct port of the hysteresis algorithm from `metrics.py` and `dem.py`
|
||||
- **DEM correction**: same HTTP API call to Open-Elevation, works from mobile
|
||||
Python (the server's extraction engine) cannot run on mobile without a specialised
|
||||
runtime. Rather than fully porting the extraction to TypeScript, the app uses a
|
||||
**tiered extraction model**:
|
||||
|
||||
The TypeScript extraction library (`bincio-extract-ts` or similar) should produce BAS-compatible JSON — the same schema that the server writes. This makes the output directly uploadable.
|
||||
### Tier 1 — On-device TypeScript extraction (always available, offline)
|
||||
|
||||
A TypeScript extraction library (`bincio-extract-ts`) runs entirely on the device:
|
||||
|
||||
- **FIT parsing**: `@garmin/fitsdk` or `fit-file-parser` (mature JS libraries)
|
||||
- **GPX/TCX parsing**: standard XML parsing (`fast-xml-parser`)
|
||||
- **Metrics**: distance (Haversine), moving time, speed, HR/power averages, lap splits
|
||||
- **Elevation**: direct port of the hysteresis algorithm from `metrics.py`
|
||||
|
||||
This produces a valid BAS JSON that the app can display immediately. It is the
|
||||
default path and works with no network.
|
||||
|
||||
### Tier 2 — Server-assisted extraction (when an instance is reachable)
|
||||
|
||||
When a Bincio instance is configured and online, the app can delegate extraction
|
||||
to the server:
|
||||
|
||||
1. Send the raw file to `POST /api/extract` (a new stateless endpoint — processes
|
||||
the file and returns BAS JSON, does **not** store anything).
|
||||
2. The server runs the full Python pipeline: FIT `enhanced_altitude` detection,
|
||||
source-aware hysteresis, DEM correction, power metrics, laps.
|
||||
3. The app stores the returned BAS JSON locally and marks it as server-extracted.
|
||||
|
||||
This gives full extraction quality without maintaining two implementations of every
|
||||
algorithm. The original file is always stored locally, so the app can re-extract
|
||||
via the server at any time (e.g. after a DEM correction improvement is deployed).
|
||||
|
||||
### Re-extraction
|
||||
|
||||
Because the original file is always on device, the app can re-run either tier at
|
||||
any time:
|
||||
|
||||
- **Re-extract offline**: apply an updated TypeScript algorithm to an existing
|
||||
original file.
|
||||
- **Re-extract via server**: send the original file to the server for higher-quality
|
||||
processing (e.g. after connecting to an instance for the first time).
|
||||
|
||||
This means extraction quality improves automatically as algorithms improve, without
|
||||
any data migration.
|
||||
|
||||
---
|
||||
|
||||
@@ -71,103 +137,165 @@ The TypeScript extraction library (`bincio-extract-ts` or similar) should produc
|
||||
```
|
||||
Bincio Mobile
|
||||
├── UI Layer (React Native / Expo)
|
||||
│ ├── Feed screen — list of local activities
|
||||
│ ├── Activity detail — map + chart + stats
|
||||
│ ├── Import screen — pick FIT/GPX/TCX from device or cloud storage
|
||||
│ ├── Sync screen — configure instance, push/pull
|
||||
│ └── Settings screen — instance URL, account, preferences
|
||||
│ ├── Feed screen — list of local activities, sorted by date
|
||||
│ ├── Activity detail — map + elevation chart + stats
|
||||
│ ├── Import screen — pick FIT/GPX/TCX from device or share sheet
|
||||
│ ├── Sync screen — configure instance URL, push/pull
|
||||
│ └── Settings screen — account, preferences, storage info
|
||||
│
|
||||
├── Extraction Engine (TypeScript)
|
||||
│ ├── FIT parser — wraps @garmin/fitsdk
|
||||
│ ├── GPX parser — XML → BAS points
|
||||
│ ├── TCX parser — XML → BAS points
|
||||
│ ├── Metrics — port of metrics.py (distance, elevation, HR, power)
|
||||
│ └── Hysteresis — port of dem.py _hysteresis_gain_loss + _moving_average
|
||||
├── Extraction Engine (TypeScript — Tier 1)
|
||||
│ ├── FIT parser — wraps @garmin/fitsdk
|
||||
│ ├── GPX parser — XML → BAS points
|
||||
│ ├── TCX parser — XML → BAS points
|
||||
│ ├── Metrics — port of metrics.py (distance, elevation, HR, power)
|
||||
│ └── Hysteresis — port of _hysteresis_gain_loss + _moving_average
|
||||
│
|
||||
├── Local Store (expo-sqlite)
|
||||
│ ├── activities — BAS detail JSON + summary fields as columns
|
||||
│ ├── timeseries — 1 Hz arrays stored as JSON blobs per activity
|
||||
│ ├── geojson — simplified GPS track per activity
|
||||
│ └── settings — instance_url, auth_token, sync preferences
|
||||
│ ├── activities — BAS detail JSON + indexed summary columns
|
||||
│ ├── timeseries — 1 Hz arrays as JSON blob per activity
|
||||
│ ├── geojson — simplified GPS track per activity
|
||||
│ ├── originals — original file paths (or blobs) per activity
|
||||
│ └── settings — instance_url, handle, auth_token, sync prefs
|
||||
│
|
||||
└── Sync Layer
|
||||
├── Auth — POST /api/auth/login → session cookie or token
|
||||
├── Manifest fetch — GET /{handle}/index.json → list of remote IDs
|
||||
├── Push — POST /api/upload (multipart, original file)
|
||||
└── Pull — GET /api/activity/{id} + timeseries + geojson
|
||||
├── Auth — POST /api/auth/login → session token
|
||||
├── Extract (Tier 2) — POST /api/extract → BAS JSON, no server storage
|
||||
├── Push — POST /api/upload (original file)
|
||||
└── Pull — GET index.json + activity/{id}.json + timeseries
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data model on device
|
||||
|
||||
Each activity is stored in SQLite with:
|
||||
```sql
|
||||
-- activities table
|
||||
id TEXT PRIMARY KEY, -- BAS ID: "2026-04-17T074238Z"
|
||||
source_hash TEXT NOT NULL, -- SHA-256 of original file (dedup key)
|
||||
detail_json TEXT NOT NULL, -- full BAS detail JSON blob
|
||||
timeseries_json TEXT, -- 1 Hz arrays (loaded lazily)
|
||||
geojson TEXT, -- simplified GPS track
|
||||
original_path TEXT, -- path to original file in app storage
|
||||
extraction_tier INTEGER, -- 1 = TypeScript, 2 = server-extracted
|
||||
synced_at INTEGER, -- unix timestamp of last push to remote (NULL = unsynced)
|
||||
origin TEXT NOT NULL, -- "local" | "remote"
|
||||
created_at INTEGER NOT NULL
|
||||
|
||||
- `id` — BAS activity ID (`2026-04-17T074238Z`)
|
||||
- `source_hash` — SHA-256 of the original file (deduplication key)
|
||||
- `detail_json` — full BAS detail JSON blob
|
||||
- `timeseries_json` — 1 Hz arrays blob (nullable — loaded lazily)
|
||||
- `geojson` — simplified GPS track (nullable)
|
||||
- `synced_at` — timestamp of last successful push to remote instance (nullable = not yet synced)
|
||||
- `origin` — `"local"` (parsed on device) | `"remote"` (pulled from instance)
|
||||
-- settings table
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
```
|
||||
|
||||
The `settings` table stores:
|
||||
**Settings keys:**
|
||||
|
||||
| Key | Value |
|
||||
| Key | Example value |
|
||||
|---|---|
|
||||
| `instance_url` | `https://bincio.org` |
|
||||
| `handle` | user's handle on the remote instance |
|
||||
| `session_token` | auth cookie/token value |
|
||||
| `last_sync_at` | ISO timestamp of last sync |
|
||||
| `handle` | `brutsalvadi` |
|
||||
| `session_token` | `abc123…` |
|
||||
| `last_sync_at` | `2026-04-24T10:00:00Z` |
|
||||
| `auto_import_path` | `/sdcard/Karoo/Rides/` (Android only) |
|
||||
|
||||
---
|
||||
|
||||
## Karoo and Android-first devices
|
||||
|
||||
Devices like the **Karoo 2** run Android and write FIT files directly to the
|
||||
filesystem (e.g. `/sdcard/Karoo/Rides/`). The app can monitor this directory and
|
||||
auto-import new files as rides complete, with no manual export step and no Hammerhead
|
||||
(or Garmin, Wahoo, etc.) cloud sync required.
|
||||
|
||||
On Karoo specifically:
|
||||
- Install the Bincio Android APK directly.
|
||||
- Configure `auto_import_path` to point at the Karoo's ride directory.
|
||||
- When a new FIT file appears, the app imports it automatically (Tier 1 extraction),
|
||||
stores the original file, and shows the ride in the feed.
|
||||
- When WiFi is available and an instance is configured, rides can be pushed to the
|
||||
instance (Tier 2 extraction for higher quality, or just raw upload).
|
||||
|
||||
This makes Bincio a complete replacement for Hammerhead's own sync infrastructure
|
||||
for users who want full control of their data.
|
||||
|
||||
---
|
||||
|
||||
## Sync protocol
|
||||
|
||||
Sync is a two-way, hash-based diff — no custom server protocol needed:
|
||||
Sync is a two-way, hash-based diff — no custom server protocol needed beyond
|
||||
the existing REST API.
|
||||
|
||||
### Push (local → server)
|
||||
|
||||
1. Fetch `{instance_url}/{handle}/index.json` to get remote activity IDs.
|
||||
2. Find activities where `synced_at IS NULL` or `origin = "local"` and `id NOT IN remote_ids`.
|
||||
3. For each unsynced activity, `POST /api/upload` with the original file (stored separately from the extracted JSON) or reconstruct a minimal file from the BAS JSON.
|
||||
2. Find local activities where `synced_at IS NULL`.
|
||||
3. For each unsynced activity, `POST /api/upload` with the original file.
|
||||
4. On 200, set `synced_at = now()`.
|
||||
|
||||
### Pull (server → local)
|
||||
|
||||
1. Fetch `{instance_url}/{handle}/index.json` (and yearly shards if present).
|
||||
2. Find IDs in the remote list that are not in the local DB.
|
||||
1. Fetch `{instance_url}/{handle}/index.json` (and yearly shards).
|
||||
2. Find remote IDs not in the local DB.
|
||||
3. For each missing activity:
|
||||
- `GET {instance_url}/activities/{id}.json` → store as `detail_json`
|
||||
- `GET {instance_url}/activities/{id}.timeseries.json` → store as `timeseries_json` (lazy, on demand)
|
||||
- `GET {instance_url}/activities/{id}.geojson` → store as `geojson`
|
||||
- `GET {instance_url}/activities/{id}.json` → `detail_json`
|
||||
- `GET {instance_url}/activities/{id}.timeseries.json` → `timeseries_json`
|
||||
- `GET {instance_url}/activities/{id}.geojson` → `geojson`
|
||||
4. Insert with `origin = "remote"`, `synced_at = now()`.
|
||||
|
||||
### Conflict handling
|
||||
|
||||
Activities are immutable once created (same philosophy as the server). Conflicts only arise if the same source file is imported on two devices before sync. The `source_hash` prevents double-counting — whichever copy arrives at the server first wins; the duplicate is rejected silently.
|
||||
Activities are immutable once created. The `source_hash` prevents double-counting —
|
||||
if the same file is imported on two devices before sync, whichever copy arrives at
|
||||
the server first wins; the duplicate is rejected with a 409.
|
||||
|
||||
---
|
||||
|
||||
## What's NOT in scope (v1)
|
||||
## New server endpoint needed: `POST /api/extract`
|
||||
|
||||
- Live activity recording (GPS track + sensor recording during a ride) — this is a separate, harder problem
|
||||
- Offline map tiles — initial version requires network for map rendering
|
||||
- Photo sync — photos are not included in the BAS timeseries, sync deferred
|
||||
- Watch / ANT+ / Bluetooth sensors — out of scope for v1
|
||||
- Editing activities on mobile — read-only feed in v1; edits happen on the web
|
||||
A stateless extraction endpoint: accepts a raw FIT/GPX/TCX file, runs the full
|
||||
Python extraction pipeline, returns BAS JSON. Does not write anything to disk.
|
||||
|
||||
```
|
||||
POST /api/extract
|
||||
Content-Type: multipart/form-data
|
||||
file: <raw activity file>
|
||||
|
||||
200 OK
|
||||
{
|
||||
"detail": { ...BAS detail JSON... },
|
||||
"timeseries": { ...1 Hz arrays... },
|
||||
"geojson": { ...simplified track... }
|
||||
}
|
||||
```
|
||||
|
||||
No authentication required (the server is just a compute service here — the result
|
||||
is not stored). Rate limiting and file size cap apply.
|
||||
|
||||
---
|
||||
|
||||
## Open questions
|
||||
## Authentication
|
||||
|
||||
1. **Original file storage**: To push to the server, we need the original FIT/GPX/TCX file. Do we always keep it, or reconstruct a minimal file from extracted BAS JSON? The `POST /api/import-bas` endpoint already exists on the edit server for this use case (see `convert/` page).
|
||||
The multi-user server currently uses HTTP session cookies. For the mobile client,
|
||||
a **Bearer token** is cleaner:
|
||||
|
||||
2. **Auth model**: The multi-user server uses HTTP session cookies. React Native handles cookies but a token-based auth (`Authorization: Bearer …`) would be cleaner for a mobile client. A `POST /api/auth/token` endpoint may be worth adding.
|
||||
```
|
||||
POST /api/auth/token
|
||||
{ "handle": "…", "password": "…" }
|
||||
→ { "token": "abc123…", "expires_at": "…" }
|
||||
```
|
||||
|
||||
3. **iOS file access**: FIT/GPX files from Garmin Connect, Komoot, etc. arrive via the share sheet or Files app. `expo-document-picker` handles this but needs testing with each source.
|
||||
The token is stored in the `settings` table and sent as
|
||||
`Authorization: Bearer abc123…` on all subsequent requests.
|
||||
|
||||
4. **Offline DEM**: DEM correction requires a network call. On mobile, the hysteresis method (offline) should be the default; DEM correction is opt-in when connected.
|
||||
---
|
||||
|
||||
## What is out of scope for v1
|
||||
|
||||
- **Live activity recording** — GPS + sensor recording during a ride. This is a
|
||||
much harder problem (background GPS, Bluetooth/ANT+ sensors, real-time display)
|
||||
and is the eventual goal for full platform independence.
|
||||
- **Offline map tiles** — v1 requires network for map rendering.
|
||||
- **Photo sync** — deferred.
|
||||
- **Watch / ANT+ / Bluetooth sensors** — deferred.
|
||||
- **Editing activities on mobile** — read-only in v1; edits happen on the web.
|
||||
|
||||
---
|
||||
|
||||
@@ -175,7 +303,9 @@ Activities are immutable once created (same philosophy as the server). Conflicts
|
||||
|
||||
| Phase | Scope |
|
||||
|---|---|
|
||||
| **0 — Foundation** | Expo project scaffold, local SQLite store, settings screen, BAS reader (load an existing `.json` file into the feed) |
|
||||
| **1 — Import** | TypeScript FIT/GPX/TCX parser + metrics engine, local feed, activity detail with map and chart |
|
||||
| **2 — Sync** | Auth, push/pull sync with bincio.org or a self-hosted instance |
|
||||
| **3 — Polish** | Offline map tiles, share sheet integration, widgets, performance |
|
||||
| **0 — Foundation** | Expo project scaffold, SQLite store, settings screen, file picker, display a BAS JSON read from disk |
|
||||
| **1 — Import** | TypeScript FIT/GPX/TCX parser + metrics engine (Tier 1), local feed, activity detail with map and chart, original file storage |
|
||||
| **2 — Karoo integration** | Auto-import from a watched directory, Android-specific file access |
|
||||
| **3 — Sync** | `POST /api/extract` endpoint, Bearer token auth, push/pull sync with an instance |
|
||||
| **4 — Polish** | Offline map tiles, share sheet, home screen widget, performance |
|
||||
| **Future** | Live recording, Bluetooth sensors, full Garmin/Wahoo replacement |
|
||||
|
||||
Reference in New Issue
Block a user