44a70f4c18
Import screen: - Add "Scan for new rides" button (green) when auto_import_path is set; shows the configured path, lets user trigger manually in addition to the automatic scan on app open. - Detect ActivityNotFoundException from DocumentPicker (Karoo and other stripped Android devices have no DocumentsUI app) and show a friendly message directing users to set a Watch directory instead. - "No new rides found" feedback when manual scan finds nothing. Docs (docs/mobile-app.md): - Phase 1 marked complete with implementation notes (wheel delivery, timeseries workaround, source_path dedup). - Phase 2 updated to reflect what is actually implemented (on-open scan, not background task) vs what remains (true background polling). - Batch import moved from Phase 5 todo to done. - Data model updated with source_path column. - Known gaps section revised to remove fixed stubs. - New Karoo sideloading section with debuggableVariants and armeabi-v7a troubleshooting notes.
907 lines
36 KiB
Markdown
907 lines
36 KiB
Markdown
# Bincio Mobile App — Design Document
|
||
|
||
## Vision
|
||
|
||
The long-term goal is full independence from Garmin Connect, Strava, Hammerhead,
|
||
and similar platforms. Today those platforms act as mandatory intermediaries: your
|
||
device syncs to their cloud, you authorise third parties to pull from their API,
|
||
and your data effectively lives on their servers.
|
||
|
||
The Bincio mobile app removes that dependency:
|
||
|
||
- Your FIT/GPX/TCX files live on your device.
|
||
- The app reads them directly — no platform sync required.
|
||
- A Bincio instance (bincio.org or self-hosted) is an optional upgrade for backup,
|
||
sharing, and web access — not a prerequisite.
|
||
- Devices like the **Karoo 2** (Android-based) are a first-class target: activities
|
||
are already saved locally as FIT files, so the app can pick them up directly from
|
||
the filesystem without any export step.
|
||
|
||
This initial version focuses on **post-ride import and local storage**. Live
|
||
recording (GPS + sensors during a ride) is the long-term goal that would complete
|
||
full platform independence, but it is out of scope until the foundation is solid.
|
||
|
||
---
|
||
|
||
## Philosophy
|
||
|
||
**Local-first.** All activity data lives on the device. The app works fully offline
|
||
— no account, no internet connection, no platform authorisation required.
|
||
|
||
**Original files as source of truth.** The raw FIT/GPX/TCX file is always stored on
|
||
device alongside the extracted BAS JSON. This means:
|
||
|
||
- You can re-extract at any time (e.g. when the algorithm improves, or to apply DEM
|
||
correction after connecting to an instance).
|
||
- Sync to a remote instance is just pushing the original file — the server
|
||
re-extracts with the full Python pipeline.
|
||
- No data is ever locked into a proprietary representation.
|
||
|
||
**The algorithm travels to the data — not the other way around.** When internet is
|
||
available, the app downloads a fresh copy of the extraction algorithm from bincio.org
|
||
and runs it locally. Your activity files never touch the server. Only the Python
|
||
wheel (the code) is downloaded; the data stays on device.
|
||
|
||
**Sync is optional and explicit.** Connecting to a Bincio instance adds cloud
|
||
backup, the web feed, and sharing. The app never silently overwrites local data.
|
||
Sync is user-initiated.
|
||
|
||
**Open format.** Activities are stored in the BAS schema — the same JSON format the
|
||
server uses. Any tool in any language can read them.
|
||
|
||
---
|
||
|
||
## Development setup
|
||
|
||
### Two build modes: Expo Go vs Development Build
|
||
|
||
This is the most important thing to understand before starting.
|
||
|
||
**Expo Go** is the Expo app available on the Play Store / App Store. It runs any
|
||
Expo project by scanning a QR code — no compilation step. However, it only supports
|
||
Expo's own built-in modules. It does **not** support third-party native modules.
|
||
|
||
**Development Build** is a custom version of the Expo Go app compiled specifically
|
||
for this project. It includes all third-party native modules (react-native-webview,
|
||
maplibre, react-native-svg). It is installed once on the device; after that, code
|
||
changes still update instantly via Metro (the JS bundler) — no rebuild needed.
|
||
|
||
| | Expo Go | Development Build |
|
||
|---|---|---|
|
||
| Setup | Scan QR, instant | Build APK once (local or EAS cloud) |
|
||
| expo-sqlite | ✅ | ✅ |
|
||
| expo-document-picker | ✅ | ✅ |
|
||
| react-native-webview (Pyodide) | ✗ | ✅ |
|
||
| @maplibre/maplibre-react-native | ✗ | ✅ |
|
||
| react-native-svg | ✗ | ✅ |
|
||
| Code changes | instant (Metro) | instant (Metro) |
|
||
| Native changes | need new Expo Go release | rebuild APK |
|
||
|
||
**Phase 0 and 0.5** only use built-in Expo modules — Expo Go works. **Phase 1**
|
||
(Pyodide) and **Phase 4** (MapLibre maps) require a Development Build because
|
||
`react-native-webview` and `@maplibre/maplibre-react-native` are native modules.
|
||
|
||
The preferred path for Phase 1+: connect the phone via USB and run
|
||
`npx expo run:android` once. After that, JS changes still update instantly via Metro
|
||
— no rebuild needed unless you change native code.
|
||
|
||
---
|
||
|
||
### Prerequisites
|
||
|
||
| Tool | Required for | Install |
|
||
|---|---|---|
|
||
| Node.js 20 LTS | everything | [nodejs.org](https://nodejs.org) or `nvm install 20` |
|
||
| npm | everything | ships with Node |
|
||
| JDK 17 | Android builds | `brew install --cask zulu@17` |
|
||
| Android Studio + SDK | Android dev build / emulator | [developer.android.com/studio](https://developer.android.com/studio) |
|
||
| Xcode 15+ | iOS only, macOS only | App Store → `xcode-select --install` |
|
||
| EAS CLI | cloud builds (optional) | `npm install -g eas-cli` |
|
||
|
||
You do **not** need a physical Android device to start. The Android emulator
|
||
(AVD Manager inside Android Studio) works fine for development.
|
||
|
||
After installing Android Studio, create `mobile/android/local.properties`:
|
||
|
||
```
|
||
sdk.dir=/path/to/Library/Android/sdk
|
||
```
|
||
|
||
---
|
||
|
||
### First-time setup
|
||
|
||
```bash
|
||
# From repo root:
|
||
cd mobile && npm install
|
||
```
|
||
|
||
---
|
||
|
||
### Local dev with a phone on WiFi
|
||
|
||
To test sync against a locally running bincio instance from a phone on the same
|
||
WiFi network:
|
||
|
||
```bash
|
||
./scripts/dev_test.py --mobile
|
||
```
|
||
|
||
This binds the API server to `0.0.0.0` (all interfaces) instead of `127.0.0.1` and
|
||
prints the LAN IP to use as the instance URL in the app settings. The mobile option
|
||
detects the local IP automatically via a UDP socket trick.
|
||
|
||
---
|
||
|
||
### Phase 0 — Expo Go (quickest start)
|
||
|
||
Since Phase 0 uses only built-in Expo modules, you can start with Expo Go:
|
||
|
||
```bash
|
||
cd mobile
|
||
npx expo start
|
||
```
|
||
|
||
1. Install **Expo Go** on your Android phone from the Play Store.
|
||
2. Scan the QR code printed in the terminal.
|
||
3. The app loads instantly. Code changes in your editor appear on the phone
|
||
within a second or two.
|
||
|
||
> **Limitation**: once you add the Pyodide WebView in Phase 1, you must switch to
|
||
> a Development Build. Expo Go will show an error for `react-native-webview`.
|
||
|
||
---
|
||
|
||
### Phase 1+ — Development Build
|
||
|
||
#### Option A: local build (Android Studio required)
|
||
|
||
Plug in an Android device via USB (or start an emulator in Android Studio), then:
|
||
|
||
```bash
|
||
cd mobile
|
||
npx expo run:android # builds APK, installs it, starts Metro
|
||
```
|
||
|
||
This compiles the full native project once (~3–5 min). After that, JS changes
|
||
reflect instantly without rebuilding.
|
||
|
||
For the emulator, create an AVD in Android Studio with API 33+ and start it before
|
||
running the command.
|
||
|
||
#### Option B: local EAS build (no Android Studio, no external cloud)
|
||
|
||
`eas build --local` runs the entire build pipeline on your own machine (or VPS):
|
||
|
||
```bash
|
||
npm install -g eas-cli
|
||
eas build -p android --profile development --local
|
||
```
|
||
|
||
This produces an `.apk` you can transfer to the device via any means (USB, VPS
|
||
download link, AirDrop). No Expo account or cloud service required.
|
||
|
||
---
|
||
|
||
### iOS development (macOS only)
|
||
|
||
```bash
|
||
cd mobile
|
||
npx expo run:ios --udid <device-udid>
|
||
```
|
||
|
||
Requires Xcode 15+ and an iOS platform SDK matching the device's iOS version
|
||
(Xcode Settings → Platforms). After initial build, JS changes update via Metro.
|
||
|
||
MapLibre on iOS requires the following in `ios/Podfile`'s `post_install` block:
|
||
|
||
```ruby
|
||
$MLRN.post_install(installer)
|
||
```
|
||
|
||
This lets MapLibre inject its Swift Package Manager framework dependency. Without
|
||
it, the build fails with `MLNNetworkConfiguration.h not found`.
|
||
|
||
Cloud builds via EAS:
|
||
|
||
```bash
|
||
eas build -p ios --profile development # requires Apple Developer account ($99/yr)
|
||
```
|
||
|
||
---
|
||
|
||
### Where Pyodide comes from
|
||
|
||
The hidden WebView loads Pyodide from the **jsDelivr CDN** — the same source
|
||
as the `/convert/` page on the web:
|
||
|
||
```
|
||
https://cdn.jsdelivr.net/pyodide/v0.26.4/full/pyodide.js (~30 MB)
|
||
```
|
||
|
||
On first extraction after install, the WebView downloads and caches this
|
||
runtime in `expo-file-system`'s document directory. Subsequent extractions use
|
||
the cached copy — no internet required.
|
||
|
||
The **bincio wheel** (~50 KB) is fetched from:
|
||
|
||
```
|
||
GET {instance_url}/api/wheel/version → { version, url, api_url }
|
||
GET {instance_url}/bincio-{version}-py3-none-any.whl (nginx, prod)
|
||
GET {instance_url}/api/wheel/download (FastAPI, local dev)
|
||
```
|
||
|
||
If no instance is configured, it falls back to `https://bincio.org`. The wheel
|
||
is also cached locally and re-downloaded only when the version changes.
|
||
|
||
**Summary of what touches the network:**
|
||
|
||
| Asset | Size | When | Cached |
|
||
|---|---|---|---|
|
||
| Pyodide runtime | ~30 MB | once (first extraction ever) | ✅ permanently |
|
||
| Common packages | ~5 MB | once | ✅ permanently |
|
||
| bincio wheel | ~50 KB | on version bump (bundled fallback in assets/) | ✅ until next update |
|
||
| Map tiles | per-tile | on pan/zoom | ✅ by MapLibre |
|
||
|
||
Everything else — the activity files, the extracted BAS JSON — stays on device.
|
||
|
||
---
|
||
|
||
### Building a standalone APK
|
||
|
||
A standalone APK is a **self-contained application binary** that runs without Expo Go and doesn't rely on development servers. Use this to distribute to friends, devices, or the Karoo 2.
|
||
|
||
#### Option A: EAS cloud build (recommended, no local tools needed)
|
||
|
||
```bash
|
||
npm install -g eas-cli
|
||
eas build -p android --profile preview # produces a standalone APK
|
||
```
|
||
|
||
The APK is available for download from the EAS dashboard or via:
|
||
```bash
|
||
eas build -p android --profile preview --wait
|
||
eas artifact download # follow the prompt
|
||
```
|
||
|
||
#### Option B: Local build with prebuild (requires Android Studio)
|
||
|
||
```bash
|
||
cd mobile
|
||
npx expo prebuild --clean # generates Android native project
|
||
cd android
|
||
./gradlew assembleRelease # builds release APK
|
||
```
|
||
|
||
The APK is at `android/app/build/outputs/apk/release/app-release.apk`.
|
||
|
||
**Note:** Release APKs must be signed. If signing fails, use `assembleDebug` instead to produce `app-debug.apk` (same as `npx expo run:android`).
|
||
|
||
#### Karoo 2 sideloading
|
||
|
||
The Karoo 2 (Hammerhead, Android 8.1, armeabi-v7a) is supported. Two build fixes
|
||
are required and already applied to `android/app/build.gradle`:
|
||
|
||
1. **JS bundle in debug APK** — `debuggableVariants = []` in the `react {}` block.
|
||
Without this, the debug APK looks for Metro on `localhost:8081`, which doesn't
|
||
exist on the Karoo, and the app hangs on the splash screen with
|
||
`Unable to load script`.
|
||
|
||
2. **armeabi-v7a native modules** — `splits.abi` must include `"armeabi-v7a"`.
|
||
Without it, the CMake build for `libappmodules.so` (the TurboModule registry)
|
||
only runs for arm64-v8a. On the Karoo the app crashes with
|
||
`PlatformConstants could not be found`.
|
||
|
||
To build and install on a connected Karoo:
|
||
|
||
```bash
|
||
cd mobile/android
|
||
./gradlew assembleDebug
|
||
adb -s <karoo-serial> install -r app/build/outputs/apk/debug/app-universal-debug.apk
|
||
```
|
||
|
||
Find the Karoo serial with `adb devices -l`.
|
||
|
||
#### Troubleshooting
|
||
|
||
If your friend's APK won't start:
|
||
|
||
1. **Check device logs:**
|
||
```bash
|
||
adb logcat -s ReactNativeJS AndroidRuntime # requires Android SDK tools
|
||
```
|
||
|
||
2. **Ensure minimum Android version:** The app requires Android 5.0 (API 21) or higher.
|
||
|
||
3. **`Unable to load script` (splash hang):** Debug APK is trying to reach Metro.
|
||
Ensure the build was compiled with `debuggableVariants = []` in `build.gradle`.
|
||
|
||
4. **`PlatformConstants could not be found` (crash on start):** `libappmodules.so`
|
||
is missing for the device's ABI. Add the ABI to `splits.abi` in `build.gradle`.
|
||
|
||
5. **Verify the APK is actually installed:**
|
||
```bash
|
||
adb install /path/to/app.apk
|
||
```
|
||
|
||
---
|
||
|
||
### Distributing the app
|
||
|
||
| Target | Method |
|
||
|---|---|
|
||
| Your own Android phone | `npx expo run:android` via USB, or `eas build --local` |
|
||
| Friends or testing | Standalone APK (release or debug, see above) — no Expo Go needed |
|
||
| Karoo 2 | `eas build -p android --profile preview` → sideload via `adb install` |
|
||
| Other Android users | Share the standalone APK download link |
|
||
| Play Store | `eas build -p android --profile preview` → upload `.aab` to Play Console |
|
||
| iOS users | `eas build -p ios --profile preview` → TestFlight (beta) or App Store |
|
||
|
||
---
|
||
|
||
## Repository layout
|
||
|
||
```
|
||
bincio_activity/
|
||
├── bincio/ — Python server + extractor
|
||
│ └── serve/server.py — FastAPI server (includes mobile API endpoints)
|
||
├── site/ — Astro web frontend
|
||
├── mobile/ — Expo React Native app ← this document
|
||
│ ├── app/
|
||
│ │ ├── (tabs)/
|
||
│ │ │ ├── index.tsx — Feed screen
|
||
│ │ │ ├── import.tsx — Import screen
|
||
│ │ │ └── settings.tsx — Settings screen
|
||
│ │ └── activity/[id].tsx — Activity detail screen
|
||
│ └── db/
|
||
│ ├── index.ts — SQLite schema migrations
|
||
│ ├── queries.ts — typed query helpers + React hooks
|
||
│ └── sync.ts — bidirectional sync logic
|
||
├── scripts/
|
||
│ └── dev_test.py — local dev runner (--mobile binds to 0.0.0.0)
|
||
└── docs/
|
||
└── mobile-app.md — this document
|
||
```
|
||
|
||
---
|
||
|
||
## What already exists
|
||
|
||
| Piece | Where | Notes |
|
||
|---|---|---|
|
||
| BAS schema | `docs/schema.md` | The on-device data format — identical to the server format |
|
||
| Pyodide-based extraction | `site/src/pages/convert/` | FIT/GPX/TCX parsing via CPython→WASM in the browser — **the proof of concept for mobile extraction**. A hidden WebView uses the same mechanism. |
|
||
| Bincio wheel | `dist/bincio-0.1.0-py3-none-any.whl`, served at `/bincio-0.1.0-py3-none-any.whl` | Pure-Python wheel already downloaded and run by the `/convert/` page |
|
||
| Local storage concept | `site/src/pages/convert/` | IndexedDB + service worker in the web app. Mobile uses SQLite instead. |
|
||
| Content-addressed dedup | `bincio/extract/dedup.py` | `source_hash` (SHA-256 of raw file) prevents duplicates |
|
||
| REST API | `bincio/serve/server.py` | Login, upload, activity detail, feed — sync primitives |
|
||
| Settings table | `bincio/serve/db.py` | Key/value settings in the server DB; same pattern used on device |
|
||
|
||
---
|
||
|
||
## Technology
|
||
|
||
### Framework: Expo (React Native)
|
||
|
||
- TypeScript throughout
|
||
- `expo-sqlite` v2 — on-device SQLite with WAL mode
|
||
- `expo-document-picker` — file picking from device storage
|
||
- `expo-file-system/legacy` — filesystem access (legacy import required with Expo SDK 54+)
|
||
- `react-native-webview` — hidden WebView for Pyodide (Phase 1)
|
||
- `@maplibre/maplibre-react-native` v11 — interactive maps, dark CartoDB tiles
|
||
- `react-native-svg` ≥ 15.15.4 — SVG area charts; **must be ≥ 15.15.4** (earlier
|
||
versions crash in React Native New Architecture / Fabric)
|
||
- `expo-background-fetch` + `expo-task-manager` — background directory polling (Android, Phase 2)
|
||
- `expo-notifications` — import notifications (Phase 2)
|
||
- EAS Build — iOS and Android binaries; APK sideloading for Karoo
|
||
|
||
### React Native New Architecture (Fabric)
|
||
|
||
The app runs with `newArchEnabled=true` (React Native New Architecture). This is
|
||
required for compatibility with the MapLibre native module. Key implication:
|
||
`react-native-svg` must be **≥ 15.15.4** — earlier versions crash on startup in
|
||
Fabric with a `RNSVGUseProps::~RNSVGUseProps()` / `folly::dynamic::destroy()` error.
|
||
|
||
### MapLibre v11 API
|
||
|
||
`@maplibre/maplibre-react-native` v11 changed from a default export to named exports:
|
||
|
||
```typescript
|
||
// v11 — named imports required
|
||
import { Camera, GeoJSONSource, Layer, Map } from '@maplibre/maplibre-react-native';
|
||
|
||
// Map props: mapStyle (not styleURL), dragPan/touchZoom/touchPitch/touchRotate
|
||
// Camera: initialViewState.bounds = [west, south, east, north]
|
||
// No scrollEnabled prop — use dragPan={false} instead
|
||
```
|
||
|
||
---
|
||
|
||
## Extraction: Pyodide in a hidden WebView
|
||
|
||
The `/convert/` page already demonstrates that the full Python extraction pipeline
|
||
runs in a browser via **Pyodide** (CPython compiled to WebAssembly). A React Native
|
||
app can host a hidden `WebView` running the exact same environment. No rewrite of
|
||
the extraction logic is required.
|
||
|
||
### Package stack (proven in /convert/ today)
|
||
|
||
```
|
||
Pyodide v0.26 (CPython → WASM, ~30 MB)
|
||
├── lxml — pre-compiled WASM in Pyodide (XML / GPX parsing)
|
||
├── fitdecode — pure Python, installed via micropip (FIT parsing)
|
||
├── gpxpy — pure Python, installed via micropip (GPX parsing)
|
||
├── pyyaml — pure Python, installed via micropip
|
||
└── bincio wheel — pure Python, fetched from bincio.org
|
||
```
|
||
|
||
Every dependency is either pre-compiled in Pyodide or **pure Python with no C
|
||
extensions**. Nothing needs recompilation for mobile.
|
||
|
||
### Data flow
|
||
|
||
```
|
||
React Native
|
||
1. Read file bytes from device filesystem (expo-file-system)
|
||
2. postMessage({ type: 'extract', filename, bytes }) → hidden WebView
|
||
|
||
Hidden WebView (Pyodide)
|
||
3. Write bytes to Pyodide virtual FS (/tmp/activity.fit)
|
||
4. Run Python extraction → BAS dict (detail + timeseries + geojson)
|
||
5. postMessage({ type: 'result', detail, timeseries, geojson }) → RN
|
||
|
||
React Native
|
||
6. Store detail_json, timeseries_json, geojson in SQLite
|
||
7. Copy original file to app storage → record path in DB
|
||
```
|
||
|
||
**Data never leaves the device.** Network traffic: only the Pyodide runtime
|
||
(~30 MB, CDN, cached once) and the bincio wheel (~50 KB, from bincio.org, updated
|
||
on version bump).
|
||
|
||
### Performance
|
||
|
||
| Scenario | Time |
|
||
|---|---|
|
||
| First extraction (cold Pyodide + packages) | ~5–8 s |
|
||
| First extraction in session (warm WebView) | ~1–3 s |
|
||
| Subsequent extractions (warm WebView) | ~0.5–1 s |
|
||
| Pyodide RAM while active | ~100–150 MB |
|
||
|
||
For batch import the WebView is kept alive across files; per-file cost drops to
|
||
the Python execution time only.
|
||
|
||
---
|
||
|
||
## Android vs iOS: platform divergences
|
||
|
||
### Filesystem access
|
||
|
||
| | Android | iOS |
|
||
|---|---|---|
|
||
| App sandbox | App has its own private directory | App has its own private directory |
|
||
| External paths | Can read arbitrary paths on the filesystem with `READ_EXTERNAL_STORAGE` (≤ Android 12) or `READ_MEDIA_*` scoped permissions (Android 13+) | **Fully sandboxed.** No access to paths outside the app container or Files app |
|
||
| Karoo rides dir | `expo-file-system` can read `/sdcard/Karoo/Rides/` directly once permission is granted | Not possible |
|
||
| Manual import | Document picker or share sheet | Document picker or share sheet |
|
||
|
||
### Auto-import (Phase 2)
|
||
|
||
| | Android | iOS |
|
||
|---|---|---|
|
||
| Mechanism | Poll a configured directory path every few minutes via a background task | Not possible — iOS apps cannot read external directories |
|
||
| Background execution | `expo-background-fetch` fires reliably; Android allows longer background windows | Background fetch is capped at ~30 s and is not guaranteed to fire; effectively unavailable |
|
||
| Import trigger | Automatic on new FIT file detected in watched directory | Manual: user shares file via Files app or "Open with Bincio" |
|
||
| Karoo auto-import | ✅ Full support — configure path once, rides appear automatically | ✗ Not applicable (Karoo is Android) |
|
||
|
||
### Summary: what is Android-only
|
||
|
||
- Auto-import from a watched directory (Phase 2)
|
||
- `auto_import_path` setting (hidden in the UI on iOS)
|
||
- APK sideloading (for Karoo)
|
||
|
||
Everything else — extraction, local feed, activity detail, sync — is identical on
|
||
both platforms.
|
||
|
||
---
|
||
|
||
## Data model on device
|
||
|
||
```sql
|
||
CREATE TABLE activities (
|
||
id TEXT PRIMARY KEY, -- BAS ID: "2026-04-17T074238Z"
|
||
source_hash TEXT NOT NULL, -- SHA-256 of raw file (dedup key)
|
||
detail_json TEXT NOT NULL, -- full BAS detail JSON blob
|
||
timeseries_json TEXT, -- 1 Hz arrays, loaded lazily
|
||
geojson TEXT, -- simplified GPS track
|
||
original_path TEXT, -- path in app storage (NULL if pulled from server)
|
||
source_path TEXT, -- original filesystem path before copy
|
||
-- e.g. /sdcard/Karoo/Rides/ride.fit
|
||
-- used for watch-folder deduplication (migration v2)
|
||
synced_at INTEGER, -- unix timestamp of last push (NULL = unsynced)
|
||
origin TEXT NOT NULL -- "local" | "remote"
|
||
CHECK(origin IN ('local', 'remote')),
|
||
created_at INTEGER NOT NULL DEFAULT (unixepoch())
|
||
);
|
||
|
||
CREATE TABLE settings (
|
||
key TEXT PRIMARY KEY,
|
||
value TEXT NOT NULL
|
||
);
|
||
```
|
||
|
||
**Settings keys:**
|
||
|
||
| Key | Description | Platform |
|
||
|---|---|---|
|
||
| `instance_url` | e.g. `https://bincio.org` | Both |
|
||
| `handle` | User's handle on the remote instance | Both |
|
||
| `api_token` | Bearer token for API auth (obtained via Connect, never stored in plaintext long-term) | Both |
|
||
| `sync_mode` | `"summaries"` (default) or `"full"` — controls whether geojson+timeseries are downloaded during sync | Both |
|
||
| `sync_upload` | `"true"` or `"false"` — whether to push local activities during sync | Both |
|
||
| `auto_import_path` | Directory to watch for new FIT files | **Android only** |
|
||
|
||
---
|
||
|
||
## Sync protocol
|
||
|
||
### Download (server → local)
|
||
|
||
Implemented in `mobile/db/sync.ts` → `syncFeed()`.
|
||
|
||
1. `GET {instance_url}/api/feed` with `Authorization: Bearer <token>` — returns all
|
||
activity summaries as `{ activities: [...] }`.
|
||
2. Upsert each summary into the local `activities` table with `origin = 'remote'`.
|
||
Returns `changes > 0` (new or updated) per row; counts synced entries.
|
||
3. **Summaries mode** (default): stops here. Geojson and timeseries are fetched
|
||
on-demand when the user opens an activity detail screen.
|
||
4. **Full mode**: for every activity where `geojson` or `timeseries_json` is NULL,
|
||
fetches `GET /api/activity/{id}/geojson` and `GET /api/activity/{id}/timeseries`.
|
||
Uses `COALESCE` in the UPDATE to avoid overwriting already-stored data.
|
||
|
||
### Upload (local → server)
|
||
|
||
Implemented in `mobile/db/sync.ts` → `uploadLocalActivities()`. Enabled when
|
||
`sync_upload = "true"` in settings.
|
||
|
||
1. Query `activities WHERE origin = 'local' AND synced_at IS NULL`.
|
||
2. For each: read the BAS JSON from `original_path` via `expo-file-system`.
|
||
3. `POST {instance_url}/api/upload/bas` with body `{ activity, timeseries?, geojson? }`.
|
||
4. On 200/duplicate: set `synced_at = unixepoch()`.
|
||
|
||
The server endpoint (`bincio/serve/server.py` → `POST /api/upload/bas`) accepts
|
||
pre-extracted BAS JSON rather than raw FIT/GPX/TCX. It deduplicates by checking
|
||
if the activity file already exists, writes geojson and timeseries if provided, then
|
||
calls `merge_all()` to refresh the server's merged feed.
|
||
|
||
### Conflict handling
|
||
|
||
Activities are immutable. Dedup on download: `ON CONFLICT(id) DO UPDATE` only
|
||
updates `detail_json` and `synced_at`. Dedup on upload: the server returns
|
||
`{ ok: true, status: "duplicate" }` (still HTTP 200) if the file already exists.
|
||
|
||
---
|
||
|
||
## Authentication
|
||
|
||
The server exposes `POST /api/auth/token`:
|
||
|
||
```
|
||
POST /api/auth/token
|
||
Body: { "handle": "…", "password": "…" }
|
||
→ { "token": "abc123…", "display_name": "…" }
|
||
```
|
||
|
||
The token is stored in the `settings` table as `api_token` and sent as
|
||
`Authorization: Bearer abc123…` on all sync requests. The password is used once and
|
||
immediately discarded — it is never persisted.
|
||
|
||
The server's `_require_auth` helper accepts both cookie-based session auth (web) and
|
||
Bearer token auth (mobile), so no separate mobile-specific middleware is needed.
|
||
|
||
---
|
||
|
||
## Mobile API endpoints
|
||
|
||
All endpoints require `Authorization: Bearer <token>` from the mobile client.
|
||
|
||
| Method | Path | Description |
|
||
|---|---|---|
|
||
| `POST` | `/api/auth/token` | Password login → Bearer token |
|
||
| `GET` | `/api/feed` | All activity summaries for the authenticated user |
|
||
| `GET` | `/api/activity/{id}/geojson` | GeoJSON route for one activity |
|
||
| `GET` | `/api/activity/{id}/timeseries` | Timeseries JSON for one activity |
|
||
| `POST` | `/api/upload/bas` | Upload a pre-extracted BAS JSON activity |
|
||
| `GET` | `/api/wheel/version` | Latest bincio wheel version + URL (public) |
|
||
| `GET` | `/api/wheel/download` | Serve the wheel file (dev mode fallback) |
|
||
|
||
---
|
||
|
||
## Implementation plan
|
||
|
||
### Phase 0 — Foundation ✅
|
||
*Goal: app launches, settings can be configured, a BAS JSON file can be picked
|
||
and displayed as an activity card. No extraction yet.*
|
||
|
||
- Expo Router with three tabs: **Feed**, **Import**, **Settings**
|
||
- `expo-sqlite` initialised; `activities` and `settings` tables created on first launch
|
||
- Settings screen: instance URL and handle, saved to `settings` table
|
||
- Import screen: `expo-document-picker`; BAS `.json` files parsed and inserted into feed
|
||
- Feed screen: activity cards sorted by `started_at`, sport icon, distance, elevation
|
||
- `GET /api/wheel/version` server endpoint (public, no auth)
|
||
|
||
**Done when:** App launches, user picks a `.json` BAS file, sees it in the Feed. ✅
|
||
|
||
---
|
||
|
||
### Phase 0.5 — Remote feed sync ✅
|
||
*Goal: pull all activities from a remote bincio instance into the local feed.*
|
||
|
||
- `POST /api/auth/token` — password login returning a Bearer token (stored in
|
||
SQLite; password forgotten immediately after)
|
||
- `GET /api/feed` — auth-gated; reads `_merged/index.json` shards and returns
|
||
all activity summaries as JSON
|
||
- Settings screen: Connect section (password field + Connect button + status);
|
||
Disconnect button clears the stored token
|
||
- Feed screen: **↓ Sync** button and pull-to-refresh; "cloud" badge on remote
|
||
activities; `syncFeed()` upserts remote summaries without overwriting local imports
|
||
|
||
**Done when:** Tap Connect, tap Sync, all instance activities appear in the Feed. ✅
|
||
|
||
---
|
||
|
||
### Phase 4 — Activity detail: map + metric charts ✅
|
||
*Goal: every activity shows a route map and metric charts.*
|
||
|
||
> Phase 4 was implemented before Phase 1 because it only requires the Development
|
||
> Build already needed for MapLibre — no Pyodide work required.
|
||
|
||
**Map (MapLibre v11):**
|
||
- Dark CartoDB tile base map via `mapStyle` prop
|
||
- Route drawn as a GeoJSON `LineLayer` (`line-color: #60a5fa`, `line-width: 3`)
|
||
- Camera auto-fits the track bounding box via `initialViewState.bounds`
|
||
- Thumbnail (non-interactive) shown inline; tap **⤢ tap to explore** to open a
|
||
full-screen modal with pan/zoom/pitch/rotate enabled
|
||
- On-demand fetch for remote activities: `GET /api/activity/{id}/geojson` with
|
||
Bearer auth; result cached in memory for the session
|
||
|
||
**Metric charts (react-native-svg):**
|
||
- Tabbed interface: Elevation / Speed / HR / Cadence / Power
|
||
- Only tabs with non-null data are shown
|
||
- Each chart: SVG area chart with gradient fill and coloured stroke; min/max labels
|
||
- Downsampled to ≤300 points for performance
|
||
- Tab colours: elevation `#00c8ff`, speed `#ff6b35`, HR `#f87171`, cadence `#a78bfa`, power `#facc15`
|
||
- On-demand fetch for remote activities: `GET /api/activity/{id}/timeseries`
|
||
|
||
**Stats grid:** distance, moving time, elevation gain/loss, avg speed, avg HR, avg power.
|
||
|
||
**Done when:** Open any synced or locally imported activity — map and charts visible. ✅
|
||
|
||
---
|
||
|
||
### Phase 0.6 — Sync mode + bidirectional sync ✅
|
||
*Goal: control download depth and push local activities back to the server.*
|
||
|
||
> Implemented as an addition to Phase 0.5 sync, ahead of Phase 1.
|
||
|
||
**Download mode (Settings → Sync → Download):**
|
||
- **Summaries only** (default): sync downloads activity summaries only. Map and
|
||
charts are fetched on first open.
|
||
- **Full data**: sync downloads geojson and timeseries for every activity that is
|
||
missing them. Uses more storage; takes longer.
|
||
|
||
**Upload (Settings → Sync → Upload):**
|
||
- **Off** (default): local activities stay on device only.
|
||
- **Upload local activities**: during each sync, unsynced local activities are
|
||
POSTed to `POST /api/upload/bas` as pre-extracted BAS JSON (not original FIT).
|
||
The server deduplicates, writes the files, and re-merges the feed.
|
||
|
||
**Settings → Data:**
|
||
- **Reset synced data**: two-tap confirm button to delete all remote activities from
|
||
the local DB. Useful when switching instances or re-syncing from scratch.
|
||
|
||
**Feed sync message** shows counts: `Synced: 3 new, 2 full datasets, 1 uploaded (47 total)`.
|
||
|
||
**Done when:** Sync pushes local activities to the instance and reports them in the feed. ✅
|
||
|
||
---
|
||
|
||
### Phase 1 — Local FIT/GPX/TCX extraction via Pyodide ✅
|
||
*Goal: pick a FIT/GPX/TCX file, extract it on-device in ~5 s.*
|
||
|
||
**Extraction engine (`mobile/extraction/`):**
|
||
- `PyodideWebView.tsx` — hidden `WebView` (mounted in the Import tab) rendering
|
||
an inline HTML page that bootstraps Pyodide from jsDelivr CDN. The WebView is
|
||
kept alive between files because Expo Router keeps tabs mounted after first visit.
|
||
- `extractActivity.ts` — module-level singleton `pyodideRef`; encodes file bytes
|
||
as base64, injects `window._bincioExtract(params)` into the WebView, awaits
|
||
`{ id, detail, timeseries, geojson, sourceHash }` via `onMessage`. Serial queue
|
||
enforced via `isExtracting` guard — only one extraction runs at a time.
|
||
|
||
**Wheel delivery:**
|
||
- The bincio wheel is fetched by React Native networking (not inside the WebView),
|
||
because WKWebView on iOS blocks HTTP requests. `GET /api/wheel/version` returns
|
||
the canonical URL; the wheel bytes are passed into the WebView as base64 and
|
||
installed via `emfs://` (blob: URLs are not recognised by micropip).
|
||
- In-memory wheel cache (`_cachedWheel`) avoids re-downloading within a session.
|
||
|
||
**Import screen:**
|
||
- Picks one or more FIT/GPX/TCX/.json files; processes them sequentially.
|
||
- Copies original file to `{documentDirectory}/originals/{id}.{ext}`.
|
||
- Stores `detail_json`, `timeseries_json`, `geojson`, `source_hash`, `source_path`
|
||
in SQLite.
|
||
|
||
**Known bug in wheel (worked around):**
|
||
`write_activity()` in the installed wheel silently skips writing the timeseries
|
||
file (an uncaught exception path). The extraction snippet checks `ts_path.exists()`
|
||
after `write_activity()` and, if missing, calls `build_timeseries()` directly and
|
||
writes the file itself. Without this fix, all locally imported activities showed
|
||
stats but no elevation chart or speed graph.
|
||
|
||
**Done when:** Pick a FIT file from the Karoo rides folder, see full stats in
|
||
the Feed, including map and elevation profile. ✅
|
||
|
||
---
|
||
|
||
### Phase 2 — Karoo auto-import *(Android only)*
|
||
*Goal: finish a ride, open the app, activities appear automatically.*
|
||
|
||
> **Partially implemented.** The watch-folder scan runs on Import tab mount and
|
||
> on every app foreground event, which covers the primary Karoo use case (open the
|
||
> app after a ride). True background polling (fires while the app is closed) is not
|
||
> yet implemented — that would require `expo-background-fetch` + `expo-task-manager`,
|
||
> but background tasks cannot access the Pyodide WebView (a UI component), so this
|
||
> requires a different architectural approach for the extraction step.
|
||
|
||
**What's implemented:**
|
||
- `auto_import_path` setting in Settings (Android only) ✅
|
||
- On Import tab mount and on `AppState` → `'active'`: reads `auto_import_path`,
|
||
requests `READ_EXTERNAL_STORAGE` permission, lists `.fit` files in the directory,
|
||
filters out files whose `source_path` is already in the DB, and automatically
|
||
imports new files through the same Pyodide extraction pipeline.
|
||
- New `source_path` column in `activities` (migration v2): stores the original
|
||
filesystem path (`/sdcard/Karoo/Rides/ride.fit`) for O(1) deduplication without
|
||
re-reading files.
|
||
- Batch import: picks multiple files at once (`multiple: true`), processes them
|
||
sequentially, shows "File N of M" progress, ends with a count + per-file errors.
|
||
|
||
**iOS (alternative flow):**
|
||
- Share Extension config so "Open with Bincio" appears in the iOS Files app
|
||
- Tapping it hands the file to the app, which runs extraction immediately
|
||
- No background polling; user-initiated but one-tap
|
||
|
||
**Done when (Karoo):** Finish a ride, open the Bincio app → new FIT files from
|
||
`/sdcard/Karoo/Rides` import automatically with no further action. ✅ (on-open)
|
||
|
||
**Remaining (background):** true background polling while app is closed — deferred.
|
||
|
||
---
|
||
|
||
### Phase 3 — Push sync (original files)
|
||
*Goal: upload original FIT/GPX/TCX files to the server for server-side re-extraction.*
|
||
|
||
Phase 0.6 already pushes **pre-extracted BAS JSON** to the server, which covers
|
||
the common case. Phase 3 adds upload of the **original raw file** so the server
|
||
can re-extract with its full pipeline (DEM correction, hysteresis, etc.).
|
||
|
||
**Server:**
|
||
- Extend `POST /api/upload` to accept raw FIT/GPX/TCX with Bearer token auth
|
||
(currently cookie-only)
|
||
|
||
**App:**
|
||
- When `original_path` is set, prefer uploading the raw file; fall back to BAS JSON
|
||
- Progress indicator per activity for first push with many files
|
||
|
||
**Done when:** Tap **Sync**, locally imported FIT files are uploaded to the instance
|
||
and re-extracted server-side.
|
||
|
||
---
|
||
|
||
### Phase 5 — Polish *(ongoing)*
|
||
|
||
- **Offline map tiles** — bundle or download an MBTiles file for a region;
|
||
MapLibre supports offline tile sources
|
||
- **Batch import** ✅ — `multiple: true` in document picker; sequential processing with "File N of M" progress and per-file error summary
|
||
- **Share sheet** — Android intent filter for incoming `.fit`/`.gpx`/`.tcx` files
|
||
- **Re-extract** — button to re-run Pyodide extraction from the stored original file
|
||
*(requires Phase 1)*
|
||
- **App icon and splash screen** — currently using Expo defaults
|
||
- **Feed search / filter** — by sport, date range, or title; necessary as the
|
||
feed grows past ~50 activities
|
||
- **Individual activity deletion** — currently users can only "Reset synced data"
|
||
(all remote) or nothing (local). No way to delete a single activity.
|
||
- **Token expiry / reconnect prompt** — when the server returns 401, the app shows
|
||
an error in the sync toast but leaves the user to navigate to Settings manually.
|
||
Should auto-show a reconnect prompt instead.
|
||
|
||
---
|
||
|
||
## Known gaps and technical debt
|
||
|
||
This section documents mismatches between what the plan describes and what is
|
||
actually implemented, plus features not yet in the plan.
|
||
|
||
### Remaining stubs
|
||
|
||
**`source_hash` for BAS JSON import is not SHA-256** (`mobile/app/(tabs)/import.tsx`)
|
||
|
||
BAS JSON import records `source_hash = "${detail.id}-${text.length}"` — a rough
|
||
stand-in. FIT/GPX/TCX imports (via Pyodide) correctly compute SHA-256 of the file
|
||
bytes. The BAS JSON path still uses the stub; dedup works in practice (activity IDs
|
||
are unique) but the hash is not a real content fingerprint.
|
||
|
||
**`auto_import_path` only triggers on app open, not in background**
|
||
|
||
The watch-folder scan runs when the Import tab mounts and when the app comes to
|
||
foreground (`AppState` → `'active'`). There is no true background task that fires
|
||
while the app is closed. Full background polling would require `expo-background-fetch`
|
||
but cannot use the Pyodide WebView (a UI component).
|
||
|
||
### Missing from the plan entirely
|
||
|
||
**Feed pagination**
|
||
|
||
`useActivities()` in `mobile/db/queries.ts` calls `getAllSync` with no `LIMIT`.
|
||
This is fine for tens of activities, but will cause noticeable lag at hundreds.
|
||
Should add cursor-based pagination or a virtual list with lazy loading.
|
||
|
||
**Individual activity deletion**
|
||
|
||
There is no way to delete a single activity, local or remote. "Reset synced data"
|
||
nukes all remote activities at once. A long-press or swipe-to-delete gesture on
|
||
activity cards is needed, with a server-side `DELETE /api/activity/{id}` endpoint
|
||
for remote activities.
|
||
|
||
**Feed search and filter**
|
||
|
||
No search bar, no sport filter, no date picker. The feed is a flat reverse-
|
||
chronological list. As the feed grows this becomes a usability problem.
|
||
|
||
**Token expiry and the reconnect flow**
|
||
|
||
Bearer tokens are stored indefinitely and never refreshed. When the server rejects
|
||
one (HTTP 401 during sync), the error toast says "Session expired — reconnect in
|
||
Settings." The user must navigate there manually. A better flow: on 401, show a
|
||
modal with a password field so the user can reconnect inline without leaving the
|
||
sync flow.
|
||
|
||
**App icon and splash screen**
|
||
|
||
The app uses Expo's default purple icon and white splash. These need to be replaced
|
||
with Bincio branding before any public distribution.
|
||
|
||
**Upload only works for activities imported from a file**
|
||
|
||
`uploadLocalActivities()` skips rows where `original_path IS NULL`. Activities
|
||
synced from the server (`origin = 'remote'`) are never uploaded back even if
|
||
upload is enabled. This is correct behaviour, but it also means a locally created
|
||
activity that somehow lacks an `original_path` (e.g. from a future recording
|
||
feature) would be silently skipped. Worth documenting as an invariant.
|
||
|
||
---
|
||
|
||
## Out of scope for v1
|
||
|
||
- **Live activity recording** — GPS track + sensor data during a ride. This is the
|
||
eventual goal for complete platform independence but requires significant additional
|
||
work (background GPS, Bluetooth/ANT+ sensor integration, real-time display).
|
||
- **Editing activities** — read-only in v1; edits happen via the web interface.
|
||
- **Photo sync** — deferred.
|
||
|
||
---
|
||
|
||
## Future: toward full platform independence
|
||
|
||
Once live recording is implemented, the stack becomes:
|
||
|
||
```
|
||
Ride starts → Bincio records GPS + sensors (BLE power meter, HR strap, etc.)
|
||
Ride ends → Bincio extracts the activity locally (Pyodide or native)
|
||
→ Activity visible in the mobile feed immediately
|
||
→ Original FIT file saved on device
|
||
→ Optional: push to bincio.org for web access
|
||
```
|
||
|
||
At that point Garmin Connect, Hammerhead sync, and Strava become entirely optional.
|
||
The Karoo (or any Android head unit running the app) becomes a self-contained
|
||
training ecosystem.
|