feat(mobile): watch-folder scan button + Karoo file picker fix + docs

Import screen:
- Add "Scan for new rides" button (green) when auto_import_path is set;
  shows the configured path, lets user trigger manually in addition to
  the automatic scan on app open.
- Detect ActivityNotFoundException from DocumentPicker (Karoo and other
  stripped Android devices have no DocumentsUI app) and show a friendly
  message directing users to set a Watch directory instead.
- "No new rides found" feedback when manual scan finds nothing.

Docs (docs/mobile-app.md):
- Phase 1 marked complete with implementation notes (wheel delivery,
  timeseries workaround, source_path dedup).
- Phase 2 updated to reflect what is actually implemented (on-open scan,
  not background task) vs what remains (true background polling).
- Batch import moved from Phase 5 todo to done.
- Data model updated with source_path column.
- Known gaps section revised to remove fixed stubs.
- New Karoo sideloading section with debuggableVariants and armeabi-v7a
  troubleshooting notes.
This commit is contained in:
Davide Scaini
2026-04-25 21:52:03 +02:00
parent 749d90c79d
commit 44a70f4c18
2 changed files with 162 additions and 69 deletions
+99 -58
View File
@@ -277,18 +277,49 @@ The APK is at `android/app/build/outputs/apk/release/app-release.apk`.
**Note:** Release APKs must be signed. If signing fails, use `assembleDebug` instead to produce `app-debug.apk` (same as `npx expo run:android`). **Note:** Release APKs must be signed. If signing fails, use `assembleDebug` instead to produce `app-debug.apk` (same as `npx expo run:android`).
#### Karoo 2 sideloading
The Karoo 2 (Hammerhead, Android 8.1, armeabi-v7a) is supported. Two build fixes
are required and already applied to `android/app/build.gradle`:
1. **JS bundle in debug APK**`debuggableVariants = []` in the `react {}` block.
Without this, the debug APK looks for Metro on `localhost:8081`, which doesn't
exist on the Karoo, and the app hangs on the splash screen with
`Unable to load script`.
2. **armeabi-v7a native modules**`splits.abi` must include `"armeabi-v7a"`.
Without it, the CMake build for `libappmodules.so` (the TurboModule registry)
only runs for arm64-v8a. On the Karoo the app crashes with
`PlatformConstants could not be found`.
To build and install on a connected Karoo:
```bash
cd mobile/android
./gradlew assembleDebug
adb -s <karoo-serial> install -r app/build/outputs/apk/debug/app-universal-debug.apk
```
Find the Karoo serial with `adb devices -l`.
#### Troubleshooting #### Troubleshooting
If your friend's APK won't start: If your friend's APK won't start:
1. **Check device logs:** 1. **Check device logs:**
```bash ```bash
adb logcat | grep -i react # requires Android SDK tools adb logcat -s ReactNativeJS AndroidRuntime # requires Android SDK tools
``` ```
2. **Ensure minimum Android version:** The app requires Android 5.0 (API 21) or higher. 2. **Ensure minimum Android version:** The app requires Android 5.0 (API 21) or higher.
3. **Verify the APK is actually installed:** 3. **`Unable to load script` (splash hang):** Debug APK is trying to reach Metro.
Ensure the build was compiled with `debuggableVariants = []` in `build.gradle`.
4. **`PlatformConstants could not be found` (crash on start):** `libappmodules.so`
is missing for the device's ABI. Add the ABI to `splits.abi` in `build.gradle`.
5. **Verify the APK is actually installed:**
```bash ```bash
adb install /path/to/app.apk adb install /path/to/app.apk
``` ```
@@ -483,6 +514,9 @@ CREATE TABLE activities (
timeseries_json TEXT, -- 1 Hz arrays, loaded lazily timeseries_json TEXT, -- 1 Hz arrays, loaded lazily
geojson TEXT, -- simplified GPS track geojson TEXT, -- simplified GPS track
original_path TEXT, -- path in app storage (NULL if pulled from server) original_path TEXT, -- path in app storage (NULL if pulled from server)
source_path TEXT, -- original filesystem path before copy
-- e.g. /sdcard/Karoo/Rides/ride.fit
-- used for watch-folder deduplication (migration v2)
synced_at INTEGER, -- unix timestamp of last push (NULL = unsynced) synced_at INTEGER, -- unix timestamp of last push (NULL = unsynced)
origin TEXT NOT NULL -- "local" | "remote" origin TEXT NOT NULL -- "local" | "remote"
CHECK(origin IN ('local', 'remote')), CHECK(origin IN ('local', 'remote')),
@@ -671,61 +705,74 @@ and displayed as an activity card. No extraction yet.*
--- ---
### Phase 1 — Local FIT/GPX/TCX extraction via Pyodide ### Phase 1 — Local FIT/GPX/TCX extraction via Pyodide
*Goal: pick a FIT/GPX/TCX file, extract it on-device in ~5 s.* *Goal: pick a FIT/GPX/TCX file, extract it on-device in ~5 s.*
> **This is the most important unbuilt feature.** Without it, local import only
> works with pre-extracted BAS JSON files — which requires already having a server.
> It undermines the "works offline without an instance" pitch. Phase 1 is also a
> hard prerequisite for the re-extract button (Phase 5), proper SHA-256 dedup
> (currently stubbed), and Phase 2 (auto-import needs extraction to be fast).
**Requires a Development Build** (`npx expo run:android` via USB, or
`eas build --local`). Expo Go does not support `react-native-webview`.
**Extraction engine (`mobile/extraction/`):** **Extraction engine (`mobile/extraction/`):**
- `PyodideWebView.tsx` — hidden `WebView` rendering an inline HTML page that - `PyodideWebView.tsx` — hidden `WebView` (mounted in the Import tab) rendering
bootstraps Pyodide an inline HTML page that bootstraps Pyodide from jsDelivr CDN. The WebView is
- `wheelCache.ts` — on startup, `GET /api/wheel/version`; if version changed, kept alive between files because Expo Router keeps tabs mounted after first visit.
download and store wheel in `expo-file-system` app directory; falls back to - `extractActivity.ts` — module-level singleton `pyodideRef`; encodes file bytes
bundled `assets/bincio.whl` for offline / pre-deploy use as base64, injects `window._bincioExtract(params)` into the WebView, awaits
- `extractActivity.ts` — encodes file bytes as base64, sends via `postMessage`, `{ id, detail, timeseries, geojson, sourceHash }` via `onMessage`. Serial queue
awaits `{ detail, timeseries, geojson }` response enforced via `isExtracting` guard — only one extraction runs at a time.
- Loading state: "Warming up extractor…" shown only on very first use
**Import screen (full):** **Wheel delivery:**
- Picks FIT/GPX/TCX, passes to `extractActivity`, stores result in SQLite - The bincio wheel is fetched by React Native networking (not inside the WebView),
(`detail_json`, `timeseries_json`, `geojson` columns) because WKWebView on iOS blocks HTTP requests. `GET /api/wheel/version` returns
- Copies original file to `{documentDirectory}/originals/{id}.{ext}` the canonical URL; the wheel bytes are passed into the WebView as base64 and
- Duplicate detection via `source_hash` before extraction installed via `emfs://` (blob: URLs are not recognised by micropip).
- In-memory wheel cache (`_cachedWheel`) avoids re-downloading within a session.
**Import screen:**
- Picks one or more FIT/GPX/TCX/.json files; processes them sequentially.
- Copies original file to `{documentDirectory}/originals/{id}.{ext}`.
- Stores `detail_json`, `timeseries_json`, `geojson`, `source_hash`, `source_path`
in SQLite.
**Known bug in wheel (worked around):**
`write_activity()` in the installed wheel silently skips writing the timeseries
file (an uncaught exception path). The extraction snippet checks `ts_path.exists()`
after `write_activity()` and, if missing, calls `build_timeseries()` directly and
writes the file itself. Without this fix, all locally imported activities showed
stats but no elevation chart or speed graph.
**Done when:** Pick a FIT file from the Karoo rides folder, see full stats in **Done when:** Pick a FIT file from the Karoo rides folder, see full stats in
the Feed within ~5 s, including map and elevation profile. the Feed, including map and elevation profile.
--- ---
### Phase 2 — Karoo auto-import *(Android only)* ### Phase 2 — Karoo auto-import *(Android only)*
*Goal: finish a ride, connect to WiFi, the activity appears in Bincio automatically.* *Goal: finish a ride, open the app, activities appear automatically.*
> **Partially stubbed.** The `auto_import_path` field exists in the Settings UI > **Partially implemented.** The watch-folder scan runs on Import tab mount and
> (Android only) and is saved to the DB, but the background task is not registered. > on every app foreground event, which covers the primary Karoo use case (open the
> The field accepts input but does nothing. Phase 1 (extraction) must be complete > app after a ride). True background polling (fires while the app is closed) is not
> first — there is no point watching a directory if you cannot extract what's in it. > yet implemented — that would require `expo-background-fetch` + `expo-task-manager`,
> but background tasks cannot access the Pyodide WebView (a UI component), so this
> requires a different architectural approach for the extraction step.
**Android:** **What's implemented:**
- Settings screen: `auto_import_path` field already present (Android only, hidden on iOS) ✅ - `auto_import_path` setting in Settings (Android only) ✅
- `expo-task-manager` background task registered at app startup - On Import tab mount and on `AppState` → `'active'`: reads `auto_import_path`,
- Task polls `auto_import_path` every 5 minutes; for each `.fit` file whose requests `READ_EXTERNAL_STORAGE` permission, lists `.fit` files in the directory,
`source_hash` is not in the DB, triggers extraction and import filters out files whose `source_path` is already in the DB, and automatically
- `expo-notifications` sends a local notification: "New ride: Morning Ride — 45 km" imports new files through the same Pyodide extraction pipeline.
- New `source_path` column in `activities` (migration v2): stores the original
filesystem path (`/sdcard/Karoo/Rides/ride.fit`) for O(1) deduplication without
re-reading files.
- Batch import: picks multiple files at once (`multiple: true`), processes them
sequentially, shows "File N of M" progress, ends with a count + per-file errors.
**iOS (alternative flow for Phase 2):** **iOS (alternative flow):**
- Share Extension config so "Open with Bincio" appears in the iOS Files app - Share Extension config so "Open with Bincio" appears in the iOS Files app
- Tapping it hands the file to the app, which runs extraction immediately - Tapping it hands the file to the app, which runs extraction immediately
- No background polling; user-initiated but one-tap - No background polling; user-initiated but one-tap
**Done when (Android):** Finish a ride on the Karoo, the activity appears in **Done when (Karoo):** Finish a ride, open the Bincio app → new FIT files from
Bincio within 5 minutes of connecting to WiFi, with no manual action. `/sdcard/Karoo/Rides` import automatically with no further action. ✅ (on-open)
**Remaining (background):** true background polling while app is closed — deferred.
--- ---
@@ -753,8 +800,7 @@ and re-extracted server-side.
- **Offline map tiles** — bundle or download an MBTiles file for a region; - **Offline map tiles** — bundle or download an MBTiles file for a region;
MapLibre supports offline tile sources MapLibre supports offline tile sources
- **Batch import** — pick a folder (Strava export, Garmin bulk export); import all - **Batch import** ✅ — `multiple: true` in document picker; sequential processing with "File N of M" progress and per-file error summary
FIT/GPX files found, with progress bar and per-file status
- **Share sheet** — Android intent filter for incoming `.fit`/`.gpx`/`.tcx` files - **Share sheet** — Android intent filter for incoming `.fit`/`.gpx`/`.tcx` files
- **Re-extract** — button to re-run Pyodide extraction from the stored original file - **Re-extract** — button to re-run Pyodide extraction from the stored original file
*(requires Phase 1)* *(requires Phase 1)*
@@ -774,26 +820,21 @@ and re-extracted server-side.
This section documents mismatches between what the plan describes and what is This section documents mismatches between what the plan describes and what is
actually implemented, plus features not yet in the plan. actually implemented, plus features not yet in the plan.
### Stubs in the current code ### Remaining stubs
**`source_hash` is not SHA-256** (`mobile/app/(tabs)/import.tsx`) **`source_hash` for BAS JSON import is not SHA-256** (`mobile/app/(tabs)/import.tsx`)
The import screen records `source_hash = "${detail.id}-${text.length}"` — a rough BAS JSON import records `source_hash = "${detail.id}-${text.length}"` — a rough
stand-in, not a real content hash. The dedup guarantee (`INSERT OR IGNORE`) works stand-in. FIT/GPX/TCX imports (via Pyodide) correctly compute SHA-256 of the file
correctly today because activity IDs are unique, but the hash column's intended bytes. The BAS JSON path still uses the stub; dedup works in practice (activity IDs
purpose (detect the same raw file imported twice under a different name) is not are unique) but the hash is not a real content fingerprint.
delivered. Phase 1 will replace this with SHA-256 of the original file bytes.
**FIT/GPX/TCX import is a placeholder** (`mobile/app/(tabs)/import.tsx`) **`auto_import_path` only triggers on app open, not in background**
Picking a FIT, GPX, or TCX file shows an alert: "Extraction coming in Phase 1." The watch-folder scan runs when the Import tab mounts and when the app comes to
No extraction happens. The Import screen effectively only works with pre-extracted foreground (`AppState` → `'active'`). There is no true background task that fires
BAS JSON files until Phase 1 is built. while the app is closed. Full background polling would require `expo-background-fetch`
but cannot use the Pyodide WebView (a UI component).
**`auto_import_path` setting has no backend** (`mobile/app/(tabs)/settings.tsx`)
The Android-only "Watch directory" field in Settings saves its value to SQLite but
nothing reads it. No background task is registered. Phase 2 must wire this up.
### Missing from the plan entirely ### Missing from the plan entirely
+63 -11
View File
@@ -3,7 +3,7 @@ import * as FileSystem from 'expo-file-system/legacy';
import { useSQLiteContext } from 'expo-sqlite'; import { useSQLiteContext } from 'expo-sqlite';
import { useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { AppState, PermissionsAndroid, Platform, Pressable, ScrollView, StyleSheet, Text, View } from 'react-native'; import { AppState, PermissionsAndroid, Platform, Pressable, ScrollView, StyleSheet, Text, View } from 'react-native';
import { insertActivity, isSourcePathImported, getSetting } from '@/db/queries'; import { insertActivity, isSourcePathImported, getSetting, useSetting } from '@/db/queries';
import { PyodideWebView } from '@/extraction/PyodideWebView'; import { PyodideWebView } from '@/extraction/PyodideWebView';
import { extractFile } from '@/extraction/extractActivity'; import { extractFile } from '@/extraction/extractActivity';
import { useTheme } from '@/ThemeContext'; import { useTheme } from '@/ThemeContext';
@@ -23,6 +23,7 @@ export default function ImportScreen() {
const theme = useTheme(); const theme = useTheme();
const [state, setState] = useState<ImportState>({ status: 'idle' }); const [state, setState] = useState<ImportState>({ status: 'idle' });
const isImporting = useRef(false); const isImporting = useRef(false);
const watchPath = Platform.OS === 'android' ? (useSetting('auto_import_path') ?? '') : '';
// Auto-scan watch folder on mount and when app comes to foreground. // Auto-scan watch folder on mount and when app comes to foreground.
useEffect(() => { useEffect(() => {
@@ -38,10 +39,10 @@ export default function ImportScreen() {
async function runAutoScan() { async function runAutoScan() {
if (isImporting.current) return; if (isImporting.current) return;
const watchPath = await getSetting(db, 'auto_import_path'); const path = await getSetting(db, 'auto_import_path');
if (!watchPath) return; if (!path) return;
const newFiles = await discoverNewFiles(db, watchPath); const newFiles = await discoverNewFiles(db, path);
if (newFiles.length === 0) return; if (newFiles.length === 0) return;
isImporting.current = true; isImporting.current = true;
@@ -52,6 +53,26 @@ export default function ImportScreen() {
} }
} }
async function manualScan() {
if (isImporting.current) return;
const path = await getSetting(db, 'auto_import_path');
if (!path) return;
setState({ status: 'loading', msg: 'Scanning…', current: 0, total: 0 });
const newFiles = await discoverNewFiles(db, path);
if (newFiles.length === 0) {
setState({ status: 'done', count: 0, errors: [] });
return;
}
isImporting.current = true;
try {
await processBatch(newFiles.map(f => ({ uri: `file://${f}`, name: f.split('/').pop() ?? f, sourcePath: f })));
} finally {
isImporting.current = false;
}
}
async function pickFiles() { async function pickFiles() {
if (isImporting.current) return; if (isImporting.current) return;
setState({ status: 'loading', msg: 'Picking files…', current: 0, total: 0 }); setState({ status: 'loading', msg: 'Picking files…', current: 0, total: 0 });
@@ -72,8 +93,15 @@ export default function ImportScreen() {
isImporting.current = false; isImporting.current = false;
} }
} catch (e: unknown) { } catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e); const raw = e instanceof Error ? e.message : String(e);
setState({ status: 'error', message: msg }); // Karoo and other stripped-down Android devices have no DocumentsUI app
const noPickerAvailable = raw.includes('ActivityNotFoundException') || raw.includes('No Activity found');
setState({
status: 'error',
message: noPickerAvailable
? 'No file picker available on this device. Set a Watch directory in Settings to import from a folder automatically.'
: raw,
});
isImporting.current = false; isImporting.current = false;
} }
} }
@@ -205,9 +233,23 @@ export default function ImportScreen() {
<Text style={styles.body}> <Text style={styles.body}>
Import FIT, GPX, or TCX files extracted on your device, nothing uploaded. Import FIT, GPX, or TCX files extracted on your device, nothing uploaded.
You can also import pre-extracted BAS <Text style={[styles.code, { color: theme.accent }]}>.json</Text> files. You can also import pre-extracted BAS <Text style={[styles.code, { color: theme.accent }]}>.json</Text> files.
Select multiple files at once to import in batch.
</Text> </Text>
{watchPath ? (
<View style={styles.watchBox}>
<Text style={styles.watchLabel}>Watch folder</Text>
<Text style={styles.watchPath} numberOfLines={2}>{watchPath}</Text>
<Pressable
style={[styles.button, styles.buttonWatch, state.status === 'loading' && styles.buttonDisabled]}
onPress={state.status !== 'loading' ? manualScan : undefined}
>
<Text style={styles.buttonText}>
{state.status === 'loading' ? 'Working…' : '↺ Scan for new rides'}
</Text>
</Pressable>
</View>
) : null}
<Pressable <Pressable
style={[styles.button, state.status === 'loading' && styles.buttonDisabled]} style={[styles.button, state.status === 'loading' && styles.buttonDisabled]}
onPress={state.status !== 'loading' ? pickFiles : undefined} onPress={state.status !== 'loading' ? pickFiles : undefined}
@@ -232,15 +274,17 @@ export default function ImportScreen() {
)} )}
{state.status === 'done' && ( {state.status === 'done' && (
<View style={styles.success}> <View style={[styles.success, state.count === 0 && state.errors.length === 0 && styles.successEmpty]}>
<Text style={styles.successText}> <Text style={styles.successText}>
Imported {state.count} {state.count === 1 ? 'activity' : 'activities'} {state.count === 0 && state.errors.length === 0
? 'No new rides found'
: `✓ Imported ${state.count} ${state.count === 1 ? 'activity' : 'activities'}`}
</Text> </Text>
{state.errors.map((e, i) => ( {state.errors.map((e, i) => (
<Text key={i} style={styles.batchError}> {e.name}: {e.message}</Text> <Text key={i} style={styles.batchError}> {e.name}: {e.message}</Text>
))} ))}
<Pressable onPress={() => setState({ status: 'idle' })}> <Pressable onPress={() => setState({ status: 'idle' })}>
<Text style={styles.errorRetry}>Import more</Text> <Text style={styles.errorRetry}>Dismiss</Text>
</Pressable> </Pressable>
</View> </View>
)} )}
@@ -387,10 +431,17 @@ const styles = StyleSheet.create({
header: { color: '#fff', fontSize: 22, fontWeight: '700', marginBottom: 12 }, header: { color: '#fff', fontSize: 22, fontWeight: '700', marginBottom: 12 },
body: { color: '#a1a1aa', fontSize: 14, lineHeight: 20, marginBottom: 24 }, body: { color: '#a1a1aa', fontSize: 14, lineHeight: 20, marginBottom: 24 },
code: { color: '#60a5fa', fontFamily: 'monospace' }, code: { color: '#60a5fa', fontFamily: 'monospace' },
watchBox: {
backgroundColor: '#18181b', borderRadius: 10, borderWidth: 1,
borderColor: '#27272a', padding: 14, marginBottom: 16, gap: 8,
},
watchLabel: { color: '#71717a', fontSize: 11, fontWeight: '600', letterSpacing: 0.5 },
watchPath: { color: '#a1a1aa', fontSize: 13, fontFamily: 'monospace' },
button: { button: {
backgroundColor: '#2563eb', borderRadius: 10, backgroundColor: '#2563eb', borderRadius: 10,
paddingVertical: 14, alignItems: 'center', marginBottom: 16, paddingVertical: 14, alignItems: 'center', marginBottom: 16,
}, },
buttonWatch: { backgroundColor: '#16a34a', marginBottom: 0 },
buttonDisabled: { opacity: 0.5 }, buttonDisabled: { opacity: 0.5 },
buttonText: { color: '#fff', fontWeight: '600', fontSize: 16 }, buttonText: { color: '#fff', fontWeight: '600', fontSize: 16 },
statusBox: { statusBox: {
@@ -403,7 +454,8 @@ const styles = StyleSheet.create({
success: { success: {
backgroundColor: '#14532d', borderRadius: 8, padding: 12, marginBottom: 16, gap: 6, backgroundColor: '#14532d', borderRadius: 8, padding: 12, marginBottom: 16, gap: 6,
}, },
successText: { color: '#86efac', fontSize: 14 }, successEmpty: { backgroundColor: '#1c1c1e' },
successText: { color: '#86efac', fontSize: 14 },
batchError: { color: '#fca5a5', fontSize: 12 }, batchError: { color: '#fca5a5', fontSize: 12 },
error: { error: {
backgroundColor: '#450a0a', borderRadius: 8, padding: 12, marginBottom: 16, gap: 8, backgroundColor: '#450a0a', borderRadius: 8, padding: 12, marginBottom: 16, gap: 8,