When the dev file-watcher and an upload endpoint both trigger merge_all on the
same user directory at the same time, the shutil.rmtree/_merged/activities/
sequence collides: one thread deletes the directory while the other is mid-walk,
causing ENOTEMPTY or ENOENT crashes.
Added a per-user-dir threading.Lock (_merge_lock) at the top of merge.py so
all callers (upload endpoints, watcher, admin rebuild) are serialized without
changing any call site. Also made the rmtree ignore_errors=True and the
symlink_to conditional to survive any stale leftover from a prior crash.
Three bugs fixed:
- /api/upload/bas and /api/upload/raw never updated user_dir/index.json, so
merge_all couldn't include uploaded activities in year shards — they existed
on disk but were invisible to the browser feed. Fixed by _upsert_index_summary()
called before merge_all().
- Silent catch {} in uploadLocalActivities swallowed all per-activity errors;
replaced with console.warn so failures are visible in Expo logs.
- After a server wipe, synced_at flags on the device caused "Nothing to upload"
forever. uploadFeed() now reconciles against GET /api/feed at the start of each
upload: local activities not found on the server get synced_at cleared.
Also: live upload progress ("Uploading N / M…"), failed count in result message,
onProgress callback on uploadFeed(), countPendingUploads() helper.
merge_all(user_dir) updates the per-user _merged/ shard but the home page
loads feed.json first via loadCombinedFeed. write_combined_feed was only
called by the CLI render command, not by the API upload endpoints or the
dev watcher, leaving feed.json permanently stale after any runtime upload.
Add write_combined_feed(_get_data_dir()) after every merge_all call in
/api/upload/bas, /api/upload/raw, the dev.py file watcher, and dev startup.
- Skip MapLibre on Android <29 (Karoo): SELinux denies kgsl-3d0 access
from untrusted_app context, crashing the GPU driver on any OpenGL
surface. Replace with SvgRouteView — equirectangular SVG route trace
using react-native-svg, no native GL surface needed.
- Add +/- zoom buttons to full-screen MapLibre map on modern devices
via Camera ref and onRegionDidChange.
- Skip PyodideWebView on Android <29: same GPU driver conflict; set
_engineUnavailable at module init via API level gate (< 29).
- Add engine_unavailable fast path in PyodideWebView: post message
immediately if WebAssembly.Global is absent (Chrome <69) instead of
attempting 30 MB Pyodide download.
- Add server-side extraction fallback (extractServer.ts): when engine
unavailable, POST raw file as base64 to /api/upload/raw; server runs
full Python pipeline and returns extracted data.
- Add /api/upload/raw endpoint in server.py.
- Add pre-flight auth check (checkServerAuth) before batch import so
an expired token errors immediately rather than after N files.
- Fix uploadLocalActivities in sync.ts: was reading original_path as
JSON (binary FIT file, always threw), silently skipping every upload.
Now reads detail_json from DB directly.
- Redesign Feed header: replace single Sync button with Upload /
Download / Refresh. Pull-to-refresh and Refresh button are local-only.
Auto-refresh on tab focus via useFocusEffect.
- Replace ActivityIndicator with plain Text everywhere (native animation
also crashes Karoo GPU driver).
- Raise macOS open-file limit in dev_test.py to prevent EMFILE errors
from Astro file watcher.
- Document all Karoo hardware constraints in docs/mobile-app.md.
Root causes identified via logcat:
Chrome <71: globalThis not defined → ReferenceError in Pyodide factory,
loadPyodide stays as {} (set by UMD wrapper), "not a function"
Chrome <63: dynamic import() is a parse-time SyntaxError
Chrome <63: for-await-of is a parse-time SyntaxError
Fix: when Chrome <80 is detected, fetch v0.18.1 pyodide.js as text and apply
three split/join patches before injecting via Blob URL:
1. prepend globalThis polyfill
2. import( → __loadScript( (script-tag shim)
3. for await( → for( (getFsHandles is never called in our flow)
Chrome 80+ keeps the original clean <script>-tag path with v0.26.4.
The regex /\bimport\(/g inside a template literal corrupts \b into a
backspace and \( into (, producing an unterminated-group SyntaxError in the
WebView. Use split().join() instead — no escape sequences, no corruption.
Modern WebViews (Chrome 63+) keep the original clean <script>-tag path.
Old WebViews (Karoo Chrome 61) take the fetch+patch+Blob path.
Chrome 61 (Karoo WebView) cannot parse dynamic import() — a SyntaxError at
parse time prevents loadPyodide from ever being defined.
Fix: fetch pyodide.js as text, replace every import( with a __loadScript(
shim that uses <script> tag injection, then inject via Blob URL. The Blob
script is never pre-scanned for module syntax so the patch is invisible to
the parser.
Also: expose waitForEngine() from extractActivity so callers can await
engine readiness before batching files — manual scan now shows "Preparing
extraction engine…" instead of flooding with N individual failures.
Settings: watch directory field is hidden behind a warning if no instance
URL is saved yet, making the dependency explicit before the user sets a
path.
Import: runAutoScan silently skips (no errors) when instance URL is
missing; manualScan shows a single clear message instead of one failure
per file.
The Save button is anchored to the Instance section near the top. Moving
the auto_import_path write to onBlur on the field itself means the value
is persisted as soon as the user leaves the input — no need to scroll up
and hit Save.
- useFocusEffect re-reads auto_import_path from DB every time the Import
tab is focused, so the scan button appears immediately after saving in
Settings (was broken by conditional hook call which violated Rules of
Hooks and never re-fired on setting changes)
- Nested inner try/catch isolates DocumentPicker.getDocumentAsync so
ActivityNotFoundException (Karoo has no DocumentsUI) shows a friendly
message instead of crashing the tab
- Settings watch-directory placeholder updated to /sdcard/FitFiles
Import screen:
- Add "Scan for new rides" button (green) when auto_import_path is set;
shows the configured path, lets user trigger manually in addition to
the automatic scan on app open.
- Detect ActivityNotFoundException from DocumentPicker (Karoo and other
stripped Android devices have no DocumentsUI app) and show a friendly
message directing users to set a Watch directory instead.
- "No new rides found" feedback when manual scan finds nothing.
Docs (docs/mobile-app.md):
- Phase 1 marked complete with implementation notes (wheel delivery,
timeseries workaround, source_path dedup).
- Phase 2 updated to reflect what is actually implemented (on-open scan,
not background task) vs what remains (true background polling).
- Batch import moved from Phase 5 todo to done.
- Data model updated with source_path column.
- Known gaps section revised to remove fixed stubs.
- New Karoo sideloading section with debuggableVariants and armeabi-v7a
troubleshooting notes.
Two issues prevented the app from running on the Karoo 2 (armeabi-v7a, Android 8.1):
1. Debug builds loaded the JS bundle from Metro (localhost:8081) rather than
embedding it. Fixed by setting debuggableVariants = [] in the react {} block,
which tells the RN Gradle plugin to bundle JS for all variants including debug.
2. splits.abi only included arm64-v8a, so the CMake/NDK build (which generates
libappmodules.so — the TurboModule registry) never ran for armeabi-v7a.
Result: PlatformConstants TurboModule not found, app stuck on splash.
Fixed by adding armeabi-v7a to the splits.abi include list.
- Import tab now accepts multiple files at once (DocumentPicker multiple:true),
processes them sequentially through Pyodide, and shows a summary with per-file
errors on completion.
- DB migration v2 adds source_path column (original filesystem path before copy)
and an index on it, enabling O(1) deduplication for watch-folder imports.
- On Android, if auto_import_path is set, the Import tab scans the directory on
mount and on AppState 'active' (app foreground), then automatically imports any
FIT files not yet in the DB. Designed for Karoo: finish a ride, open the app,
new files import without any manual steps.
- insertActivity now accepts optional source_path; both importBasJson and
importNativeFile pass it through (null for files picked via DocumentPicker,
real path for watch-folder files).
Diagnosed via on-device debug: build_timeseries produces 3271 points
correctly, but the installed wheel's write_activity has a silent
exception path that skips writing the timeseries file. The workaround
calls build_timeseries directly and writes the file if missing.
Also moves useTheme import to @/ThemeContext across all tab screens.
Replace useSetting()-based useTheme() with a React context (ThemeProvider
+ useTheme/usePaletteControl). The context holds palette key in state so
pressing a palette button in Settings re-renders all screens instantly.
Persists to SQLite and reloads the stored value on mount.
Inline script in Base.astro sets --accent / --accent-dim CSS variables
before first paint based on the current date. Switches to pink (Giro),
yellow (Tour), or red (Vuelta) during each Grand Tour window; falls back
to the default blue. Also aligns default --accent with the mobile app
(#60a5fa instead of #00c8ff).
- theme.ts: useTheme() hook with race calendar (May–Sep windows),
auto-detects Giro/Tour/Vuelta by date; stores override in SQLite
- All screens (feed, import, activity, tab bar) now use accent/dim
from useTheme() instead of hardcoded #60a5fa
- Settings: Palette section with Auto/Default/Giro/Tour/Vuelta buttons
to override the auto-detected palette for testing
- Detail screen: Delete button (top-right, red) with confirmation alert;
deletes SQLite row and original file via expo-file-system
- Feed screen: long-press card to enter select mode; checkbox + blue
border on selected cards; bottom action bar with bulk Delete N button;
header switches to show count + Cancel
- db/queries: deleteActivity (returns original_path) and deleteActivities
(bulk, returns all original paths)
The dedup cache (.bincio_cache.json) persists source hashes across runs.
If the activities/ directory is wiped (--fresh in another session, or macOS
clearing /tmp after a restart) while the cache file survives at the user dir
level, re-running extract skips all files as "already extracted" and leaves
activities/ empty. Now only hashes whose corresponding .json file is present
on disk are treated as known, so missing files are always re-extracted.
In Hermes release builds, useEffect captures closure values from the first
render. If instanceUrl or token were empty at that moment (before SQLite reads
complete), no fetch ran and map/graphs never loaded. Adding them to the
dependency array ensures the effect re-runs once the values are available;
guards on existing geojson/timeseries state prevent double-fetching.
Also add @maplibre/maplibre-react-native to the Expo plugins array so that
expo prebuild applies the library's required Gradle property configuration to
the Android project.
Android release builds block cleartext HTTP by default (debug builds override
this via the debug manifest overlay). Add usesCleartextTraffic=true to app.json
so expo prebuild includes it in the generated manifest — required to reach local
Bincio instances over HTTP.
In bincio dev (Astro dev server), /activity/<id>/ routes 404 because
getStaticPaths() returns [] and there is no nginx try_files fallback. Add a Vite
middleware plugin to astro.config.mjs that rewrites /activity/<id>/ to /activity/
in dev, matching what nginx does in production.
micropip requires the full PEP 427 wheel filename (name-version-py-abi-plat.whl)
— writing the file as bincio.whl caused InvalidWheelFilename. The wheel URL from
/api/wheel/version now provides the basename; it flows through fetchWheelBase64 →
extractFile → WebView where the file is written with the correct name and
_wheel_path is set as a Pyodide global before PY_INSTALL_WHEEL runs.
Layout fix: WebView as a sibling in the root layout breaks flex geometry even
with position:absolute. Moving it inside the Import tab screen (which Expo Router
keeps mounted after first visit) eliminates the issue entirely and restores the
original simple root layout.
micropip fix: blob: URLs are not a recognised scheme in micropip — they are parsed
as package requirement strings, producing InvalidRequirement. Write the wheel bytes
to Pyodide's Emscripten FS (/tmp/bincio.whl) and install via emfs:/// instead.
WKWebView blocks HTTP requests (ATS) even when NSAllowsLocalNetworking is set for
the app's own networking — so fetch(http://192.168.x.x/api/wheel/download) inside
the WebView always fails with 'Load failed' on iOS.
- extractActivity.ts: rename wheelUrl param to wheelBase64; WebView now receives
the wheel as pre-fetched base64 bytes rather than a URL to fetch itself
- PyodideWebView.tsx: decode wheelBase64 → Uint8Array → Blob → blob URL for
micropip.install; fix baseUrl '' → 'https://localhost' (null origin blocks fetch
on iOS)
- import.tsx: add fetchWheelBase64() that resolves the wheel URL via
/api/wheel/version then fetches with native networking (HTTP works); caches
result in memory so repeated imports in one session don't re-download
- extraction/PyodideWebView.tsx: hidden WebView (1×1 px, off-screen) that
bootstraps Pyodide v0.26.4 from jsDelivr CDN on app startup; loads lxml,
pyyaml, micropip, fitdecode, gpxpy automatically; installs the bincio wheel
lazily on the first extraction call via a blob URL (avoids startup delay)
- extraction/extractActivity.ts: typed bridge — extractFile(filename, base64,
wheelUrl, onStatus) injects JS into the WebView, tracks pending promises by
request ID, resolves with { id, detail, timeseries, geojson, sourceHash }
- app/_layout.tsx: mounts <PyodideWebView> outside SQLiteProvider at root so
the runtime warms up as soon as the app opens
- app/(tabs)/import.tsx: replaces the placeholder alert with real extraction;
reads files as base64, calls extractFile with a progress callback, stores
detail_json + timeseries_json + geojson + real SHA-256 source_hash; resolves
wheel URL via GET /api/wheel/version with fallback to /api/wheel/download;
falls back to bincio.org if no instance is configured
- Mark Phase 1 as the critical unbuilt feature; note its prerequisite chain
(SHA-256 dedup, re-extract button, Phase 2 auto-import all depend on it)
- Flag Phase 2 auto_import_path as stubbed in UI but unimplemented in background
- Add "Known gaps and technical debt" section covering:
- source_hash stub (id+length, not SHA-256)
- FIT/GPX/TCX import placeholder alert
- auto_import_path field with no backend task
- feed pagination (getAllSync with no LIMIT)
- individual activity deletion (missing from plan)
- feed search and filter (missing from plan)
- token expiry / inline reconnect flow (missing from plan)
- app icon and splash screen (Expo defaults)
- upload skip behaviour for origin=remote rows
- Add Phase 5 items not previously in the plan: app icon, feed search/filter,
individual deletion, token reconnect prompt
- Server: POST /api/upload/bas accepts pre-extracted BAS JSON (activity + optional timeseries/geojson), writes files and triggers merge_all
- sync.ts: uploadLocalActivities reads unsynced local activities by original_path, POSTs to /api/upload/bas, marks synced_at on success
- Settings: Upload toggle (Off / Upload local activities) in Sync section with subLabel dividers for Download / Upload groups
- Feed: sync message includes uploaded count when activities are pushed
- Add /api/activity/{id}/geojson and /api/activity/{id}/timeseries endpoints
(bearer-token-gated, falls back from _merged to raw activities dir)
- Rewrite activity detail screen with MapLibreGL v11 API (Map, Camera,
GeoJSONSource, Layer) and react-native-svg area chart with gradient fill
- On-demand fetch for remote activities that have no local geojson/timeseries
- Add react-native-svg dependency; requires dev build (npx expo run:android)
_merged/index.json is a shard manifest with activities:[] when the user
has >FEED_PAGE_SIZE activities. The endpoint now collects from all
index-{year}.json shard files before returning.
SyncResult gains a `total` field (activities received from server) so the
feed screen can distinguish "server returned nothing" from "all already
stored locally". Messages: "No activities on instance" / "Up to date (N)"
/ "X of N activities synced".
Server (bincio/serve/server.py):
- Add _require_auth: accepts session cookie OR Authorization: Bearer token
- POST /api/auth/token: same as /api/auth/login but returns token in body
(password used once, not stored; mobile stores only the session token)
- GET /api/feed: auth-gated; reads _merged/index.json for the user and
returns the activities array as JSON
Mobile:
- db/sync.ts: syncFeed(db) fetches /api/feed, upserts each summary into
local SQLite as origin='remote'; skips locally-imported activities
- db/queries.ts: add upsertRemoteActivity (INSERT ... ON CONFLICT DO UPDATE
WHERE origin='remote' — never overwrites local imports); fix feed sort
order to started_at DESC instead of insertion order
- settings.tsx: Connect section — password field (not persisted) + Connect
button calls POST /api/auth/token and stores token; Disconnect clears it
- index.tsx: ↓ Sync button + pull-to-refresh both trigger syncFeed; cloud
badge on remote activities; empty state updated
expo-router@6.0.23 declares react-dom as peerOptional; without an explicit
pin npm resolves react-dom@19.2.5 (latest) which conflicts with react@19.1.0.
The SDK 54 template pins both to 19.1.0 — mirroring that here.
react-dom@19.2.5 (pulled by @expo/router-server) requires react@^19.2.5;
@react-native/virtualized-lists@0.85.2 requires @types/react@^19.2.0.
npm install now completes without errors or warnings.
- Add GET /api/wheel/download to serve/server.py and edit/server.py:
serves dist/bincio-*.whl via FileResponse; in production nginx takes
the request before FastAPI, so this is a no-op there but works locally
- wheel_version response now includes api_url: "/api/wheel/download"
alongside the nginx-served url field
- Bundle mobile/assets/bincio.whl (built from dist/) as an offline
fallback for Pyodide testing before the first instance sync
- docs/mobile-app.md: document dev setup — bundled asset, local server
endpoint, and how to refresh the bundle with uv build + cp
Davide Scaini