From b633d72258ce87c8c22aa986198d555bb20d0dca Mon Sep 17 00:00:00 2001 From: Davide Scaini Date: Mon, 6 Apr 2026 20:52:31 +0200 Subject: [PATCH] iterating on the plan --- ARCHITECTURE.md | 108 +++++++++++++++++++++++------------------------- 1 file changed, 51 insertions(+), 57 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 02d6e60..71d991d 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -159,7 +159,7 @@ sequenceDiagram User->>Page: opens /convert/ Page->>Pyodide: load Pyodide + packages\n(lxml, fitdecode, bincio wheel) - Note over Page,Pyodide: ~8MB, cached after first visit + Note over Page,Pyodide: ~10MB, cached after first visit User->>Page: selects GPX / FIT / TCX file Page->>Pyodide: write file to virtual FS\nrun parse → metrics → write @@ -212,7 +212,8 @@ flowchart TB A2 -->|BAS JSON URL| C B2 -->|BAS JSON URL| C - Note1["Bob adds Alice's index.json URL\nto his followed feeds.\nNo accounts. No central server.\nJust a URL."] + %% Bob adds Alice's index.json URL to his followed feeds. + %% No accounts. No central server. Just a URL. ``` Federation is a planned feature — the data format (BAS JSON) is designed for it, but the site UI doesn't yet support adding followed feeds. @@ -230,8 +231,7 @@ The existing Astro/Svelte site is wrapped in a **Capacitor** native shell to pro | Background GPS (iOS) | ✗ killed by OS | ✅ native entitlement | | Background GPS (Android) | ⚠️ limited | ✅ foreground service | | Filesystem access | ✗ sandboxed | ✅ full device storage | -| Install without App Store | ✅ | iOS: App Store / TestFlight | -| Local HTTP server | ✗ | ✅ (future, via native plugin) | +| Install without App Store | ✅ (Android sideload) | iOS: App Store or TestFlight | ### Architecture @@ -293,10 +293,10 @@ Phone app: /convert/ → user picks GPX file Phone app: /record/ → GPS recording (native Geolocation) → export GPX → /convert/ /convert/ → Pyodide converts (loaded from local cache) - → writes BAS JSON to device storage (Filesystem plugin) -Local Node server → serves BAS JSON from device storage to WebView -WebView → feed + activity pages read from local server - → edit drawer saves sidecars to device storage + → saveActivityLocally() writes to IndexedDB +Data loader → loadIndex() / loadActivity(id) read from IndexedDB +WebView → feed + activity pages populated from local data + → edit drawer writes sidecars (merge logic in JS) ``` See "Fully offline — missing pieces" below for implementation status. @@ -335,74 +335,68 @@ Total cold-start download: ~10 MB. Subsequent visits: instant (all cached). ### Fully offline — missing pieces -Five things need to be in place before Workflow 4 works end-to-end. +#### 1. Data access abstraction *(do this first)* -#### 1. Pyodide available offline +The prior design asked "how do we serve `/data/*` to the WebView from local storage?" — but that question skips a cleaner solution: eliminate the `fetch('/data/*')` calls from the hot path entirely. -Currently loads from CDN on first use. Two options: +The Svelte components (`ActivityFeed`, `ActivityDetail`, etc.) currently call `fetch(...)` directly. Replacing those with a thin loader module (`loadIndex()`, `loadActivity(id)`) with two implementations decouples the data source from the UI: -- **Service worker cache** *(recommended)* — on first online visit the service worker caches all Pyodide assets; subsequent visits work offline. Standard PWA pattern, no app size increase. -- **Bundle in app assets** — copy `pyodide/` into `site/public/` at build time. Adds ~25 MB to the app but works with zero network dependency. Use if targeting environments with no initial connectivity. +``` +src/lib/dataloader.ts + loadIndex() → fetch('/data/index.json') [cloud build] + → idbGet('/data/local-index') [app build] + loadActivity(id) → fetch('/data/activities/...') [cloud build] + → idbGet('/data/activities/...') [app build] +``` + +The build variant is selected by an env var (`PUBLIC_DATA_MODE=cloud|local`) at Astro build time. No service worker required. No iOS WKWebView question. Useful regardless of offline support (testability, mocking, future federation transports). + +**This is the first thing to build.** The service worker work already done (`sw.js`, `localstore.ts`) provides a working fallback and is useful for the browser, but the abstraction layer is cleaner and should land before investing more in the SW path. Status: not yet implemented. -#### 2. Write converted activities to device storage +#### 2. Pyodide available offline -`@capacitor/filesystem` is already installed. After Pyodide converts a file, instead of (or in addition to) POSTing to the edit server, the convert page should: -1. Write `{id}.json` and `{id}.geojson` to a local `activities/` directory via the Filesystem plugin -2. Append the summary to a local `index.json` - -This is self-contained JS work — no native changes needed. +Currently loads from CDN (~10 MB) on first use. A service worker can cache the CDN assets on first online visit so all subsequent visits work offline. Standard PWA pattern, no app size increase, no code changes to the convert page. Status: not yet implemented. -#### 3. Serve local activity data to the WebView *(the core problem)* +#### 3. Sidecar merge in JS -The Astro site reads `/data/index.json` and `/data/activities/*.json` via `fetch()`. In the current setup these come from either bundled static assets (baked in at build time) or a remote server. There is no built-in mechanism for a WebView to dynamically serve locally-stored files at those URLs. +The edit drawer currently calls the edit server's `POST /api/activity/{id}` which triggers the Python `merge_all()`. For offline editing this needs to run locally. Port to JS (~150 lines) — **do not use Pyodide for this**. -Two approaches: - -**Option A — Service worker interception** *(web-only, no native code)* -A service worker intercepts `fetch('/data/*')` and responds with data from IndexedDB. No native plugin required. Limitation: iOS WKWebView has historically restricted service worker scope in Capacitor apps, though this has improved in recent iOS versions. Best option if cross-platform parity is not critical. - -**Option B — Local HTTP server via `capacitor-nodejs`** *(recommended for iOS reliability)* -[`capacitor-nodejs`](https://github.com/hampoelz/Capacitor-NodeJS) embeds a Node.js runtime inside the Capacitor app. A small Express/Fastify server: -- Serves the static site assets (or proxies to the WebView asset loader) -- Serves BAS JSON from device storage at `/data/*` -- Handles the edit/save API (`/api/import-bas`, `/api/activity/{id}`, etc.) -The WebView points to `http://localhost:PORT/`. Reliable on both iOS and Android. - -Status: not yet implemented. Option B is the recommended path. - -#### 4. Port edit server API to local JS - -Once Option B above is in place, the Node.js server needs to implement the same API surface as the current FastAPI edit server — at minimum: - -| Endpoint | Purpose | -|---|---| -| `POST /api/import-bas` | Save converted activity to device storage | -| `GET /api/activity/{id}` | Read activity + sidecar | -| `POST /api/activity/{id}` | Write sidecar to device storage | -| `GET /data/index.json` | Serve merged index from device storage | -| `GET /data/activities/*` | Serve individual activity files | - -The sidecar merge logic (`merge_all`) can either be reimplemented in JS (~150 lines) or invoked via Pyodide (already present in the app). - -Strava sync and image upload require network access by definition — they are out of scope for offline mode. +Pyodide is lazy-loaded on `/convert/`. It is not warm just because the app is open. A user tapping Edit → change title → Save would trigger a full ~10 MB Pyodide cold start for a trivial one-line edit. The edit drawer must remain decoupled from the convert page's machinery. Status: not yet implemented. -#### 5. Summary +#### 4. If service workers fail on iOS + +If testing in a Capacitor iOS WebView reveals that service workers are blocked, fall back to one of: + +1. **Data access abstraction only** (§1 above) — if all `fetch('/data/*')` calls are gone, the SW question is moot. +2. **Native Swift/Kotlin micro-server** — a small HTTP server embedded in the native layer, serving from device storage. No third-party plugins. +3. **Re-run `cap sync` on import** — bundle updated static data files and resync. Crude but reliable for low-frequency imports. + +`capacitor-nodejs` is not a fallback option. It embeds a full Node runtime, has spotty iOS support, and carries significant long-term maintenance cost. + +#### 5. Open questions — data lifecycle on device + +These don't need solutions today but will surface during implementation: + +- **Storage limits:** iOS may evict IndexedDB under storage pressure. `@capacitor/filesystem` stores in the app's Documents directory (not evictable) — may be preferable for activity data. +- **Uninstall / reinstall:** IndexedDB is wiped on uninstall. Documents directory survives reinstall on iOS. Strategy TBD. +- **Sync conflicts:** if the same activity exists locally and on a cloud instance (e.g. uploaded via Strava, also recorded in-app), the merge strategy is undefined. Likely: server wins on pull, local wins on push, user resolves conflicts manually. + +#### 6. Implementation order | Step | Effort | Status | |---|---|---| +| Data access abstraction (`dataloader.ts`) | Medium | Not started — **do first** | +| Sidecar merge in JS | Medium | Not started | | Pyodide service worker cache | Medium | Not started | -| Write to Filesystem after convert | Easy | Not started | -| Local Node.js server (`capacitor-nodejs`) | Hard | Not started | -| Port edit API to local JS | Medium | Not started | -| Sidecar merge in JS (or via Pyodide) | Medium | Not started | - -The local Node.js server (step 3B) is the critical path item — it unblocks steps 4 and makes the feed dynamic. Steps 1 and 2 are independent and can be done in any order. +| Test SW in Capacitor Android WebView | Low | Not started | +| Test SW on iOS | Low | Not started | +| Native micro-server (iOS fallback, if needed) | Hard | Contingency only | ---