feat(mobile/upload): upload_format setting + Option A local update from server response
Settings → Sync → Upload format: "Original file" (default) / "Extracted JSON". - raw (default): reads original_path as base64, POSTs to /api/upload/raw; after success, overwrites local detail_json/timeseries_json/geojson/source_hash with the server's DEM-corrected extraction (Option A). Falls back to bas if the file is missing. - bas: POSTs pre-extracted JSON to /api/upload/bas, faster, no DEM correction. Switching modes is safe — the server deduplicates by activity id so a previous raw upload will return status:"duplicate" on a subsequent bas attempt.
This commit is contained in:
Davide Scaini
+10
-8
@@ -673,6 +673,7 @@ CREATE TABLE settings (
|
||||
| `api_token` | Bearer token for API auth (obtained via Connect, never stored in plaintext long-term) | Both |
|
||||
| `sync_mode` | `"summaries"` (default) or `"full"` — controls whether geojson+timeseries are downloaded during sync | Both |
|
||||
| `sync_upload` | `"true"` or `"false"` — whether to push local activities during sync | Both |
|
||||
| `upload_format` | `"raw"` (default) or `"bas"` — whether to upload the original FIT/GPX/TCX file (server re-extracts with DEM) or the pre-extracted JSON | Both |
|
||||
| `auto_import_path` | Directory to watch for new FIT files | **Android only** |
|
||||
|
||||
---
|
||||
@@ -700,17 +701,18 @@ Implemented in `mobile/db/sync.ts` → `uploadFeed()` / `uploadLocalActivities()
|
||||
|
||||
1. **Reconcile** against the server: fetch `GET /api/feed` and compare its activity IDs against local rows where `synced_at IS NOT NULL`. Any local activity that is marked as synced but absent from the server (e.g. server was wiped) has its `synced_at` cleared so it re-enters the upload queue. This is best-effort — if the feed fetch fails, upload proceeds with whatever is currently queued.
|
||||
2. Query `activities WHERE origin = 'local' AND synced_at IS NULL`.
|
||||
3. For each: parse `detail_json` from the DB row and construct `{ id: row.id, ...detail }`.
|
||||
4. `POST {instance_url}/api/upload/bas` with body `{ activity, timeseries?, geojson? }`.
|
||||
5. On 200 (including `{ status: "duplicate" }`): set `synced_at = unixepoch()`. On error: log to console, count as failed, continue with the next activity.
|
||||
3. For each, choose the upload path based on the `upload_format` setting (default `'raw'`):
|
||||
- **`raw` (default):** if `original_path` is set and the file still exists on disk, read it as base64 and `POST {instance_url}/api/upload/raw { filename, base64 }`. The server re-extracts the file with DEM elevation correction and returns `{ id, detail, timeseries, geojson, source_hash }`. After a successful upload, the local row's `detail_json`, `timeseries_json`, `geojson`, and `source_hash` are updated with the server's better data (Option A). Falls back to `bas` if the file is missing.
|
||||
- **`bas`:** `POST {instance_url}/api/upload/bas { activity, timeseries?, geojson? }` with the pre-extracted JSON from the DB. Faster, but no DEM elevation correction.
|
||||
4. On 200 (including `{ status: "duplicate" }`): set `synced_at = unixepoch()`. On error: log to console, count as failed, continue with next activity.
|
||||
|
||||
The UI shows live progress ("Uploading N / M…") during the batch and reports failures separately ("X uploaded, Y failed").
|
||||
|
||||
The server endpoint (`bincio/serve/server.py` → `POST /api/upload/bas`) accepts
|
||||
pre-extracted BAS JSON rather than raw FIT/GPX/TCX. It writes the activity file,
|
||||
**updates `user_dir/index.json`** with a summary entry (so `merge_all` can include
|
||||
the activity in year shards and the browser feed), writes geojson and timeseries if
|
||||
provided, then calls `merge_all()` + `write_combined_feed()`.
|
||||
**Upload format setting (`upload_format`):** controls whether to prefer the original raw file or the pre-extracted JSON. `raw` is the default and is recommended — it produces DEM-corrected elevation data on the server and back-fills the local copy. Switching between modes is safe: the server deduplicates by activity id, so switching from `raw` to `bas` just results in the server returning `{ status: "duplicate" }` (HTTP 200) and the client marking the activity as synced.
|
||||
|
||||
**Server endpoint for BAS JSON** (`POST /api/upload/bas`): accepts pre-extracted BAS JSON, writes the activity file, updates `user_dir/index.json` with a summary entry (so `merge_all` can include the activity in year shards and the browser feed), writes geojson and timeseries if provided, then calls `merge_all()` + `write_combined_feed()`.
|
||||
|
||||
**Server endpoint for raw files** (`POST /api/upload/raw`): accepts a base64-encoded FIT/GPX/TCX file, runs full server-side extraction with DEM correction, stores the result, updates the index, calls `merge_all()` + `write_combined_feed()`, and returns the full extracted data to the client.
|
||||
|
||||
> **Bug that was fixed:** earlier versions of both `/api/upload/bas` and `/api/upload/raw`
|
||||
> wrote activity files to disk but never updated `user_dir/index.json`. Since `merge_all`
|
||||
|
||||
@@ -17,12 +17,14 @@ export default function SettingsScreen() {
|
||||
const storedToken = useSetting('api_token');
|
||||
const storedSyncMode = (useSetting('sync_mode') ?? 'summaries') as 'summaries' | 'full';
|
||||
const storedSyncUpload = useSetting('sync_upload') === 'true';
|
||||
const storedUploadFormat = (useSetting('upload_format') ?? 'raw') as 'raw' | 'bas';
|
||||
|
||||
const [instanceUrl, setInstanceUrl] = useState(storedUrl);
|
||||
const [handle, setHandle] = useState(storedHandle);
|
||||
const [autoPath, setAutoPath] = useState(storedPath);
|
||||
const [syncMode, setSyncMode] = useState(storedSyncMode);
|
||||
const [syncUpload, setSyncUpload] = useState(storedSyncUpload);
|
||||
const [uploadFormat, setUploadFormat] = useState(storedUploadFormat);
|
||||
const [saved, setSaved] = useState(false);
|
||||
const theme = useTheme();
|
||||
const { paletteKey: palette, setPaletteOverride } = usePaletteControl();
|
||||
@@ -213,6 +215,18 @@ export default function SettingsScreen() {
|
||||
? 'Local activities are uploaded to the instance during sync.'
|
||||
: 'Local activities stay on device only.'}
|
||||
</Text>
|
||||
<Text style={[styles.subLabel, { borderTopWidth: 1, borderTopColor: '#27272a', paddingTop: 12 }]}>Upload format</Text>
|
||||
<View style={styles.modeRow}>
|
||||
<ModeButton label="Original file" active={uploadFormat === 'raw'} accent={theme.accent} dim={theme.dim}
|
||||
onPress={() => { setUploadFormat('raw'); setSetting(db, 'upload_format', 'raw'); }} />
|
||||
<ModeButton label="Extracted JSON" active={uploadFormat === 'bas'} accent={theme.accent} dim={theme.dim}
|
||||
onPress={() => { setUploadFormat('bas'); setSetting(db, 'upload_format', 'bas'); }} />
|
||||
</View>
|
||||
<Text style={styles.hint}>
|
||||
{uploadFormat === 'raw'
|
||||
? 'Uploads the original FIT/GPX/TCX file. The server re-extracts it with DEM elevation correction and updates your local copy.'
|
||||
: 'Uploads the pre-extracted JSON. Faster, but no DEM elevation correction.'}
|
||||
</Text>
|
||||
</Section>
|
||||
|
||||
<Section title="Palette">
|
||||
|
||||
+37
-3
@@ -170,6 +170,7 @@ async function uploadLocalActivities(
|
||||
FROM activities WHERE origin = 'local' AND synced_at IS NULL`,
|
||||
);
|
||||
|
||||
const preferRaw = (await getSetting(db, 'upload_format') ?? 'raw') === 'raw';
|
||||
const headers = { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' };
|
||||
let uploaded = 0;
|
||||
let failed = 0;
|
||||
@@ -182,9 +183,10 @@ async function uploadLocalActivities(
|
||||
try {
|
||||
let resp: Response;
|
||||
|
||||
// Prefer raw upload when the original FIT/GPX/TCX file is still on disk.
|
||||
// The server re-extracts it with DEM elevation correction, producing better data.
|
||||
const useRaw = row.original_path !== null &&
|
||||
// When preferRaw is set and the original file is still on disk, send the raw
|
||||
// bytes to /api/upload/raw so the server re-extracts with DEM elevation correction.
|
||||
const useRaw = preferRaw &&
|
||||
row.original_path !== null &&
|
||||
(await FileSystem.getInfoAsync(row.original_path)).exists;
|
||||
|
||||
if (useRaw) {
|
||||
@@ -211,6 +213,38 @@ async function uploadLocalActivities(
|
||||
|
||||
if (resp.ok) {
|
||||
await db.runAsync(`UPDATE activities SET synced_at = ? WHERE id = ?`, [now, row.id]);
|
||||
// Option A: after a raw upload, update local detail/timeseries/geojson with the
|
||||
// server's DEM-corrected extraction so the app shows better elevation data.
|
||||
if (useRaw) {
|
||||
try {
|
||||
const data = await resp.json() as {
|
||||
id: string;
|
||||
detail: object;
|
||||
timeseries: object | null;
|
||||
geojson: object | null;
|
||||
source_hash: string;
|
||||
};
|
||||
if (data.id === row.id) {
|
||||
await db.runAsync(
|
||||
`UPDATE activities
|
||||
SET detail_json = ?,
|
||||
timeseries_json = COALESCE(?, timeseries_json),
|
||||
geojson = COALESCE(?, geojson),
|
||||
source_hash = ?
|
||||
WHERE id = ?`,
|
||||
[
|
||||
JSON.stringify(data.detail),
|
||||
data.timeseries ? JSON.stringify(data.timeseries) : null,
|
||||
data.geojson ? JSON.stringify(data.geojson) : null,
|
||||
data.source_hash,
|
||||
row.id,
|
||||
],
|
||||
);
|
||||
}
|
||||
} catch {
|
||||
// Non-fatal: synced_at is already set, local data stays as-is
|
||||
}
|
||||
}
|
||||
uploaded++;
|
||||
} else {
|
||||
console.warn(`upload ${row.id}: HTTP ${resp.status}`);
|
||||
|
||||
Reference in New Issue
Block a user