Add MkDocs documentation: architecture, deployment, development, decisions
This commit is contained in:
brutsalvadi
@@ -0,0 +1,115 @@
|
||||
# Authentication & Invites
|
||||
|
||||
## Shared database
|
||||
|
||||
BincioWiki shares its user database with `bincio_activity`. Both services read from and write to the same SQLite file at `/var/bincio/data/instance.db`. There is no separate user store for the wiki.
|
||||
|
||||
The relevant tables:
|
||||
|
||||
### `users`
|
||||
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
handle TEXT PRIMARY KEY,
|
||||
display_name TEXT NOT NULL DEFAULT '',
|
||||
password_hash TEXT NOT NULL,
|
||||
is_admin INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL,
|
||||
wiki_access INTEGER NOT NULL DEFAULT 1,
|
||||
activity_access INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
```
|
||||
|
||||
`wiki_access = 1` grants access to BincioWiki. `activity_access = 1` grants access to BincioActivity. The two flags are independent; a user can have one, both, or neither.
|
||||
|
||||
### `sessions`
|
||||
|
||||
```sql
|
||||
CREATE TABLE sessions (
|
||||
token TEXT PRIMARY KEY,
|
||||
handle TEXT NOT NULL REFERENCES users(handle) ON DELETE CASCADE,
|
||||
created_at INTEGER NOT NULL,
|
||||
expires_at INTEGER NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
Sessions expire after **30 days**. The cookie is `bincio_session`, HTTP-only, `SameSite=Lax`, domain `.bincio.org` (shared across all subdomains).
|
||||
|
||||
### `invites`
|
||||
|
||||
```sql
|
||||
CREATE TABLE invites (
|
||||
code TEXT PRIMARY KEY,
|
||||
created_by TEXT NOT NULL REFERENCES users(handle) ON DELETE CASCADE,
|
||||
used_by TEXT REFERENCES users(handle) ON DELETE SET NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
used_at INTEGER,
|
||||
grants_activity INTEGER NOT NULL DEFAULT 0,
|
||||
grants_wiki INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
```
|
||||
|
||||
Wiki invites are created with `grants_wiki = 1, grants_activity = 0`. The `grants_wiki` column was added by a migration in `edit/server.py` (`_migrate_db()` runs at sidecar startup).
|
||||
|
||||
## Session flow
|
||||
|
||||
```
|
||||
POST /api/auth/login → bcrypt.checkpw → INSERT sessions → Set-Cookie: bincio_session
|
||||
GET /api/me → SELECT sessions JOIN users → 200 + user JSON | 401
|
||||
POST /api/auth/logout → DELETE sessions → Clear cookie
|
||||
```
|
||||
|
||||
Every protected Astro page calls `GET /api/me` on load via inline JavaScript. A 401 or 403 response triggers an immediate `window.location.replace('/login/')`.
|
||||
|
||||
## Security model
|
||||
|
||||
This is **client-side enforcement only**. The static HTML is served publicly by nginx without any HTTP-level authentication. A user without JavaScript, or one who crafts direct HTTP requests, can read the raw HTML.
|
||||
|
||||
This is an intentional, documented tradeoff. The content is community memories (cycling routes, gear notes, trip reports) — not financial or medical data. The goal is keeping search engine crawlers and casual visitors out, not resisting determined attackers. All members of the community are trusted.
|
||||
|
||||
For future pages that require stricter enforcement, the architecture supports adding an Astro SSR mode with server-side cookie validation before any HTML is rendered. This would require migrating from `output: "static"` to `output: "server"`.
|
||||
|
||||
## Invite system
|
||||
|
||||
### Creating an invite
|
||||
|
||||
Any wiki user can generate an invite link from `/invites/`. The sidecar creates a single-use token:
|
||||
|
||||
```
|
||||
POST /api/invites
|
||||
→ { "code": "abc123def456ghi7" }
|
||||
```
|
||||
|
||||
The invite URL is `https://wiki.bincio.org/join/?code=<token>`.
|
||||
|
||||
### Registration
|
||||
|
||||
The invitee visits the URL and fills in a handle, optional display name, and password. On submit:
|
||||
|
||||
```
|
||||
POST /api/auth/register
|
||||
{ code, handle, display_name, password }
|
||||
```
|
||||
|
||||
The sidecar:
|
||||
|
||||
1. Validates the code exists and `used_by` is null.
|
||||
2. Checks `grants_wiki = 1`.
|
||||
3. Validates the handle (lowercase, 2–20 chars, `[a-z][a-z0-9_-]{1,19}`).
|
||||
4. Validates password length ≥ 8 characters.
|
||||
5. Checks handle uniqueness.
|
||||
6. Inserts the user (`wiki_access=1, activity_access=0`) and marks the invite as used — in a single transaction.
|
||||
7. Creates a session and sets the cookie. The user is immediately logged in.
|
||||
|
||||
### Revoking an invite
|
||||
|
||||
A user can revoke any of their own unused invites via the `/invites/` page. Used invites cannot be revoked.
|
||||
|
||||
### User limits
|
||||
|
||||
The `settings` table stores a `max_wiki_users` value (currently `100`). The `/api/auth/register` endpoint reads this value and rejects registration if the current wiki user count has reached the limit.
|
||||
|
||||
```sql
|
||||
SELECT value FROM settings WHERE key = 'max_wiki_users';
|
||||
-- 100
|
||||
```
|
||||
@@ -0,0 +1,83 @@
|
||||
# Content Model
|
||||
|
||||
## Collections
|
||||
|
||||
Astro 6 content collections are defined in `site/src/content.config.ts`. All three load files from **outside** the `site/` submodule — Vite is configured with `server.fs.allow: ['..']` to allow this.
|
||||
|
||||
| Collection | Source directory | URL pattern |
|
||||
|------------|-----------------|-------------|
|
||||
| `entries` | `pages/*.md` | `/entries/{slug}/` |
|
||||
| `blog` | `blog/*.md` | `/blog/{slug}/` |
|
||||
| `index` | `config/*.md` | internal only (wikibonsai tree) |
|
||||
|
||||
### `entries` — wiki pages
|
||||
|
||||
Plain Markdown with minimal frontmatter:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Nome della pagina
|
||||
---
|
||||
|
||||
Contenuto in markdown...
|
||||
```
|
||||
|
||||
Slugs are derived from the filename minus extension. Subdirectory slugs are supported: `pages/bincio-tech/gps-e-ciclocomputer.md` → slug `bincio-tech/gps-e-ciclocomputer` → URL `/entries/bincio-tech/gps-e-ciclocomputer/`.
|
||||
|
||||
A `generateId` override in `content.config.ts` strips the `.md` extension and preserves slashes, so Astro does not apply any githubSlug mangling.
|
||||
|
||||
### `blog` — stories and posts
|
||||
|
||||
Blog frontmatter is richer:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Titolo del post
|
||||
description: Breve descrizione
|
||||
pubDate: 2024-05-01
|
||||
heroImage: /assets/foto.jpg # optional
|
||||
---
|
||||
```
|
||||
|
||||
### `index` — wikibonsai semantic tree
|
||||
|
||||
`config/i.bonsai.md` defines the hierarchical semantic tree used by the wikibonsai integration. It is not rendered as a page; Astro reads it to resolve `[[WikiLink]]` relationships and generate forward/back reference data.
|
||||
|
||||
## Sections
|
||||
|
||||
`config/sections.json` defines the top-level wiki sections (e.g. BincioTech, BincioOfficina) and their subsections. This file is imported by:
|
||||
|
||||
- The Astro pages/entries index to group and display pages
|
||||
- The `PageEditor.svelte` component to populate the section picker when creating a new page
|
||||
|
||||
Current sections: `bincio-tech`, `bincio-officina`, `bincio-tour`, `bincio-corsa`, `bincio-abbigliamento`.
|
||||
|
||||
## WikiLinks and wikibonsai
|
||||
|
||||
Pages can reference each other with `[[WikiLink]]` syntax. The remark pipeline resolves these at build time:
|
||||
|
||||
1. `remark-caml` parses CAML attribute syntax in frontmatter.
|
||||
2. `remark-wikirefs` resolves `[[target]]` to `/entries/{slug}/` links using `resolveHtmlHref` and `resolveHtmlText` from `site/src/wikibonsai/wikirefs.ts`.
|
||||
3. `generateForeRefsRemarkPlugin` collects all forward references so they can be stored in the content store.
|
||||
4. Back-references (pages that link to the current page) are computed at build time in `site/src/wikibonsai/backrefs.ts` and rendered by the `BackRefs.astro` component.
|
||||
|
||||
Invalid wikilinks (targets that don't exist) are rendered as dimmed grey text rather than broken links.
|
||||
|
||||
## Assets
|
||||
|
||||
User-uploaded images live in `assets/` at the repo root. This directory is:
|
||||
|
||||
- **Gitignored** — not versioned, not part of any git commit.
|
||||
- **Served** by FastAPI's `StaticFiles` mount at `/assets/{filename}` (nginx proxies `/assets/` to FastAPI in production).
|
||||
- **Uploaded** via `POST /api/assets` (multipart, images only, max 10 MB). The editor inserts `` into the Markdown on upload.
|
||||
|
||||
On the VPS, assets are separate from the git push/pull cycle and must be managed independently if ever migrated.
|
||||
|
||||
## Delete behaviour
|
||||
|
||||
Deleting a page via the editor calls `DELETE /pages/{slug}` or `DELETE /stories/{slug}`. The sidecar:
|
||||
|
||||
1. Unlinks the file.
|
||||
2. Commits the deletion with `git rm`.
|
||||
|
||||
In Astro dev mode, a `contentDeleteWatcher` Vite plugin watches `pages/` and `blog/` for `unlink` events. When a `.md` file is deleted it clears `.astro/data-store.json` and restarts the Astro dev server, forcing a full re-scan. Without this, Astro's persisted data store would keep serving the deleted entry.
|
||||
@@ -0,0 +1,129 @@
|
||||
# Edit Sidecar
|
||||
|
||||
The edit sidecar (`edit/server.py`) is a FastAPI application that handles all write operations. It runs alongside the static Astro build and is the only process that modifies content on disk.
|
||||
|
||||
## API endpoints
|
||||
|
||||
### Auth
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/api/me` | session | Return current user info |
|
||||
| `POST` | `/api/auth/login` | — | Create session, set cookie |
|
||||
| `POST` | `/api/auth/logout` | session | Destroy session, clear cookie |
|
||||
| `POST` | `/api/auth/register` | — | Register via invite token |
|
||||
|
||||
### Wiki pages
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/pages` | List all page slugs |
|
||||
| `GET` | `/pages/{slug}` | Get page content + `base_hash` |
|
||||
| `POST` | `/pages/{slug}` | Save page (with 3-way merge) |
|
||||
| `DELETE` | `/pages/{slug}` | Delete page |
|
||||
|
||||
### Blog / stories
|
||||
|
||||
Same as pages, with `/stories` prefix.
|
||||
|
||||
### Invites
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/invites` | List own invites |
|
||||
| `POST` | `/api/invites` | Create a new wiki invite |
|
||||
| `DELETE` | `/api/invites/{code}` | Revoke an unused invite |
|
||||
|
||||
### Other
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/api/assets` | Upload image (multipart) |
|
||||
| `POST` | `/rebuild` | Trigger Astro rebuild + rsync |
|
||||
| `GET` | `/api/log` | Last 50 web-editor commits |
|
||||
|
||||
## Git attribution
|
||||
|
||||
Every page save or delete is committed to git with the editor's handle as the author:
|
||||
|
||||
```
|
||||
git add pages/bincio-tech/gps-e-ciclocomputer.md
|
||||
git commit -m "brut: edited gps-e-ciclocomputer" \
|
||||
--author="brut <brut@bincio.wiki>"
|
||||
git update-ref refs/heads/main HEAD
|
||||
```
|
||||
|
||||
The committer is always `bincio-wiki <wiki@bincio.wiki>`. The author field carries the human attribution.
|
||||
|
||||
`GIT_COMMITTER_NAME` and `GIT_COMMITTER_EMAIL` are set in the environment passed to each subprocess. On the VPS, `GIT_DIR` points to the bare repo at `/opt/bincio-wiki-repo.git` and `GIT_WORK_TREE` to `/opt/bincio_wiki`.
|
||||
|
||||
### The `update-ref` fix
|
||||
|
||||
The VPS post-receive hook runs `git checkout -f <SHA>` to check out the new tree. This detaches `HEAD` in the bare repo: subsequent sidecar commits advance the detached HEAD pointer but not `refs/heads/main`. On the next `git push` from a developer's machine, those sidecar commits become unreachable and are eventually garbage-collected.
|
||||
|
||||
The fix: after every `git commit`, the sidecar immediately runs:
|
||||
|
||||
```
|
||||
git update-ref refs/heads/main HEAD
|
||||
```
|
||||
|
||||
This keeps `refs/heads/main` in sync with the detached HEAD, so sidecar commits are always reachable via the branch and survive a future push.
|
||||
|
||||
### Serialisation
|
||||
|
||||
All git operations are serialised with a module-level `asyncio.Lock()`. This prevents two concurrent saves from interleaving `git add` + `git commit` calls and producing a corrupted index or commit history.
|
||||
|
||||
## Optimistic concurrency
|
||||
|
||||
When a page is opened for editing, the sidecar returns the current HEAD commit hash alongside the content:
|
||||
|
||||
```json
|
||||
{ "slug": "gps-e-ciclocomputer", "content": "...", "base_hash": "a85a2ee" }
|
||||
```
|
||||
|
||||
The editor holds this hash. On save, it sends it back:
|
||||
|
||||
```json
|
||||
{ "content": "...", "base_hash": "a85a2ee" }
|
||||
```
|
||||
|
||||
The sidecar compares `base_hash` against the current HEAD for that file. If they differ, another edit landed while the user was writing.
|
||||
|
||||
## 3-way merge
|
||||
|
||||
Rather than rejecting the save with a 409, the sidecar attempts a 3-way merge using `git merge-file`:
|
||||
|
||||
```
|
||||
base = content at base_hash (what the user started with)
|
||||
current = content at HEAD (what's on disk now)
|
||||
user = content from request (what the user wrote)
|
||||
```
|
||||
|
||||
If the edits touched different lines, `git merge-file` produces a clean merged result and the save proceeds transparently. If the same lines were changed by both, `git merge-file` exits with a non-zero code and embeds conflict markers (`<<<<<<< attuale / ======= / >>>>>>> handle`). The sidecar returns a 409 with the conflict-marked content so the user can resolve manually.
|
||||
|
||||
```python
|
||||
# exit code 0 → clean merge; > 0 → conflicts present
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"git", "merge-file",
|
||||
"-L", "attuale", "-L", "base", "-L", handle,
|
||||
str(p_current), str(p_base), str(p_user),
|
||||
...
|
||||
)
|
||||
await proc.wait()
|
||||
merged_content = p_current.read_text()
|
||||
has_conflict = proc.returncode > 0
|
||||
```
|
||||
|
||||
## Rebuild
|
||||
|
||||
`POST /rebuild` does three things:
|
||||
|
||||
1. Deletes `.astro/data-store.json` — Astro's persisted content cache. Without this, changes to files in `pages/` and `blog/` (which live outside `site/`) are not picked up by a cached build.
|
||||
2. Runs `npm run build -- --force` in `site/`. The `--force` flag tells Astro to ignore any remaining stale cache.
|
||||
3. If `WIKI_WEBROOT` is set, rsyncs `site/dist/` to the webroot so nginx serves the updated build immediately.
|
||||
|
||||
The rebuild is triggered automatically by the `PageEditor.svelte` after every successful save.
|
||||
|
||||
## Slug validation
|
||||
|
||||
Slugs are validated against `^[a-zA-Z0-9_][a-zA-Z0-9_\-/]*$` and checked for path traversal before any file operation. The resolved path must be a child of the configured base directory.
|
||||
@@ -0,0 +1,57 @@
|
||||
# System Overview
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
Browser
|
||||
│
|
||||
│ HTTPS (443)
|
||||
▼
|
||||
nginx ─── static files ──► /var/www/bincio-wiki/ (Astro build output)
|
||||
│
|
||||
│ proxy_pass (127.0.0.1:4042)
|
||||
▼
|
||||
FastAPI sidecar (edit/server.py)
|
||||
├── reads/writes ──► pages/ (wiki markdown)
|
||||
├── reads/writes ──► blog/ (blog markdown)
|
||||
├── reads/writes ──► assets/ (uploaded images)
|
||||
├── git add + commit ──► /opt/bincio-wiki-repo.git (bare repo)
|
||||
└── reads/writes ──► /var/bincio/data/instance.db (shared SQLite)
|
||||
```
|
||||
|
||||
## Request flow
|
||||
|
||||
1. Browser requests `wiki.bincio.org/entries/some-page/`.
|
||||
2. nginx serves the pre-built static HTML from `/var/www/bincio-wiki/`.
|
||||
3. The page's inline JavaScript calls `GET /api/me` on load. nginx proxies this to FastAPI.
|
||||
4. If FastAPI returns 401/403 (no valid session), JS redirects to `/login/`.
|
||||
5. On a page edit, the browser `POST /pages/{slug}` with new Markdown content. FastAPI writes the file, runs `git commit`, then `POST /rebuild` triggers `astro build --force`.
|
||||
6. nginx serves the rebuilt static output immediately on next request.
|
||||
|
||||
## Technology stack
|
||||
|
||||
| Layer | Technology |
|
||||
|-------|------------|
|
||||
| Frontend framework | [Astro 6](https://astro.build) (static output) |
|
||||
| CSS | Tailwind CSS |
|
||||
| Interactive editor | Svelte (PageEditor component) |
|
||||
| Markdown extensions | `remark-wikirefs`, `remark-caml` |
|
||||
| Wikibonsai integration | Custom TypeScript (`site/src/wikibonsai/`) |
|
||||
| API / sidecar | FastAPI + uvicorn |
|
||||
| Python packaging | uv |
|
||||
| Database | SQLite (shared with `bincio_activity`) |
|
||||
| Passwords | bcrypt |
|
||||
| VCS | Git (two bare repos on VPS) |
|
||||
| Web server | nginx + Let's Encrypt (Certbot) |
|
||||
| Process manager | systemd |
|
||||
| VPS | Hetzner, Debian 12 |
|
||||
|
||||
## Port map
|
||||
|
||||
| Port | Service |
|
||||
|------|---------|
|
||||
| 443 | nginx (HTTPS, wiki.bincio.org) |
|
||||
| 4042 | bincio-wiki FastAPI (production) |
|
||||
| 8001 | bincio-wiki FastAPI (local dev) |
|
||||
| 4321 | Astro dev server (local dev only) |
|
||||
| 4041 | bincio_activity FastAPI |
|
||||
Reference in New Issue
Block a user