feat: NDJSON dispatcher, SFTP area, async jobs
SSH: one port now answers three ways. An interactive terminal gets the
menu; a command (`ssh host hash input=hi`) or a pipe with no terminal gets
a newline-delimited JSON dispatcher over a closed vocabulary with declared,
validated arguments; the sftp subsystem gets a private per-key file area
that tools read with file=NAME. Slow tools (net, links, data, path) can run
as background jobs (async=true, then `job id=...` from any connection with
the same key). Jobs live in memory.
Anonymous sessions are now throwaway identities: defaults for the
connection, nothing saved, no file area. Keying them by client IP made
everyone behind one NAT share and overwrite each other's settings.
HTTP: X-Forwarded-For / X-Real-IP are believed only from trusted proxies
(server.trustedProxies, loopback by default), read right to left. Weather
requests are rate limited per client IP (429 + Retry-After), dispatcher
calls per key, and geocoder calls per UTC day (geo.dailyLookupLimit) to
protect the OpenCage quota.
OCR is removed everywhere: ocrpdf package, menu entry, config keys,
translations and the ocrmypdf/tesseract packages in the Dockerfile. Old
configs carrying ocr* keys will fail at startup; use the ones in deploy/.
The storage purge is generalised to the file area (sweepStorageLoop).
Tests added for the HTTP server, client IP resolution, rate limiting,
dispatcher, jobs, file area and the SSH modes end to end. README and the
CloudPanel runbook describe the current ports, modes and limits, and
`make go-test-zig` runs the cgo tests without a C compiler.
author
Pablo Murad
date
24 Sep 2026, 14:36 -0300 commit
5ed49b87939f272adfdfbf2afb98e22d90509f52 parent
82964b8f1741f96aaa3fccb57e7ed94d37570178 changes
45 files +4611 −924
@@ -86,10 +86,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ ca-certificates \
iputils-ping \
mtr-tiny \
- ocrmypdf \
- tesseract-ocr \
- tesseract-ocr-por \
- tesseract-ocr-eng \
&& rm -rf /var/lib/apt/lists/*
COPY --from=host-tools /usr/local/bin/lychee /usr/local/bin/lychee
@@ -108,10 +104,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ ca-certificates \
iputils-ping \
mtr-tiny \
- ocrmypdf \
- tesseract-ocr \
- tesseract-ocr-por \
- tesseract-ocr-eng \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --create-home --uid 1000 --shell /usr/sbin/nologin wttr \
&& mkdir -p /app/cache /app/share/datasets \
@@ -12,3 +12,7 @@ true
go-test:
go test ./...
+
+# cgo without a C compiler (e.g. Windows): zig stands in for gcc.
+go-test-zig:
+ CGO_ENABLED=1 CC="zig cc -target x86_64-windows-gnu" go test ./...
@@ -20,16 +20,17 @@ ## Status
Hammer grew out of a Go reimplementation of [wttr.in](https://github.com/chubin/wttr.in).
-That heritage is still visible: the HTTP weather service is complete and is what most of
-this document describes.
+That heritage is still visible in the HTTP weather service, which is what most of this
+document describes.
| Part | State |
|---|---|
-| HTTP weather service | working — views, formats, 74 languages |
+| HTTP weather service | working: views, formats, 74 languages, per-IP rate limit |
| SSH front-end (menu, profiles, saved cities, tools) | working |
-| SSH tools (net, qr, hash, jq, age; optional lychee/DuckDB/MTR) | working |
-| NDJSON dispatcher for scripts and agents | planned |
-| File area over SFTP, async jobs | planned |
+| SSH tools (net, qr, hash, jq, age, csv, base64, ids; optional lychee/DuckDB/MTR) | working |
+| NDJSON dispatcher for scripts and agents | working |
+| Per-key file area over SFTP | working |
+| Background (async) jobs | working, in memory |
The direction is described under [Roadmap](#roadmap). Anything not listed as working
above does not exist yet, and this file will not pretend otherwise.
@@ -38,33 +39,135 @@ ---
## The SSH front-end
-Connect with any SSH client. There is no password and no account to create:
+One SSH port answers three ways:
+
+| You send | You get |
+|---|---|
+| an interactive terminal, no command | the menu |
+| a command (`ssh host hash`) or a pipe with no terminal | the [NDJSON dispatcher](#scripts-and-agents-ndjson) |
+| the `sftp` subsystem | your [file area](#file-area-sftp) |
+
+The port is set by `bbs.port` (2222 when absent; the public instance uses 2323):
```bash
-ssh -p 2222 your-server
+ssh -p 2323 your-server
```
-You are identified by your public key. On first connection a profile is created and, when
-GeoIP is configured, seeded with your city. Your default city, saved cities, units,
-language, view and emoji preference are stored against that key, so they survive across
-sessions.
+**With a key** you are identified by your public key. There is no password and no account
+to create. On first connection a profile is created and, when GeoIP is configured, seeded
+with your city. Your default city, saved cities, units, language, view and emoji
+preference are stored against that key, so they survive across sessions. The key also
+owns your file area and your background jobs.
+
+**Without a key** (`ssh -p 2323 guest@your-server`) you get in only when
+`bbs.allowAnonymous` is on. An anonymous connection is a throwaway identity: it starts
+from defaults, nothing you change is saved, it has no file area, and its jobs are visible
+only on that connection. Hammer does not key anonymous visitors by IP address, because
+everyone behind one NAT (an office, a campus, mobile CGNAT) would then share and
+overwrite each other's settings.
The menu offers:
-- **weather** — now, search a city, and saved cities
-- **net** — DNS, TLS, HTTP and RDAP for a public host
-- **qr** — ANSI QR code
-- **hash** — SHA, MIME, entropy over pasted bytes
-- **jq** — JSON/YAML transform
-- **age** — encrypt, decrypt, generate a key
-- **links / data / path** — lychee, DuckDB and ping/MTR, listed only when the binaries exist on the host
-- **settings** — units, language, view, emoji
-- **about** — project information
+- **weather**: now, search a city, and saved cities
+- **net**: DNS, TLS, HTTP and RDAP for a public host
+- **qr**: ANSI QR code
+- **hash**: SHA, MIME, entropy over pasted bytes
+- **jq**: JSON/YAML transform
+- **age**: encrypt, decrypt, generate a key
+- **links / data / path**: lychee, DuckDB and ping/MTR, listed only when the binaries exist on the host
+- **more**: CSV summary, Base64, UUIDs
+- **settings**: units, language, view, emoji
+- **about**: project information
The host key is generated on first start at `hostKeyPath`. Profiles live in the SQLite
database at `profilesDb`. The SSH front-end runs alongside the HTTP server on its own
port, so both are available from one process.
+Guardrails on the public port: at most `maxConnectionsPerIP` concurrent sessions per
+client address (5 by default), and tool calls through the dispatcher are rate limited
+per key, or per IP for anonymous callers (60 a minute, bursts of 20).
+
+### Scripts and agents (NDJSON)
+
+The dispatcher is a closed vocabulary: every tool declares its arguments, and a request
+that names an unknown tool or passes an unknown, mistyped, oversized or out-of-range
+argument is refused before anything runs. There is no shell behind it.
+
+As a command, the first word names the tool and the rest are `key=value` pairs. The
+answer is one JSON line, and the exit status is 0 on success and 1 on error:
+
+```bash
+ssh -p 2323 your-server hash input=hello
+ssh -p 2323 your-server weather city=Lisbon view=j1
+cat data.json | ssh -p 2323 your-server jq filter=.items
+ssh -p 2323 your-server help # the vocabulary, readable
+ssh -p 2323 your-server tools # the same, as JSON
+```
+
+Tools that take a payload read it from `input=...`, from `file=NAME` in your file area, or
+from stdin. Add `output=text` to print the human-readable lines instead of JSON.
+
+As a pipe, send one request per line and read one response per line, in order:
+
+```bash
+printf '%s\n' \
+ '{"id":1,"tool":"b64.encode","args":{"input":"hi"}}' \
+ '{"id":2,"tool":"ids","args":{"count":3}}' \
+ | ssh -T -p 2323 your-server
+```
+
+```json
+{"id":1,"ok":true,"tool":"b64.encode","result":{"data":{"base64":"aGk="},"lines":["aGk="]}}
+{"id":2,"ok":true,"tool":"ids","result":{"data":{"uuids":["…","…","…"]},"lines":["…","…","…"]}}
+```
+
+Every result carries `data` (structured, for programs) and usually `lines` (what the menu
+would print). Errors look like
+`{"ok":false,"error":{"code":"bad_args","message":"…"}}`. The codes are `bad_request`,
+`unknown_tool`, `bad_args`, `forbidden`, `not_found`, `rate_limited` (with
+`retry_after` in seconds), `busy` and `failed`.
+
+The vocabulary: `weather`, `net`, `qr`, `hash`, `jq`, `age.encrypt`, `age.decrypt`,
+`age.keygen`, `b64.encode`, `b64.decode`, `csv`, `ids`, plus `links`, `data` and `path`
+when their binaries are installed, and the built-ins `tools`, `files`, `job` and `jobs`.
+Ask the server with `tools` for the exact arguments each one takes.
+
+### Background jobs
+
+Slow tools (`net`, `links`, `data`, `path`) can run in the background. Add `async=true`
+(or `"async":true` in NDJSON) and you get a job id back straight away:
+
+```bash
+ssh -p 2323 your-server links file=README.md async=true
+ssh -p 2323 your-server job id=j_4f1c… output=text # later, from any connection with the same key
+ssh -p 2323 your-server jobs # your jobs, newest first
+```
+
+Jobs live in memory, so a restart forgets them. Results are kept for `bbs.jobs.ttl` (20
+minutes), a job may run for `bbs.jobs.timeout` (2 minutes), each key may have
+`maxPerKey` unfinished jobs (5), and `workers` (4) run at once across everyone.
+
+### File area (SFTP)
+
+Each key gets a private directory, reachable with any SFTP client:
+
+```bash
+sftp -P 2323 your-server
+sftp> put report.csv
+```
+
+Then point a tool at it:
+
+```bash
+ssh -p 2323 your-server csv file=report.csv
+```
+
+Paths are confined to your directory and links are refused. A key may hold
+`bbs.files.quotaBytes` (50 MiB), everyone together may hold `bbs.storage.globalQuotaBytes`,
+and tools read files up to `bbs.files.maxFileInput` (8 MiB). Files are deleted
+`bbs.storage.ttl` after their last upload (20 minutes by default), so treat the area as
+a workbench, not storage. Anonymous connections have no file area.
+
---
## The HTTP weather service
@@ -306,6 +409,22 @@ For a terminal without emoji support, `?d` restricts output to glyphs present in standard
console fonts, and the `v2d` and `v2n` views use Nerd Fonts.
+### Behind a proxy, and rate limits
+
+The client address decides the default location and units, so Hammer does not take it
+from request headers blindly. `X-Forwarded-For` and `X-Real-IP` are believed only when
+the connection comes from a trusted proxy, listed in `server.trustedProxies` as addresses
+or CIDR prefixes. The default is loopback (`127.0.0.0/8`, `::1/128`), which fits a
+reverse proxy on the same host; an empty list ignores the headers entirely. The forwarded
+chain is read from the right, skipping trusted hops, so entries a client writes itself
+are ignored.
+
+Weather requests are limited per client address (`server.rateLimit`, 60 a minute with
+bursts of 20 by default). Past that the answer is `429 Too Many Requests` with a
+`Retry-After` header. Static files are not limited. Every unseen city name costs one call
+to the geocoder, so `geo.dailyLookupLimit` also caps geocoder calls per UTC day across
+all clients; names already in the location cache do not count.
+
---
## Building
@@ -343,6 +462,16 @@ Other targets: `build.sh assets` re-syncs the embedded assets, `build.sh gen` regenerates
the options parser from `share/defs/`, and `build.sh clean` removes both.
+Run the tests with `go test ./...`. The SQLite-backed packages (`bbs`, `weather`, `server`,
+`location` and others) need cgo, like the build. On a machine without a C compiler,
+[zig](https://ziglang.org) can stand in for one:
+
+```bash
+CGO_ENABLED=1 CC="zig cc -target x86_64-windows-gnu" go test ./...
+```
+
+`make go-test-zig` runs the same thing. Drop the `-target` part on Linux or macOS.
+
---
## Configuration
@@ -373,6 +502,7 @@ - name: opencage type: opencage
url: https://api.opencagedata.com/geocode/v1/json
token: "YOUR_OPENCAGE_TOKEN"
+ dailyLookupLimit: 2000 # geocoder calls per UTC day; 0 = unlimited
weather:
wwo:
@@ -383,12 +513,33 @@ portHttp: 8080 # portHttps: 8443
# tlsCertFile: /hammer/etc/fullchain.pem
# tlsKeyFile: /hammer/etc/privkey.pem
+ trustedProxies: ["127.0.0.0/8", "::1/128"] # whose X-Forwarded-For to believe
+ rateLimit:
+ requestsPerMinute: 60 # per client IP; 0 turns it off
+ burst: 20
bbs:
enabled: true
port: 2222
+ allowAnonymous: false # true lets keyless clients in (nothing is saved)
+ maxConnectionsPerIP: 5
hostKeyPath: /hammer/etc/hammer_ed25519
profilesDb: /hammer/cache/hammer.db
+ storage:
+ globalQuotaBytes: 1073741824 # all file areas together
+ ttl: 20m # uploads are deleted this long after arriving
+ files:
+ dir: /hammer/cache/files # one private directory per key
+ quotaBytes: 52428800
+ maxFileInput: 8388608
+ dispatch:
+ requestsPerMinute: 60 # tool calls per key (per IP when anonymous)
+ burst: 20
+ jobs:
+ workers: 4
+ maxPerKey: 5
+ ttl: 20m
+ timeout: 2m
```
```bash
@@ -421,21 +572,14 @@ [research into which tasks justify being remote SSH workers](https://murad.gg/research/2026-08-26-pesquisa-130-workers-ssh-servicos-processamento-remoto): a dispatcher with a closed vocabulary and validated arguments, never an unrestricted
shell.
-The shape it is heading toward is one SSH server answering three ways on the same port and
-the same key. A PTY with no command gets the interactive menu. A command or a pipe gets a
-newline-delimited JSON protocol, which is what n8n, shell scripts and AI agents can drive.
-The `sftp` subsystem gets a per-key file area, which is what makes document and image
-tools usable at all.
+The shape described there now exists. One SSH server answers three ways on the same port
+and the same key: the menu for people, NDJSON for n8n, shell scripts and AI agents, and an
+SFTP file area that document and image tools can read from. Tools declare which binaries
+they need, and the menu and the dispatcher hide the missing ones in the same way.
-The interactive SSH menu lists weather as one entry (now / search / saved cities) and
-puts the tools on the main menu. Native ones (DNS/TLS/HTTP/RDAP, QR, hashing, jq/YAML,
-age) are always listed. lychee, DuckDB and ping/MTR appear only when their binaries are
-on `PATH`. DuckDB runs curated SQL from `datasetsDir` (see `share/datasets/`); it does
-not take arbitrary queries from the session.
-
-The NDJSON dispatcher, SFTP file area and async jobs are still planned. Tools will keep
-declaring which binaries they need so a later dispatcher can hide or refuse missing ones
-the same way the menu does.
+Still open: jobs are held in memory and do not survive a restart, results cannot yet be
+written back into the file area, and the catalogue itself is small. New tools should
+arrive through the dispatcher first, with the menu entry as a view over it.
---
@@ -10,8 +10,14 @@ → **`http://127.0.0.1:8081`**. Port **8080 is taken by CloudPanel's own nginx**, so the container's HTTP (8080 inside) is published on host
**127.0.0.1:8081**. CloudPanel terminates TLS (Let's Encrypt).
- **SSH BBS:** `ssh -p 2323 guest@hammer.pablomurad.com`. Published directly on
- TCP **2323** (no reverse proxy — SSH is not HTTP). `guest` is cosmetic; auth
- is by SSH public key.
+ TCP **2323** (no reverse proxy — SSH is not HTTP). The user name is cosmetic.
+ A client offering an SSH key is identified by it (saved profile, SFTP file
+ area, jobs). `bbs.allowAnonymous: true` in `config.docker.yaml` also lets
+ keyless clients in; they get defaults for that connection only, nothing is
+ saved, and they have no file area.
+- **Same port, three modes:** interactive terminal → menu; a command or a pipe
+ → NDJSON dispatcher; `sftp -P 2323` → the caller's file area
+ (`/app/cache/files`, inside the `deploy/cache` volume).
- **Port 22** stays with the box's admin sshd — do not touch it.
## Prerequisites
@@ -95,7 +101,15 @@ ### 8. Verify ```bash
curl -sS https://hammer.pablomurad.com/London | head # web weather via TLS
ssh -p 2323 guest@hammer.pablomurad.com # BBS splash
+ssh -p 2323 hammer.pablomurad.com hash input=hello # dispatcher: one JSON line
+echo put /etc/hostname | sftp -b - -P 2323 hammer.pablomurad.com # file area (needs a key)
```
+The client IP the weather service sees must be the visitor's, not the Docker
+gateway: `server.trustedProxies` in `config.docker.yaml` trusts loopback and
+`172.16.0.0/12` (Docker's bridge ranges), which is where CloudPanel's proxy
+arrives from inside the container. If the compose network lives elsewhere, add
+its gateway there, or visitors will all look like one address (one location,
+one rate-limit bucket).
## Redeploy / update
```bash
@@ -114,6 +128,12 @@ host key / profiles DB. - **`curl -L`** for the mmdb — MaxMind 302-redirects to R2.
- **`cap_add: NET_RAW, NET_ADMIN`** are required for the `path` tool (ping/MTR);
already set in the compose. Do NOT use `privileged: true`.
+- **Unknown config keys stop the service.** A `config.docker.yaml` from before the
+ OCR removal (`ocrInbox`, `ocrWorkDir`, `ocrTTL`, `ocrmypdfPath`) fails at startup;
+ use the one in the repo.
+- **Rate limits:** HTTP weather is limited per visitor IP (`server.rateLimit`),
+ dispatcher calls per key (`bbs.dispatch`), and geocoder calls per UTC day for
+ everyone (`geo.dailyLookupLimit`, set under the OpenCage free tier).
- **Never commit** `deploy/.env`, the `.mmdb`, or the CloudPanel site password.
-- Hammer refuses SSH connections presenting no public key; keyless anonymous
- guests would be a code change (new stdio/auth mode).
+- Profiles saved by older builds for anonymous visitors (keyed `ip:…`) are no
+ longer read and can stay in `hammer.db` harmlessly.
@@ -23,6 +23,7 @@ - name: opencage type: opencage
url: https://api.opencagedata.com/geocode/v1/json
token: "${OPENCAGE_TOKEN}"
+ dailyLookupLimit: 2000 # OpenCage free tier is 2,500/day; cached names don't count
ip:
ipCacheType: db
@@ -46,24 +47,40 @@ # HTTP weather — listens on 8080 inside the container; compose publishes it on # host 127.0.0.1:8081 (8080 belongs to CloudPanel's nginx).
server:
portHttp: 8080
+ # X-Forwarded-For is believed only from these peers. Inside the container
+ # the CloudPanel proxy arrives from the Docker bridge gateway, not loopback;
+ # the port is published on host loopback only, so nothing else reaches it.
+ trustedProxies: ["127.0.0.0/8", "::1/128", "172.16.0.0/12"]
+ rateLimit:
+ requestsPerMinute: 60 # per client IP
+ burst: 20
# SSH BBS — listens on 2323 inside the container; compose publishes 2323:2323.
bbs:
enabled: true
port: 2323
- allowAnonymous: true # keyless entry: ssh -p 2323 guest@host
+ allowAnonymous: true # keyless entry: ssh -p 2323 guest@host (nothing saved)
maxConnectionsPerIP: 5 # concurrent-session guardrail (0 = unlimited)
storage:
globalQuotaBytes: 1073741824 # ~1 GB shared by everyone
ttl: 20m # purge drains the quota within this window
hostKeyPath: /app/etc/hammer_ed25519
profilesDb: /app/cache/hammer.db
+ files:
+ dir: /app/cache/files # one private SFTP area per SSH key
+ quotaBytes: 52428800 # 50 MiB per key (the global quota above still applies)
+ maxFileInput: 8388608 # largest file a tool reads via file=NAME
+ dispatch:
+ requestsPerMinute: 60 # tool calls per key (per IP when anonymous)
+ burst: 20
+ jobs:
+ workers: 4 # background jobs running at once, all keys
+ maxPerKey: 5 # unfinished jobs per key
+ ttl: 20m # results kept this long
+ timeout: 2m # longest a job may run
tools:
datasetsDir: /app/share/datasets
maxPasteBytes: 65536
netTimeout: 12s
mtrCycles: 5
pingCount: 5
- ocrInbox: /app/cache/inbox
- ocrWorkDir: /app/cache/ocr
- ocrTTL: 20m
@@ -22,6 +22,7 @@ - name: opencage type: opencage
url: https://api.opencagedata.com/geocode/v1/json
token: "${OPENCAGE_TOKEN}"
+ dailyLookupLimit: 2000 # OpenCage free tier is 2,500/day; cached names don't count
ip:
ipCacheType: db
@@ -44,25 +45,39 @@ # HTTP weather service — kept on loopback-reachable 8080, fronted by nginx.
server:
portHttp: 8080
+ # nginx on this host is the only proxy; X-Forwarded-For from anyone else is ignored.
+ trustedProxies: ["127.0.0.0/8", "::1/128"]
+ rateLimit:
+ requestsPerMinute: 60 # per client IP
+ burst: 20
# SSH BBS — owns port 22 on this host so `ssh guest@hammer.pablomurad.com`
# works with no -p flag. Requires CAP_NET_BIND_SERVICE (see hammer.service).
bbs:
enabled: true
port: 22
- allowAnonymous: true # keyless entry: ssh guest@host
+ allowAnonymous: true # keyless entry: ssh guest@host (nothing saved)
maxConnectionsPerIP: 5 # concurrent-session guardrail (0 = unlimited)
storage:
globalQuotaBytes: 1073741824 # ~1 GB shared by everyone
ttl: 20m # purge drains the quota within this window
hostKeyPath: /hammer/etc/hammer_ed25519
profilesDb: /hammer/cache/hammer.db
+ files:
+ dir: /hammer/cache/files # one private SFTP area per SSH key
+ quotaBytes: 52428800 # 50 MiB per key (the global quota above still applies)
+ maxFileInput: 8388608 # largest file a tool reads via file=NAME
+ dispatch:
+ requestsPerMinute: 60 # tool calls per key (per IP when anonymous)
+ burst: 20
+ jobs:
+ workers: 4 # background jobs running at once, all keys
+ maxPerKey: 5 # unfinished jobs per key
+ ttl: 20m # results kept this long
+ timeout: 2m # longest a job may run
tools:
datasetsDir: /hammer/share/datasets
maxPasteBytes: 65536
netTimeout: 12s
mtrCycles: 5
pingCount: 5
- ocrInbox: /hammer/cache/inbox
- ocrWorkDir: /hammer/cache/ocr
- ocrTTL: 20m
@@ -20,6 +20,7 @@ github.com/kixorz/suncalc v1.0.0 github.com/klauspost/lctime v0.1.0
github.com/nathan-osman/go-sunrise v1.1.0
github.com/oschwald/geoip2-golang/v2 v2.1.0
+ github.com/pkg/sftp v1.13.11
github.com/rcarmo/go-te v0.1.0
github.com/ringsaturn/tzf v1.0.4
github.com/rivo/uniseg v0.4.7
@@ -40,6 +41,7 @@ github.com/clipperhouse/uax29/v2 v2.7.0 // indirect github.com/davecgh/go-spew v1.1.1 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/itchyny/timefmt-go v0.1.7 // indirect
+ github.com/kr/fs v0.1.0 // indirect
github.com/mattn/go-runewidth v0.0.19 // indirect
github.com/mattn/go-sqlite3 v2.0.3+incompatible // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
@@ -67,6 +67,8 @@ github.com/kixorz/suncalc v1.0.0/go.mod h1:oje1Y/vU97KtaYBjnQpJEQQxQUL+0tQ7ea/ekU3YrYg= github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/lctime v0.1.0 h1:nINsuFc860M9cyYhT6vfg6U1USh7kiVBj/s/2b04U70=
github.com/klauspost/lctime v0.1.0/go.mod h1:OwdMhr8tbQvusAsnilqkkgDQqivWlqyg0w5cfXkLiDk=
+github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
+github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
@@ -93,6 +95,8 @@ github.com/paulmach/orb v0.12.0 h1:z+zOwjmG3MyEEqzv92UN49Lg1JFYx0L9GpGKNVDKk1s= github.com/paulmach/orb v0.12.0/go.mod h1:5mULz1xQfs3bmQm63QEJA6lNGujuRafwA5S/EnuLaLU=
github.com/paulmach/protoscan v0.2.1/go.mod h1:SpcSwydNLrxUGSDvXvO0P7g7AuhJ7lcKfDlhJCDw2gY=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/sftp v1.13.11 h1:0N92SLTB8JqASJB14ZLHHzFnBV8mG9zw4K7jghEFWuE=
+github.com/pkg/sftp v1.13.11/go.mod h1:uNkH9roSXglNJqM+glJJi+TQXQUm0fXFWqCFmT8hsN0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rcarmo/go-te v0.1.0 h1:BH9Ub+A0AVBY5Q00El4QMVaWAMbycVHgMHQI2Kz8J/o=
@@ -1,16 +1,11 @@ package bbs
import (
- "net"
"os"
"path/filepath"
"strings"
"testing"
"time"
-
- gossh "golang.org/x/crypto/ssh"
-
- "github.com/runawaydevil/hammer/internal/tools/ocrpdf"
)
func TestIdentity(t *testing.T) {
@@ -18,15 +13,15 @@ if got := identityForFingerprint("SHA256:abc"); got != "fp:SHA256:abc" { t.Errorf("fingerprint identity = %q", got)
}
- a := identityForIP("1.2.3.4")
- if !strings.HasPrefix(a, "ip:") {
- t.Errorf("ip identity missing prefix: %q", a)
+ a, b := anonymousIdentity(), anonymousIdentity()
+ if !strings.HasPrefix(a, "anon:") || !isAnonymous(a) {
+ t.Errorf("anonymous identity = %q", a)
}
- if b := identityForIP("1.2.3.4"); a != b {
- t.Errorf("same IP must map to same identity: %q != %q", a, b)
+ if a == b {
+ t.Error("two anonymous connections must not share an identity")
}
- if c := identityForIP("1.2.3.5"); a == c {
- t.Errorf("different IPs must differ: both %q", a)
+ if isAnonymous(identityForFingerprint("SHA256:abc")) {
+ t.Error("a keyed identity is not anonymous")
}
}
@@ -58,148 +53,68 @@ } }
func TestStorageQuotaRefusesAndDrains(t *testing.T) {
- dir := t.TempDir()
- inbox := filepath.Join(dir, "inbox")
- if err := os.MkdirAll(inbox, 0o700); err != nil {
+ root := t.TempDir()
+ area := filepath.Join(root, "abc123")
+ if err := os.MkdirAll(filepath.Join(area, "docs"), 0o700); err != nil {
t.Fatal(err)
}
- blob := filepath.Join(inbox, "big.bin")
+ blob := filepath.Join(area, "docs", "big.bin")
if err := os.WriteFile(blob, make([]byte, 2000), 0o600); err != nil {
t.Fatal(err)
}
+ fresh := filepath.Join(area, "fresh.txt")
+ if err := os.WriteFile(fresh, []byte("new"), 0o600); err != nil {
+ t.Fatal(err)
+ }
- acc := newStorageAccountant(1000, inbox) // quota 1000 < usage 2000
- if acc.CanWrite() {
- t.Fatal("quota is full, CanWrite must be false")
+ acc := newStorageAccountant(1000, root) // quota 1000 < usage 2003
+ if acc.CanWrite() || acc.Remaining() != 0 {
+ t.Fatalf("quota is full: CanWrite=%v Remaining=%d", acc.CanWrite(), acc.Remaining())
}
- // Age the file past the TTL and run the same purge the server runs.
+ // Age the big file and its directory past the TTL, then purge.
old := time.Now().Add(-1 * time.Hour)
- if err := os.Chtimes(blob, old, old); err != nil {
- t.Fatal(err)
- }
- if err := ocrpdf.Sweep(inbox, "", 20*time.Minute, time.Now()); err != nil {
- t.Fatalf("sweep: %v", err)
+ for _, p := range []string{blob, filepath.Join(area, "docs")} {
+ if err := os.Chtimes(p, old, old); err != nil {
+ t.Fatal(err)
+ }
}
+ acc.sweep(20*time.Minute, time.Now())
if !acc.CanWrite() {
t.Fatalf("after the purge drained the quota, CanWrite must be true (usage=%d)", acc.Usage())
}
+ if _, err := os.Stat(fresh); err != nil {
+ t.Error("a file younger than the TTL must survive the purge")
+ }
+ if _, err := os.Stat(root); err != nil {
+ t.Error("the storage root itself must never be removed")
+ }
// A non-positive quota is always writable.
- if !newStorageAccountant(0, inbox).CanWrite() {
+ if !newStorageAccountant(0, root).CanWrite() {
t.Fatal("quota 0 must mean unlimited")
}
}
-func TestProfilePersistsByIPIdentity(t *testing.T) {
- dir := t.TempDir()
- store, err := NewProfileStore(filepath.Join(dir, "profiles.db"))
- if err != nil {
- t.Fatalf("store: %v", err)
- }
- t.Cleanup(func() { store.Close() })
+func TestAnonymousProfileIsNotStored(t *testing.T) {
+ srv := testServer(t, &Config{AllowAnonymous: true})
- id := identityForIP("203.0.113.7")
- if err := store.Save(&Profile{Fingerprint: id, DefaultCity: "Paris", Units: "m", View: "card"}); err != nil {
- t.Fatalf("save: %v", err)
- }
-
- got, err := store.Get(id)
+ id := anonymousIdentity()
+ p, err := srv.loadOrCreateProfile(id, "203.0.113.7:5555")
if err != nil {
- t.Fatalf("get: %v", err)
+ t.Fatal(err)
}
- if got.DefaultCity != "Paris" {
- t.Errorf("expected persisted city Paris, got %q", got.DefaultCity)
+ if p.View != "card" || p.Units != "m" {
+ t.Errorf("anonymous profile should start with defaults, got %+v", p)
}
-}
-// TestSSHAnonymousKeylessSession connects with NO public key (keyboard-
-// interactive) and expects the splash, plus a profile keyed by the client IP.
-func TestSSHAnonymousKeylessSession(t *testing.T) {
- dir := t.TempDir()
+ st := &session{srv: srv, profile: p, fp: id, anon: true}
+ p.DefaultCity = "Paris"
+ st.save()
- store, err := NewProfileStore(filepath.Join(dir, "profiles.db"))
- if err != nil {
- t.Fatalf("store: %v", err)
- }
- t.Cleanup(func() { store.Close() })
-
- srv := &Server{
- cfg: &Config{HostKeyPath: filepath.Join(dir, "hostkey"), AllowAnonymous: true},
- adapter: NewWeatherAdapter(fakeRenderer{}, fakeLocator{}),
- store: store,
- }
-
- sshSrv, err := srv.sshServer()
- if err != nil {
- t.Fatalf("sshServer: %v", err)
- }
-
- ln, err := net.Listen("tcp", "127.0.0.1:0")
- if err != nil {
- t.Fatalf("listen: %v", err)
- }
- t.Cleanup(func() { ln.Close() })
-
- go sshSrv.Serve(ln)
-
- client, err := gossh.Dial("tcp", ln.Addr().String(), &gossh.ClientConfig{
- User: "guest",
- Auth: []gossh.AuthMethod{
- gossh.KeyboardInteractive(func(_, _ string, questions []string, _ []bool) ([]string, error) {
- return make([]string, len(questions)), nil // answer nothing
- }),
- },
- HostKeyCallback: gossh.InsecureIgnoreHostKey(),
- Timeout: 5 * time.Second,
- })
- if err != nil {
- t.Fatalf("keyless dial should succeed with anonymous access on: %v", err)
- }
- defer client.Close()
-
- sess, err := client.NewSession()
- if err != nil {
- t.Fatalf("session: %v", err)
- }
-
- out := &syncBuf{}
- sess.Stdout = out
- sess.Stderr = out
-
- stdin, err := sess.StdinPipe()
- if err != nil {
- t.Fatalf("stdin: %v", err)
- }
-
- if err := sess.RequestPty("xterm", 40, 100, gossh.TerminalModes{}); err != nil {
- t.Fatalf("pty: %v", err)
- }
- if err := sess.Shell(); err != nil {
- t.Fatalf("shell: %v", err)
- }
-
- stdin.Write([]byte("\r"))
- time.Sleep(300 * time.Millisecond)
- stdin.Write([]byte("q\r"))
-
- done := make(chan error, 1)
- go func() { done <- sess.Wait() }()
-
- select {
- case <-done:
- case <-time.After(5 * time.Second):
- t.Fatal("session did not end after quit")
- }
-
- if got := out.String(); !strings.Contains(got, "HAMMER") {
- t.Errorf("keyless session missing splash/menu:\n%s", got)
- }
-
- // The keyless session must have persisted a profile keyed by the client IP.
- if _, err := store.Get(identityForIP("127.0.0.1")); err != nil {
- t.Errorf("expected IP-keyed profile for 127.0.0.1: %v", err)
+ if _, err := srv.store.Get(id); err == nil {
+ t.Fatal("an anonymous profile must never be written to the store")
}
}
@@ -4,6 +4,7 @@ import ( "fmt"
"time"
+ "github.com/runawaydevil/hammer/internal/jobs"
"github.com/runawaydevil/hammer/internal/tools"
)
@@ -16,6 +17,12 @@ defaultMaxConnPerIP = 5 // defaultStorageTTL drives the background purge that drains the storage
// quota, so space frees up on its own.
defaultStorageTTL = 20 * time.Minute
+
+ defaultFilesDir = "cache/files"
+ defaultFilesQuota = 50 << 20 // per key
+ defaultMaxFileInput = 8 << 20 // largest file a tool reads from the area
+ defaultDispatchRate = 60 // tool calls per minute per identity
+ defaultDispatchBurst = 20
)
// Config is the "bbs" section of the root configuration.
@@ -27,7 +34,8 @@ ProfilesDB string `yaml:"profilesDb"`
// AllowAnonymous lets clients connect without a public key (keyboard-
// interactive / password auth answered with no challenge). Off by default.
- // When on, preferences persist best-effort by client IP (see identity.go).
+ // Anonymous sessions get defaults that last only for the connection: no
+ // saved preferences, no file area, jobs visible only on that connection.
AllowAnonymous bool `yaml:"allowAnonymous"`
// MaxConnPerIP caps simultaneous sessions from one client IP. Absent = 5;
@@ -36,6 +44,12 @@ MaxConnPerIP *int `yaml:"maxConnectionsPerIP"`
// Storage bounds the total writable footprint of the BBS (see storage.go).
Storage *StorageConfig `yaml:"storage,omitempty"`
+ // Files is the per-key file area served over SFTP.
+ Files *FilesConfig `yaml:"files,omitempty"`
+ // Dispatch configures the NDJSON / command interface.
+ Dispatch *DispatchConfig `yaml:"dispatch,omitempty"`
+ // Jobs bounds background (async) tool calls.
+ Jobs *JobsConfig `yaml:"jobs,omitempty"`
Tools *ToolsConfig `yaml:"tools,omitempty"`
@@ -50,11 +64,38 @@ // unlimited. When reached, new writes are refused until the TTL purge frees // space.
GlobalQuotaBytes int64 `yaml:"globalQuotaBytes"`
- // TTL is how long generated/uploaded files live before the background purge
- // deletes them, draining the quota. Absent = 20m.
+ // TTL is how long uploaded files live before the background purge deletes
+ // them, draining the quota. Absent = 20m.
TTL string `yaml:"ttl"`
}
+// FilesConfig is the optional "bbs.files" block.
+type FilesConfig struct {
+ // Dir holds one private directory per SSH key. Absent = cache/files.
+ Dir string `yaml:"dir"`
+ // QuotaBytes caps one key's area. Absent or 0 = 50 MiB.
+ QuotaBytes int64 `yaml:"quotaBytes"`
+ // MaxFileInput is the largest file a tool will read. Absent or 0 = 8 MiB.
+ MaxFileInput int64 `yaml:"maxFileInput"`
+}
+
+// DispatchConfig is the optional "bbs.dispatch" block.
+type DispatchConfig struct {
+ // RequestsPerMinute caps tool calls per identity (listings and job polls
+ // are free). Absent = 60; an explicit 0 = unlimited.
+ RequestsPerMinute *int `yaml:"requestsPerMinute"`
+ // Burst is how many calls may arrive at once. Absent or 0 = 20.
+ Burst int `yaml:"burst"`
+}
+
+// JobsConfig is the optional "bbs.jobs" block. Zero values take defaults.
+type JobsConfig struct {
+ Workers int `yaml:"workers"` // jobs running at once, all keys (4)
+ MaxPerKey int `yaml:"maxPerKey"` // unfinished jobs per identity (5)
+ TTL string `yaml:"ttl"` // how long results are kept (20m)
+ Timeout string `yaml:"timeout"` // longest a job may run (2m)
+}
+
// IsAnonymousAllowed reports whether keyless sessions may connect.
func (c *Config) IsAnonymousAllowed() bool {
return c != nil && c.AllowAnonymous
@@ -98,6 +139,19 @@ func (c *Config) Validate() error { if c == nil {
return nil
}
+ if c.Jobs != nil {
+ for name, v := range map[string]string{"ttl": c.Jobs.TTL, "timeout": c.Jobs.Timeout} {
+ if v == "" {
+ continue
+ }
+ if _, err := time.ParseDuration(v); err != nil {
+ return fmt.Errorf("bbs.jobs.%s %q is not a valid duration: %w", name, v, err)
+ }
+ }
+ }
+ if c.Files != nil && (c.Files.QuotaBytes < 0 || c.Files.MaxFileInput < 0) {
+ return fmt.Errorf("bbs.files sizes must be >= 0")
+ }
if c.Storage != nil {
if c.Storage.GlobalQuotaBytes < 0 {
return fmt.Errorf("bbs.storage.globalQuotaBytes must be >= 0, got %d", c.Storage.GlobalQuotaBytes)
@@ -123,10 +177,6 @@ LycheePath string `yaml:"lycheePath"` DuckDBPath string `yaml:"duckdbPath"`
MtrPath string `yaml:"mtrPath"`
PingPath string `yaml:"pingPath"`
- OcrmypdfPath string `yaml:"ocrmypdfPath"`
- OCRInbox string `yaml:"ocrInbox"`
- OCRWorkDir string `yaml:"ocrWorkDir"`
- OCRTTL string `yaml:"ocrTTL"`
}
func (c *Config) ListenPort() int {
@@ -148,12 +198,6 @@ if d, err := time.ParseDuration(c.Tools.NetTimeout); err == nil && d > 0 { timeout = d
}
}
- ttl := 20 * time.Minute
- if c.Tools.OCRTTL != "" {
- if d, err := time.ParseDuration(c.Tools.OCRTTL); err == nil && d > 0 {
- ttl = d
- }
- }
return tools.Config{
DatasetsDir: c.Tools.DatasetsDir,
@@ -165,9 +209,64 @@ LycheePath: c.Tools.LycheePath, DuckDBPath: c.Tools.DuckDBPath,
MtrPath: c.Tools.MtrPath,
PingPath: c.Tools.PingPath,
- OcrmypdfPath: c.Tools.OcrmypdfPath,
- OCRInboxDir: c.Tools.OCRInbox,
- OCRWorkDir: c.Tools.OCRWorkDir,
- OCRTTL: ttl,
+ }
+}
+
+// FilesDir is where per-key file areas live.
+func (c *Config) FilesDir() string {
+ if c == nil || c.Files == nil || c.Files.Dir == "" {
+ return defaultFilesDir
+ }
+
+ return c.Files.Dir
+}
+
+// FilesQuota is one key's area quota in bytes.
+func (c *Config) FilesQuota() int64 {
+ if c == nil || c.Files == nil || c.Files.QuotaBytes <= 0 {
+ return defaultFilesQuota
}
+
+ return c.Files.QuotaBytes
+}
+
+// MaxFileInput is the largest file a tool reads from the area.
+func (c *Config) MaxFileInput() int64 {
+ if c == nil || c.Files == nil || c.Files.MaxFileInput <= 0 {
+ return defaultMaxFileInput
+ }
+
+ return c.Files.MaxFileInput
+}
+
+// DispatchLimits returns tool calls per minute (0 = unlimited) and burst.
+func (c *Config) DispatchLimits() (perMinute, burst int) {
+ perMinute, burst = defaultDispatchRate, defaultDispatchBurst
+ if c == nil || c.Dispatch == nil {
+ return perMinute, burst
+ }
+ if c.Dispatch.RequestsPerMinute != nil {
+ perMinute = max(*c.Dispatch.RequestsPerMinute, 0)
+ }
+ if c.Dispatch.Burst > 0 {
+ burst = c.Dispatch.Burst
+ }
+
+ return perMinute, burst
+}
+
+// jobsConfig resolves the "bbs.jobs" block; unset fields keep the defaults.
+func (c *Config) jobsConfig() jobs.Config {
+ if c == nil || c.Jobs == nil {
+ return jobs.Config{}
+ }
+ cfg := jobs.Config{Workers: c.Jobs.Workers, MaxPerOwner: c.Jobs.MaxPerKey}
+ if d, err := time.ParseDuration(c.Jobs.TTL); err == nil {
+ cfg.TTL = d
+ }
+ if d, err := time.ParseDuration(c.Jobs.Timeout); err == nil {
+ cfg.Timeout = d
+ }
+
+ return cfg
}
@@ -0,0 +1,546 @@ +package bbs
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "log"
+ "strings"
+ "time"
+ "unicode/utf8"
+
+ "github.com/gliderlabs/ssh"
+ "github.com/pkg/sftp"
+
+ "github.com/runawaydevil/hammer/internal/dispatch"
+ "github.com/runawaydevil/hammer/internal/filearea"
+ "github.com/runawaydevil/hammer/internal/localization"
+ "github.com/runawaydevil/hammer/internal/tools/agecrypt"
+ "github.com/runawaydevil/hammer/internal/tools/b64"
+ "github.com/runawaydevil/hammer/internal/tools/csvstat"
+ "github.com/runawaydevil/hammer/internal/tools/dataquery"
+ "github.com/runawaydevil/hammer/internal/tools/hash"
+ "github.com/runawaydevil/hammer/internal/tools/ids"
+ "github.com/runawaydevil/hammer/internal/tools/links"
+ "github.com/runawaydevil/hammer/internal/tools/netdiag"
+ "github.com/runawaydevil/hammer/internal/tools/netpath"
+ qrtool "github.com/runawaydevil/hammer/internal/tools/qr"
+ "github.com/runawaydevil/hammer/internal/tools/transform"
+)
+
+// caller is what the bbs tools need to know about who is asking; it travels
+// in dispatch.Env.Caller so it survives into background jobs.
+type caller struct {
+ identity string
+ clientIP string
+ profile *Profile
+}
+
+func callerOf(c *dispatch.Call) *caller {
+ cl, _ := c.Env.Caller.(*caller)
+ if cl == nil {
+ return &caller{profile: &Profile{}}
+ }
+
+ return cl
+}
+
+// serveDispatch answers a session that carries a command (`ssh host hash`) or
+// a pipe with no terminal (NDJSON on stdin).
+func (s *Server) serveDispatch(sess ssh.Session, identity string) {
+ env, err := s.dispatchEnv(sess, identity)
+ if err != nil {
+ log.Println("bbs: dispatch env:", err)
+ io.WriteString(sess.Stderr(), "internal error, try again later.\n")
+ _ = sess.Exit(1)
+
+ return
+ }
+
+ _, _, isPty := sess.Pty()
+ var out, errOut io.Writer = sess, sess.Stderr()
+ if isPty {
+ // With -t the client terminal is raw: bare \n would not return the carriage.
+ out, errOut = crlfWriter{sess}, crlfWriter{sess.Stderr()}
+ }
+
+ argv := sess.Command()
+ if len(argv) == 0 {
+ if err := s.catalog.Serve(sess.Context(), env, sess, out, s.maxRequestLine()); err != nil && !errors.Is(err, io.EOF) {
+ log.Println("bbs: dispatch stream:", err)
+ _ = sess.Exit(1)
+
+ return
+ }
+ _ = sess.Exit(0)
+
+ return
+ }
+
+ req, text, perr := dispatch.ParseCommand(argv)
+ if _, known := s.catalog.Lookup(argv[0]); perr != nil && !known && argv[0] != "help" {
+ // An unknown tool is the more useful complaint than its arguments.
+ req, perr = dispatch.Request{Tool: argv[0]}, nil
+ }
+ var resp dispatch.Response
+ if perr != nil {
+ resp = dispatch.Response{Tool: argv[0], Error: perr}
+ } else {
+ env.Stdin = sess
+ resp = s.catalog.Handle(sess.Context(), env, req)
+ }
+
+ if text {
+ w := out
+ if !resp.OK {
+ w = errOut
+ }
+ io.WriteString(w, dispatch.FormatText(resp, env.Hint))
+ } else {
+ enc := json.NewEncoder(out)
+ enc.SetEscapeHTML(false)
+ _ = enc.Encode(resp)
+ }
+
+ if !resp.OK {
+ _ = sess.Exit(1)
+
+ return
+ }
+ _ = sess.Exit(0)
+}
+
+// dispatchEnv describes the caller to the dispatcher. Keyed callers get their
+// file area and saved profile; anonymous ones are rate limited by IP, since a
+// fresh connection is a fresh identity.
+func (s *Server) dispatchEnv(sess ssh.Session, identity string) (*dispatch.Env, error) {
+ clientIP := hostOf(sess.RemoteAddr().String())
+
+ profile, err := s.loadOrCreateProfile(identity, sess.RemoteAddr().String())
+ if err != nil {
+ return nil, err
+ }
+
+ rateKey := identity
+ if isAnonymous(identity) {
+ rateKey = "ip:" + clientIP
+ }
+
+ env := &dispatch.Env{
+ Owner: identity,
+ MaxInput: s.tools.Cfg().MaxPasteBytes,
+ MaxFile: s.cfg.MaxFileInput(),
+ Jobs: s.jobs,
+ Allow: func() (bool, time.Duration) { return s.calls.Allow(rateKey) },
+ Hint: fmt.Sprintf("ssh -p %d <host>", s.cfg.ListenPort()),
+ Caller: &caller{identity: identity, clientIP: clientIP, profile: profile},
+ }
+
+ if !isAnonymous(identity) {
+ area, err := filearea.Open(s.cfg.FilesDir(), identity, s.cfg.FilesQuota(), s.storage)
+ if err != nil {
+ return nil, err
+ }
+ env.Files = area
+ }
+
+ return env, nil
+}
+
+// maxRequestLine bounds one NDJSON line: the paste limit, allowing for JSON
+// escaping of every byte, plus room for the envelope.
+func (s *Server) maxRequestLine() int {
+ return s.tools.Cfg().MaxPasteBytes*6 + 4096
+}
+
+// handleSFTP serves the caller's file area. It needs a key: an anonymous
+// connection has no stable identity to own files.
+func (s *Server) handleSFTP(sess ssh.Session) {
+ clientIP := hostOf(sess.RemoteAddr().String())
+ if !s.conns.acquire(clientIP) {
+ io.WriteString(sess.Stderr(), "connection limit reached — close a session and try again.\n")
+ _ = sess.Exit(1)
+
+ return
+ }
+ defer s.conns.release(clientIP)
+
+ identity, ok := s.identify(sess)
+ if !ok {
+ return
+ }
+ if isAnonymous(identity) {
+ io.WriteString(sess.Stderr(), "the file area needs an SSH key; connect with one to use sftp.\n")
+ _ = sess.Exit(1)
+
+ return
+ }
+
+ area, err := filearea.Open(s.cfg.FilesDir(), identity, s.cfg.FilesQuota(), s.storage)
+ if err != nil {
+ log.Println("bbs: sftp area:", err)
+ _ = sess.Exit(1)
+
+ return
+ }
+
+ srv := sftp.NewRequestServer(sess, area.Handlers())
+ if err := srv.Serve(); err != nil && !errors.Is(err, io.EOF) {
+ log.Println("bbs: sftp:", err)
+ }
+ _ = srv.Close()
+ _ = sess.Exit(0)
+}
+
+type crlfWriter struct{ w io.Writer }
+
+func (c crlfWriter) Write(p []byte) (int, error) {
+ if _, err := c.w.Write([]byte(strings.ReplaceAll(string(p), "\n", "\r\n"))); err != nil {
+ return 0, err
+ }
+
+ return len(p), nil
+}
+
+// dispatchTools is the vocabulary: the same tools as the menu, with declared
+// arguments. Tools whose host binaries are missing are left out, exactly as
+// the menu hides them.
+func (s *Server) dispatchTools() []*dispatch.Tool {
+ cfg := s.tools.Cfg()
+ str := func(name string, max int, required bool, help string) dispatch.Arg {
+ return dispatch.Arg{Name: name, Kind: dispatch.KindString, Max: max, Required: required, Help: help}
+ }
+
+ list := []*dispatch.Tool{
+ {
+ Name: "weather",
+ Summary: "weather for a city (default: your saved or detected city)",
+ Args: []dispatch.Arg{
+ str("city", 128, false, "city, airport code, landmark or @domain"),
+ {Name: "view", Kind: dispatch.KindString, Enum: []string{"card", "v1", "v2", "line", "j1", "j2"}, Help: "layout; j1/j2 return JSON data"},
+ {Name: "units", Kind: dispatch.KindString, Enum: []string{"m", "u", "M"}, Help: "m metric, u USCS, M metric with m/s wind"},
+ str("lang", 8, false, "output language code"),
+ {Name: "width", Kind: dispatch.KindInt, Min: 40, Max: 250, Help: "terminal columns"},
+ },
+ Run: s.runWeather,
+ },
+ {
+ Name: "net",
+ Summary: "DNS, TLS, HTTP and RDAP for a public host",
+ Args: []dispatch.Arg{str("host", 253, true, "host name or URL")},
+ Async: true,
+ Run: func(ctx context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ rep, err := netdiag.Diagnose(ctx, c.Str("host"), netdiag.Options{Timeout: cfg.NetTimeout})
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{
+ Data: map[string]any{"host": rep.Host, "dns": rep.DNS, "tls": rep.TLS, "http": rep.HTTP, "rdap": rep.RDAP},
+ Lines: rep.Lines(),
+ }, nil
+ },
+ },
+ {
+ Name: "qr",
+ Summary: "QR code as ANSI blocks",
+ Args: []dispatch.Arg{
+ str("text", 1024, true, "payload"),
+ {Name: "kind", Kind: dispatch.KindString, Enum: []string{"text", "url", "phone"}, Help: "normalise as a URL or tel: link"},
+ },
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ payload := c.Str("text")
+ var err error
+ switch c.Str("kind") {
+ case "url":
+ payload, err = qrtool.URLPayload(payload)
+ case "phone":
+ payload, err = qrtool.PhonePayload(payload)
+ }
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s", err)
+ }
+ out, err := qrtool.ANSI(payload)
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{Data: map[string]any{"payload": payload}, Lines: strings.Split(out, "\n")}, nil
+ },
+ },
+ {
+ Name: "hash",
+ Summary: "SHA-256/SHA-1/MD5, MIME type and entropy of a payload",
+ Input: true,
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ rep := hash.Inspect(c.Input)
+
+ return &dispatch.Result{Data: hashData(rep), Lines: rep.Lines()}, nil
+ },
+ },
+ {
+ Name: "jq",
+ Summary: "jq filter over JSON or YAML",
+ Args: []dispatch.Arg{str("filter", 256, false, "jq expression (default .)")},
+ Input: true,
+ Run: func(ctx context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ ctx, cancel := context.WithTimeout(ctx, cfg.NetTimeout)
+ defer cancel()
+
+ res, err := transform.Apply(ctx, c.Input, c.Str("filter"))
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s", err)
+ }
+ data := map[string]any{"input_kind": res.InputKind, "output": res.Output}
+ var parsed any
+ if json.Unmarshal([]byte(res.Output), &parsed) == nil {
+ data["value"] = parsed
+ }
+
+ return &dispatch.Result{Data: data, Lines: strings.Split(res.Output, "\n")}, nil
+ },
+ },
+ {
+ Name: "age.encrypt",
+ Summary: "encrypt a payload to an age recipient (armored)",
+ Args: []dispatch.Arg{str("recipient", 128, true, "age1...")},
+ Input: true,
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ out, err := agecrypt.Encrypt(c.Input, c.Str("recipient"))
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s", err)
+ }
+
+ return &dispatch.Result{Data: map[string]any{"ciphertext": out}, Lines: strings.Split(strings.TrimRight(out, "\n"), "\n")}, nil
+ },
+ },
+ {
+ Name: "age.decrypt",
+ Summary: "decrypt an age payload with an identity",
+ Args: []dispatch.Arg{str("identity", 200, true, "AGE-SECRET-KEY-...")},
+ Input: true,
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ plain, err := agecrypt.Decrypt(c.Input, c.Str("identity"))
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s", err)
+ }
+
+ return bytesResult("plaintext", plain), nil
+ },
+ },
+ {
+ Name: "age.keygen",
+ Summary: "new age identity (not stored anywhere)",
+ Run: func(context.Context, *dispatch.Call) (*dispatch.Result, error) {
+ pair, err := agecrypt.Generate()
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{
+ Data: map[string]any{"recipient": pair.Recipient, "identity": pair.Identity},
+ Lines: []string{pair.Recipient, pair.Identity},
+ }, nil
+ },
+ },
+ {
+ Name: "b64.encode",
+ Summary: "Base64-encode a payload",
+ Input: true,
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ out, err := b64.Encode(c.Input)
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{Data: map[string]any{"base64": out}, Lines: wrapBase64(out)}, nil
+ },
+ },
+ {
+ Name: "b64.decode",
+ Summary: "decode Base64 (text is returned as text, binary is inspected)",
+ Input: true,
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ raw, err := b64.Decode(string(c.Input))
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s", err)
+ }
+
+ return bytesResult("text", raw), nil
+ },
+ },
+ {
+ Name: "csv",
+ Summary: "rows, columns, header and value counts of a CSV",
+ Input: true,
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ rep, err := csvstat.Analyze(c.Input)
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s", err)
+ }
+ counts := make([]map[string]any, 0, len(rep.Counts))
+ for _, n := range rep.Counts {
+ counts = append(counts, map[string]any{"value": n.Value, "n": n.N})
+ }
+
+ return &dispatch.Result{
+ Data: map[string]any{
+ "rows": rep.Rows, "cols": rep.Cols, "header": rep.Header,
+ "preview": rep.Preview, "count_column": rep.CountCol, "counts": counts,
+ },
+ Lines: rep.Lines(),
+ }, nil
+ },
+ },
+ {
+ Name: "ids",
+ Summary: "random UUID v4s",
+ Args: []dispatch.Arg{{Name: "count", Kind: dispatch.KindInt, Min: 1, Max: 100, Help: "how many (default 8)"}},
+ Run: func(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ got, err := ids.UUIDv4(c.Int("count"))
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{Data: map[string]any{"uuids": got}, Lines: got}, nil
+ },
+ },
+ }
+
+ if bin := s.tools.Lychee(); bin != "" {
+ list = append(list, &dispatch.Tool{
+ Name: "links",
+ Summary: "check every link in a URL, Markdown or HTML payload (lychee)",
+ Input: true,
+ Async: true,
+ Run: func(ctx context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ lines, err := links.Check(ctx, bin, string(c.Input), cfg.NetTimeout)
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{Data: map[string]any{"report": lines}, Lines: lines}, nil
+ },
+ })
+ }
+
+ if s.tools.HasData() {
+ if queries, err := dataquery.LoadQueries(cfg.DatasetsDir); err == nil && len(queries) > 0 {
+ byID := map[string]dataquery.Query{}
+ enum := make([]string, 0, len(queries))
+ for _, q := range queries {
+ byID[q.ID] = q
+ enum = append(enum, q.ID)
+ }
+ list = append(list, &dispatch.Tool{
+ Name: "data",
+ Summary: "run a curated DuckDB query over the bundled datasets",
+ Args: []dispatch.Arg{{Name: "query", Kind: dispatch.KindString, Required: true, Enum: enum, Help: "query id"}},
+ Async: true,
+ Run: func(ctx context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ raw, err := dataquery.Run(ctx, s.tools.DuckDB(), cfg.DatasetsDir, byID[c.Str("query")], cfg.NetTimeout)
+ if err != nil {
+ return nil, err
+ }
+ var rows any
+ if json.Unmarshal([]byte(raw), &rows) != nil {
+ rows = raw
+ }
+
+ return &dispatch.Result{Data: map[string]any{"rows": rows}, Lines: dataquery.FormatJSONTable(raw)}, nil
+ },
+ })
+ }
+ }
+
+ if s.tools.HasPath() {
+ list = append(list, &dispatch.Tool{
+ Name: "path",
+ Summary: "ping and MTR to a public host",
+ Args: []dispatch.Arg{str("host", 253, true, "host name or IP")},
+ Async: true,
+ Run: func(ctx context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ lines, err := netpath.Probe(ctx, c.Str("host"), s.tools.Ping(), s.tools.Mtr(), cfg.PingCount, cfg.MtrCycles, cfg.NetTimeout)
+ if err != nil {
+ return nil, err
+ }
+
+ return &dispatch.Result{Data: map[string]any{"report": lines}, Lines: lines}, nil
+ },
+ })
+ }
+
+ return list
+}
+
+func (s *Server) runWeather(_ context.Context, c *dispatch.Call) (*dispatch.Result, error) {
+ cl := callerOf(c)
+ p := cl.profile
+
+ city := c.Str("city")
+ if city == "" {
+ city = p.DefaultCity
+ }
+ if city == "" {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, `no city: pass city=... (no saved or detected city for you)`)
+ }
+
+ req := WeatherRequest{
+ City: city,
+ View: pick(c.Str("view"), p.View, "card"),
+ Lang: localization.Canonical(pick(c.Str("lang"), p.Lang, "en")),
+ Units: pick(c.Str("units"), p.Units, "m"),
+ Emoji: pick(p.Emoji, "unicode"),
+ Width: c.Int("width"),
+ }
+ if req.Width == 0 {
+ req.Width = 80
+ }
+
+ out, err := s.adapter.Render(req, cl.clientIP)
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeFailed, "weather for %s: %s", city, err)
+ }
+
+ data := map[string]any{"city": city, "view": req.View, "units": req.Units, "lang": req.Lang}
+ if req.View == "j1" || req.View == "j2" {
+ var parsed any
+ if json.Unmarshal(out, &parsed) == nil {
+ data["weather"] = parsed
+
+ return &dispatch.Result{Data: data}, nil
+ }
+ }
+
+ return &dispatch.Result{Data: data, Lines: strings.Split(rebrandFooter(out), "\n")}, nil
+}
+
+// bytesResult returns text as text, and inspects anything that is not.
+func bytesResult(key string, raw []byte) *dispatch.Result {
+ if utf8.Valid(raw) && !strings.ContainsRune(string(raw), 0) {
+ return &dispatch.Result{Data: map[string]any{key: string(raw)}, Lines: strings.Split(string(raw), "\n")}
+ }
+ rep := hash.Inspect(raw)
+
+ return &dispatch.Result{Data: map[string]any{"binary": true, "inspect": hashData(rep)}, Lines: rep.Lines()}
+}
+
+func hashData(r hash.Report) map[string]any {
+ return map[string]any{
+ "bytes": r.Bytes, "sha256": r.SHA256, "sha1": r.SHA1, "md5": r.MD5,
+ "mime": r.MIME, "kind": r.Kind, "utf8": r.UTF8, "entropy": r.Entropy,
+ }
+}
+
+// pick returns the first non-empty value.
+func pick(vals ...string) string {
+ for _, v := range vals {
+ if v != "" {
+ return v
+ }
+ }
+
+ return ""
+}
@@ -97,6 +97,8 @@ "BBS_OFF": "off", "BBS_ABOUT_BLURB": "An interactive SSH front-end for tools.",
"BBS_VERSION": "version",
"BBS_YOUR_KEY": "your key",
+ "BBS_ANONYMOUS": "anonymous (no key)",
+ "BBS_ANON_NOTE": "anonymous session — settings last until you disconnect; connect with an SSH key to keep them",
"BBS_TIP_WIDE": "(tip: maximize your terminal for the full layout)",
"BBS_PROMPT_WX": "weather> ",
"BBS_PROMPT_SET": "settings> ",
@@ -131,14 +133,6 @@ "BBS_B64_PROMPT": "b64> ", "BBS_PASTE_B64": "paste Base64",
"BBS_IDS": "ids",
"BBS_IDS_SUM": "UUID v4",
- "BBS_OCR": "ocr",
- "BBS_OCR_SUM": "searchable PDF",
- "BBS_OCR_PROMPT": "ocr> ",
- "BBS_NO_OCR": "ocrmypdf is not installed on this host",
- "BBS_NO_INBOX": "no PDFs in %s — copy a file there, wait 2s, try again",
- "BBS_OCR_RUNNING": "OCR %s...",
- "BBS_OCR_DONE": "searchable PDF (copy it out before it expires)",
- "BBS_OCR_TTL": "deleted after %d minutes",
"BBS_HOST_URL": "host or URL: ",
"BBS_PROBING": "probing %s...",
"BBS_TEXT": "text: ",
@@ -178,5 +172,4 @@ "BBS_QR_PHONE": "phone", "BBS_QR_PHONE_SUM": "tel: link for the dialer",
"BBS_QR_URL_PROMPT": "url: ",
"BBS_QR_PHONE_PROMPT": "phone: ",
- "BBS_STORAGE_FULL": "storage is full right now — try again in ~20 min",
}
@@ -35,7 +35,7 @@ "BBS_LANGUAGE": "idioma", "BBS_VIEW": "visão",
"BBS_CONTINUE": "pressione qualquer tecla para continuar...",
"BBS_MORE_TOOLS": "mais",
- "BBS_OCR": "ocr",
+ "BBS_ANONYMOUS": "anônimo (sem chave)",
}
for key, want := range cases {
if got := st.t(key); got != want {
@@ -46,11 +46,10 @@ }
func TestMainNumericRangeNativeTools(t *testing.T) {
st := &session{srv: &Server{tools: tools.NewRegistry(tools.Config{
- LycheePath: "/no/lychee",
- DuckDBPath: "/no/duckdb",
- MtrPath: "/no/mtr",
- PingPath: "/no/ping",
- OcrmypdfPath: "/no/ocrmypdf",
+ LycheePath: "/no/lychee",
+ DuckDBPath: "/no/duckdb",
+ MtrPath: "/no/mtr",
+ PingPath: "/no/ping",
})}}
if got := st.mainNumericRange(); got != "1-6" {
t.Fatalf("range = %q, want 1-6", got)
@@ -1,24 +1,30 @@ package bbs
import (
- "crypto/sha256"
+ "crypto/rand"
"encoding/hex"
)
-// Profiles are keyed by a stable identity string. Keyed sessions use the SSH
-// public-key fingerprint; anonymous (keyless) sessions fall back to a hash of
-// the client IP.
+// Profiles, file areas and jobs are keyed by a stable identity string, which
+// only an SSH public key provides.
//
-// The IP fallback is best-effort: clients behind the same NAT share one
-// identity, and a client whose IP changes loses its saved preferences. This is
-// an accepted limitation of keyless access.
-
+// Anonymous (keyless) sessions get no stable identity. Keying them by client
+// IP would make everyone behind one NAT (an office, a campus, mobile CGNAT)
+// share and overwrite each other's preferences, so each anonymous connection
+// is its own throwaway identity instead: defaults for the session, nothing
+// saved, no file area.
func identityForFingerprint(fingerprint string) string {
return "fp:" + fingerprint
}
-func identityForIP(clientIP string) string {
- sum := sha256.Sum256([]byte(clientIP))
+// anonymousIdentity returns a fresh identity for one keyless connection.
+func anonymousIdentity() string {
+ var b [8]byte
+ _, _ = rand.Read(b[:])
+
+ return "anon:" + hex.EncodeToString(b[:])
+}
- return "ip:" + hex.EncodeToString(sum[:])
+func isAnonymous(identity string) bool {
+ return len(identity) >= 5 && identity[:5] == "anon:"
}
@@ -15,10 +15,12 @@ "github.com/gliderlabs/ssh"
gossh "golang.org/x/crypto/ssh"
+ "github.com/runawaydevil/hammer/internal/dispatch"
"github.com/runawaydevil/hammer/internal/domain"
+ "github.com/runawaydevil/hammer/internal/jobs"
"github.com/runawaydevil/hammer/internal/localization"
+ "github.com/runawaydevil/hammer/internal/ratelimit"
"github.com/runawaydevil/hammer/internal/tools"
- "github.com/runawaydevil/hammer/internal/tools/ocrpdf"
"github.com/runawaydevil/hammer/internal/weather"
)
@@ -44,6 +46,9 @@ localizer localization.Localizer storage *storageAccountant
conns *connLimiter
purgeTTL time.Duration
+ catalog *dispatch.Catalog
+ jobs *jobs.Manager
+ calls *ratelimit.Limiter // tool calls per identity (dispatcher)
}
func New(cfg *Config, ws *weather.WeatherService, ip ipDataLocator) (*Server, error) {
@@ -63,21 +68,41 @@ langs = ws.Localizer.Languages() loc = ws.Localizer
}
+ s, err := newServer(cfg, NewWeatherAdapter(ws, ws.Locator), store, ip)
+ if err != nil {
+ store.Close()
+
+ return nil, err
+ }
+ s.langs, s.localizer = langs, loc
+
+ return s, nil
+}
+
+// newServer wires everything except the localizer; tests build servers with
+// it directly around fake weather and a temporary profile store.
+func newServer(cfg *Config, adapter *WeatherAdapter, store *ProfileStore, ip ipDataLocator) (*Server, error) {
reg := tools.NewRegistry(cfg.toolsConfig())
- rc := reg.Cfg()
+ s := &Server{
+ cfg: cfg,
+ adapter: adapter,
+ store: store,
+ ip: ip,
+ tools: reg,
+ storage: newStorageAccountant(cfg.QuotaBytes(), cfg.FilesDir()),
+ conns: newConnLimiter(cfg.MaxConnectionsPerIP()),
+ purgeTTL: cfg.QuotaTTL(),
+ jobs: jobs.New(cfg.jobsConfig()),
+ calls: ratelimit.New(cfg.DispatchLimits()),
+ }
- return &Server{
- cfg: cfg,
- adapter: NewWeatherAdapter(ws, ws.Locator),
- store: store,
- ip: ip,
- tools: reg,
- langs: langs,
- localizer: loc,
- storage: newStorageAccountant(cfg.QuotaBytes(), rc.OCRInboxDir, rc.OCRWorkDir),
- conns: newConnLimiter(cfg.MaxConnectionsPerIP()),
- purgeTTL: cfg.QuotaTTL(),
- }, nil
+ catalog, err := dispatch.NewCatalog(s.dispatchTools()...)
+ if err != nil {
+ return nil, err
+ }
+ s.catalog = catalog
+
+ return s, nil
}
func (s *Server) sshServer() (*ssh.Server, error) {
@@ -95,6 +120,9 @@ PublicKeyHandler: func(ctx ssh.Context, key ssh.PublicKey) bool { ctx.SetValue(fingerprintKey{}, gossh.FingerprintSHA256(key))
return true
+ },
+ SubsystemHandlers: map[string]ssh.SubsystemHandler{
+ "sftp": s.handleSFTP,
},
}
@@ -121,39 +149,24 @@ if err != nil { return err
}
- go s.sweepOCRLoop()
+ go s.sweepStorageLoop()
log.Printf("hammer listening on %s\n", srv.Addr)
return srv.ListenAndServe()
}
-func (s *Server) sweepOCRLoop() {
- s.ensureOCRDirs()
- s.sweepOCR()
+// sweepStorageLoop purges files older than the storage TTL every minute, which
+// is what drains the global quota.
+func (s *Server) sweepStorageLoop() {
+ _ = os.MkdirAll(s.cfg.FilesDir(), 0o700)
+
+ s.storage.sweep(s.purgeTTL, time.Now())
t := time.NewTicker(time.Minute)
defer t.Stop()
+
for range t.C {
- s.sweepOCR()
- }
-}
-
-func (s *Server) ensureOCRDirs() {
- cfg := s.tools.Cfg()
- _ = os.MkdirAll(cfg.OCRInboxDir, 0o700)
- _ = os.MkdirAll(cfg.OCRWorkDir, 0o700)
-}
-
-func (s *Server) sweepOCR() {
- cfg := s.tools.Cfg()
- // The global storage TTL drives quota drainage across all writable roots;
- // fall back to the tool's OCR TTL when it is not configured.
- ttl := s.purgeTTL
- if ttl <= 0 {
- ttl = cfg.OCRTTL
- }
- if err := ocrpdf.Sweep(cfg.OCRInboxDir, cfg.OCRWorkDir, ttl, time.Now()); err != nil {
- log.Println("bbs: ocr sweep:", err)
+ s.storage.sweep(s.purgeTTL, time.Now())
}
}
@@ -217,28 +230,40 @@ return data.City
}
-func (s *Server) loadOrCreateProfile(fingerprint, remoteAddr string) (*Profile, error) {
- p, err := s.store.Get(fingerprint)
- if err == nil {
- return p, nil
- }
- if !errors.Is(err, ErrProfileNotFound) {
- return nil, err
- }
-
- p = &Profile{
- Fingerprint: fingerprint,
+// defaultProfile is what a new identity starts with, seeded with the city
+// GeoIP guesses for the connecting address.
+func (s *Server) defaultProfile(identity, remoteAddr string) *Profile {
+ p := &Profile{
+ Fingerprint: identity,
Units: "m",
Lang: "en",
View: "card",
Emoji: "unicode",
}
-
if city := s.geoCity(remoteAddr); city != "" {
p.DefaultCity = city
p.Cities = city
}
+ return p
+}
+
+// loadOrCreateProfile returns identity's saved profile, creating it on first
+// use. Anonymous identities get a fresh profile that is never stored.
+func (s *Server) loadOrCreateProfile(identity, remoteAddr string) (*Profile, error) {
+ if isAnonymous(identity) {
+ return s.defaultProfile(identity, remoteAddr), nil
+ }
+
+ p, err := s.store.Get(identity)
+ if err == nil {
+ return p, nil
+ }
+ if !errors.Is(err, ErrProfileNotFound) {
+ return nil, err
+ }
+
+ p = s.defaultProfile(identity, remoteAddr)
if err := s.store.Save(p); err != nil {
return nil, err
}
@@ -247,5 +272,7 @@ return p, nil }
func (s *Server) Close() error {
+ s.jobs.Close()
+
return s.store.Close()
}
@@ -2,7 +2,13 @@ package bbs
import (
"bytes"
+ "context"
"crypto/ed25519"
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "io"
"net"
"path/filepath"
"strings"
@@ -10,15 +16,17 @@ "sync" "testing"
"time"
+ "github.com/pkg/sftp"
gossh "golang.org/x/crypto/ssh"
+ "github.com/runawaydevil/hammer/internal/dispatch"
"github.com/runawaydevil/hammer/internal/domain"
)
type fakeRenderer struct{}
func (fakeRenderer) RenderQuery(q *domain.Query) (*domain.RenderOutput, error) {
- return &domain.RenderOutput{Content: []byte("WEATHER OUTPUT\n")}, nil
+ return &domain.RenderOutput{Content: []byte("WEATHER OUTPUT for " + q.Location.Name + "\n")}, nil
}
type fakeLocator struct{}
@@ -46,21 +54,40 @@ return s.b.String()
}
-func TestSSHSessionEndToEnd(t *testing.T) {
+// testServer builds a server around fake weather and temporary state. Host
+// binaries are pointed at paths that do not exist, so optional tools stay
+// hidden whatever is installed on the machine running the tests.
+func testServer(t *testing.T, cfg *Config) *Server {
+ t.Helper()
dir := t.TempDir()
+
+ if cfg.HostKeyPath == "" {
+ cfg.HostKeyPath = filepath.Join(dir, "hostkey")
+ }
+ if cfg.Files == nil {
+ cfg.Files = &FilesConfig{Dir: filepath.Join(dir, "files")}
+ }
+ if cfg.Tools == nil {
+ cfg.Tools = &ToolsConfig{LycheePath: "/no/lychee", DuckDBPath: "/no/duckdb", MtrPath: "/no/mtr", PingPath: "/no/ping"}
+ }
store, err := NewProfileStore(filepath.Join(dir, "profiles.db"))
if err != nil {
t.Fatalf("store: %v", err)
}
- t.Cleanup(func() { store.Close() })
- srv := &Server{
- cfg: &Config{HostKeyPath: filepath.Join(dir, "hostkey")},
- adapter: NewWeatherAdapter(fakeRenderer{}, fakeLocator{}),
- store: store,
+ srv, err := newServer(cfg, NewWeatherAdapter(fakeRenderer{}, fakeLocator{}), store, nil)
+ if err != nil {
+ t.Fatalf("newServer: %v", err)
}
+ t.Cleanup(func() { srv.Close() })
+ return srv
+}
+
+// listen serves srv on a loopback port and returns its address.
+func listen(t *testing.T, srv *Server) string {
+ t.Helper()
sshSrv, err := srv.sshServer()
if err != nil {
t.Fatalf("sshServer: %v", err)
@@ -74,27 +101,94 @@ t.Cleanup(func() { ln.Close() })
go sshSrv.Serve(ln)
+ return ln.Addr().String()
+}
+
+func newSigner(t *testing.T) gossh.Signer {
+ t.Helper()
_, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatalf("client key: %v", err)
}
-
signer, err := gossh.NewSignerFromKey(priv)
if err != nil {
t.Fatalf("signer: %v", err)
}
- client, err := gossh.Dial("tcp", ln.Addr().String(), &gossh.ClientConfig{
- User: "tester",
- Auth: []gossh.AuthMethod{gossh.PublicKeys(signer)},
+ return signer
+}
+
+// dial connects with signer, or keyless when signer is nil.
+func dial(t *testing.T, addr string, signer gossh.Signer) *gossh.Client {
+ t.Helper()
+ auth := []gossh.AuthMethod{
+ gossh.KeyboardInteractive(func(_, _ string, questions []string, _ []bool) ([]string, error) {
+ return make([]string, len(questions)), nil // answer nothing
+ }),
+ }
+ user := "guest"
+ if signer != nil {
+ auth, user = []gossh.AuthMethod{gossh.PublicKeys(signer)}, "tester"
+ }
+
+ client, err := gossh.Dial("tcp", addr, &gossh.ClientConfig{
+ User: user,
+ Auth: auth,
HostKeyCallback: gossh.InsecureIgnoreHostKey(),
Timeout: 5 * time.Second,
})
if err != nil {
t.Fatalf("dial: %v", err)
}
- defer client.Close()
+ t.Cleanup(func() { client.Close() })
+
+ return client
+}
+
+// run executes one command, feeding stdin, and returns stdout and the exit code.
+func run(t *testing.T, client *gossh.Client, cmd, stdin string) (string, int) {
+ t.Helper()
+ sess, err := client.NewSession()
+ if err != nil {
+ t.Fatalf("session: %v", err)
+ }
+ defer sess.Close()
+
+ var out, errOut bytes.Buffer
+ sess.Stdout, sess.Stderr = &out, &errOut
+ sess.Stdin = strings.NewReader(stdin)
+
+ code := 0
+ if err := sess.Run(cmd); err != nil {
+ var exit *gossh.ExitError
+ if !errors.As(err, &exit) {
+ t.Fatalf("%s: %v", cmd, err)
+ }
+ code = exit.ExitStatus()
+ }
+
+ return out.String() + errOut.String(), code
+}
+
+func decode(t *testing.T, line string) dispatch.Response {
+ t.Helper()
+ var resp dispatch.Response
+ if err := json.Unmarshal([]byte(strings.TrimSpace(line)), &resp); err != nil {
+ t.Fatalf("not a JSON response: %q", line)
+ }
+
+ return resp
+}
+
+func sha(s string) string {
+ sum := sha256.Sum256([]byte(s))
+ return hex.EncodeToString(sum[:])
+}
+
+// menu opens a PTY shell, dismisses the splash, quits, and returns the output.
+func menu(t *testing.T, client *gossh.Client) string {
+ t.Helper()
sess, err := client.NewSession()
if err != nil {
t.Fatalf("session: %v", err)
@@ -108,7 +202,6 @@ stdin, err := sess.StdinPipe() if err != nil {
t.Fatalf("stdin: %v", err)
}
-
if err := sess.RequestPty("xterm", 40, 100, gossh.TerminalModes{}); err != nil {
t.Fatalf("pty: %v", err)
}
@@ -129,22 +222,295 @@ case <-time.After(5 * time.Second): t.Fatal("session did not end after quit")
}
- got := out.String()
+ return out.String()
+}
+
+func TestSSHSessionEndToEnd(t *testing.T) {
+ srv := testServer(t, &Config{})
+ signer := newSigner(t)
+ got := menu(t, dial(t, listen(t, srv), signer))
+
+ for _, want := range []string{"HAMMER", "weather", "net", "see you"} {
+ if !strings.Contains(got, want) {
+ t.Errorf("output missing %q:\n%s", want, got)
+ }
+ }
+ if strings.Contains(got, "anonymous session") {
+ t.Error("a keyed session must not get the anonymous notice")
+ }
+
+ fp := gossh.FingerprintSHA256(signer.PublicKey())
+ if _, err := srv.store.Get(identityForFingerprint(fp)); err != nil {
+ t.Errorf("expected persisted profile for %s: %v", identityForFingerprint(fp), err)
+ }
+}
+
+// TestSSHAnonymousKeylessSession connects with NO public key and expects the
+// menu plus the notice that nothing will be saved.
+func TestSSHAnonymousKeylessSession(t *testing.T) {
+ srv := testServer(t, &Config{AllowAnonymous: true})
+ got := menu(t, dial(t, listen(t, srv), nil))
+
if !strings.Contains(got, "HAMMER") {
- t.Errorf("output missing main menu:\n%s", got)
+ t.Errorf("keyless session missing splash/menu:\n%s", got)
+ }
+ if !strings.Contains(got, "anonymous session") {
+ t.Errorf("keyless session missing the not-saved notice:\n%s", got)
+ }
+}
+
+func TestKeylessRefusedWhenAnonymousOff(t *testing.T) {
+ srv := testServer(t, &Config{})
+ _, err := gossh.Dial("tcp", listen(t, srv), &gossh.ClientConfig{
+ User: "guest",
+ Auth: []gossh.AuthMethod{gossh.KeyboardInteractive(func(_, _ string, q []string, _ []bool) ([]string, error) {
+ return make([]string, len(q)), nil
+ })},
+ HostKeyCallback: gossh.InsecureIgnoreHostKey(),
+ Timeout: 5 * time.Second,
+ })
+ if err == nil {
+ t.Fatal("keyless auth must fail with anonymous access off")
}
- if !strings.Contains(got, "weather") {
- t.Errorf("output missing weather menu entry:\n%s", got)
+}
+
+func TestDispatchCommandMode(t *testing.T) {
+ srv := testServer(t, &Config{})
+ client := dial(t, listen(t, srv), newSigner(t))
+
+ out, code := run(t, client, "hash input=hello", "")
+ resp := decode(t, out)
+ if code != 0 || !resp.OK {
+ t.Fatalf("hash: code=%d out=%s", code, out)
+ }
+ if got := resp.Result.Data.(map[string]any)["sha256"]; got != sha("hello") {
+ t.Errorf("sha256 = %v", got)
}
- if !strings.Contains(got, "net") {
- t.Errorf("output missing net tool on main menu:\n%s", got)
+
+ // Payload from stdin, human output.
+ out, code = run(t, client, "hash output=text", "piped bytes")
+ if code != 0 || !strings.Contains(out, sha("piped bytes")) {
+ t.Errorf("stdin hash: code=%d out=%s", code, out)
}
- if !strings.Contains(got, "see you") {
- t.Errorf("output missing quit message:\n%s", got)
+
+ // Quoted arguments survive the shell-style split.
+ out, _ = run(t, client, `jq filter=".a" input='{"a": [1, 2]}'`, "")
+ if resp := decode(t, out); !resp.OK || !strings.Contains(strings.Join(resp.Result.Lines, ""), "1") {
+ t.Errorf("jq: %s", out)
}
- fp := gossh.FingerprintSHA256(signer.PublicKey())
- if _, err := store.Get(identityForFingerprint(fp)); err != nil {
- t.Errorf("expected persisted profile for %s: %v", identityForFingerprint(fp), err)
+ out, code = run(t, client, "rm -rf /", "")
+ if code == 0 || decode(t, out).Error.Code != dispatch.CodeUnknownTool {
+ t.Errorf("unknown tool: code=%d out=%s", code, out)
+ }
+ // A known tool with malformed arguments is refused, not run without them.
+ out, code = run(t, client, "ids -n 5", "")
+ if code == 0 || decode(t, out).Error.Code != dispatch.CodeBadArgs {
+ t.Errorf("malformed args: code=%d out=%s", code, out)
+ }
+
+ out, code = run(t, client, "weather city=Lisbon output=text", "")
+ if code != 0 || !strings.Contains(out, "WEATHER OUTPUT for Lisbon") {
+ t.Errorf("weather: code=%d out=%s", code, out)
+ }
+
+ out, code = run(t, client, "help", "")
+ if code != 0 || !strings.Contains(out, "age.encrypt") || !strings.Contains(out, "weather") {
+ t.Errorf("help: code=%d out=%s", code, out)
+ }
+ if strings.Contains(out, "links") || strings.Contains(out, "path ") {
+ t.Errorf("tools whose binaries are missing must be hidden:\n%s", out)
+ }
+}
+
+func TestDispatchStreamMode(t *testing.T) {
+ srv := testServer(t, &Config{AllowAnonymous: true})
+ client := dial(t, listen(t, srv), nil)
+
+ sess, err := client.NewSession()
+ if err != nil {
+ t.Fatal(err)
+ }
+ var out bytes.Buffer
+ sess.Stdout = &out
+ sess.Stdin = strings.NewReader(strings.Join([]string{
+ `{"id":1,"tool":"b64.encode","args":{"input":"hi"}}`,
+ `{"id":2,"tool":"ids","args":{"count":3}}`,
+ `{"id":3,"tool":"files"}`,
+ `{"id":4,"tool":"weather"}`,
+ }, "\n") + "\n")
+
+ // A pipe with no terminal and no command speaks NDJSON.
+ if err := sess.Shell(); err != nil {
+ t.Fatal(err)
+ }
+ if err := sess.Wait(); err != nil {
+ t.Fatalf("stream session: %v", err)
+ }
+
+ lines := strings.Split(strings.TrimSpace(out.String()), "\n")
+ if len(lines) != 4 {
+ t.Fatalf("got %d responses:\n%s", len(lines), out.String())
+ }
+
+ if r := decode(t, lines[0]); !r.OK || r.Result.Data.(map[string]any)["base64"] != "aGk=" {
+ t.Errorf("b64: %s", lines[0])
+ }
+ if r := decode(t, lines[1]); !r.OK || len(r.Result.Data.(map[string]any)["uuids"].([]any)) != 3 {
+ t.Errorf("ids: %s", lines[1])
+ }
+ if r := decode(t, lines[2]); r.OK || r.Error.Code != dispatch.CodeForbidden {
+ t.Errorf("anonymous files must be refused: %s", lines[2])
+ }
+ if r := decode(t, lines[3]); r.OK || r.Error.Code != dispatch.CodeBadArgs {
+ t.Errorf("anonymous weather with no city must ask for one: %s", lines[3])
+ }
+}
+
+func TestDispatchRateLimit(t *testing.T) {
+ one := 1
+ srv := testServer(t, &Config{Dispatch: &DispatchConfig{RequestsPerMinute: &one, Burst: 1}})
+ client := dial(t, listen(t, srv), newSigner(t))
+
+ if out, code := run(t, client, "ids count=1", ""); code != 0 {
+ t.Fatalf("first call: %s", out)
+ }
+ out, code := run(t, client, "ids count=1", "")
+ if r := decode(t, out); code == 0 || r.Error.Code != dispatch.CodeRateLimited {
+ t.Fatalf("second call must be rate limited: %s", out)
+ }
+ // Listings are free, and the limit follows the key across connections.
+ if out, code := run(t, client, "tools", ""); code != 0 {
+ t.Fatalf("tools must not be limited: %s", out)
+ }
+ if out, code := run(t, dial(t, listen(t, srv), newSigner(t)), "ids count=1", ""); code != 0 {
+ t.Fatalf("another key has its own budget: %s", out)
+ }
+}
+
+func TestAsyncJobAcrossConnections(t *testing.T) {
+ srv := testServer(t, &Config{})
+ release := make(chan struct{})
+ catalog, err := dispatch.NewCatalog(append(srv.dispatchTools(), &dispatch.Tool{
+ Name: "slow",
+ Async: true,
+ Run: func(ctx context.Context, _ *dispatch.Call) (*dispatch.Result, error) {
+ <-release
+
+ return &dispatch.Result{Lines: []string{"slow result"}}, nil
+ },
+ })...)
+ if err != nil {
+ t.Fatal(err)
+ }
+ srv.catalog = catalog
+
+ addr := listen(t, srv)
+ signer := newSigner(t)
+
+ out, _ := run(t, dial(t, addr, signer), "slow async=true", "")
+ submitted := decode(t, out)
+ if !submitted.OK || submitted.Job == nil {
+ t.Fatalf("submit: %s", out)
+ }
+ id := submitted.Job.ID
+ close(release)
+
+ // A later connection with the same key collects the result.
+ later := dial(t, addr, signer)
+ deadline := time.Now().Add(5 * time.Second)
+ for {
+ out, _ = run(t, later, "job id="+id+" output=text", "")
+ if strings.Contains(out, "slow result") {
+ break
+ }
+ if time.Now().After(deadline) {
+ t.Fatalf("job never finished: %s", out)
+ }
+ time.Sleep(20 * time.Millisecond)
+ }
+
+ // Another key cannot see it.
+ out, code := run(t, dial(t, addr, newSigner(t)), "job id="+id, "")
+ if r := decode(t, out); code == 0 || r.Error.Code != dispatch.CodeNotFound {
+ t.Fatalf("another key read the job: %s", out)
+ }
+
+ // Tools that are quick refuse async.
+ out, _ = run(t, later, "ids async=true", "")
+ if r := decode(t, out); r.OK {
+ t.Fatalf("ids must not run as a job: %s", out)
+ }
+}
+
+func TestSFTPFileAreaFeedsTools(t *testing.T) {
+ srv := testServer(t, &Config{AllowAnonymous: true, Files: &FilesConfig{Dir: filepath.Join(t.TempDir(), "files"), QuotaBytes: 64}})
+ addr := listen(t, srv)
+ signer := newSigner(t)
+ client := dial(t, addr, signer)
+
+ fc, err := sftp.NewClient(client)
+ if err != nil {
+ t.Fatalf("sftp: %v", err)
+ }
+ defer fc.Close()
+
+ f, err := fc.Create("/report.csv")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := f.Write([]byte("name,n\na,1\nb,2\n")); err != nil {
+ t.Fatal(err)
+ }
+ f.Close()
+
+ // Over quota: 64 bytes per key.
+ if big, err := fc.Create("/big.bin"); err == nil {
+ _, werr := big.Write(make([]byte, 100))
+ cerr := big.Close()
+ if werr == nil && cerr == nil {
+ t.Error("an upload past the per-key quota must fail")
+ }
+ }
+
+ out, code := run(t, client, "csv file=report.csv", "")
+ if r := decode(t, out); code != 0 || !r.OK || r.Result.Data.(map[string]any)["rows"].(float64) < 2 {
+ t.Fatalf("csv from file: %s", out)
+ }
+ out, _ = run(t, client, "files", "")
+ if r := decode(t, out); !r.OK || !strings.Contains(out, "report.csv") {
+ t.Fatalf("files: %s", out)
+ }
+
+ // Another key has its own, empty area.
+ other := dial(t, addr, newSigner(t))
+ out, _ = run(t, other, "hash file=report.csv", "")
+ if r := decode(t, out); r.OK || r.Error.Code != dispatch.CodeNotFound {
+ t.Fatalf("another key read the file: %s", out)
+ }
+
+ // Anonymous connections get no file area.
+ if _, err := sftp.NewClient(dial(t, addr, nil)); err == nil {
+ t.Fatal("anonymous sftp must be refused")
+ }
+}
+
+func TestSFTPPathEscapeRefused(t *testing.T) {
+ srv := testServer(t, &Config{})
+ client := dial(t, listen(t, srv), newSigner(t))
+ fc, err := sftp.NewClient(client)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer fc.Close()
+
+ f, err := fc.Open("/../../../profiles.db")
+ if err == nil {
+ head := make([]byte, 16)
+ n, _ := io.ReadFull(f, head)
+ f.Close()
+ if strings.HasPrefix(string(head[:n]), "SQLite") {
+ t.Fatal("sftp escaped the file area")
+ }
}
}
@@ -36,6 +36,7 @@ srv *Server sess ssh.Session
profile *Profile
fp string
+ anon bool // keyless: nothing is saved
mu sync.Mutex
width int
height int
@@ -46,45 +47,42 @@ err error }
func (s *Server) handleSession(sess ssh.Session) {
- ptyReq, winCh, isPty := sess.Pty()
-
- st := &session{srv: s, sess: sess, width: 80, height: 24}
- if isPty && ptyReq.Window.Width > 0 {
- st.width = ptyReq.Window.Width
- }
-
- if isPty && ptyReq.Window.Height > 0 {
- st.height = ptyReq.Window.Height
- }
-
- clientIP := st.remoteIP()
+ clientIP := hostOf(sess.RemoteAddr().String())
// Light guardrail on the public port: cap concurrent sessions per IP. The
// defer guarantees the count drains even on error, disconnect or panic.
if !s.conns.acquire(clientIP) {
- io.WriteString(sess, fmt.Sprintf(
+ io.WriteString(sess.Stderr(), fmt.Sprintf(
"connection limit reached (max %d per IP) — close a session and try again.\r\n",
s.cfg.MaxConnectionsPerIP()))
+ _ = sess.Exit(1)
return
}
defer s.conns.release(clientIP)
- // Identity keys the persistent profile: the SSH key fingerprint when the
- // client has one, otherwise a hash of the client IP for keyless sessions.
- fp, _ := sess.Context().Value(fingerprintKey{}).(string)
- var identity string
- switch {
- case fp != "":
- identity = identityForFingerprint(fp)
- case s.cfg.IsAnonymousAllowed():
- identity = identityForIP(clientIP)
- default:
- io.WriteString(sess, "hammer requires a public key. disconnecting.\r\n")
+ identity, ok := s.identify(sess)
+ if !ok {
+ return
+ }
+
+ ptyReq, winCh, isPty := sess.Pty()
+
+ // A command, or a pipe with no terminal, speaks the NDJSON dispatcher; only
+ // an interactive terminal with no command gets the menu.
+ if sess.RawCommand() != "" || !isPty {
+ s.serveDispatch(sess, identity)
return
}
- st.fp = identity
+
+ st := &session{srv: s, sess: sess, width: 80, height: 24, fp: identity, anon: isAnonymous(identity)}
+ if ptyReq.Window.Width > 0 {
+ st.width = ptyReq.Window.Width
+ }
+ if ptyReq.Window.Height > 0 {
+ st.height = ptyReq.Window.Height
+ }
profile, err := s.loadOrCreateProfile(identity, sess.RemoteAddr().String())
if err != nil {
@@ -94,16 +92,44 @@ return
}
st.profile = profile
+ if st.anon {
+ st.note = st.t("BBS_ANON_NOTE")
+ }
- if isPty {
- go func() {
- for win := range winCh {
- st.setSize(win.Width, win.Height)
- }
- }()
+ go func() {
+ for win := range winCh {
+ st.setSize(win.Width, win.Height)
+ }
+ }()
+
+ st.run()
+}
+
+// identify returns the session's identity: the SSH key fingerprint, or a
+// throwaway anonymous identity when keyless access is on. It tells the client
+// why when neither applies.
+func (s *Server) identify(sess ssh.Session) (string, bool) {
+ if fp, _ := sess.Context().Value(fingerprintKey{}).(string); fp != "" {
+ return identityForFingerprint(fp), true
+ }
+ if s.cfg.IsAnonymousAllowed() {
+ return anonymousIdentity(), true
+ }
+
+ io.WriteString(sess.Stderr(), "hammer requires a public key. disconnecting.\r\n")
+ _ = sess.Exit(1)
+
+ return "", false
+}
+
+// hostOf strips the port from a remote address.
+func hostOf(addr string) string {
+ host, _, err := net.SplitHostPort(addr)
+ if err != nil {
+ return addr
}
- st.run()
+ return host
}
func (st *session) run() {
@@ -564,6 +590,9 @@ } }
func (st *session) save() {
+ if st.anon {
+ return // anonymous preferences last only for this connection
+ }
if err := st.srv.store.Save(st.profile); err != nil {
log.Println("bbs: profile save:", err)
}
@@ -761,12 +790,7 @@ } }
func (st *session) remoteIP() string {
- host, _, err := net.SplitHostPort(st.sess.RemoteAddr().String())
- if err != nil {
- return st.sess.RemoteAddr().String()
- }
-
- return host
+ return hostOf(st.sess.RemoteAddr().String())
}
func (st *session) searchCity() {
@@ -1050,12 +1074,21 @@ ui.Accent + st.t("BBS_ABOUT_BLURB") + ui.Reset, "",
ui.Dim + st.t("BBS_VERSION") + " " + ui.Reset + ui.Value + st.srv.cfg.Version + ui.Reset,
ui.Dim + st.t("BBS_VIEW") + " " + ui.Reset + ui.Value + st.viewLabel(st.profile.View) + ui.Reset,
- ui.Dim + st.t("BBS_YOUR_KEY") + " " + ui.Reset + ui.Value + shortFingerprint(st.fp) + ui.Reset,
+ ui.Dim + st.t("BBS_YOUR_KEY") + " " + ui.Reset + ui.Value + st.keyLabel() + ui.Reset,
"",
ui.Accent + attributionAuthor + ui.Reset,
}, w))
st.write([]string{""})
st.pause("")
+}
+
+// keyLabel names the session's identity for the about screen.
+func (st *session) keyLabel() string {
+ if st.anon {
+ return st.t("BBS_ANONYMOUS")
+ }
+
+ return shortFingerprint(st.fp)
}
// shortFingerprint keeps a fingerprint readable on one line.
@@ -3,10 +3,8 @@ import (
"context"
"errors"
- "fmt"
"strconv"
"strings"
- "time"
"unicode/utf8"
"github.com/runawaydevil/hammer/internal/tools/agecrypt"
@@ -18,7 +16,6 @@ "github.com/runawaydevil/hammer/internal/tools/ids" "github.com/runawaydevil/hammer/internal/tools/links"
"github.com/runawaydevil/hammer/internal/tools/netdiag"
"github.com/runawaydevil/hammer/internal/tools/netpath"
- "github.com/runawaydevil/hammer/internal/tools/ocrpdf"
qrtool "github.com/runawaydevil/hammer/internal/tools/qr"
"github.com/runawaydevil/hammer/internal/tools/transform"
"github.com/runawaydevil/hammer/internal/ui"
@@ -359,20 +356,15 @@ st.showReport(lines) }
func (st *session) toolMore() {
- hasOCR := st.srv != nil && st.srv.tools.HasOCR()
st.clear()
lines := []string{
"",
menuEntry("1", st.t("BBS_CSV"), st.t("BBS_CSV_SUM")),
menuEntry("2", st.t("BBS_B64"), st.t("BBS_B64_SUM")),
menuEntry("3", st.t("BBS_IDS"), st.t("BBS_IDS_SUM")),
+ "",
}
- rangeHint := "1-3"
- if hasOCR {
- lines = append(lines, menuEntry("4", st.t("BBS_OCR"), st.t("BBS_OCR_SUM")))
- rangeHint = "1-4"
- }
- lines = append(lines, "")
+ const rangeHint = "1-3"
box := ui.TitledBoxANSI(ui.Title+"MORE"+ui.Reset, lines, menuInner, ui.Rounded(), ui.Frame)
box = append(box, ui.HintBar([][2]string{{rangeHint, st.t("BBS_CHOOSE")}, {"b", st.t("BBS_BACK")}}))
st.screen(box)
@@ -384,87 +376,11 @@ case '2': st.toolB64()
case '3':
st.toolIDs()
- case '4':
- if hasOCR {
- st.toolOCR()
- } else {
- st.note = st.tf("BBS_UNKNOWN_MORE", rangeHint)
- }
case 'b', 'B':
return
default:
st.note = st.tf("BBS_UNKNOWN_MORE", rangeHint)
}
-}
-
-func (st *session) toolOCR() {
- if !st.srv.tools.HasOCR() {
- st.fail(st.t("BBS_NO_OCR"))
-
- return
- }
- cfg := st.srv.tools.Cfg()
- files, err := ocrpdf.ListInbox(cfg.OCRInboxDir, time.Now())
- if err != nil {
- st.fail(err.Error())
-
- return
- }
- if len(files) == 0 {
- st.fail(st.tf("BBS_NO_INBOX", cfg.OCRInboxDir))
-
- return
- }
-
- st.clear()
- rows := []string{""}
- for i, f := range files {
- extra := fmt.Sprintf("%d KiB", f.Size/1024)
- if f.Size < 1024 {
- extra = fmt.Sprintf("%d B", f.Size)
- }
- rows = append(rows, menuEntry(strconv.Itoa(i+1), f.Name, extra))
- }
- rows = append(rows, "")
- box := ui.TitledBoxANSI(ui.Title+"OCR"+ui.Reset, rows, 48, ui.Rounded(), ui.Frame)
- box = append(box, ui.HintBar([][2]string{{"n", st.t("BBS_RUN")}, {"b", st.t("BBS_BACK")}}))
- st.screen(box)
-
- choice := strings.TrimSpace(st.readLine(st.t("BBS_OCR_PROMPT")))
- if choice == "" || strings.EqualFold(choice, "b") {
- return
- }
- n, err := strconv.Atoi(choice)
- if err != nil || n < 1 || n > len(files) {
- st.fail(st.tf("BBS_NO_QUERY", choice))
-
- return
- }
-
- // OCR writes a new PDF into the work dir; refuse if the global storage quota
- // is full. The TTL purge drains it, so it recovers on its own.
- if !st.srv.storage.CanWrite() {
- st.fail(st.t("BBS_STORAGE_FULL"))
-
- return
- }
-
- st.clear()
- st.writeString(ui.Dim + st.tf("BBS_OCR_RUNNING", files[n-1].Name) + ui.Reset + "\r\n")
-
- res, err := ocrpdf.Run(context.Background(), st.srv.tools.Ocrmypdf(), cfg.OCRInboxDir, cfg.OCRWorkDir, files[n-1].Name)
- if err != nil {
- st.fail(err.Error())
-
- return
- }
- st.showReport([]string{
- st.t("BBS_OCR_DONE"),
- "",
- res.OutPDF,
- "",
- st.tf("BBS_OCR_TTL", int(cfg.OCRTTL.Minutes())),
- })
}
func (st *session) toolCSV() {
@@ -1,18 +1,21 @@ package bbs
import (
+ "io/fs"
+ "math"
"os"
"path/filepath"
+ "time"
)
// storageAccountant sums the bytes used across the BBS's writable roots and
// enforces a single global quota shared by everyone (not per user). A nil
// accountant, or a non-positive quota, means unlimited.
//
-// It counts every regular file under the configured roots (ocrInbox,
-// ocrWorkDir, and any other generated-file dirs). Pastes are held in memory and
-// are separately bounded by tools.maxPasteBytes, so they never hit these roots.
-// The space is drained by the background TTL purge (see sweepStorageLoop), so a
+// Today the only writable root is the SFTP file area (bbs.files.dir); each key
+// also has its own smaller quota there (see package filearea). Pastes are held
+// in memory and bounded by tools.maxPasteBytes, so they never touch disk.
+// Space is drained by the background TTL purge (see sweepStorageLoop), so a
// full quota recovers on its own within the TTL.
type storageAccountant struct {
roots []string
@@ -35,7 +38,6 @@ func (a *storageAccountant) Usage() int64 { if a == nil {
return 0
}
-
var total int64
for _, root := range a.roots {
_ = filepath.Walk(root, func(_ string, info os.FileInfo, err error) error {
@@ -56,9 +58,53 @@ // CanWrite reports whether the global quota still has room. Unlimited when the
// accountant is nil or the quota is not positive.
func (a *storageAccountant) CanWrite() bool {
+ return a.Remaining() > 0
+}
+
+// Remaining is how many bytes may still be written under the global quota.
+func (a *storageAccountant) Remaining() int64 {
if a == nil || a.quota <= 0 {
- return true
+ return math.MaxInt64
}
- return a.Usage() < a.quota
+ return max(a.quota-a.Usage(), 0)
+}
+
+// sweep deletes regular files under the roots last modified more than ttl ago,
+// then removes stale empty directories (never the roots themselves).
+func (a *storageAccountant) sweep(ttl time.Duration, now time.Time) {
+ if a == nil || ttl <= 0 {
+ return
+ }
+ for _, root := range a.roots {
+ var dirs []string
+ _ = filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
+ if err != nil {
+ return nil
+ }
+ if d.IsDir() {
+ // Only directories untouched for a whole TTL are candidates, so
+ // a key's area is not pulled out from under an active upload.
+ if info, err := d.Info(); err == nil && p != root && now.Sub(info.ModTime()) > ttl {
+ dirs = append(dirs, p)
+ }
+
+ return nil
+ }
+ info, err := d.Info()
+ if err != nil {
+ return nil
+ }
+ // Links are never created by the file area; remove any found.
+ if !info.Mode().IsRegular() || now.Sub(info.ModTime()) > ttl {
+ _ = os.Remove(p)
+ }
+
+ return nil
+ })
+ // Deepest first, so parents empty out after their children.
+ for i := len(dirs) - 1; i >= 0; i-- {
+ _ = os.Remove(dirs[i]) // fails harmlessly while not empty
+ }
+ }
}
@@ -0,0 +1,127 @@ +// Package clientip decides which address an HTTP request really came from.
+// Forwarding headers are honoured only when the direct peer is a trusted
+// proxy; otherwise anyone could claim any address by sending X-Forwarded-For.
+package clientip
+
+import (
+ "context"
+ "fmt"
+ "net"
+ "net/http"
+ "net/netip"
+ "strings"
+)
+
+// DefaultTrusted is used when no proxy list is configured: only a reverse
+// proxy on the same host (CloudPanel, nginx on loopback) is believed.
+var DefaultTrusted = []string{"127.0.0.0/8", "::1/128"}
+
+// Resolver extracts the client address from a request.
+type Resolver struct {
+ trusted []netip.Prefix
+}
+
+// NewResolver builds a resolver trusting the given proxies. Each entry is an
+// IP address or a CIDR prefix. An empty list trusts nobody, so forwarding
+// headers are always ignored.
+func NewResolver(proxies []string) (*Resolver, error) {
+ r := &Resolver{}
+ for _, p := range proxies {
+ p = strings.TrimSpace(p)
+ if p == "" {
+ continue
+ }
+ if !strings.Contains(p, "/") {
+ addr, err := netip.ParseAddr(p)
+ if err != nil {
+ return nil, fmt.Errorf("trusted proxy %q: %w", p, err)
+ }
+ r.trusted = append(r.trusted, netip.PrefixFrom(addr.Unmap(), addr.Unmap().BitLen()))
+
+ continue
+ }
+ prefix, err := netip.ParsePrefix(p)
+ if err != nil {
+ return nil, fmt.Errorf("trusted proxy %q: %w", p, err)
+ }
+ r.trusted = append(r.trusted, prefix.Masked())
+ }
+
+ return r, nil
+}
+
+func (r *Resolver) isTrusted(addr netip.Addr) bool {
+ addr = addr.Unmap()
+ for _, p := range r.trusted {
+ if p.Contains(addr) {
+ return true
+ }
+ }
+
+ return false
+}
+
+// ClientIP returns the address of the client behind any trusted proxies.
+//
+// The direct peer is the only address that cannot be forged. When it is a
+// trusted proxy, X-Forwarded-For is walked from the right (the entry the proxy
+// itself appended) skipping further trusted hops; the first untrusted address
+// is the client. Entries to its left were written by the client and are
+// ignored. X-Real-IP is consulted only when X-Forwarded-For is absent.
+func (r *Resolver) ClientIP(req *http.Request) string {
+ peer := hostOnly(req.RemoteAddr)
+ peerAddr, err := netip.ParseAddr(peer)
+ if err != nil || !r.isTrusted(peerAddr) {
+ return peer
+ }
+
+ if xff := req.Header.Values("X-Forwarded-For"); len(xff) > 0 {
+ hops := strings.Split(strings.Join(xff, ","), ",")
+ for i := len(hops) - 1; i >= 0; i-- {
+ addr, err := netip.ParseAddr(hostOnly(strings.TrimSpace(hops[i])))
+ if err != nil {
+ // A malformed hop means the chain cannot be trusted past here.
+ return peer
+ }
+ if !r.isTrusted(addr) {
+ return addr.Unmap().String()
+ }
+ }
+
+ return peer
+ }
+
+ if real := strings.TrimSpace(req.Header.Get("X-Real-IP")); real != "" {
+ if addr, err := netip.ParseAddr(real); err == nil {
+ return addr.Unmap().String()
+ }
+ }
+
+ return peer
+}
+
+// hostOnly strips an optional port (and IPv6 brackets) from an address.
+func hostOnly(s string) string {
+ if host, _, err := net.SplitHostPort(s); err == nil {
+ return host
+ }
+
+ return strings.Trim(s, "[]")
+}
+
+type ctxKey struct{}
+
+// WithIP stores the resolved client address in ctx.
+func WithIP(ctx context.Context, ip string) context.Context {
+ return context.WithValue(ctx, ctxKey{}, ip)
+}
+
+// FromRequest returns the address stored by WithIP, falling back to the direct
+// peer. It never reads forwarding headers itself.
+func FromRequest(req *http.Request) string {
+ if ip, ok := req.Context().Value(ctxKey{}).(string); ok && ip != "" {
+ return ip
+ }
+
+ return hostOnly(req.RemoteAddr)
+}
@@ -0,0 +1,98 @@ +package clientip
+
+import (
+ "net/http/httptest"
+ "testing"
+)
+
+func TestClientIP(t *testing.T) {
+ r, err := NewResolver(DefaultTrusted)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ cases := []struct {
+ name string
+ remote string
+ xff []string
+ realIP string
+ want string
+ }{
+ {"direct client, no headers", "203.0.113.9:5555", nil, "", "203.0.113.9"},
+ {"direct client forging XFF", "203.0.113.9:5555", []string{"8.8.8.8"}, "", "203.0.113.9"},
+ {"direct client forging X-Real-IP", "203.0.113.9:5555", nil, "8.8.8.8", "203.0.113.9"},
+ {"trusted proxy appends client", "127.0.0.1:40000", []string{"198.51.100.4"}, "", "198.51.100.4"},
+ {"client-supplied entries left of the proxy's are ignored", "127.0.0.1:40000", []string{"8.8.8.8, 198.51.100.4"}, "", "198.51.100.4"},
+ {"multiple header lines are one chain", "127.0.0.1:40000", []string{"8.8.8.8", "198.51.100.4"}, "", "198.51.100.4"},
+ {"trusted hops are skipped", "127.0.0.1:40000", []string{"198.51.100.4, 127.0.0.1"}, "", "198.51.100.4"},
+ {"only trusted hops falls back to peer", "127.0.0.1:40000", []string{"127.0.0.2"}, "", "127.0.0.1"},
+ {"malformed hop falls back to peer", "127.0.0.1:40000", []string{"garbage"}, "", "127.0.0.1"},
+ {"trusted proxy with X-Real-IP only", "127.0.0.1:40000", nil, "198.51.100.4", "198.51.100.4"},
+ {"XFF wins over X-Real-IP", "127.0.0.1:40000", []string{"198.51.100.4"}, "8.8.8.8", "198.51.100.4"},
+ {"IPv6 loopback proxy", "[::1]:40000", []string{"2001:db8::7"}, "", "2001:db8::7"},
+ {"IPv4-mapped peer is recognised", "[::ffff:127.0.0.1]:40000", []string{"198.51.100.4"}, "", "198.51.100.4"},
+ {"hop with port", "127.0.0.1:40000", []string{"198.51.100.4:1234"}, "", "198.51.100.4"},
+ }
+
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ req := httptest.NewRequest("GET", "/", nil)
+ req.RemoteAddr = tc.remote
+ for _, v := range tc.xff {
+ req.Header.Add("X-Forwarded-For", v)
+ }
+ if tc.realIP != "" {
+ req.Header.Set("X-Real-IP", tc.realIP)
+ }
+ if got := r.ClientIP(req); got != tc.want {
+ t.Errorf("ClientIP = %q, want %q", got, tc.want)
+ }
+ })
+ }
+}
+
+func TestEmptyListTrustsNobody(t *testing.T) {
+ r, err := NewResolver(nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ req := httptest.NewRequest("GET", "/", nil)
+ req.RemoteAddr = "127.0.0.1:1"
+ req.Header.Set("X-Forwarded-For", "8.8.8.8")
+ if got := r.ClientIP(req); got != "127.0.0.1" {
+ t.Fatalf("got %q, headers must be ignored with no trusted proxies", got)
+ }
+}
+
+func TestNewResolverRejectsGarbage(t *testing.T) {
+ if _, err := NewResolver([]string{"not-an-ip"}); err == nil {
+ t.Fatal("expected an error for an invalid proxy entry")
+ }
+ if _, err := NewResolver([]string{"10.0.0.0/99"}); err == nil {
+ t.Fatal("expected an error for an invalid prefix")
+ }
+ r, err := NewResolver([]string{"10.1.2.3", "192.168.0.0/16"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ req := httptest.NewRequest("GET", "/", nil)
+ req.RemoteAddr = "192.168.5.5:1"
+ req.Header.Set("X-Forwarded-For", "198.51.100.4")
+ if got := r.ClientIP(req); got != "198.51.100.4" {
+ t.Fatalf("got %q", got)
+ }
+}
+
+func TestFromRequest(t *testing.T) {
+ req := httptest.NewRequest("GET", "/", nil)
+ req.RemoteAddr = "203.0.113.9:5555"
+ req.Header.Set("X-Forwarded-For", "8.8.8.8")
+ if got := FromRequest(req); got != "203.0.113.9" {
+ t.Fatalf("without a stored IP, FromRequest must use the peer, got %q", got)
+ }
+
+ req = req.WithContext(WithIP(req.Context(), "198.51.100.4"))
+ if got := FromRequest(req); got != "198.51.100.4" {
+ t.Fatalf("got %q", got)
+ }
+}
@@ -0,0 +1,631 @@ +// Package dispatch is the non-interactive face of the SSH front-end: a closed
+// vocabulary of tools, each declaring the arguments it accepts, spoken as
+// newline-delimited JSON. Nothing here runs a shell; a request can only name a
+// catalogued tool and pass values that survive validation.
+//
+// One request per line:
+//
+// {"id":1,"tool":"hash","args":{"input":"hello"}}
+// {"id":2,"tool":"links","args":{"file":"README.md"},"async":true}
+//
+// One response per line, in order:
+//
+// {"id":1,"ok":true,"tool":"hash","result":{"data":{...},"lines":[...]}}
+// {"id":2,"ok":true,"tool":"links","job":{"id":"j_…","status":"queued",...}}
+// {"id":3,"ok":false,"tool":"nope","error":{"code":"unknown_tool","message":"..."}}
+package dispatch
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "sort"
+ "strconv"
+ "strings"
+ "time"
+ "unicode"
+ "unicode/utf8"
+
+ "github.com/runawaydevil/hammer/internal/jobs"
+)
+
+// Kind is an argument's type.
+type Kind string
+
+const (
+ KindString Kind = "string"
+ KindInt Kind = "int"
+ KindBool Kind = "bool"
+)
+
+// Arg declares one argument a tool accepts.
+type Arg struct {
+ Name string `json:"name"`
+ Kind Kind `json:"type"`
+ Required bool `json:"required,omitempty"`
+ Min int `json:"min,omitempty"` // int: smallest value
+ Max int `json:"max,omitempty"` // string: max characters; int: largest value
+ Enum []string `json:"enum,omitempty"`
+ Help string `json:"help,omitempty"`
+}
+
+// Tool is one entry in the vocabulary.
+type Tool struct {
+ Name string
+ Summary string
+ Args []Arg
+ // Input tools take a byte payload from the "input" or "file" argument, or
+ // from stdin when invoked as a command.
+ Input bool
+ // Async tools may be submitted as background jobs.
+ Async bool
+ // Free tools are not rate limited (listings, job polling).
+ Free bool
+ Run func(ctx context.Context, c *Call) (*Result, error)
+}
+
+// Reserved argument names added to every Input tool.
+const (
+ ArgInput = "input"
+ ArgFile = "file"
+)
+
+// FileInfo describes one file in a caller's file area.
+type FileInfo struct {
+ Name string `json:"name"`
+ Size int64 `json:"size"`
+ Modified time.Time `json:"modified"`
+}
+
+// Files is the caller's file area (the one reachable over SFTP).
+type Files interface {
+ List() ([]FileInfo, error)
+ ReadFile(name string, max int64) ([]byte, error)
+}
+
+// Env is everything a request may use beyond its own arguments.
+type Env struct {
+ Owner string // job owner; one SSH identity or connection
+ MaxInput int // bytes accepted inline or from stdin
+ MaxFile int64 // bytes read from the file area
+ Files Files // nil when the caller has no file area
+ Jobs *jobs.Manager // nil disables async
+ Allow func() (bool, time.Duration) // rate limit; nil = unlimited
+ Stdin io.Reader // command mode only: default payload source
+ Hint string // how to reach this host, for help text
+ Caller any // opaque details for tools about who is asking
+}
+
+// Call is a validated request, handed to Tool.Run.
+type Call struct {
+ Args map[string]any
+ Input []byte
+ Env *Env
+}
+
+// Str returns a string argument ("" when absent).
+func (c *Call) Str(name string) string { s, _ := c.Args[name].(string); return s }
+
+// Int returns an int argument (0 when absent).
+func (c *Call) Int(name string) int { n, _ := c.Args[name].(int); return n }
+
+// Bool returns a bool argument (false when absent).
+func (c *Call) Bool(name string) bool { b, _ := c.Args[name].(bool); return b }
+
+// Has reports whether the caller passed the argument.
+func (c *Call) Has(name string) bool { _, ok := c.Args[name]; return ok }
+
+// Result is a tool's output: structured data for programs, lines for people.
+type Result struct {
+ Data any `json:"data,omitempty"`
+ Lines []string `json:"lines,omitempty"`
+}
+
+// Request is one line of input.
+type Request struct {
+ ID json.RawMessage `json:"id,omitempty"`
+ Tool string `json:"tool"`
+ Args map[string]any `json:"args,omitempty"`
+ Async bool `json:"async,omitempty"`
+}
+
+// Response is one line of output.
+type Response struct {
+ ID json.RawMessage `json:"id,omitempty"`
+ OK bool `json:"ok"`
+ Tool string `json:"tool,omitempty"`
+ Result *Result `json:"result,omitempty"`
+ Job *jobs.View `json:"job,omitempty"`
+ Error *Error `json:"error,omitempty"`
+}
+
+// Error codes.
+const (
+ CodeBadRequest = "bad_request"
+ CodeUnknownTool = "unknown_tool"
+ CodeBadArgs = "bad_args"
+ CodeForbidden = "forbidden"
+ CodeNotFound = "not_found"
+ CodeRateLimited = "rate_limited"
+ CodeBusy = "busy"
+ CodeFailed = "failed"
+)
+
+// Error is a protocol-level failure.
+type Error struct {
+ Code string `json:"code"`
+ Message string `json:"message"`
+ RetryAfter float64 `json:"retry_after,omitempty"` // seconds, for rate_limited
+}
+
+func (e *Error) Error() string { return e.Code + ": " + e.Message }
+
+// Errf builds an *Error; tools return it to choose their error code.
+func Errf(code, format string, args ...any) *Error {
+ return &Error{Code: code, Message: fmt.Sprintf(format, args...)}
+}
+
+// Catalog is the closed vocabulary.
+type Catalog struct {
+ byName map[string]*Tool
+ order []*Tool
+}
+
+// NewCatalog registers tools plus the built-ins (tools, files, job, jobs).
+func NewCatalog(tools ...*Tool) (*Catalog, error) {
+ c := &Catalog{byName: map[string]*Tool{}}
+ all := append(builtins(c), tools...)
+ for _, orig := range all {
+ t := new(Tool)
+ *t = *orig
+ t.Args = append([]Arg(nil), orig.Args...)
+ if t.Name == "" || t.Run == nil {
+ return nil, fmt.Errorf("dispatch: tool %q is incomplete", t.Name)
+ }
+ if _, dup := c.byName[t.Name]; dup {
+ return nil, fmt.Errorf("dispatch: duplicate tool %q", t.Name)
+ }
+ if t.Input {
+ t.Args = append(t.Args,
+ Arg{Name: ArgInput, Kind: KindString, Help: "payload, inline"},
+ Arg{Name: ArgFile, Kind: KindString, Max: 255, Help: "payload, from your SFTP file area"},
+ )
+ }
+ seen := map[string]bool{}
+ for _, a := range t.Args {
+ if seen[a.Name] {
+ return nil, fmt.Errorf("dispatch: tool %q declares %q twice", t.Name, a.Name)
+ }
+ seen[a.Name] = true
+ }
+ c.byName[t.Name] = t
+ c.order = append(c.order, t)
+ }
+
+ return c, nil
+}
+
+// Lookup returns a tool by name.
+func (c *Catalog) Lookup(name string) (*Tool, bool) {
+ t, ok := c.byName[name]
+
+ return t, ok
+}
+
+// Handle validates and runs one request.
+func (c *Catalog) Handle(ctx context.Context, env *Env, req Request) (resp Response) {
+ resp = Response{ID: req.ID, Tool: req.Tool}
+ fail := func(e *Error) Response {
+ resp.OK, resp.Error = false, e
+
+ return resp
+ }
+
+ t, ok := c.byName[req.Tool]
+ if !ok {
+ if req.Tool == "" {
+ return fail(Errf(CodeBadRequest, `missing "tool"; send {"tool":"tools"} for the vocabulary`))
+ }
+
+ return fail(Errf(CodeUnknownTool, `no tool %q; send {"tool":"tools"} for the vocabulary`, req.Tool))
+ }
+
+ args, err := validate(t, req.Args)
+ if err != nil {
+ return fail(err)
+ }
+
+ if req.Async && !t.Async {
+ return fail(Errf(CodeBadRequest, "%s does not run as a job; drop \"async\"", t.Name))
+ }
+ if req.Async && env.Jobs == nil {
+ return fail(Errf(CodeForbidden, "background jobs are not available on this connection"))
+ }
+
+ if !t.Free && env.Allow != nil {
+ if ok, wait := env.Allow(); !ok {
+ e := Errf(CodeRateLimited, "too many requests; retry in %.0f s", wait.Seconds()+0.5)
+ e.RetryAfter = wait.Seconds()
+
+ return fail(e)
+ }
+ }
+
+ var input []byte
+ if t.Input {
+ if input, err = resolveInput(args, env); err != nil {
+ return fail(err)
+ }
+ }
+
+ call := &Call{Args: args, Input: input, Env: env}
+
+ if req.Async {
+ v, err := env.Jobs.Submit(env.Owner, t.Name, func(ctx context.Context) (any, error) {
+ return t.Run(ctx, call)
+ })
+ if errors.Is(err, jobs.ErrBusy) {
+ return fail(Errf(CodeBusy, "%s", err))
+ }
+ if err != nil {
+ return fail(Errf(CodeFailed, "%s", err))
+ }
+ resp.OK, resp.Job = true, &v
+
+ return resp
+ }
+
+ res, runErr := run(ctx, t, call)
+ if runErr != nil {
+ var e *Error
+ if errors.As(runErr, &e) {
+ return fail(e)
+ }
+
+ return fail(Errf(CodeFailed, "%s", runErr))
+ }
+ if res == nil {
+ res = &Result{}
+ }
+ resp.OK, resp.Result = true, res
+
+ return resp
+}
+
+// run calls the tool, turning a panic into an error so one bad request cannot
+// take the connection down.
+func run(ctx context.Context, t *Tool, call *Call) (res *Result, err error) {
+ defer func() {
+ if r := recover(); r != nil {
+ err = fmt.Errorf("%s crashed: %v", t.Name, r)
+ }
+ }()
+
+ return t.Run(ctx, call)
+}
+
+// validate checks req args against the tool's declaration and normalises
+// their types (string, int, bool).
+func validate(t *Tool, raw map[string]any) (map[string]any, *Error) {
+ decl := make(map[string]Arg, len(t.Args))
+ for _, a := range t.Args {
+ decl[a.Name] = a
+ }
+
+ out := make(map[string]any, len(raw))
+ for name, v := range raw {
+ a, ok := decl[name]
+ if !ok {
+ return nil, Errf(CodeBadArgs, "%s: unknown argument %q (accepted: %s)", t.Name, name, argNames(t))
+ }
+ if v == nil {
+ continue
+ }
+ val, err := coerce(a, v)
+ if err != nil {
+ return nil, Errf(CodeBadArgs, "%s: %s: %s", t.Name, name, err)
+ }
+ out[name] = val
+ }
+ for _, a := range t.Args {
+ if _, ok := out[a.Name]; a.Required && !ok {
+ return nil, Errf(CodeBadArgs, "%s: missing required argument %q", t.Name, a.Name)
+ }
+ }
+
+ return out, nil
+}
+
+func coerce(a Arg, v any) (any, error) {
+ switch a.Kind {
+ case KindString:
+ s, ok := v.(string)
+ if !ok {
+ return nil, errors.New("must be a string")
+ }
+ if !utf8.ValidString(s) {
+ return nil, errors.New("must be valid UTF-8")
+ }
+ if a.Name != ArgInput {
+ if a.Max > 0 && utf8.RuneCountInString(s) > a.Max {
+ return nil, fmt.Errorf("longer than %d characters", a.Max)
+ }
+ for _, r := range s {
+ if unicode.IsControl(r) {
+ return nil, errors.New("must not contain control characters")
+ }
+ }
+ }
+ if len(a.Enum) > 0 && !contains(a.Enum, s) {
+ return nil, fmt.Errorf("must be one of %s", strings.Join(a.Enum, ", "))
+ }
+
+ return s, nil
+
+ case KindInt:
+ var n int64
+ var err error
+ switch x := v.(type) {
+ case json.Number:
+ n, err = x.Int64()
+ case float64:
+ if x != float64(int64(x)) {
+ err = errors.New("not an integer")
+ }
+ n = int64(x)
+ case string:
+ n, err = strconv.ParseInt(strings.TrimSpace(x), 10, 64)
+ default:
+ err = errors.New("must be an integer")
+ }
+ if err != nil {
+ return nil, errors.New("must be an integer")
+ }
+ if n < int64(a.Min) || (a.Max != 0 && n > int64(a.Max)) {
+ return nil, fmt.Errorf("must be between %d and %d", a.Min, a.Max)
+ }
+
+ return int(n), nil
+
+ case KindBool:
+ switch x := v.(type) {
+ case bool:
+ return x, nil
+ case string:
+ switch strings.ToLower(strings.TrimSpace(x)) {
+ case "true", "1", "yes", "on":
+ return true, nil
+ case "false", "0", "no", "off":
+ return false, nil
+ }
+ }
+
+ return nil, errors.New("must be true or false")
+ }
+
+ return nil, fmt.Errorf("unsupported type %q", a.Kind)
+}
+
+// resolveInput picks the payload: inline "input", a "file" from the caller's
+// area, or stdin in command mode — exactly one of them.
+func resolveInput(args map[string]any, env *Env) ([]byte, *Error) {
+ inline, hasInline := args[ArgInput].(string)
+ file, hasFile := args[ArgFile].(string)
+
+ switch {
+ case hasInline && hasFile:
+ return nil, Errf(CodeBadArgs, `pass "input" or "file", not both`)
+
+ case hasInline:
+ if len(inline) > env.MaxInput {
+ return nil, Errf(CodeBadArgs, "input is larger than %d bytes; upload it over SFTP and pass \"file\"", env.MaxInput)
+ }
+ if inline == "" {
+ return nil, Errf(CodeBadArgs, "input is empty")
+ }
+
+ return []byte(inline), nil
+
+ case hasFile:
+ if env.Files == nil {
+ return nil, Errf(CodeForbidden, "no file area on this connection; connect with an SSH key")
+ }
+ data, err := env.Files.ReadFile(file, env.MaxFile)
+ if err != nil {
+ var e *Error
+ if errors.As(err, &e) {
+ return nil, e
+ }
+
+ return nil, Errf(CodeNotFound, "%s", err)
+ }
+
+ return data, nil
+
+ case env.Stdin != nil:
+ data, err := io.ReadAll(io.LimitReader(env.Stdin, int64(env.MaxInput)+1))
+ if err != nil {
+ return nil, Errf(CodeBadRequest, "reading stdin: %s", err)
+ }
+ if len(data) > env.MaxInput {
+ return nil, Errf(CodeBadArgs, "stdin is larger than %d bytes; upload it over SFTP and pass file=NAME", env.MaxInput)
+ }
+ if len(data) == 0 {
+ return nil, Errf(CodeBadArgs, `no input: pipe it to stdin, or pass input=... or file=NAME`)
+ }
+
+ return data, nil
+ }
+
+ return nil, Errf(CodeBadArgs, `no input: pass "input" or "file"`)
+}
+
+func argNames(t *Tool) string {
+ if len(t.Args) == 0 {
+ return "none"
+ }
+ names := make([]string, len(t.Args))
+ for i, a := range t.Args {
+ names[i] = a.Name
+ }
+
+ return strings.Join(names, ", ")
+}
+
+func contains(list []string, s string) bool {
+ for _, x := range list {
+ if x == s {
+ return true
+ }
+ }
+
+ return false
+}
+
+// ToolInfo is how the "tools" built-in describes an entry.
+type ToolInfo struct {
+ Name string `json:"name"`
+ Summary string `json:"summary"`
+ Args []Arg `json:"args"`
+ Input bool `json:"input,omitempty"`
+ Async bool `json:"async,omitempty"`
+}
+
+// Describe returns the vocabulary in registration order.
+func (c *Catalog) Describe() []ToolInfo {
+ out := make([]ToolInfo, 0, len(c.order))
+ for _, t := range c.order {
+ args := t.Args
+ if args == nil {
+ args = []Arg{}
+ }
+ out = append(out, ToolInfo{Name: t.Name, Summary: t.Summary, Args: args, Input: t.Input, Async: t.Async})
+ }
+
+ return out
+}
+
+func builtins(c *Catalog) []*Tool {
+ return []*Tool{
+ {
+ Name: "tools",
+ Summary: "this list: every tool and the arguments it accepts",
+ Free: true,
+ Run: func(context.Context, *Call) (*Result, error) {
+ info := c.Describe()
+ width := 0
+ for _, t := range info {
+ width = max(width, len(t.Name))
+ }
+ lines := make([]string, 0, len(info))
+ for _, t := range info {
+ var args []string
+ for _, a := range t.Args {
+ s := a.Name
+ if !a.Required {
+ s = "[" + s + "]"
+ }
+ args = append(args, s)
+ }
+ flag := ""
+ if t.Async {
+ flag = " (async)"
+ }
+ lines = append(lines, fmt.Sprintf("%-*s %s%s", width, t.Name, t.Summary, flag))
+ if len(args) > 0 {
+ lines = append(lines, fmt.Sprintf("%-*s %s", width, "", strings.Join(args, " ")))
+ }
+ }
+
+ return &Result{Data: info, Lines: lines}, nil
+ },
+ },
+ {
+ Name: "files",
+ Summary: "list your SFTP file area",
+ Free: true,
+ Run: func(_ context.Context, call *Call) (*Result, error) {
+ if call.Env.Files == nil {
+ return nil, Errf(CodeForbidden, "no file area on this connection; connect with an SSH key")
+ }
+ files, err := call.Env.Files.List()
+ if err != nil {
+ return nil, err
+ }
+ lines := make([]string, 0, len(files))
+ for _, f := range files {
+ lines = append(lines, fmt.Sprintf("%10d %s %s", f.Size, f.Modified.UTC().Format(time.DateTime), f.Name))
+ }
+ if len(lines) == 0 {
+ lines = []string{"(empty)"}
+ }
+ if files == nil {
+ files = []FileInfo{}
+ }
+
+ return &Result{Data: files, Lines: lines}, nil
+ },
+ },
+ {
+ Name: "job",
+ Summary: "status and result of a background job",
+ Args: []Arg{{Name: "id", Kind: KindString, Required: true, Max: 40}},
+ Free: true,
+ Run: func(_ context.Context, call *Call) (*Result, error) {
+ if call.Env.Jobs == nil {
+ return nil, Errf(CodeForbidden, "background jobs are not available on this connection")
+ }
+ v, ok := call.Env.Jobs.Get(call.Env.Owner, call.Str("id"))
+ if !ok {
+ return nil, Errf(CodeNotFound, "no job %q (finished jobs are kept for %s)", call.Str("id"), call.Env.Jobs.Config().TTL)
+ }
+ lines := []string{fmt.Sprintf("%s %s %s", v.ID, v.Tool, v.Status)}
+ if v.Error != "" {
+ lines = append(lines, "error: "+v.Error)
+ }
+ if r, ok := v.Result.(*Result); ok && r != nil {
+ lines = append(lines, "")
+ lines = append(lines, r.Lines...)
+ }
+
+ return &Result{Data: v, Lines: lines}, nil
+ },
+ },
+ {
+ Name: "jobs",
+ Summary: "your background jobs, newest first",
+ Free: true,
+ Run: func(_ context.Context, call *Call) (*Result, error) {
+ if call.Env.Jobs == nil {
+ return nil, Errf(CodeForbidden, "background jobs are not available on this connection")
+ }
+ list := call.Env.Jobs.List(call.Env.Owner)
+ lines := make([]string, 0, len(list))
+ for _, v := range list {
+ lines = append(lines, fmt.Sprintf("%s %-8s %-7s %s", v.ID, v.Tool, v.Status, v.Created.UTC().Format(time.DateTime)))
+ }
+ if len(lines) == 0 {
+ lines = []string{"(no jobs)"}
+ }
+ if list == nil {
+ list = []jobs.View{}
+ }
+
+ return &Result{Data: list, Lines: lines}, nil
+ },
+ },
+ }
+}
+
+// Names lists every tool name, sorted, for error messages and completion.
+func (c *Catalog) Names() []string {
+ names := make([]string, 0, len(c.order))
+ for _, t := range c.order {
+ names = append(names, t.Name)
+ }
+ sort.Strings(names)
+
+ return names
+}
@@ -0,0 +1,357 @@ +package dispatch
+
+import (
+ "bufio"
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/runawaydevil/hammer/internal/jobs"
+)
+
+type memFiles map[string]string
+
+func (m memFiles) List() ([]FileInfo, error) {
+ var out []FileInfo
+ for name, body := range m {
+ out = append(out, FileInfo{Name: name, Size: int64(len(body))})
+ }
+
+ return out, nil
+}
+
+func (m memFiles) ReadFile(name string, max int64) ([]byte, error) {
+ body, ok := m[name]
+ if !ok {
+ return nil, errors.New("no such file")
+ }
+ if int64(len(body)) > max {
+ return nil, Errf(CodeBadArgs, "too big")
+ }
+
+ return []byte(body), nil
+}
+
+func testCatalog(t *testing.T) *Catalog {
+ t.Helper()
+ c, err := NewCatalog(
+ &Tool{
+ Name: "echo",
+ Input: true,
+ Async: true,
+ Args: []Arg{{Name: "upper", Kind: KindBool}},
+ Run: func(_ context.Context, call *Call) (*Result, error) {
+ s := string(call.Input)
+ if call.Bool("upper") {
+ s = strings.ToUpper(s)
+ }
+
+ return &Result{Data: map[string]string{"text": s}, Lines: []string{s}}, nil
+ },
+ },
+ &Tool{
+ Name: "ids",
+ Args: []Arg{
+ {Name: "count", Kind: KindInt, Min: 1, Max: 10},
+ {Name: "kind", Kind: KindString, Enum: []string{"uuid", "ulid"}},
+ {Name: "host", Kind: KindString, Max: 8, Required: true},
+ },
+ Run: func(_ context.Context, call *Call) (*Result, error) {
+ return &Result{Data: map[string]any{"count": call.Int("count"), "kind": call.Str("kind")}}, nil
+ },
+ },
+ &Tool{
+ Name: "boom",
+ Run: func(context.Context, *Call) (*Result, error) { panic("nope") },
+ },
+ &Tool{
+ Name: "denied",
+ Run: func(context.Context, *Call) (*Result, error) {
+ return nil, Errf(CodeForbidden, "not for you")
+ },
+ },
+ )
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ return c
+}
+
+func env() *Env {
+ return &Env{Owner: "o", MaxInput: 64, MaxFile: 1 << 20}
+}
+
+func TestHandleValidation(t *testing.T) {
+ c := testCatalog(t)
+ ctx := context.Background()
+
+ cases := []struct {
+ name string
+ req Request
+ code string
+ }{
+ {"missing tool", Request{}, CodeBadRequest},
+ {"unknown tool", Request{Tool: "rm"}, CodeUnknownTool},
+ {"unknown arg", Request{Tool: "ids", Args: map[string]any{"host": "a", "shell": "x"}}, CodeBadArgs},
+ {"missing required", Request{Tool: "ids"}, CodeBadArgs},
+ {"string too long", Request{Tool: "ids", Args: map[string]any{"host": "123456789"}}, CodeBadArgs},
+ {"control characters", Request{Tool: "ids", Args: map[string]any{"host": "a\nb"}}, CodeBadArgs},
+ {"wrong type", Request{Tool: "ids", Args: map[string]any{"host": 5}}, CodeBadArgs},
+ {"int out of range", Request{Tool: "ids", Args: map[string]any{"host": "a", "count": json.Number("11")}}, CodeBadArgs},
+ {"int not integral", Request{Tool: "ids", Args: map[string]any{"host": "a", "count": 1.5}}, CodeBadArgs},
+ {"enum", Request{Tool: "ids", Args: map[string]any{"host": "a", "kind": "guid"}}, CodeBadArgs},
+ {"bool", Request{Tool: "echo", Args: map[string]any{"input": "x", "upper": "maybe"}}, CodeBadArgs},
+ {"no input", Request{Tool: "echo"}, CodeBadArgs},
+ {"input and file", Request{Tool: "echo", Args: map[string]any{"input": "x", "file": "f"}}, CodeBadArgs},
+ {"input too large", Request{Tool: "echo", Args: map[string]any{"input": strings.Repeat("x", 65)}}, CodeBadArgs},
+ {"file without area", Request{Tool: "echo", Args: map[string]any{"file": "f"}}, CodeForbidden},
+ {"async not allowed", Request{Tool: "ids", Args: map[string]any{"host": "a"}, Async: true}, CodeBadRequest},
+ {"async without jobs", Request{Tool: "echo", Args: map[string]any{"input": "x"}, Async: true}, CodeForbidden},
+ {"panic", Request{Tool: "boom"}, CodeFailed},
+ {"tool error code", Request{Tool: "denied"}, CodeForbidden},
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ resp := c.Handle(ctx, env(), tc.req)
+ if resp.OK || resp.Error == nil || resp.Error.Code != tc.code {
+ t.Fatalf("resp = %+v, want error %s", resp, tc.code)
+ }
+ })
+ }
+}
+
+func TestHandleSuccessAndCoercion(t *testing.T) {
+ c := testCatalog(t)
+ resp := c.Handle(context.Background(), env(), Request{
+ ID: json.RawMessage(`7`),
+ Tool: "ids",
+ Args: map[string]any{"host": "a", "count": "3", "kind": "uuid"},
+ })
+ if !resp.OK || string(resp.ID) != "7" {
+ t.Fatalf("resp = %+v", resp)
+ }
+ data := resp.Result.Data.(map[string]any)
+ if data["count"] != 3 || data["kind"] != "uuid" {
+ t.Fatalf("args not coerced: %+v", data)
+ }
+
+ resp = c.Handle(context.Background(), env(), Request{Tool: "echo", Args: map[string]any{"input": "hi", "upper": "yes"}})
+ if !resp.OK || resp.Result.Lines[0] != "HI" {
+ t.Fatalf("echo = %+v", resp)
+ }
+}
+
+func TestInputFromFileAndStdin(t *testing.T) {
+ c := testCatalog(t)
+ e := env()
+ e.Files = memFiles{"notes.txt": "from file"}
+
+ resp := c.Handle(context.Background(), e, Request{Tool: "echo", Args: map[string]any{"file": "notes.txt"}})
+ if !resp.OK || resp.Result.Lines[0] != "from file" {
+ t.Fatalf("file input = %+v", resp)
+ }
+ resp = c.Handle(context.Background(), e, Request{Tool: "echo", Args: map[string]any{"file": "missing"}})
+ if resp.OK || resp.Error.Code != CodeNotFound {
+ t.Fatalf("missing file = %+v", resp)
+ }
+
+ e.Stdin = strings.NewReader("from stdin")
+ resp = c.Handle(context.Background(), e, Request{Tool: "echo"})
+ if !resp.OK || resp.Result.Lines[0] != "from stdin" {
+ t.Fatalf("stdin input = %+v", resp)
+ }
+
+ e.Stdin = strings.NewReader(strings.Repeat("x", 100))
+ resp = c.Handle(context.Background(), e, Request{Tool: "echo"})
+ if resp.OK || resp.Error.Code != CodeBadArgs {
+ t.Fatalf("oversized stdin = %+v", resp)
+ }
+}
+
+func TestRateLimitSkipsFreeTools(t *testing.T) {
+ c := testCatalog(t)
+ e := env()
+ e.Allow = func() (bool, time.Duration) { return false, 3 * time.Second }
+
+ resp := c.Handle(context.Background(), e, Request{Tool: "echo", Args: map[string]any{"input": "x"}})
+ if resp.OK || resp.Error.Code != CodeRateLimited || resp.Error.RetryAfter != 3 {
+ t.Fatalf("limited call = %+v", resp)
+ }
+ if resp := c.Handle(context.Background(), e, Request{Tool: "tools"}); !resp.OK {
+ t.Fatalf("tools must not be rate limited: %+v", resp)
+ }
+}
+
+func TestAsyncJob(t *testing.T) {
+ c := testCatalog(t)
+ e := env()
+ e.Jobs = jobs.New(jobs.Config{})
+ defer e.Jobs.Close()
+
+ resp := c.Handle(context.Background(), e, Request{Tool: "echo", Args: map[string]any{"input": "later"}, Async: true})
+ if !resp.OK || resp.Job == nil {
+ t.Fatalf("submit = %+v", resp)
+ }
+
+ deadline := time.Now().Add(5 * time.Second)
+ for time.Now().Before(deadline) {
+ poll := c.Handle(context.Background(), e, Request{Tool: "job", Args: map[string]any{"id": resp.Job.ID}})
+ if !poll.OK {
+ t.Fatalf("poll = %+v", poll)
+ }
+ v := poll.Result.Data.(jobs.View)
+ if v.Status == jobs.Done {
+ if poll.Result.Lines[len(poll.Result.Lines)-1] != "later" {
+ t.Fatalf("job lines = %v", poll.Result.Lines)
+ }
+
+ other := &Env{Owner: "someone-else", Jobs: e.Jobs}
+ if r := c.Handle(context.Background(), other, Request{Tool: "job", Args: map[string]any{"id": resp.Job.ID}}); r.OK || r.Error.Code != CodeNotFound {
+ t.Fatalf("another owner read the job: %+v", r)
+ }
+
+ list := c.Handle(context.Background(), e, Request{Tool: "jobs"})
+ if !list.OK || len(list.Result.Data.([]jobs.View)) != 1 {
+ t.Fatalf("jobs = %+v", list)
+ }
+
+ return
+ }
+ time.Sleep(5 * time.Millisecond)
+ }
+ t.Fatal("job did not finish")
+}
+
+func TestToolsListing(t *testing.T) {
+ c := testCatalog(t)
+ resp := c.Handle(context.Background(), env(), Request{Tool: "tools"})
+ info := resp.Result.Data.([]ToolInfo)
+ names := map[string]ToolInfo{}
+ for _, ti := range info {
+ names[ti.Name] = ti
+ }
+ for _, want := range []string{"tools", "files", "job", "jobs", "echo", "ids"} {
+ if _, ok := names[want]; !ok {
+ t.Errorf("tools listing is missing %q", want)
+ }
+ }
+ echo := names["echo"]
+ if !echo.Input || !echo.Async || len(echo.Args) != 3 {
+ t.Errorf("echo description = %+v (want upper, input, file)", echo)
+ }
+}
+
+func TestCatalogRejectsBadDefinitions(t *testing.T) {
+ run := func(context.Context, *Call) (*Result, error) { return nil, nil }
+ if _, err := NewCatalog(&Tool{Name: "tools", Run: run}); err == nil {
+ t.Error("a tool shadowing a built-in must be rejected")
+ }
+ if _, err := NewCatalog(&Tool{Name: "x"}); err == nil {
+ t.Error("a tool without Run must be rejected")
+ }
+ if _, err := NewCatalog(&Tool{Name: "x", Input: true, Args: []Arg{{Name: "input", Kind: KindString}}, Run: run}); err == nil {
+ t.Error("an Input tool redeclaring input must be rejected")
+ }
+
+ // Building twice from the same definitions must not accumulate args.
+ def := &Tool{Name: "x", Input: true, Run: run}
+ for i := 0; i < 2; i++ {
+ if _, err := NewCatalog(def); err != nil {
+ t.Fatalf("build %d: %v", i+1, err)
+ }
+ }
+}
+
+func TestServeStream(t *testing.T) {
+ c := testCatalog(t)
+ in := strings.Join([]string{
+ `{"id":1,"tool":"echo","args":{"input":"a"}}`,
+ ``,
+ `not json`,
+ `{"id":2,"tool":"echo","args":{"input":"b"},"extra":1}`,
+ `{"id":"three","tool":"ids","args":{"host":"h","count":2}}`,
+ `{"id":4,"tool":"echo","args":{"input":"` + strings.Repeat("x", 300) + `"}}`,
+ `{"id":5,"tool":"echo","args":{"input":"last"}}`,
+ }, "\n")
+
+ var out bytes.Buffer
+ if err := c.Serve(context.Background(), env(), strings.NewReader(in), &out, 200); err != nil {
+ t.Fatal(err)
+ }
+
+ var resps []Response
+ sc := bufio.NewScanner(&out)
+ for sc.Scan() {
+ var r Response
+ if err := json.Unmarshal(sc.Bytes(), &r); err != nil {
+ t.Fatalf("response is not JSON: %q", sc.Text())
+ }
+ resps = append(resps, r)
+ }
+
+ if len(resps) != 6 {
+ t.Fatalf("got %d responses, want 6 (blank lines are skipped):\n%s", len(resps), out.String())
+ }
+ checks := []struct {
+ ok bool
+ id string
+ code string
+ }{
+ {true, "1", ""},
+ {false, "", CodeBadRequest},
+ {false, "", CodeBadRequest}, // unknown top-level field
+ {true, `"three"`, ""},
+ {false, "", CodeBadRequest}, // line too long
+ {true, "5", ""},
+ }
+ for i, want := range checks {
+ r := resps[i]
+ if r.OK != want.ok || string(r.ID) != want.id || (want.code != "" && r.Error.Code != want.code) {
+ t.Errorf("response %d = %+v, want %+v", i, r, want)
+ }
+ }
+}
+
+func TestParseCommand(t *testing.T) {
+ req, text, err := ParseCommand([]string{"ids", "host=h", "count=2", "output=text"})
+ if err != nil || req.Tool != "ids" || !text || req.Args["count"] != "2" || req.Args["host"] != "h" {
+ t.Fatalf("req=%+v text=%v err=%v", req, text, err)
+ }
+
+ req, text, err = ParseCommand([]string{"help"})
+ if err != nil || req.Tool != "tools" || !text {
+ t.Fatalf("help: req=%+v text=%v err=%v", req, text, err)
+ }
+
+ req, _, err = ParseCommand([]string{"echo", "async=true", "input=a=b"})
+ if err != nil || !req.Async || req.Args["input"] != "a=b" {
+ t.Fatalf("async: req=%+v err=%v", req, err)
+ }
+
+ for _, bad := range [][]string{{}, {"ids", "oops"}, {"ids", "=x"}, {"ids", "a=1", "a=2"}, {"ids", "output=xml"}, {"ids", "async=perhaps"}} {
+ if _, _, err := ParseCommand(bad); err == nil {
+ t.Errorf("%v: expected an error", bad)
+ }
+ }
+}
+
+func TestFormatText(t *testing.T) {
+ if got := FormatText(Response{OK: true, Result: &Result{Lines: []string{"a", "b"}}}, "ssh h"); got != "a\nb\n" {
+ t.Errorf("lines: %q", got)
+ }
+ if got := FormatText(Response{Error: Errf(CodeBadArgs, "nope")}, "ssh h"); !strings.Contains(got, "bad_args") {
+ t.Errorf("error: %q", got)
+ }
+ job := &jobs.View{ID: "j_1", Status: jobs.Queued}
+ if got := FormatText(Response{OK: true, Job: job}, "ssh -p 2323 h"); !strings.Contains(got, "ssh -p 2323 h job id=j_1") {
+ t.Errorf("job: %q", got)
+ }
+ if got := FormatText(Response{OK: true, Result: &Result{Data: map[string]int{"n": 1}}}, ""); !strings.Contains(got, `"n": 1`) {
+ t.Errorf("data: %q", got)
+ }
+}
@@ -0,0 +1,169 @@ +package dispatch
+
+import (
+ "bufio"
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "strings"
+)
+
+// Serve reads requests line by line from r and writes one response line per
+// request to w, until EOF. A line longer than maxLine is answered with an
+// error and skipped, so one bad line never desynchronises the stream.
+func (c *Catalog) Serve(ctx context.Context, env *Env, r io.Reader, w io.Writer, maxLine int) error {
+ enc := json.NewEncoder(w)
+ enc.SetEscapeHTML(false)
+ br := bufio.NewReaderSize(r, 64<<10)
+
+ for {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+
+ line, tooLong, readErr := readLine(br, maxLine)
+ if tooLong {
+ resp := Response{Error: Errf(CodeBadRequest, "request line exceeds %d bytes; upload large payloads over SFTP and pass \"file\"", maxLine)}
+ if err := enc.Encode(resp); err != nil {
+ return err
+ }
+ } else if len(bytes.TrimSpace(line)) > 0 {
+ var resp Response
+ req, err := decodeRequest(line)
+ if err != nil {
+ resp = Response{Error: err}
+ } else {
+ resp = c.Handle(ctx, env, req)
+ }
+ if err := enc.Encode(resp); err != nil {
+ return err
+ }
+ }
+
+ if errors.Is(readErr, io.EOF) {
+ return nil
+ }
+ if readErr != nil {
+ return readErr
+ }
+ }
+}
+
+// readLine returns the next line without its terminator. When the line is
+// longer than max it is consumed and discarded, and tooLong is set.
+func readLine(br *bufio.Reader, max int) (line []byte, tooLong bool, err error) {
+ var buf []byte
+ for {
+ chunk, err := br.ReadSlice('\n')
+ if !tooLong {
+ if len(buf)+len(chunk) > max+2 { // +2 tolerates "\r\n"
+ tooLong, buf = true, nil
+ } else {
+ buf = append(buf, chunk...)
+ }
+ }
+ if errors.Is(err, bufio.ErrBufferFull) {
+ continue
+ }
+
+ return bytes.TrimRight(buf, "\r\n"), tooLong, err
+ }
+}
+
+func decodeRequest(line []byte) (Request, *Error) {
+ dec := json.NewDecoder(bytes.NewReader(line))
+ dec.UseNumber()
+ dec.DisallowUnknownFields()
+
+ var req Request
+ if err := dec.Decode(&req); err != nil {
+ return Request{}, Errf(CodeBadRequest, "invalid JSON request: %s", err)
+ }
+ if dec.More() {
+ return Request{}, Errf(CodeBadRequest, "one JSON object per line")
+ }
+
+ return req, nil
+}
+
+// ParseCommand turns an SSH command line (already split into words) into a
+// request: the first word names the tool and the rest are key=value pairs.
+// Two keys are reserved: async=true submits a job, and output=text prints
+// the human-readable lines instead of JSON. "help" is an alias for tools
+// with text output.
+func ParseCommand(argv []string) (req Request, text bool, err *Error) {
+ if len(argv) == 0 {
+ return Request{}, false, Errf(CodeBadRequest, "no command")
+ }
+
+ req.Tool = argv[0]
+ if req.Tool == "help" {
+ req.Tool, text = "tools", true
+ }
+
+ for _, kv := range argv[1:] {
+ key, value, ok := strings.Cut(kv, "=")
+ if !ok || key == "" {
+ return Request{}, false, Errf(CodeBadArgs, "expected key=value, got %q (run \"help\" for the vocabulary)", kv)
+ }
+ switch key {
+ case "async":
+ b, err := coerce(Arg{Name: key, Kind: KindBool}, value)
+ if err != nil {
+ return Request{}, false, Errf(CodeBadArgs, "async: %s", err)
+ }
+ req.Async = b.(bool)
+
+ continue
+ case "output":
+ switch value {
+ case "text":
+ text = true
+ case "json":
+ text = false
+ default:
+ return Request{}, false, Errf(CodeBadArgs, "output must be text or json")
+ }
+
+ continue
+ }
+ if req.Args == nil {
+ req.Args = map[string]any{}
+ }
+ if _, dup := req.Args[key]; dup {
+ return Request{}, false, Errf(CodeBadArgs, "argument %q given twice", key)
+ }
+ req.Args[key] = value
+ }
+
+ return req, text, nil
+}
+
+// FormatText renders a response for a person reading a terminal.
+func FormatText(resp Response, hint string) string {
+ var b strings.Builder
+ switch {
+ case !resp.OK && resp.Error != nil:
+ fmt.Fprintf(&b, "error (%s): %s\n", resp.Error.Code, resp.Error.Message)
+
+ case resp.Job != nil:
+ fmt.Fprintf(&b, "job %s %s\n", resp.Job.ID, resp.Job.Status)
+ fmt.Fprintf(&b, "check it with: %s job id=%s output=text\n", hint, resp.Job.ID)
+
+ case resp.Result != nil && len(resp.Result.Lines) > 0:
+ for _, l := range resp.Result.Lines {
+ b.WriteString(l)
+ b.WriteByte('\n')
+ }
+
+ case resp.Result != nil && resp.Result.Data != nil:
+ out, _ := json.MarshalIndent(resp.Result.Data, "", " ")
+ b.Write(out)
+ b.WriteByte('\n')
+ }
+
+ return b.String()
+}
@@ -0,0 +1,392 @@ +// Package filearea is the per-key file area: a private directory for each SSH
+// identity, reachable over SFTP and readable by tools through the dispatcher
+// ("file" argument). Paths are confined to the area, links are refused, and
+// writes stop at the per-key quota and at whatever the global quota has left.
+package filearea
+
+import (
+ "crypto/sha256"
+ "encoding/hex"
+ "errors"
+ "fmt"
+ "io"
+ "io/fs"
+ "math"
+ "os"
+ "path"
+ "path/filepath"
+ "sort"
+ "strings"
+
+ "github.com/pkg/sftp"
+
+ "github.com/runawaydevil/hammer/internal/dispatch"
+)
+
+// ErrQuota is returned when a write would exceed a quota.
+var ErrQuota = errors.New("quota exceeded: delete files or wait for them to expire")
+
+// Budget reports how many bytes may still be written across every area.
+type Budget interface {
+ Remaining() int64
+}
+
+// Area is one identity's directory.
+type Area struct {
+ root string
+ quota int64 // per-area bytes; <= 0 = unlimited
+ global Budget
+}
+
+// DirFor returns the directory name used for identity under base. The
+// identity is hashed so fingerprints never appear on disk.
+func DirFor(base, identity string) string {
+ sum := sha256.Sum256([]byte(identity))
+
+ return filepath.Join(base, hex.EncodeToString(sum[:16]))
+}
+
+// Open returns the area for identity, creating its directory.
+func Open(base, identity string, quota int64, global Budget) (*Area, error) {
+ if base == "" || identity == "" {
+ return nil, errors.New("file area not configured")
+ }
+ root, err := filepath.Abs(DirFor(base, identity))
+ if err != nil {
+ return nil, err
+ }
+ if err := os.MkdirAll(root, 0o700); err != nil {
+ return nil, err
+ }
+
+ return &Area{root: root, quota: quota, global: global}, nil
+}
+
+// Root is the area's absolute directory.
+func (a *Area) Root() string { return a.root }
+
+// resolve maps a client path ("/docs/a.txt", "docs/a.txt") to a host path
+// inside the area. No component may be a symbolic link.
+func (a *Area) resolve(p string) (string, error) {
+ // The storage purge may remove an idle, empty area; bring it back.
+ if err := os.MkdirAll(a.root, 0o700); err != nil {
+ return "", err
+ }
+ clean := path.Clean("/" + strings.ReplaceAll(p, "\\", "/"))
+ if strings.ContainsRune(clean, 0) {
+ return "", fs.ErrInvalid
+ }
+ full := filepath.Join(a.root, filepath.FromSlash(clean))
+
+ rel, err := filepath.Rel(a.root, full)
+ if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
+ return "", fs.ErrPermission
+ }
+
+ // Refuse links anywhere along the path, including the leaf.
+ cur := a.root
+ if rel != "." {
+ for _, part := range strings.Split(rel, string(filepath.Separator)) {
+ cur = filepath.Join(cur, part)
+ fi, err := os.Lstat(cur)
+ if errors.Is(err, fs.ErrNotExist) {
+ break
+ }
+ if err != nil {
+ return "", err
+ }
+ if fi.Mode()&fs.ModeSymlink != 0 {
+ return "", fs.ErrPermission
+ }
+ }
+ }
+
+ return full, nil
+}
+
+// Usage is the total size of regular files in the area.
+func (a *Area) Usage() int64 {
+ var total int64
+ _ = filepath.WalkDir(a.root, func(_ string, d fs.DirEntry, err error) error {
+ if err != nil {
+ return nil
+ }
+ if d.Type().IsRegular() {
+ if fi, err := d.Info(); err == nil {
+ total += fi.Size()
+ }
+ }
+
+ return nil
+ })
+
+ return total
+}
+
+// allowance is how many more bytes may be written right now.
+func (a *Area) allowance() int64 {
+ left := int64(math.MaxInt64)
+ if a.quota > 0 {
+ left = a.quota - a.Usage()
+ }
+ if a.global != nil {
+ left = min(left, a.global.Remaining())
+ }
+
+ return left
+}
+
+// List returns every regular file in the area, with slash-separated names
+// relative to its root.
+func (a *Area) List() ([]dispatch.FileInfo, error) {
+ var out []dispatch.FileInfo
+ err := filepath.WalkDir(a.root, func(p string, d fs.DirEntry, err error) error {
+ if err != nil {
+ return nil
+ }
+ if !d.Type().IsRegular() {
+ return nil
+ }
+ fi, err := d.Info()
+ if err != nil {
+ return nil
+ }
+ rel, _ := filepath.Rel(a.root, p)
+ out = append(out, dispatch.FileInfo{Name: filepath.ToSlash(rel), Size: fi.Size(), Modified: fi.ModTime()})
+
+ return nil
+ })
+ sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+
+ return out, err
+}
+
+// ReadFile returns a file's contents, refusing files larger than max bytes.
+func (a *Area) ReadFile(name string, max int64) ([]byte, error) {
+ full, err := a.resolve(name)
+ if err != nil {
+ return nil, dispatch.Errf(dispatch.CodeForbidden, "%s: path not allowed", name)
+ }
+ fi, err := os.Lstat(full)
+ if err != nil || !fi.Mode().IsRegular() {
+ return nil, dispatch.Errf(dispatch.CodeNotFound, "%s: no such file in your area (see the files tool)", name)
+ }
+ if fi.Size() > max {
+ return nil, dispatch.Errf(dispatch.CodeBadArgs, "%s is %d bytes; tools read files up to %d bytes", name, fi.Size(), max)
+ }
+
+ return os.ReadFile(full)
+}
+
+// Handlers exposes the area to an SFTP request server.
+func (a *Area) Handlers() sftp.Handlers {
+ h := &handlers{a: a}
+
+ return sftp.Handlers{FileGet: h, FilePut: h, FileCmd: h, FileList: h}
+}
+
+type handlers struct{ a *Area }
+
+func (h *handlers) Fileread(r *sftp.Request) (io.ReaderAt, error) {
+ full, err := h.a.resolve(r.Filepath)
+ if err != nil {
+ return nil, sftp.ErrSSHFxPermissionDenied
+ }
+ fi, err := os.Lstat(full)
+ if err != nil {
+ return nil, sftp.ErrSSHFxNoSuchFile
+ }
+ if !fi.Mode().IsRegular() {
+ return nil, sftp.ErrSSHFxFailure
+ }
+
+ return os.Open(full)
+}
+
+func (h *handlers) Filewrite(r *sftp.Request) (io.WriterAt, error) {
+ full, err := h.a.resolve(r.Filepath)
+ if err != nil || full == h.a.root {
+ return nil, sftp.ErrSSHFxPermissionDenied
+ }
+
+ flags := r.Pflags()
+ var existing int64
+ if fi, err := os.Lstat(full); err == nil {
+ if !fi.Mode().IsRegular() {
+ return nil, sftp.ErrSSHFxFailure
+ }
+ if flags.Excl {
+ return nil, os.ErrExist
+ }
+ existing = fi.Size()
+ }
+
+ allow := h.a.allowance()
+ if flags.Trunc {
+ // Overwriting frees the old contents first.
+ allow += existing
+ existing = 0
+ }
+ if allow <= 0 {
+ return nil, ErrQuota
+ }
+
+ mode := os.O_WRONLY | os.O_CREATE
+ if flags.Trunc {
+ mode |= os.O_TRUNC
+ }
+ if flags.Excl {
+ mode |= os.O_EXCL
+ }
+ f, err := os.OpenFile(full, mode, 0o600)
+ if err != nil {
+ return nil, err
+ }
+
+ return &cappedWriter{f: f, limit: sat(existing, allow)}, nil
+}
+
+// cappedWriter refuses writes that would grow the file past limit.
+type cappedWriter struct {
+ f *os.File
+ limit int64
+}
+
+func (w *cappedWriter) WriteAt(p []byte, off int64) (int, error) {
+ if off < 0 || off > w.limit || int64(len(p)) > w.limit-off {
+ return 0, ErrQuota
+ }
+
+ return w.f.WriteAt(p, off)
+}
+
+func (w *cappedWriter) Close() error { return w.f.Close() }
+
+func (h *handlers) Filecmd(r *sftp.Request) error {
+ full, err := h.a.resolve(r.Filepath)
+ if err != nil {
+ return sftp.ErrSSHFxPermissionDenied
+ }
+
+ switch r.Method {
+ case "Setstat":
+ // Permissions and times are ours to decide; only shrinking is honoured,
+ // so a client cannot grow a file past the quota with truncate().
+ if r.AttrFlags().Size {
+ fi, err := os.Lstat(full)
+ if err != nil {
+ return sftp.ErrSSHFxNoSuchFile
+ }
+ if size := int64(r.Attributes().Size); size < fi.Size() {
+ return os.Truncate(full, size)
+ } else if size > fi.Size() {
+ return ErrQuota
+ }
+ }
+
+ return nil
+
+ case "Rename":
+ target, err := h.a.resolve(r.Target)
+ if err != nil || full == h.a.root || target == h.a.root {
+ return sftp.ErrSSHFxPermissionDenied
+ }
+
+ return os.Rename(full, target)
+
+ case "Mkdir":
+ if full == h.a.root {
+ return os.ErrExist
+ }
+
+ return os.Mkdir(full, 0o700)
+
+ case "Rmdir":
+ if full == h.a.root {
+ return sftp.ErrSSHFxPermissionDenied
+ }
+ fi, err := os.Lstat(full)
+ if err != nil {
+ return sftp.ErrSSHFxNoSuchFile
+ }
+ if !fi.IsDir() {
+ return fmt.Errorf("%s is not a directory", r.Filepath)
+ }
+
+ return os.Remove(full)
+
+ case "Remove":
+ fi, err := os.Lstat(full)
+ if err != nil {
+ return sftp.ErrSSHFxNoSuchFile
+ }
+ if fi.IsDir() {
+ return fmt.Errorf("%s is a directory; use rmdir", r.Filepath)
+ }
+
+ return os.Remove(full)
+ }
+
+ // Link, Symlink and anything newer.
+ return sftp.ErrSSHFxOpUnsupported
+}
+
+func (h *handlers) Filelist(r *sftp.Request) (sftp.ListerAt, error) {
+ full, err := h.a.resolve(r.Filepath)
+ if err != nil {
+ return nil, sftp.ErrSSHFxPermissionDenied
+ }
+
+ switch r.Method {
+ case "List":
+ entries, err := os.ReadDir(full)
+ if err != nil {
+ return nil, sftp.ErrSSHFxNoSuchFile
+ }
+ var infos []os.FileInfo
+ for _, e := range entries {
+ if e.Type()&fs.ModeSymlink != 0 {
+ continue
+ }
+ if fi, err := e.Info(); err == nil {
+ infos = append(infos, fi)
+ }
+ }
+
+ return listerAt(infos), nil
+
+ case "Stat":
+ fi, err := os.Lstat(full)
+ if err != nil {
+ return nil, sftp.ErrSSHFxNoSuchFile
+ }
+
+ return listerAt{fi}, nil
+ }
+
+ // Readlink: there are no links to read.
+ return nil, sftp.ErrSSHFxOpUnsupported
+}
+
+type listerAt []os.FileInfo
+
+func (l listerAt) ListAt(dst []os.FileInfo, off int64) (int, error) {
+ if off >= int64(len(l)) {
+ return 0, io.EOF
+ }
+ n := copy(dst, l[off:])
+ if n < len(dst) {
+ return n, io.EOF
+ }
+
+ return n, nil
+}
+
+// sat adds without overflowing.
+func sat(a, b int64) int64 {
+ if b > math.MaxInt64-a {
+ return math.MaxInt64
+ }
+
+ return a + b
+}
@@ -0,0 +1,222 @@ +package filearea
+
+import (
+ "errors"
+ "io"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/pkg/sftp"
+
+ "github.com/runawaydevil/hammer/internal/dispatch"
+)
+
+type fixedBudget int64
+
+func (b fixedBudget) Remaining() int64 { return int64(b) }
+
+// client connects an in-memory SFTP client to the area's request server.
+func client(t *testing.T, a *Area) *sftp.Client {
+ t.Helper()
+ cr, sw := io.Pipe()
+ sr, cw := io.Pipe()
+
+ srv := sftp.NewRequestServer(struct {
+ io.Reader
+ io.WriteCloser
+ }{sr, sw}, a.Handlers())
+ go srv.Serve()
+
+ c, err := sftp.NewClientPipe(cr, cw)
+ if err != nil {
+ t.Fatal(err)
+ }
+ // Closing the pipes first unblocks both ends; Close would otherwise wait
+ // forever for a peer that is still reading.
+ t.Cleanup(func() {
+ cw.Close()
+ sw.Close()
+ c.Close()
+ srv.Close()
+ })
+
+ return c
+}
+
+func put(c *sftp.Client, name, body string) error {
+ f, err := c.Create(name)
+ if err != nil {
+ return err
+ }
+ if _, err := f.Write([]byte(body)); err != nil {
+ f.Close()
+
+ return err
+ }
+
+ return f.Close()
+}
+
+func TestIdentitiesGetSeparateDirs(t *testing.T) {
+ base := t.TempDir()
+ a, err := Open(base, "fp:SHA256:alice", 0, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ b, err := Open(base, "fp:SHA256:bob", 0, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if a.Root() == b.Root() {
+ t.Fatal("two identities share a directory")
+ }
+ if strings.Contains(a.Root(), "alice") {
+ t.Fatal("the identity must not appear on disk")
+ }
+ if _, err := Open("", "x", 0, nil); err == nil {
+ t.Fatal("an unconfigured base must be refused")
+ }
+}
+
+func TestSFTPRoundTripAndDispatchRead(t *testing.T) {
+ a, _ := Open(t.TempDir(), "id", 0, nil)
+ c := client(t, a)
+
+ if err := c.Mkdir("/docs"); err != nil {
+ t.Fatal(err)
+ }
+ if err := put(c, "/docs/a.txt", "hello"); err != nil {
+ t.Fatal(err)
+ }
+
+ f, err := c.Open("docs/a.txt")
+ if err != nil {
+ t.Fatal(err)
+ }
+ got, _ := io.ReadAll(f)
+ f.Close()
+ if string(got) != "hello" {
+ t.Fatalf("read back %q", got)
+ }
+
+ entries, err := c.ReadDir("/docs")
+ if err != nil || len(entries) != 1 || entries[0].Name() != "a.txt" {
+ t.Fatalf("readdir = %v, %v", entries, err)
+ }
+
+ data, err := a.ReadFile("docs/a.txt", 100)
+ if err != nil || string(data) != "hello" {
+ t.Fatalf("ReadFile = %q, %v", data, err)
+ }
+ var de *dispatch.Error
+ if _, err := a.ReadFile("docs/a.txt", 2); !errors.As(err, &de) || de.Code != dispatch.CodeBadArgs {
+ t.Fatalf("oversized ReadFile: %v", err)
+ }
+ if _, err := a.ReadFile("nope", 100); !errors.As(err, &de) || de.Code != dispatch.CodeNotFound {
+ t.Fatalf("missing ReadFile: %v", err)
+ }
+
+ list, err := a.List()
+ if err != nil || len(list) != 1 || list[0].Name != "docs/a.txt" || list[0].Size != 5 {
+ t.Fatalf("List = %+v, %v", list, err)
+ }
+
+ if err := c.Rename("/docs/a.txt", "/b.txt"); err != nil {
+ t.Fatal(err)
+ }
+ if err := c.Remove("/b.txt"); err != nil {
+ t.Fatal(err)
+ }
+ if err := c.RemoveDirectory("/docs"); err != nil {
+ t.Fatal(err)
+ }
+ if list, _ := a.List(); len(list) != 0 {
+ t.Fatalf("area not empty after cleanup: %+v", list)
+ }
+}
+
+func TestConfinement(t *testing.T) {
+ base := t.TempDir()
+ outside := filepath.Join(base, "secret.txt")
+ if err := os.WriteFile(outside, []byte("secret"), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ a, _ := Open(filepath.Join(base, "areas"), "id", 0, nil)
+ c := client(t, a)
+
+ for _, p := range []string{"../../secret.txt", "/../../secret.txt", "..\\..\\secret.txt"} {
+ if f, err := c.Open(p); err == nil {
+ body, _ := io.ReadAll(f)
+ f.Close()
+ if string(body) == "secret" {
+ t.Fatalf("%s escaped the area", p)
+ }
+ }
+ }
+
+ // Paths that climb above the root collapse onto it.
+ if got, err := a.resolve("../../x"); err != nil || got != filepath.Join(a.Root(), "x") {
+ t.Fatalf("resolve(../../x) = %q, %v", got, err)
+ }
+
+ if err := c.Symlink("/etc/passwd", "/link"); err == nil {
+ t.Fatal("symlink creation must be refused")
+ }
+
+ // A link planted on disk (not via SFTP) is still refused.
+ if err := os.Symlink(outside, filepath.Join(a.Root(), "planted")); err == nil {
+ if _, err := c.Open("/planted"); err == nil {
+ t.Fatal("reading through a planted symlink must be refused")
+ }
+ if _, err := a.ReadFile("planted", 100); err == nil {
+ t.Fatal("ReadFile through a planted symlink must be refused")
+ }
+ }
+
+ if err := c.RemoveDirectory("/"); err == nil {
+ t.Fatal("removing the area root must be refused")
+ }
+}
+
+func TestPerAreaQuota(t *testing.T) {
+ a, _ := Open(t.TempDir(), "id", 10, nil)
+ c := client(t, a)
+
+ if err := put(c, "/a", "12345678"); err != nil {
+ t.Fatalf("within quota: %v", err)
+ }
+ if err := put(c, "/b", "12345"); err == nil {
+ t.Fatal("a write past the per-area quota must fail")
+ }
+ // Overwriting a file frees its old size first.
+ if err := put(c, "/a", "1234567890"); err != nil {
+ t.Fatalf("overwrite within quota: %v", err)
+ }
+
+ // truncate() may shrink but never grow.
+ if err := c.Truncate("/a", 4); err != nil {
+ t.Fatalf("shrink: %v", err)
+ }
+ if err := c.Truncate("/a", 1<<20); err == nil {
+ t.Fatal("growing a file with truncate must be refused")
+ }
+}
+
+func TestGlobalBudget(t *testing.T) {
+ a, _ := Open(t.TempDir(), "id", 0, fixedBudget(4))
+ c := client(t, a)
+
+ if err := put(c, "/a", "1234"); err != nil {
+ t.Fatalf("within budget: %v", err)
+ }
+ if err := put(c, "/b", "12345"); err == nil {
+ t.Fatal("a write past the global budget must fail")
+ }
+
+ full, _ := Open(t.TempDir(), "id", 0, fixedBudget(0))
+ if err := put(client(t, full), "/a", "x"); err == nil {
+ t.Fatal("with no budget left, uploads must be refused")
+ }
+}
@@ -0,0 +1,274 @@ +// Package jobs runs slow tool calls in the background so a script can submit
+// one, disconnect, and collect the result later with the same SSH key.
+//
+// Jobs live in memory: a restart forgets them. Each owner (an SSH identity)
+// sees only its own jobs.
+package jobs
+
+import (
+ "context"
+ "crypto/rand"
+ "encoding/hex"
+ "errors"
+ "fmt"
+ "sort"
+ "sync"
+ "time"
+)
+
+// Status is the lifecycle stage of a job.
+type Status string
+
+const (
+ Queued Status = "queued"
+ Running Status = "running"
+ Done Status = "done"
+ Failed Status = "failed"
+)
+
+// ErrBusy is returned when an owner already has MaxPerOwner unfinished jobs.
+var ErrBusy = errors.New("too many unfinished jobs; wait for one to finish")
+
+// Config bounds the manager. Zero values take the defaults.
+type Config struct {
+ Workers int // jobs running at once, across all owners
+ MaxPerOwner int // unfinished jobs one owner may have
+ KeepPerKey int // finished jobs retained per owner
+ TTL time.Duration // how long a finished job's result is kept
+ Timeout time.Duration // upper bound on one job's run time
+}
+
+const (
+ defaultWorkers = 4
+ defaultMaxPerOwner = 5
+ defaultKeep = 50
+ defaultTTL = 20 * time.Minute
+ defaultTimeout = 2 * time.Minute
+)
+
+func (c Config) withDefaults() Config {
+ if c.Workers <= 0 {
+ c.Workers = defaultWorkers
+ }
+ if c.MaxPerOwner <= 0 {
+ c.MaxPerOwner = defaultMaxPerOwner
+ }
+ if c.KeepPerKey <= 0 {
+ c.KeepPerKey = defaultKeep
+ }
+ if c.TTL <= 0 {
+ c.TTL = defaultTTL
+ }
+ if c.Timeout <= 0 {
+ c.Timeout = defaultTimeout
+ }
+
+ return c
+}
+
+// Func is the work a job performs. Its result must be JSON-serialisable.
+type Func func(ctx context.Context) (any, error)
+
+// View is a snapshot of a job, safe to marshal and hand to a client.
+type View struct {
+ ID string `json:"id"`
+ Tool string `json:"tool"`
+ Status Status `json:"status"`
+ Created time.Time `json:"created"`
+ Started *time.Time `json:"started,omitempty"`
+ Finished *time.Time `json:"finished,omitempty"`
+ Result any `json:"result,omitempty"`
+ Error string `json:"error,omitempty"`
+}
+
+type job struct {
+ View
+ owner string
+}
+
+// Manager owns every job and the worker slots they share.
+type Manager struct {
+ cfg Config
+ slots chan struct{}
+ ctx context.Context
+ cancel context.CancelFunc
+ now func() time.Time
+
+ mu sync.Mutex
+ jobs map[string]*job
+}
+
+// New returns a manager ready to accept jobs.
+func New(cfg Config) *Manager {
+ cfg = cfg.withDefaults()
+ ctx, cancel := context.WithCancel(context.Background())
+
+ return &Manager{
+ cfg: cfg,
+ slots: make(chan struct{}, cfg.Workers),
+ ctx: ctx,
+ cancel: cancel,
+ now: time.Now,
+ jobs: make(map[string]*job),
+ }
+}
+
+// Config reports the effective limits.
+func (m *Manager) Config() Config { return m.cfg }
+
+// Submit queues fn for owner and returns immediately.
+func (m *Manager) Submit(owner, tool string, fn Func) (View, error) {
+ id, err := newID()
+ if err != nil {
+ return View{}, err
+ }
+
+ m.mu.Lock()
+ m.pruneLocked()
+ unfinished := 0
+ for _, j := range m.jobs {
+ if j.owner == owner && (j.Status == Queued || j.Status == Running) {
+ unfinished++
+ }
+ }
+ if unfinished >= m.cfg.MaxPerOwner {
+ m.mu.Unlock()
+
+ return View{}, ErrBusy
+ }
+ j := &job{owner: owner, View: View{ID: id, Tool: tool, Status: Queued, Created: m.now()}}
+ m.jobs[id] = j
+ v := j.View
+ m.mu.Unlock()
+
+ go m.run(j, fn)
+
+ return v, nil
+}
+
+func (m *Manager) run(j *job, fn Func) {
+ select {
+ case m.slots <- struct{}{}:
+ case <-m.ctx.Done():
+ m.finish(j, nil, m.ctx.Err())
+
+ return
+ }
+ defer func() { <-m.slots }()
+
+ m.mu.Lock()
+ started := m.now()
+ j.Status, j.Started = Running, &started
+ m.mu.Unlock()
+
+ ctx, cancel := context.WithTimeout(m.ctx, m.cfg.Timeout)
+ defer cancel()
+
+ var (
+ res any
+ err error
+ )
+ func() {
+ defer func() {
+ if r := recover(); r != nil {
+ err = fmt.Errorf("job crashed: %v", r)
+ }
+ }()
+ res, err = fn(ctx)
+ }()
+ if err == nil && ctx.Err() != nil {
+ err = ctx.Err()
+ }
+ if errors.Is(err, context.DeadlineExceeded) {
+ err = fmt.Errorf("timed out after %s", m.cfg.Timeout)
+ }
+ m.finish(j, res, err)
+}
+
+func (m *Manager) finish(j *job, res any, err error) {
+ m.mu.Lock()
+ defer m.mu.Unlock()
+
+ done := m.now()
+ j.Finished = &done
+ if err != nil {
+ j.Status, j.Error = Failed, err.Error()
+
+ return
+ }
+ j.Status, j.Result = Done, res
+}
+
+// Get returns one of owner's jobs. Another owner's job is reported as absent.
+func (m *Manager) Get(owner, id string) (View, bool) {
+ m.mu.Lock()
+ defer m.mu.Unlock()
+
+ m.pruneLocked()
+ j, ok := m.jobs[id]
+ if !ok || j.owner != owner {
+ return View{}, false
+ }
+
+ return j.View, true
+}
+
+// List returns owner's jobs, newest first, without their results.
+func (m *Manager) List(owner string) []View {
+ m.mu.Lock()
+ defer m.mu.Unlock()
+
+ m.pruneLocked()
+ var out []View
+ for _, j := range m.jobs {
+ if j.owner == owner {
+ v := j.View
+ v.Result = nil
+ out = append(out, v)
+ }
+ }
+ sort.Slice(out, func(a, b int) bool { return out[a].Created.After(out[b].Created) })
+
+ return out
+}
+
+// Close cancels running jobs and refuses queued ones.
+func (m *Manager) Close() {
+ m.cancel()
+}
+
+// pruneLocked forgets finished jobs past their TTL, and the oldest finished
+// jobs of any owner holding more than KeepPerKey of them.
+func (m *Manager) pruneLocked() {
+ now := m.now()
+ finished := map[string][]*job{}
+ for id, j := range m.jobs {
+ if j.Finished == nil {
+ continue
+ }
+ if now.Sub(*j.Finished) > m.cfg.TTL {
+ delete(m.jobs, id)
+
+ continue
+ }
+ finished[j.owner] = append(finished[j.owner], j)
+ }
+ for _, list := range finished {
+ if len(list) <= m.cfg.KeepPerKey {
+ continue
+ }
+ sort.Slice(list, func(a, b int) bool { return list[a].Finished.Before(*list[b].Finished) })
+ for _, j := range list[:len(list)-m.cfg.KeepPerKey] {
+ delete(m.jobs, j.ID)
+ }
+ }
+}
+
+func newID() (string, error) {
+ var b [8]byte
+ if _, err := rand.Read(b[:]); err != nil {
+ return "", err
+ }
+
+ return "j_" + hex.EncodeToString(b[:]), nil
+}
@@ -0,0 +1,153 @@ +package jobs
+
+import (
+ "context"
+ "errors"
+ "strings"
+ "testing"
+ "time"
+)
+
+// wait polls until the job leaves the queued/running states.
+func wait(t *testing.T, m *Manager, owner, id string) View {
+ t.Helper()
+ deadline := time.Now().Add(5 * time.Second)
+ for time.Now().Before(deadline) {
+ v, ok := m.Get(owner, id)
+ if !ok {
+ t.Fatalf("job %s vanished", id)
+ }
+ if v.Status == Done || v.Status == Failed {
+ return v
+ }
+ time.Sleep(5 * time.Millisecond)
+ }
+ t.Fatalf("job %s did not finish", id)
+
+ return View{}
+}
+
+func TestSubmitAndCollect(t *testing.T) {
+ m := New(Config{})
+ defer m.Close()
+
+ v, err := m.Submit("alice", "links", func(context.Context) (any, error) { return "result", nil })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if v.Status != Queued || !strings.HasPrefix(v.ID, "j_") {
+ t.Fatalf("submitted view = %+v", v)
+ }
+
+ got := wait(t, m, "alice", v.ID)
+ if got.Status != Done || got.Result != "result" || got.Started == nil || got.Finished == nil {
+ t.Fatalf("finished view = %+v", got)
+ }
+}
+
+func TestFailureAndPanic(t *testing.T) {
+ m := New(Config{})
+ defer m.Close()
+
+ v, _ := m.Submit("a", "x", func(context.Context) (any, error) { return nil, errors.New("nope") })
+ if got := wait(t, m, "a", v.ID); got.Status != Failed || got.Error != "nope" {
+ t.Fatalf("failed job = %+v", got)
+ }
+
+ v, _ = m.Submit("a", "x", func(context.Context) (any, error) { panic("kaboom") })
+ if got := wait(t, m, "a", v.ID); got.Status != Failed || !strings.Contains(got.Error, "kaboom") {
+ t.Fatalf("panicking job = %+v", got)
+ }
+}
+
+func TestTimeout(t *testing.T) {
+ m := New(Config{Timeout: 20 * time.Millisecond})
+ defer m.Close()
+
+ v, _ := m.Submit("a", "slow", func(ctx context.Context) (any, error) {
+ <-ctx.Done()
+
+ return nil, ctx.Err()
+ })
+ if got := wait(t, m, "a", v.ID); got.Status != Failed || !strings.Contains(got.Error, "timed out") {
+ t.Fatalf("timed-out job = %+v", got)
+ }
+}
+
+func TestOwnersAreIsolated(t *testing.T) {
+ m := New(Config{})
+ defer m.Close()
+
+ v, _ := m.Submit("alice", "x", func(context.Context) (any, error) { return 1, nil })
+ wait(t, m, "alice", v.ID)
+
+ if _, ok := m.Get("bob", v.ID); ok {
+ t.Fatal("bob can read alice's job")
+ }
+ if len(m.List("bob")) != 0 {
+ t.Fatal("bob's list shows alice's job")
+ }
+ if l := m.List("alice"); len(l) != 1 || l[0].Result != nil {
+ t.Fatalf("alice's list = %+v (results must be omitted)", l)
+ }
+}
+
+func TestPerOwnerCap(t *testing.T) {
+ m := New(Config{MaxPerOwner: 2, Workers: 1})
+ defer m.Close()
+
+ release := make(chan struct{})
+ block := func(context.Context) (any, error) { <-release; return nil, nil }
+
+ for i := 0; i < 2; i++ {
+ if _, err := m.Submit("a", "x", block); err != nil {
+ t.Fatalf("job %d: %v", i+1, err)
+ }
+ }
+ if _, err := m.Submit("a", "x", block); !errors.Is(err, ErrBusy) {
+ t.Fatalf("third unfinished job: err = %v, want ErrBusy", err)
+ }
+ if _, err := m.Submit("b", "x", block); err != nil {
+ t.Fatalf("another owner must not be blocked: %v", err)
+ }
+ close(release)
+}
+
+func TestPruneByTTLAndCount(t *testing.T) {
+ m := New(Config{TTL: time.Minute, KeepPerKey: 2})
+ defer m.Close()
+
+ var ids []string
+ for i := 0; i < 3; i++ {
+ v, _ := m.Submit("a", "x", func(context.Context) (any, error) { return nil, nil })
+ wait(t, m, "a", v.ID)
+ ids = append(ids, v.ID)
+ time.Sleep(2 * time.Millisecond) // distinct finish times
+ }
+ if n := len(m.List("a")); n != 2 {
+ t.Fatalf("kept %d finished jobs, want 2", n)
+ }
+ if _, ok := m.Get("a", ids[0]); ok {
+ t.Fatal("the oldest finished job should have been dropped")
+ }
+
+ later := time.Now().Add(2 * time.Minute)
+ m.now = func() time.Time { return later }
+ if n := len(m.List("a")); n != 0 {
+ t.Fatalf("%d jobs survived past the TTL", n)
+ }
+}
+
+func TestCloseFailsQueuedJobs(t *testing.T) {
+ m := New(Config{Workers: 1})
+ release := make(chan struct{})
+ m.Submit("a", "x", func(context.Context) (any, error) { <-release; return nil, nil })
+ time.Sleep(20 * time.Millisecond) // let the first job take the only slot
+ v, _ := m.Submit("a", "x", func(context.Context) (any, error) { return nil, nil })
+
+ m.Close()
+ if got := wait(t, m, "a", v.ID); got.Status != Failed {
+ t.Fatalf("queued job after Close = %+v", got)
+ }
+ close(release)
+}
@@ -10,4 +10,9 @@ Type string `yaml:"type"` URL string `yaml:"url"`
Token string `yaml:"token"`
} `yaml:"nominatim"`
+
+ // DailyLookupLimit caps calls to the external geocoders per UTC day,
+ // across every client, so a flood of unseen city names cannot exhaust
+ // the provider's quota. Cached names do not count. 0 = unlimited.
+ DailyLookupLimit int `yaml:"dailyLookupLimit"`
}
@@ -1,6 +1,15 @@ package location
-import "github.com/sirupsen/logrus"
+import (
+ "errors"
+ "sync"
+ "time"
+
+ "github.com/sirupsen/logrus"
+)
+
+// ErrLookupBudgetExhausted is returned once the daily geocoding budget is spent.
+var ErrLookupBudgetExhausted = errors.New("daily geocoding limit reached, try again tomorrow or use a city that was looked up before")
type Provider interface {
Name() string
@@ -9,6 +18,7 @@ }
type Searcher struct {
providers []Provider
+ budget *dailyBudget
}
// NewSearcher returns a new Searcher for the specified config.
@@ -20,6 +30,7 @@ }
return &Searcher{
providers: providers,
+ budget: newDailyBudget(config.DailyLookupLimit),
}
}
@@ -32,6 +43,12 @@ err error result *Location
)
+ if !s.budget.spend() {
+ logrus.Warnln("geocoding: daily lookup limit reached, refusing", location)
+
+ return nil, ErrLookupBudgetExhausted
+ }
+
for _, p := range s.providers {
logrus.Debugln("querying ", p.Name())
result, err = p.Query(location)
@@ -42,3 +59,39 @@ }
return result, err
}
+
+// dailyBudget counts external lookups per UTC day. A nil budget is unlimited.
+type dailyBudget struct {
+ mu sync.Mutex
+ limit int
+ day string
+ used int
+ now func() time.Time
+}
+
+func newDailyBudget(limit int) *dailyBudget {
+ if limit <= 0 {
+ return nil
+ }
+
+ return &dailyBudget{limit: limit, now: time.Now}
+}
+
+func (b *dailyBudget) spend() bool {
+ if b == nil {
+ return true
+ }
+
+ b.mu.Lock()
+ defer b.mu.Unlock()
+
+ if day := b.now().UTC().Format(time.DateOnly); day != b.day {
+ b.day, b.used = day, 0
+ }
+ if b.used >= b.limit {
+ return false
+ }
+ b.used++
+
+ return true
+}
@@ -0,0 +1,53 @@ +package location
+
+import (
+ "errors"
+ "testing"
+ "time"
+)
+
+type countingProvider struct{ calls int }
+
+func (p *countingProvider) Name() string { return "counting" }
+
+func (p *countingProvider) Query(location string) (*Location, error) {
+ p.calls++
+
+ return &Location{Name: location}, nil
+}
+
+func TestSearchDailyBudget(t *testing.T) {
+ prov := &countingProvider{}
+ s := &Searcher{providers: []Provider{prov}, budget: newDailyBudget(2)}
+ day := time.Date(2026, 9, 24, 23, 0, 0, 0, time.UTC)
+ s.budget.now = func() time.Time { return day }
+
+ for i := 0; i < 2; i++ {
+ if _, err := s.Search("somewhere"); err != nil {
+ t.Fatalf("lookup %d within budget: %v", i+1, err)
+ }
+ }
+ if _, err := s.Search("somewhere-else"); !errors.Is(err, ErrLookupBudgetExhausted) {
+ t.Fatalf("lookup past the budget: err = %v", err)
+ }
+ if prov.calls != 2 {
+ t.Fatalf("provider called %d times, want 2", prov.calls)
+ }
+
+ // A new UTC day resets the budget.
+ day = day.Add(2 * time.Hour)
+ if _, err := s.Search("tomorrow"); err != nil {
+ t.Fatalf("budget did not reset on a new day: %v", err)
+ }
+}
+
+func TestSearchUnlimitedByDefault(t *testing.T) {
+ prov := &countingProvider{}
+ s := NewSearcher(&Config{})
+ s.providers = []Provider{prov}
+ for i := 0; i < 50; i++ {
+ if _, err := s.Search("x"); err != nil {
+ t.Fatalf("unlimited searcher refused: %v", err)
+ }
+ }
+}
@@ -0,0 +1,96 @@ +// Package ratelimit is a small per-key token bucket, used to cap how fast one
+// client (an IP, an SSH identity) can spend upstream quota.
+package ratelimit
+
+import (
+ "math"
+ "sync"
+ "time"
+)
+
+// sweepEvery bounds how often idle buckets are dropped, so the map cannot grow
+// without limit under a stream of distinct keys.
+const sweepEvery = time.Minute
+
+type bucket struct {
+ tokens float64
+ last time.Time
+}
+
+// Limiter admits up to burst requests at once per key, refilled at a steady
+// rate. A nil Limiter admits everything.
+type Limiter struct {
+ mu sync.Mutex
+ rate float64 // tokens per second
+ burst float64
+ buckets map[string]*bucket
+ lastSweep time.Time
+ now func() time.Time
+}
+
+// New returns a limiter allowing perMinute requests per key on average with
+// bursts up to burst. perMinute <= 0 disables limiting (nil limiter). A burst
+// below 1 is raised to 1.
+func New(perMinute, burst int) *Limiter {
+ if perMinute <= 0 {
+ return nil
+ }
+ if burst < 1 {
+ burst = 1
+ }
+
+ return &Limiter{
+ rate: float64(perMinute) / 60,
+ burst: float64(burst),
+ buckets: make(map[string]*bucket),
+ now: time.Now,
+ }
+}
+
+// Allow spends one token for key. When the bucket is empty it returns false
+// and how long until the next token is available.
+func (l *Limiter) Allow(key string) (bool, time.Duration) {
+ if l == nil {
+ return true, 0
+ }
+
+ l.mu.Lock()
+ defer l.mu.Unlock()
+
+ now := l.now()
+ l.sweep(now)
+
+ b, ok := l.buckets[key]
+ if !ok {
+ b = &bucket{tokens: l.burst, last: now}
+ l.buckets[key] = b
+ }
+
+ b.tokens = math.Min(l.burst, b.tokens+now.Sub(b.last).Seconds()*l.rate)
+ b.last = now
+
+ if b.tokens < 1 {
+ wait := time.Duration((1 - b.tokens) / l.rate * float64(time.Second))
+
+ return false, wait
+ }
+ b.tokens--
+
+ return true, 0
+}
+
+// sweep drops buckets that have refilled completely: forgetting them loses no
+// state, since a new bucket starts full.
+func (l *Limiter) sweep(now time.Time) {
+ if now.Sub(l.lastSweep) < sweepEvery {
+ return
+ }
+ l.lastSweep = now
+
+ full := time.Duration(l.burst / l.rate * float64(time.Second))
+ for k, b := range l.buckets {
+ if now.Sub(b.last) >= full {
+ delete(l.buckets, k)
+ }
+ }
+}
@@ -0,0 +1,78 @@ +package ratelimit
+
+import (
+ "testing"
+ "time"
+)
+
+func fakeClock(l *Limiter) *time.Time {
+ t := time.Date(2026, 9, 24, 12, 0, 0, 0, time.UTC)
+ l.now = func() time.Time { return t }
+
+ return &t
+}
+
+func TestBurstThenRefuse(t *testing.T) {
+ l := New(60, 3)
+ fakeClock(l)
+
+ for i := 0; i < 3; i++ {
+ if ok, _ := l.Allow("a"); !ok {
+ t.Fatalf("request %d within burst was refused", i+1)
+ }
+ }
+ ok, wait := l.Allow("a")
+ if ok {
+ t.Fatal("request past the burst must be refused")
+ }
+ if wait <= 0 || wait > time.Second {
+ t.Fatalf("retry-after = %v, want (0, 1s] at 60/min", wait)
+ }
+
+ // Another key has its own bucket.
+ if ok, _ := l.Allow("b"); !ok {
+ t.Fatal("a different key must not be affected")
+ }
+}
+
+func TestRefill(t *testing.T) {
+ l := New(60, 1)
+ clock := fakeClock(l)
+
+ if ok, _ := l.Allow("a"); !ok {
+ t.Fatal("first request refused")
+ }
+ if ok, _ := l.Allow("a"); ok {
+ t.Fatal("empty bucket admitted a request")
+ }
+ *clock = clock.Add(time.Second)
+ if ok, _ := l.Allow("a"); !ok {
+ t.Fatal("bucket did not refill after one period")
+ }
+}
+
+func TestSweepDropsIdleBuckets(t *testing.T) {
+ l := New(60, 2)
+ clock := fakeClock(l)
+
+ l.Allow("a")
+ l.Allow("b")
+ *clock = clock.Add(2 * sweepEvery)
+ l.Allow("c")
+
+ if len(l.buckets) != 1 {
+ t.Fatalf("idle buckets were not swept: %d left", len(l.buckets))
+ }
+}
+
+func TestNilAdmitsEverything(t *testing.T) {
+ l := New(0, 5)
+ if l != nil {
+ t.Fatal("perMinute 0 must disable the limiter")
+ }
+ for i := 0; i < 100; i++ {
+ if ok, _ := l.Allow("x"); !ok {
+ t.Fatal("nil limiter refused a request")
+ }
+ }
+}
@@ -1,6 +1,10 @@ package server
-import "fmt"
+import (
+ "fmt"
+
+ "github.com/runawaydevil/hammer/internal/clientip"
+)
// Config holds the server configuration loaded from YAML.
type Config struct {
@@ -16,6 +20,50 @@ // === Modern multi-certificate configuration ===
// Preferred way for multiple domains / wildcards.
TLSCerts []TLSCertConfig `yaml:"tlsCerts,omitempty"`
+
+ // TrustedProxies lists the proxies (IPs or CIDR prefixes) whose
+ // X-Forwarded-For / X-Real-IP headers are believed. Absent = loopback
+ // only, which fits a reverse proxy on the same host. An explicit empty
+ // list trusts nobody.
+ TrustedProxies *[]string `yaml:"trustedProxies,omitempty"`
+
+ // RateLimit caps weather requests per client IP. Absent = defaults.
+ RateLimit *RateLimitConfig `yaml:"rateLimit,omitempty"`
+}
+
+// RateLimitConfig is a per-client-IP token bucket for weather requests.
+type RateLimitConfig struct {
+ // RequestsPerMinute is the sustained rate. 0 disables the limit.
+ RequestsPerMinute int `yaml:"requestsPerMinute"`
+ // Burst is how many requests may arrive at once. Absent or 0 = default.
+ Burst int `yaml:"burst"`
+}
+
+const (
+ defaultRequestsPerMinute = 60
+ defaultBurst = 20
+)
+
+// Proxies returns the trusted proxy list, defaulting to loopback.
+func (c *Config) Proxies() []string {
+ if c.TrustedProxies == nil {
+ return clientip.DefaultTrusted
+ }
+
+ return *c.TrustedProxies
+}
+
+// Limits returns the per-IP rate (requests per minute) and burst.
+func (c *Config) Limits() (perMinute, burst int) {
+ if c.RateLimit == nil {
+ return defaultRequestsPerMinute, defaultBurst
+ }
+ burst = c.RateLimit.Burst
+ if burst <= 0 {
+ burst = defaultBurst
+ }
+
+ return c.RateLimit.RequestsPerMinute, burst
}
// TLSCertConfig defines one certificate + its matching domain(s).
@@ -45,6 +93,13 @@ } if c.TLSKeyFile == "" {
return fmt.Errorf("tlsKeyFile is required when tlsCertFile is set")
}
+ }
+
+ if _, err := clientip.NewResolver(c.Proxies()); err != nil {
+ return fmt.Errorf("trustedProxies: %w", err)
+ }
+ if c.RateLimit != nil && c.RateLimit.RequestsPerMinute < 0 {
+ return fmt.Errorf("rateLimit.requestsPerMinute must be >= 0, got %d", c.RateLimit.RequestsPerMinute)
}
// Validate multiple certs
@@ -7,14 +7,18 @@ "fmt" "io"
"log"
stdlog "log"
+ "math"
"net/http"
"path"
"runtime/debug"
+ "strconv"
"strings"
"time"
"github.com/runawaydevil/hammer/internal/assets"
+ "github.com/runawaydevil/hammer/internal/clientip"
"github.com/runawaydevil/hammer/internal/logging"
+ "github.com/runawaydevil/hammer/internal/ratelimit"
"github.com/runawaydevil/hammer/internal/weather"
)
@@ -228,8 +232,6 @@ }
func Serve(conf *Config, logConf *logging.Config, ws *weather.WeatherService) error {
var (
- mux = http.NewServeMux()
-
logger = logging.NewRequestLogger(
logConf.AccessLog,
time.Duration(logConf.Interval)*time.Second,
@@ -246,14 +248,18 @@ logLineStart, )
)
+ if err := conf.Validate(); err != nil {
+ return err
+ }
+
if err := errorsLog.Open(); err != nil {
return err
}
- // Register handlers with panic recovery
- mux.HandleFunc("/", panicRecovery(mainHandler(ws, logger)))
- mux.HandleFunc("/favicon.ico", panicRecovery(faviconHandler))
- mux.HandleFunc("/files/", panicRecovery(staticFilesHandler))
+ mux, err := newMux(conf, mainHandler(ws, logger))
+ if err != nil {
+ return err
+ }
if conf.PortHTTP != 0 {
go serveHTTP(mux, conf.PortHTTP, errorsLog, errs)
@@ -270,6 +276,48 @@ return ErrNoServersConfigured }
return <-errs // block until one server fails
+}
+
+// newMux wires the routes. Every request gets its client IP resolved once
+// (trusting forwarding headers only from configured proxies); weather requests
+// are then rate limited per client IP. Static assets are not limited.
+func newMux(conf *Config, weatherHandler http.HandlerFunc) (*http.ServeMux, error) {
+ resolver, err := clientip.NewResolver(conf.Proxies())
+ if err != nil {
+ return nil, fmt.Errorf("trustedProxies: %w", err)
+ }
+ limiter := ratelimit.New(conf.Limits())
+
+ withIP := func(next http.HandlerFunc) http.HandlerFunc {
+ return func(w http.ResponseWriter, r *http.Request) {
+ next(w, r.WithContext(clientip.WithIP(r.Context(), resolver.ClientIP(r))))
+ }
+ }
+
+ mux := http.NewServeMux()
+ mux.HandleFunc("/", withIP(panicRecovery(rateLimited(limiter, weatherHandler))))
+ mux.HandleFunc("/favicon.ico", withIP(panicRecovery(faviconHandler)))
+ mux.HandleFunc("/files/", withIP(panicRecovery(staticFilesHandler)))
+
+ return mux, nil
+}
+
+// rateLimited answers 429 with Retry-After once a client IP has spent its
+// burst, protecting the geocoding and weather upstream quotas.
+func rateLimited(l *ratelimit.Limiter, next http.HandlerFunc) http.HandlerFunc {
+ return func(w http.ResponseWriter, r *http.Request) {
+ if ok, wait := l.Allow(clientip.FromRequest(r)); !ok {
+ secs := int(math.Ceil(wait.Seconds()))
+ if secs < 1 {
+ secs = 1
+ }
+ w.Header().Set("Retry-After", strconv.Itoa(secs))
+ http.Error(w, fmt.Sprintf("Too many requests. Try again in %d s.", secs), http.StatusTooManyRequests)
+
+ return
+ }
+ next(w, r)
+ }
}
func mainHandler(ws *weather.WeatherService, logger *logging.RequestLogger) func(http.ResponseWriter, *http.Request) {
@@ -0,0 +1,170 @@ +package server
+
+import (
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "testing"
+
+ "github.com/runawaydevil/hammer/internal/clientip"
+)
+
+// recordingWeather stands in for the weather service and remembers the client
+// IP each request was resolved to.
+type recordingWeather struct{ ips []string }
+
+func (rw *recordingWeather) handle(w http.ResponseWriter, r *http.Request) {
+ rw.ips = append(rw.ips, clientip.FromRequest(r))
+ _, _ = w.Write([]byte("ok"))
+}
+
+func do(t *testing.T, h http.Handler, remote, path string, headers map[string]string) *httptest.ResponseRecorder {
+ t.Helper()
+ req := httptest.NewRequest("GET", path, nil)
+ req.RemoteAddr = remote
+ for k, v := range headers {
+ req.Header.Set(k, v)
+ }
+ rec := httptest.NewRecorder()
+ h.ServeHTTP(rec, req)
+
+ return rec
+}
+
+func TestClientIPOnlyFromTrustedProxy(t *testing.T) {
+ rw := &recordingWeather{}
+ mux, err := newMux(&Config{PortHTTP: 8080}, rw.handle)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ xff := map[string]string{"X-Forwarded-For": "198.51.100.4"}
+ do(t, mux, "203.0.113.9:1000", "/Paris", xff) // forged by a direct client
+ do(t, mux, "127.0.0.1:1000", "/Paris", xff) // appended by the local proxy
+
+ want := []string{"203.0.113.9", "198.51.100.4"}
+ for i, ip := range want {
+ if rw.ips[i] != ip {
+ t.Errorf("request %d resolved to %q, want %q", i, rw.ips[i], ip)
+ }
+ }
+}
+
+func TestTrustedProxiesConfigurable(t *testing.T) {
+ rw := &recordingWeather{}
+ none := []string{}
+ mux, err := newMux(&Config{PortHTTP: 8080, TrustedProxies: &none}, rw.handle)
+ if err != nil {
+ t.Fatal(err)
+ }
+ do(t, mux, "127.0.0.1:1000", "/", map[string]string{"X-Forwarded-For": "198.51.100.4"})
+ if rw.ips[0] != "127.0.0.1" {
+ t.Fatalf("with no trusted proxies the header must be ignored, got %q", rw.ips[0])
+ }
+
+ bad := []string{"nope"}
+ if _, err := newMux(&Config{TrustedProxies: &bad}, rw.handle); err == nil {
+ t.Fatal("an invalid proxy entry must fail at startup")
+ }
+}
+
+func TestRateLimitPerClientIP(t *testing.T) {
+ rw := &recordingWeather{}
+ mux, err := newMux(&Config{PortHTTP: 8080, RateLimit: &RateLimitConfig{RequestsPerMinute: 6, Burst: 2}}, rw.handle)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ for i := 0; i < 2; i++ {
+ if rec := do(t, mux, "203.0.113.9:1", "/City"+string(rune('A'+i)), nil); rec.Code != http.StatusOK {
+ t.Fatalf("request %d within burst: status %d", i+1, rec.Code)
+ }
+ }
+
+ rec := do(t, mux, "203.0.113.9:1", "/CityZ", nil)
+ if rec.Code != http.StatusTooManyRequests {
+ t.Fatalf("request past the burst: status %d, want 429", rec.Code)
+ }
+ if ra := rec.Header().Get("Retry-After"); ra == "" || ra == "0" {
+ t.Errorf("Retry-After = %q, want a positive number of seconds", ra)
+ }
+ if len(rw.ips) != 2 {
+ t.Errorf("the weather handler ran %d times, want 2", len(rw.ips))
+ }
+
+ // Another client is unaffected, and so are static assets.
+ if rec := do(t, mux, "198.51.100.4:1", "/City", nil); rec.Code != http.StatusOK {
+ t.Errorf("other client: status %d", rec.Code)
+ }
+ if rec := do(t, mux, "203.0.113.9:1", "/favicon.ico", nil); rec.Code == http.StatusTooManyRequests {
+ t.Error("static assets must not be rate limited")
+ }
+
+ // Behind the proxy the limit follows the real client, not the proxy.
+ for i := 0; i < 2; i++ {
+ do(t, mux, "127.0.0.1:1", "/", map[string]string{"X-Forwarded-For": "192.0.2.1"})
+ }
+ if rec := do(t, mux, "127.0.0.1:1", "/", map[string]string{"X-Forwarded-For": "192.0.2.2"}); rec.Code != http.StatusOK {
+ t.Errorf("a second client behind the proxy was limited by the first: %d", rec.Code)
+ }
+}
+
+func TestRateLimitDisabled(t *testing.T) {
+ rw := &recordingWeather{}
+ mux, err := newMux(&Config{PortHTTP: 8080, RateLimit: &RateLimitConfig{RequestsPerMinute: 0}}, rw.handle)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for i := 0; i < 100; i++ {
+ if rec := do(t, mux, "203.0.113.9:1", "/", nil); rec.Code != http.StatusOK {
+ t.Fatalf("requestsPerMinute 0 must disable the limit, got %d", rec.Code)
+ }
+ }
+}
+
+func TestConfigDefaultsAndValidate(t *testing.T) {
+ c := &Config{PortHTTP: 8080}
+ if pm, b := c.Limits(); pm != defaultRequestsPerMinute || b != defaultBurst {
+ t.Errorf("defaults = %d/%d", pm, b)
+ }
+ if got := c.Proxies(); len(got) != len(clientip.DefaultTrusted) {
+ t.Errorf("default proxies = %v", got)
+ }
+ if err := c.Validate(); err != nil {
+ t.Errorf("valid config rejected: %v", err)
+ }
+
+ c.RateLimit = &RateLimitConfig{RequestsPerMinute: -1}
+ if err := c.Validate(); err == nil {
+ t.Error("negative rate must be rejected")
+ }
+
+ if err := (&Config{}).Validate(); err != ErrNoServersConfigured {
+ t.Errorf("no ports: %v", err)
+ }
+}
+
+func TestStaticFilesRejectTraversal(t *testing.T) {
+ mux, err := newMux(&Config{PortHTTP: 8080}, (&recordingWeather{}).handle)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, p := range []string{"/files/", "/files/../../etc/passwd", "/files/..%2f..%2fetc%2fpasswd", "/files/nope.css"} {
+ rec := do(t, mux, "203.0.113.9:1", p, nil)
+ if rec.Code == http.StatusOK {
+ t.Errorf("%s: status 200, want an error", p)
+ }
+ if strings.Contains(rec.Body.String(), "root:") {
+ t.Errorf("%s leaked a host file", p)
+ }
+ }
+}
+
+func TestPanicRecovery(t *testing.T) {
+ h := panicRecovery(func(http.ResponseWriter, *http.Request) { panic("boom") })
+ rec := httptest.NewRecorder()
+ h(rec, httptest.NewRequest("GET", "/", nil))
+ if rec.Code != http.StatusInternalServerError {
+ t.Fatalf("status %d, want 500", rec.Code)
+ }
+}
@@ -1,288 +0,0 @@ -// Package ocrpdf runs OCRmyPDF on files dropped in a confined inbox.
-package ocrpdf
-
-import (
- "bytes"
- "context"
- "crypto/rand"
- "encoding/hex"
- "errors"
- "fmt"
- "io"
- "os"
- "path/filepath"
- "strings"
- "time"
-
- "github.com/runawaydevil/hammer/internal/tools"
- "github.com/runawaydevil/hammer/internal/tools/run"
-)
-
-const (
- maxSize = 20 << 20
- minAge = 2 * time.Second
- defaultTTL = 20 * time.Minute
- ocrTimeout = 3 * time.Minute
- pdfMagic = "%PDF-"
-)
-
-// File is one inbox PDF ready to process.
-type File struct {
- Name string
- Size int64
- Age time.Duration
-}
-
-// Result is the work-dir output of a successful OCR run.
-type Result struct {
- JobID string
- OutPDF string
-}
-
-// ListInbox returns PDFs under inbox that look complete and valid.
-func ListInbox(inbox string, now time.Time) ([]File, error) {
- if inbox == "" {
- return nil, errors.New("inbox not configured")
- }
- ents, err := os.ReadDir(inbox)
- if err != nil {
- if os.IsNotExist(err) {
- return nil, nil
- }
-
- return nil, err
- }
-
- var out []File
- for _, e := range ents {
- if e.IsDir() {
- continue
- }
- name := e.Name()
- if !strings.EqualFold(filepath.Ext(name), ".pdf") {
- continue
- }
- path, err := tools.Confine(inbox, name)
- if err != nil {
- continue
- }
- st, err := os.Stat(path)
- if err != nil || st.IsDir() {
- continue
- }
- age := now.Sub(st.ModTime())
- if age < minAge {
- continue
- }
- if st.Size() == 0 || st.Size() > maxSize {
- continue
- }
- if !isPDF(path) {
- continue
- }
- out = append(out, File{Name: name, Size: st.Size(), Age: age})
- }
-
- return out, nil
-}
-
-// Run copies name from inbox into a new job dir and runs ocrmypdf.
-func Run(ctx context.Context, bin, inbox, work, name string) (Result, error) {
- if bin == "" {
- return Result{}, errors.New("ocrmypdf not available")
- }
- src, err := tools.Confine(inbox, name)
- if err != nil {
- return Result{}, err
- }
- st, err := os.Stat(src)
- if err != nil {
- return Result{}, err
- }
- if st.Size() == 0 || st.Size() > maxSize {
- return Result{}, errors.New("file too large or empty")
- }
- if !isPDF(src) {
- return Result{}, errors.New("not a PDF")
- }
-
- id, err := jobID()
- if err != nil {
- return Result{}, err
- }
- // run.Command sets cwd to TempDir; ocrmypdf needs absolute in/out paths.
- work, err = filepath.Abs(work)
- if err != nil {
- return Result{}, err
- }
- dir := filepath.Join(work, id)
- if err := os.MkdirAll(dir, 0o700); err != nil {
- return Result{}, err
- }
- inPath := filepath.Join(dir, "in.pdf")
- outPath := filepath.Join(dir, "out.pdf")
- if err := copyFile(src, inPath); err != nil {
- _ = os.RemoveAll(dir)
-
- return Result{}, err
- }
-
- if _, err := tools.Confine(work, filepath.Join(id, "in.pdf")); err != nil {
- _ = os.RemoveAll(dir)
-
- return Result{}, err
- }
-
- if _, hasDeadline := ctx.Deadline(); !hasDeadline {
- var cancel context.CancelFunc
- ctx, cancel = context.WithTimeout(ctx, ocrTimeout)
- defer cancel()
- }
-
- res, err := run.Command(ctx, bin,
- "--quiet",
- "--jobs", "1",
- "--language", "por+eng",
- "--optimize", "0",
- "--skip-text",
- "--max-image-mpixels", "25",
- inPath,
- outPath,
- )
- if err != nil {
- return Result{}, err
- }
- if res.Code != 0 {
- msg := strings.TrimSpace(string(res.Stderr))
- if msg == "" {
- msg = strings.TrimSpace(string(res.Stdout))
- }
- if msg == "" {
- msg = fmt.Sprintf("ocrmypdf exit %d", res.Code)
- }
-
- return Result{}, errors.New(msg)
- }
- if _, err := os.Stat(outPath); err != nil {
- return Result{}, errors.New("ocrmypdf produced no output")
- }
-
- return Result{JobID: id, OutPDF: outPath}, nil
-}
-
-// Sweep deletes inbox files and OCR job dirs older than ttl.
-func Sweep(inbox, work string, ttl time.Duration, now time.Time) error {
- if ttl <= 0 {
- ttl = defaultTTL
- }
- var first error
- if inbox != "" {
- if err := sweepFiles(inbox, ttl, now); err != nil && first == nil {
- first = err
- }
- }
- if work != "" {
- if err := sweepDirs(work, ttl, now); err != nil && first == nil {
- first = err
- }
- }
-
- return first
-}
-
-func sweepFiles(dir string, ttl time.Duration, now time.Time) error {
- ents, err := os.ReadDir(dir)
- if err != nil {
- if os.IsNotExist(err) {
- return nil
- }
-
- return err
- }
- for _, e := range ents {
- if e.IsDir() {
- continue
- }
- path := filepath.Join(dir, e.Name())
- st, err := os.Stat(path)
- if err != nil {
- continue
- }
- if now.Sub(st.ModTime()) < ttl {
- continue
- }
- _ = os.Remove(path)
- }
-
- return nil
-}
-
-func sweepDirs(dir string, ttl time.Duration, now time.Time) error {
- ents, err := os.ReadDir(dir)
- if err != nil {
- if os.IsNotExist(err) {
- return nil
- }
-
- return err
- }
- for _, e := range ents {
- if !e.IsDir() {
- continue
- }
- path := filepath.Join(dir, e.Name())
- st, err := os.Stat(path)
- if err != nil {
- continue
- }
- if now.Sub(st.ModTime()) < ttl {
- continue
- }
- _ = os.RemoveAll(path)
- }
-
- return nil
-}
-
-func isPDF(path string) bool {
- f, err := os.Open(path)
- if err != nil {
- return false
- }
- defer f.Close()
- var hdr [8]byte
- n, err := io.ReadFull(f, hdr[:])
- if err != nil && n < len(pdfMagic) {
- return false
- }
-
- return bytes.HasPrefix(hdr[:n], []byte(pdfMagic))
-}
-
-func copyFile(src, dst string) error {
- in, err := os.Open(src)
- if err != nil {
- return err
- }
- defer in.Close()
- out, err := os.OpenFile(dst, os.O_CREATE|os.O_WRONLY|os.O_EXCL, 0o600)
- if err != nil {
- return err
- }
- _, err = io.Copy(out, in)
- cerr := out.Close()
- if err != nil {
- return err
- }
-
- return cerr
-}
-
-func jobID() (string, error) {
- var b [8]byte
- if _, err := rand.Read(b[:]); err != nil {
- return "", err
- }
-
- return hex.EncodeToString(b[:]), nil
-}
@@ -1,110 +0,0 @@ -package ocrpdf
-
-import (
- "os"
- "os/exec"
- "path/filepath"
- "testing"
- "time"
-)
-
-func tinyPDF() []byte {
- return []byte("%PDF-1.1\n1 0 obj<<>>endobj\ntrailer<<>>\n%%EOF\n")
-}
-
-func TestListInboxSkipsFreshAndNonPDF(t *testing.T) {
- dir := t.TempDir()
- pdf := filepath.Join(dir, "ok.pdf")
- if err := os.WriteFile(pdf, tinyPDF(), 0o600); err != nil {
- t.Fatal(err)
- }
- old := time.Now().Add(-time.Minute)
- if err := os.Chtimes(pdf, old, old); err != nil {
- t.Fatal(err)
- }
- if err := os.WriteFile(filepath.Join(dir, "nope.txt"), tinyPDF(), 0o600); err != nil {
- t.Fatal(err)
- }
- if err := os.WriteFile(filepath.Join(dir, "fake.pdf"), []byte("not a pdf"), 0o600); err != nil {
- t.Fatal(err)
- }
- _ = os.Chtimes(filepath.Join(dir, "fake.pdf"), old, old)
-
- got, err := ListInbox(dir, time.Now())
- if err != nil {
- t.Fatal(err)
- }
- if len(got) != 1 || got[0].Name != "ok.pdf" {
- t.Fatalf("%+v", got)
- }
-}
-
-func TestListInboxRejectsEscape(t *testing.T) {
- dir := t.TempDir()
- got, err := ListInbox(dir, time.Now())
- if err != nil {
- t.Fatal(err)
- }
- if len(got) != 0 {
- t.Fatalf("%+v", got)
- }
-}
-
-func TestSweepRemovesOld(t *testing.T) {
- inbox := t.TempDir()
- work := t.TempDir()
- oldPDF := filepath.Join(inbox, "old.pdf")
- newPDF := filepath.Join(inbox, "new.pdf")
- if err := os.WriteFile(oldPDF, tinyPDF(), 0o600); err != nil {
- t.Fatal(err)
- }
- if err := os.WriteFile(newPDF, tinyPDF(), 0o600); err != nil {
- t.Fatal(err)
- }
- oldJob := filepath.Join(work, "abc")
- if err := os.Mkdir(oldJob, 0o700); err != nil {
- t.Fatal(err)
- }
- if err := os.WriteFile(filepath.Join(oldJob, "out.pdf"), tinyPDF(), 0o600); err != nil {
- t.Fatal(err)
- }
-
- now := time.Now()
- old := now.Add(-30 * time.Minute)
- if err := os.Chtimes(oldPDF, old, old); err != nil {
- t.Fatal(err)
- }
- if err := os.Chtimes(oldJob, old, old); err != nil {
- t.Fatal(err)
- }
-
- if err := Sweep(inbox, work, 20*time.Minute, now); err != nil {
- t.Fatal(err)
- }
- if _, err := os.Stat(oldPDF); !os.IsNotExist(err) {
- t.Fatal("old inbox file should be gone")
- }
- if _, err := os.Stat(newPDF); err != nil {
- t.Fatal("fresh inbox file should remain")
- }
- if _, err := os.Stat(oldJob); !os.IsNotExist(err) {
- t.Fatal("old job dir should be gone")
- }
-}
-
-func TestRunIfOcrmypdfPresent(t *testing.T) {
- bin, err := exec.LookPath("ocrmypdf")
- if err != nil {
- t.Skip("ocrmypdf not on PATH")
- }
- inbox := t.TempDir()
- work := t.TempDir()
- src := filepath.Join(inbox, "scan.pdf")
- if err := os.WriteFile(src, tinyPDF(), 0o600); err != nil {
- t.Fatal(err)
- }
- _, err = Run(t.Context(), bin, inbox, work, "scan.pdf")
- if err == nil {
- t.Fatal("tiny empty PDF should fail OCR")
- }
-}
@@ -0,0 +1,32 @@ +package weather
+
+import (
+ "net/http/httptest"
+ "strings"
+ "testing"
+
+ "github.com/runawaydevil/hammer/internal/clientip"
+)
+
+// The weather service must never read forwarding headers itself: only the
+// HTTP server, which knows the trusted proxies, may resolve them.
+func TestGetClientIPIgnoresForgedHeaders(t *testing.T) {
+ req := httptest.NewRequest("GET", "/Paris", nil)
+ req.RemoteAddr = "203.0.113.9:5555"
+ req.Header.Set("X-Forwarded-For", "8.8.8.8")
+ req.Header.Set("X-Real-IP", "8.8.4.4")
+
+ if got := getClientIP(req); got != "203.0.113.9" {
+ t.Fatalf("getClientIP = %q, want the peer address", got)
+ }
+
+ req = req.WithContext(clientip.WithIP(req.Context(), "198.51.100.4"))
+ if got := getClientIP(req); got != "198.51.100.4" {
+ t.Fatalf("getClientIP = %q, want the address resolved by the server", got)
+ }
+
+ // The response cache key follows the resolved address too.
+ if key := buildCacheKey(req); !strings.Contains(key, "198.51.100.4") {
+ t.Fatalf("cache key %q does not use the resolved client IP", key)
+ }
+}
@@ -6,12 +6,12 @@ "encoding/json" "errors"
"fmt"
"log"
- "net"
"net/http"
"os"
"strings"
"time"
+ "github.com/runawaydevil/hammer/internal/clientip"
"github.com/runawaydevil/hammer/internal/domain"
"github.com/runawaydevil/hammer/internal/localization"
"github.com/runawaydevil/hammer/internal/options"
@@ -137,31 +137,11 @@ Localizer: localizer, }
}
-// Helper to get the "best guess" client IP — handles proxies/load balancers
+// getClientIP returns the client address resolved by the HTTP server, which
+// honours forwarding headers only from trusted proxies (see package clientip).
+// Called outside that server, it falls back to the direct peer.
func getClientIP(r *http.Request) string {
- // Order of preference: X-Forwarded-For (first non-proxy IP), X-Real-IP, RemoteAddr
- if forwarded := r.Header.Get("X-Forwarded-For"); forwarded != "" {
- // X-Forwarded-For can be "client-ip, proxy1, proxy2, ..."
- // Take the leftmost (original client) — but in production you should trust only known proxies
- parts := strings.Split(forwarded, ",")
- if len(parts) > 0 {
- ip := strings.TrimSpace(parts[0])
- if ip != "" {
- return ip
- }
- }
- }
-
- if realIP := r.Header.Get("X-Real-IP"); realIP != "" {
- return strings.TrimSpace(realIP)
- }
-
- // Fallback to direct connection (may be proxy/load-balancer IP)
- host, _, err := net.SplitHostPort(r.RemoteAddr)
- if err == nil {
- return host
- }
- return r.RemoteAddr // worst case
+ return clientip.FromRequest(r)
}
// WeatherHandler handles weather requests with proper cache coalescing.
@@ -58,6 +58,3 @@ maxPasteBytes: 65536 netTimeout: 12s
mtrCycles: 5
pingCount: 5
- ocrInbox: cache/inbox
- ocrWorkDir: cache/ocr
- ocrTTL: 20m
@@ -168,12 +168,6 @@ "BBS_B64_PROMPT": "b64> ", "BBS_PASTE_B64": "paste Base64",
"BBS_IDS": "ids",
"BBS_IDS_SUM": "UUID v4",
- "BBS_OCR": "ocr",
- "BBS_OCR_SUM": "searchable PDF",
- "BBS_OCR_PROMPT": "ocr> ",
- "BBS_NO_OCR": "ocrmypdf is not installed on this host",
- "BBS_NO_INBOX": "no PDFs in %s — copy a file there, wait 2s, try again",
- "BBS_OCR_RUNNING": "OCR %s...",
- "BBS_OCR_DONE": "searchable PDF (copy it out before it expires)",
- "BBS_OCR_TTL": "deleted after %d minutes"
+ "BBS_ANON_NOTE": "anonymous session — settings last until you disconnect; connect with an SSH key to keep them",
+ "BBS_ANONYMOUS": "anonymous (no key)"
}
@@ -168,12 +168,6 @@ "BBS_B64_PROMPT": "b64> ", "BBS_PASTE_B64": "cole o Base64",
"BBS_IDS": "ids",
"BBS_IDS_SUM": "UUID v4",
- "BBS_OCR": "ocr",
- "BBS_OCR_SUM": "PDF pesquisável",
- "BBS_OCR_PROMPT": "ocr> ",
- "BBS_NO_OCR": "ocrmypdf não está instalado neste host",
- "BBS_NO_INBOX": "nenhum PDF em %s — copie um ficheiro, espere 2s, tente de novo",
- "BBS_OCR_RUNNING": "OCR %s...",
- "BBS_OCR_DONE": "PDF pesquisável (copie-o antes de expirar)",
- "BBS_OCR_TTL": "apagado após %d minutos"
+ "BBS_ANON_NOTE": "sessão anônima — as configurações valem até você desconectar; conecte com uma chave SSH para guardá-las",
+ "BBS_ANONYMOUS": "anônimo (sem chave)"
}
