M21 design: anonymous egress, payload collection and static analysis
Status: proposal for review. Scope: docs/ROADMAP.md, section M21.
Every address in this document is a placeholder or comes from the documentation ranges
(192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24), except well-known public anycast resolvers used
as probe targets (1.1.1.1, 9.9.9.9).
1. Goal and invariants
Section titled “1. Goal and invariants”The owner’s home public IP must never be the source of an outbound connection made by the stack.
Today ingest sits on the egress network and talks to about fifteen external hosts directly. Its
DNS queries are forwarded by Docker’s embedded resolver to the host’s resolver, so they leave from
the home IP too.
After M21:
| # | Invariant | Enforced by |
|---|---|---|
| I1 | No container except vpn is attached to a network that has a route to the internet, apart from the two inbound-only networks that exist today (sp-ingress for sensors, sp-dashboard for api). | Compose layout + static test (§4.1) |
| I2 | vpn forwards traffic only through the WireGuard tunnel. With the tunnel down, nothing leaves. | gluetun firewall (kill switch) + host nftables rule (§4.4) |
| I3 | ingest and collector reach the internet only through the vpn HTTP proxy. A client that ignores the proxy fails, because there is no route and no external DNS. | Internal networks only + dns: 127.0.0.1 + static code test (§3.3) |
| I4 | No egress work starts unless the egress self-check passed in the last EGRESS_CHECK_INTERVAL_S. | EgressGate in the shared suckerpot_egress package (§3.4) |
| I5 | The collector never fetches a private, reserved, documentation, loopback, link-local or own address, and never follows a redirect to one. | URL validator (§5.3) + gluetun DNS rebinding protection + host nftables rule |
| I6 | Fetched files are never executed, never served raw, and are parsed only in a resource-limited child process with no network. | analyzer service on an internal network only + child process limits (§6.4) |
Unchanged: sensors answer attackers on the home IP (inbound only, sp-ingress has no NEW
outbound, see scripts/host/firewall.nft); the dashboard path (cloudflared to api on
sp-dashboard) is unchanged; host traffic (apt, image pulls, git) is out of scope.
2. The vpn service
Section titled “2. The vpn service”2.1 Image choice
Section titled “2.1 Image choice”qmcgaw/gluetun:v3.41.1@sha256:1a5bf4b4820a879cdf8d93d7ef0d2d963af56670c9ebff8981860b6804ebc8ab
(index digest checked with docker pull on 2026-10-31; Alpine based).
Why gluetun rather than a hand-made wireguard-go + iptables image:
- Userspace WireGuard (
WIREGUARD_IMPLEMENTATION=userspace, wireguard-go): the kernel module is not loaded on the VM and we do not want to ask the owner to load modules. - Kill switch built in: its firewall (iptables, default DROP) allows only the tunnel interface and UDP to the endpoint on the uplink. The firewall is up before the tunnel.
- DNS server built in, upstream through the tunnel, with DNS rebinding protection (answers in private ranges are dropped). That protection is one of our SSRF layers (§5.3).
- HTTP proxy built in with stealth mode (no
Via, noX-Forwarded-For) and basic auth. - Control server with per-route roles, so ingest and the collector can read tunnel status and public IP without being able to stop the tunnel.
- It is maintained and pinnable. A home-made image would need the same pieces and more tests.
Rejected: kernel WireGuard (needs the module, SYS_MODULE or host change), network_mode: service:vpn for ingest (it would put ingest’s DB and Redis traffic inside the VPN container’s
namespace and break the “vpn is on no backend network” property), Tor (slow, many APIs block it).
2.2 Environment mapping
Section titled “2.2 Environment mapping”.env holds VPN_WG_PRIVATE_KEY, VPN_WG_ADDRESSES, VPN_WG_DNS, VPN_WG_PEER_PUBLIC_KEY,
VPN_WG_ENDPOINT (ip:port). Compose cannot split a string, and gluetun wants the endpoint IP
and port separately, so the owner adds two derived variables, VPN_WG_ENDPOINT_IP and
VPN_WG_ENDPOINT_PORT. scripts/check-vpn-env.sh (run by just config and just up, next to
check-ports.sh) fails when they are missing, when VPN_WG_ENDPOINT is set and differs from
IP:PORT, when the endpoint IP is private, or when EGRESS_PROXY_PASSWORD is shorter than 24
characters. It prints variable names only, never values.
| gluetun variable | Value | Why |
|---|---|---|
VPN_SERVICE_PROVIDER | custom | Plain WireGuard peer |
VPN_TYPE | wireguard | |
VPN_INTERFACE | wg0 | Fixed name, used by just egress-check |
WIREGUARD_IMPLEMENTATION | userspace | No kernel module |
WIREGUARD_PRIVATE_KEY | ${VPN_WG_PRIVATE_KEY:-} | Empty makes gluetun exit: fail closed |
WIREGUARD_PUBLIC_KEY | ${VPN_WG_PEER_PUBLIC_KEY:-} | |
WIREGUARD_ADDRESSES | ${VPN_WG_ADDRESSES:-} | Comma-separated v4/32 and v6/128 |
WIREGUARD_ENDPOINT_IP / _PORT | ${VPN_WG_ENDPOINT_IP:-} / ${VPN_WG_ENDPOINT_PORT:-} | |
WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL | 25s | Keeps the NAT mapping on the ISP box |
WIREGUARD_ALLOWED_IPS | unset (all) | Everything goes into the tunnel |
| DNS upstream | plain DNS to ${VPN_WG_DNS} (inside the tunnel) | Provider’s resolver, no third party. Exact variable names to confirm against the v3.41 wiki (DNS_UPSTREAM_RESOLVER_TYPE=plain plus the plain-address variable). Fallback if not supported: keep the default DoT to Cloudflare, which also goes through the tunnel. |
DNS_UPSTREAM_IPV6 | off | |
BLOCK_MALICIOUS, BLOCK_SURVEILLANCE, BLOCK_ADS | off | Must be off: the malicious-host blocklist would stop the collector from resolving the very hosts it must fetch, and its update is extra traffic |
DNS_KEEP_NAMESERVER | off | Never fall back to Docker’s resolver |
HTTPPROXY | on | |
HTTPPROXY_LISTENING_ADDRESS | :8888 | |
HTTPPROXY_STEALTH | on | No Via / X-Forwarded-For |
HTTPPROXY_LOG | off | Do not log fetched URLs in Docker logs |
HTTPPROXY_USER / HTTPPROXY_PASSWORD | egress / ${EGRESS_PROXY_PASSWORD:-} | Only ingest and collector know it |
SHADOWSOCKS | off | |
HTTP_CONTROL_SERVER_ADDRESS | :8000 | |
HTTP_CONTROL_SERVER_LOG | off | |
HTTP_CONTROL_SERVER_AUTH_CONFIG_FILEPATH | /gluetun/auth/config.toml | Read-only bind of vpn/control-auth.toml |
HTTP_CONTROL_SERVER_AUTH_DEFAULT_ROLE | {} | Deny any route not listed |
PUBLICIP_ENABLED | on | Gluetun learns its exit IP through the tunnel; read by the self-check |
HEALTH_SMALL_CHECK_TYPE | dns | ICMP would need NET_RAW |
HEALTH_RESTART_VPN | on | |
VERSION_INFORMATION | off | No GitHub call at start |
UPDATER_PERIOD | 0 | |
FIREWALL_OUTBOUND_SUBNETS | empty, must stay empty | Would open LAN routes outside the tunnel |
FIREWALL_INPUT_PORTS | 8888,8000 | Only if gluetun does not already accept them from attached subnets (verify) |
TZ | UTC |
vpn/control-auth.toml (committed, contains no secret):
[[roles]]name = "egress-readers"routes = ["GET /v1/vpn/status", "GET /v1/publicip/ip"]auth = "none"No role can PUT /v1/vpn/status, so a compromised ingest can read the tunnel state but not stop
or reconfigure it. (Stopping it would only close the gate anyway.)
2.3 Compose block (contract for task A)
Section titled “2.3 Compose block (contract for task A)”vpn: image: qmcgaw/gluetun:v3.41.1@sha256:1a5bf4b4820a879cdf8d93d7ef0d2d963af56670c9ebff8981860b6804ebc8ab profiles: [egress] read_only: true cap_drop: [ALL] cap_add: [NET_ADMIN] devices: ["/dev/net/tun:/dev/net/tun"] security_opt: ["no-new-privileges:true"] sysctls: { net.ipv4.conf.all.src_valid_mark: "1" } tmpfs: - /tmp:rw,noexec,nosuid,nodev,size=8m - /gluetun:rw,noexec,nosuid,nodev,size=8m volumes: ["./vpn/control-auth.toml:/gluetun/auth/config.toml:ro"] networks: vpn-uplink: { gw_priority: 100 } egress-proxy: {} mem_limit: 128m memswap_limit: 128m cpus: 0.5 pids_limit: 64 restart: unless-stopped healthcheck: test: ["CMD", "/gluetun-entrypoint", "healthcheck"] interval: 15s timeout: 5s retries: 3 start_period: 30s # no ports: nothing is publishedgluetun runs as root inside the container (it programs iptables); with cap_drop: [ALL] plus
NET_ADMIN that root has no other capability. If it refuses to start with NET_ADMIN alone, the
only acceptable addition is NET_RAW, and it needs the owner’s approval (he approved NET_ADMIN
and /dev/net/tun only). If read_only breaks it, the implementer reports the exact path it
needs and adds a tmpfs for that path; read_only: false is not an option without review.
Profile egress: the owner’s .env sets COMPOSE_PROFILES=egress. A fresh clone without a VPN
runs without vpn, collector and analyzer; ingest then keeps its gate closed and every
external feature reports “no egress”. The stack is still healthy, and CI needs no VPN.
2.4 Networks
Section titled “2.4 Networks”| Network | Docker name | internal | Members | Notes |
|---|---|---|---|---|
vpn-uplink | sp-vpn (fixed bridge name) | no | vpn only | enable_icc: false, masquerading on (needed to reach the endpoint), no IPv6 |
egress-proxy | (project prefix) | yes | vpn, ingest, collector | The proxy and control server live here |
samples | (project prefix) | yes | db, collector, analyzer | Keeps collector and analyzer away from Redis, ingest and api |
egress | Removed |
ingest: [backend, egress-proxy]. collector: [samples, egress-proxy]. analyzer:
[samples]. api gains no network; it mounts the quarantine read-only (§7.3).
vpn is never on backend or samples: through the proxy one can reach whatever shares a
network with vpn, so it must share networks only with its two clients.
2.5 IPv6
Section titled “2.5 IPv6”The Docker networks have no IPv6. gluetun is expected to drop the IPv6 /128 address and the ::/0
allowed range when the container has no IPv6 support (it logs this). Either way no IPv6 packet can
leave: the container has no IPv6 route, and the host rule (§4.4) drops everything from sp-vpn
except UDP to the IPv4 endpoint. Acceptance check: logs show the IPv6 address skipped, or
ip -6 route inside vpn lists only wg0.
3. Forcing every outbound client through the proxy
Section titled “3. Forcing every outbound client through the proxy”3.1 Inventory of outbound clients in ingest today
Section titled “3.1 Inventory of outbound clients in ingest today”| Module | Destination | Change |
|---|---|---|
enrichment/downloader.py (GeoManager) | MaxMind / DB-IP downloads (redirects to a CDN) | egress_client(follow_redirects=True) |
enrichment/reputation.py | AbuseIPDB, GreyNoise | egress_client() |
enrichment/intel.py | urlhaus, threatfox, VirusTotal | egress_client() |
enrichment/credentials.py | provider key tests | egress_client() |
cve/watch.py + cve/sources.py | NVD, GitHub advisories, CISA KEV | egress_client() |
llm/worker.py | Anthropic API | egress_client() (keeps trust_env=False) |
alerts/send.py webhooks | Discord, Teams | Drop the local resolve-and-pin (ingest can no longer resolve). Post by host name through the proxy; the host is already restricted to the official list by validate_webhook, and gluetun’s rebinding protection refuses private answers |
alerts/send.py SMTP (smtplib) | SMTP_HOST | smtplib has no proxy support. Subclass SMTP / SMTP_SSL and override _get_socket to return a socket from suckerpot_egress.open_tunnel(host, port) (HTTP CONNECT through the proxy). Literal IP hosts must be public. TLS checks unchanged |
attack/ | none (bundled data) | none |
3.2 The suckerpot_egress package (new, egress/)
Section titled “3.2 The suckerpot_egress package (new, egress/)”A small shared package (deps: httpx), used by ingest and collector, so the proxy, the gate and the
tests exist once.
class EgressSettings: # from env, frozen dataclass proxy_url: str # EGRESS_PROXY_URL, default http://vpn:8888 proxy_user: str # "egress" proxy_password: str # EGRESS_PROXY_PASSWORD (repr=False) control_url: str # EGRESS_CONTROL_URL, default http://vpn:8000 echo_urls: tuple[str, ...] # EGRESS_ECHO_URLS allowed_cidrs: tuple[IPv4Network | IPv6Network, ...] # EGRESS_ALLOWED_CIDRS (or derived) check_interval_s: int # EGRESS_CHECK_INTERVAL_S, default 300
class EgressClosed(Exception): code: str # "no_vpn", "tunnel_down", "ip_mismatch", ...
@dataclass(frozen=True)class GateStatus: open: bool reason: str # constant code, never a free-form server answer egress_ip: str | None # the VPN exit address, shown in Settings checked_at: datetime
class EgressGate: def __init__(self, settings: EgressSettings, *, client: httpx.AsyncClient | None = None): ... async def check(self) -> GateStatus # one self-check (section 3.4) async def run(self) -> None # check at start, then every interval def require_open(self) -> None # raises EgressClosed @property def status(self) -> GateStatus
def egress_client(gate: EgressGate, *, timeout: httpx.Timeout, follow_redirects: bool = False, verify: bool = True, limits: httpx.Limits | None = None, headers: Mapping[str, str] | None = None) -> httpx.AsyncClient # proxy=settings.proxy_url with auth, trust_env=False, and a transport wrapper that calls # gate.require_open() before every request (redirect hops included).
async def open_tunnel(gate: EgressGate, host: str, port: int, *, timeout: float) -> socket.socket # CONNECT host:port through the proxy, returns a blocking socket for smtplib.Rules:
trust_env=Falsealways, and noHTTP(S)_PROXYvariable is set in compose. A stray client that ignores the factory has neither proxy nor route nor external DNS, so it fails loudly, and the static test below catches it before it ships.- Requests send host names to the proxy (
CONNECT host:443); ingest and collector never resolve external names themselves. EgressClosedis treated as a retryable “not now” by every worker: no quota is spent, no negative cache entry is written.
3.3 DNS inside ingest and collector
Section titled “3.3 DNS inside ingest and collector”Both get dns: ["127.0.0.1"] like api. On internal networks Docker 29’s embedded resolver
(127.0.0.11) still answers service names (db, redis, vpn) and does not forward external
names. just egress-check proves getaddrinfo("example.com") fails in both containers.
Static code test (ingest/tests/test_egress_static.py, same idea in collector/tests/): AST scan
of the package source; it fails on httpx.AsyncClient(, httpx.Client(, socket.create_connection,
socket.socket(, smtplib.SMTP( / SMTP_SSL(, urllib.request, aiohttp, requests, or
getaddrinfo anywhere except in an explicit allow-list (the smtp adapter in alerts/send.py,
which may only call open_tunnel). Test clients built in tests/ are exempt.
3.4 Egress self-check
Section titled “3.4 Egress self-check”We do not need to know the home IP, and we must not learn it (learning it means a connection from it). A positive check is stronger than a negative one: the exit IP must be one we expect from the VPN, and the home IP is never inside the provider’s range.
EgressGate.check(), in order, stopping at the first failure (gate closed, reason recorded):
GET {control_url}/v1/vpn/statusanswers{"status": "running"}. Otherwisetunnel_down, orno_vpnifvpnis unreachable (profile off, container stopped).GET {control_url}/v1/publicip/ipgives gluetun’s exit IPG(learned through the tunnel). Missing:no_public_ip.- Through the proxy,
GETeach URL inEGRESS_ECHO_URLS(default:https://api.ipify.org,https://icanhazip.com, text body, 64 bytes max, 10 s timeout). At least two must answer, and all answers must be the same addressE. Otherwiseecho_failed/echo_disagree. E == G. Otherwiseip_mismatch(the proxy is not leaving through the tunnel gluetun reports).Eis a global unicast address (ipaddress.ip_address(E).is_global).Eis insideEGRESS_ALLOWED_CIDRS. When that variable is empty, the allow-list is derived:VPN_WG_ENDPOINT_IP/24for IPv4 (many WireGuard providers exit on, or next to, the server they accept on). If the provider exits elsewhere, the gate stays closed withip_not_allowedand Settings shows the observed VPN exit address so the owner can setEGRESS_ALLOWED_CIDRS.
Cadence: at start, every EGRESS_CHECK_INTERVAL_S (default 300 s), and right away after 3
consecutive proxy errors. The gate is closed until the first check passes. Ingest writes the status
to enrichment_status (provider egress); the collector writes its own row (egress_collector).
The event pipeline never waits for the gate: events are stored, only lookups wait.
Rejected alternative: a salted hash of the home IP in .env as a deny-list. The home IP is
dynamic, the hash adds nothing over plain text inside .env, and a deny-list misses any other
non-VPN path.
4. Guardrails and tests
Section titled “4. Guardrails and tests”4.1 Static compose test (scripts/guardrails/)
Section titled “4.1 Static compose test (scripts/guardrails/)”A small Python package (pyyaml, pytest) added to the packages list in the justfile and to
the CI python job, so it runs in just test and just check. Also just guardrails alone.
It loads docker-compose.yml with yaml.safe_load (merge keys resolved), plus each overlay
(docker-compose.ci.yml, docker-compose.it.yml) merged on top, and asserts:
- Every network is
internal: true, except exactly:ingress(namesp-ingress, masquerade off, icc off),dashboard(namesp-dashboard, masquerade off) andvpn-uplink(namesp-vpn, icc off). ingressmembers are exactly thesensor-*services;dashboardonlyapi;vpn-uplinkonlyvpn;egress-proxyexactly{vpn, ingest, collector}; no network namedegress.ingest,collector,analyzerare only on internal networks and havedns: ["127.0.0.1"].vpnis on exactly{vpn-uplink, egress-proxy},cap_add == ["NET_ADMIN"],devicesis exactly/dev/net/tun,cap_drop: [ALL],no-new-privileges, noports, image pinned by digest.- No other service has
cap_add,devices,privileged,pid: host,ipc: host, or anynetwork_mode(host,service:*,container:*). - No service except sensors and
apihasports; no service setsHTTP_PROXY,HTTPS_PROXY,ALL_PROXYorNO_PROXY. analyzermountsquarantineread-only;apimounts it read-only; onlycollectorwrites it.
Unit tests of the checker itself feed it small broken compose snippets (one per rule).
4.2 Live check: just egress-check
Section titled “4.2 Live check: just egress-check”scripts/egress-check.sh runs on the live stack (profile egress up) and prints PASS/FAIL per
line; exit code non-zero on any FAIL. It runs python -m suckerpot_egress.selftest inside ingest
and collector (the module ships with the package, no test tooling in images):
| Check | Expected |
|---|---|
Direct TCP to 1.1.1.1:443 and 9.9.9.9:443 (3 s timeout) | fails (no route / unreachable) |
UDP DNS query to 1.1.1.1:53 | no answer |
getaddrinfo("example.com") | fails |
| Gate check (§3.4) | open, E == G, E in the allow-list |
Through the proxy, CONNECT to the sp-vpn gateway on the dashboard port, and to vpn:8000 by an RFC 1918 literal | refused or timed out |
| Through the proxy, a name that resolves to 127.0.0.1 (a public “localhost” test domain set in the script) | refused (rebinding protection) |
In analyzer: direct TCP to 1.1.1.1:443, and vpn:8888 | both fail (no route, no such host) |
docker compose exec vpn ip link set wg0 down, then an echo through the proxy within 2 s | fails; gate closes on next check |
| Wait for gluetun to restore the tunnel (≤ 60 s), then gate check | open again |
docker compose stop vpn, then gate check in ingest | no_vpn, ingest stays healthy, events still stored; then docker compose start vpn |
The script never prints the exit IP in full: it prints in allow-list: yes/no.
4.3 Kill switch tests
Section titled “4.3 Kill switch tests”Covered by the last three rows above (tunnel down, container stopped) plus unit tests in
egress/tests/ with a fake control server and fake proxy: every gate failure reason closes the
gate, the client factory raises EgressClosed with the gate closed, and a redirect hop is checked
too.
4.4 Host firewall (scripts/host/firewall.nft)
Section titled “4.4 Host firewall (scripts/host/firewall.nft)”The owner applies it himself. New defines and rules (bridge matched by name, like the others):
# The VPN server (from VPN_WG_ENDPOINT in .env). The vpn container may send only this.define VPN_ENDPOINT_IP = CHANGE_ME # e.g. 203.0.113.10define VPN_ENDPOINT_PORT = CHANGE_ME # e.g. 51820
chain input { ... # (4) The vpn container never talks to the host. iifname "sp-vpn" ct state new counter drop}
chain forward { ... # (5) From the vpn uplink, only WireGuard to the one endpoint. Everything else (other hosts, # the LAN, IPv6, TCP) is dropped, even if gluetun's own firewall failed. iifname "sp-vpn" ct state new ip daddr $VPN_ENDPOINT_IP udp dport $VPN_ENDPOINT_PORT accept iifname "sp-vpn" ct state new counter drop}This makes the kill switch independent of gluetun: with the tunnel down, the only packet that may
leave is a WireGuard handshake to the endpoint. scripts/host/README.md gets the verification
commands (nft list table inet suckerpot_host, counters after just egress-check).
5. Collector
Section titled “5. Collector”5.1 Placement
Section titled “5.1 Placement”Two new services built from one image (collector/), with different commands, roles and networks:
| Service | Command | Networks | DB role | Quarantine | Why |
|---|---|---|---|---|---|
collector | python -m collector fetch | samples, egress-proxy | suckerpot_collector | rw | Talks to the internet through the proxy; never parses content beyond hashing |
analyzer | python -m collector analyze | samples | suckerpot_analyzer | ro | Parses hostile files; has no path to the internet or Redis |
Not inside ingest: a bug in a fetcher or parser must not reach Redis, the events pipeline or the ingest DB role. Splitting fetch from analysis means a parser exploit gets neither the proxy nor the network. Cost: two small Python containers (~60 MB each idle).
Both are in profile egress, <<: *hardening, user 10001:10001, dns: ["127.0.0.1"], tmpfs
/tmp:rw,noexec,nosuid,nodev,size=16m. Limits: collector mem_limit: 256m, cpus: 0.5,
pids_limit: 32; analyzer mem_limit: 512m (child is capped at 384 MB), cpus: 0.5,
pids_limit: 16. collector depends on vpn (service_started) and migrate; ingest does
not depend on vpn, so the event pipeline never waits for the tunnel.
5.2 Input and queue
Section titled “5.2 Input and queue”Download URLs are already stored by ingest as indicators rows with type = 'url' and
role = 'download_host' (from download_attempt events and URLs in commands), linked to sessions
through indicator_sessions. The collector polls them; it does not read Redis, so the
single-consumer invariant of ingest is untouched.
Every COLLECTOR_POLL_S (default 15 s): insert into fetch_targets every such indicator with
demo = false that is not there yet (ON CONFLICT (url) DO NOTHING), then pick due targets
(next_attempt_at <= now(), ordered by next_attempt_at). URLs found in fetched scripts
(§6.3) are inserted by the analyzer with depth = parent depth + 1, only if
depth <= COLLECTOR_MAX_DEPTH (default 2) and at most 20 per sample.
Schemes: http and https are fetched. tftp and ftp are recorded with status unsupported.
TFTP is deferred: it is UDP, which cannot cross an HTTP proxy; supporting it means a tiny TFTP
client in the VPN namespace (network_mode: service:vpn), a second process sharing the namespace
of the one container with NET_ADMIN, and an exception in the guardrails. Not worth it for M21;
the UI says “TFTP: recorded, not fetched”.
Re-fetch policy (url is unique after normalisation: lower-case scheme and host, default port
removed, fragment dropped, max 1,024 characters):
- Success: re-fetch only if the indicator was seen again after the last fetch and the last fetch
is older than
COLLECTOR_REFETCH_HOURS(default 24), at most 7 times. Payloads behind a URL rotate, so a new hash on re-fetch is a new sample linked to the same target. - Failure (network, 5xx, timeout): retry after 15 min, 1 h, 6 h, then status
failed. - 4xx, refused URL, too large: final (
gone,refused,too_large).
5.3 URL validation (SSRF)
Section titled “5.3 URL validation (SSRF)”Checked before the first request and again on every redirect hop:
- Scheme
httporhttps; port 1 to 65535; no user-info (user:pass@). - Host is an IP literal: parsed strictly (only canonical dotted quad or bracketed IPv6; decimal,
octal, hex and short forms such as
http://3232235777/are refused). Must beis_globaland not in: RFC 1918, 100.64.0.0/10, loopback, link-local, multicast, reserved, the documentation ranges, 198.18.0.0/15, the IPv6 ULA and documentation prefixes,VPN_WG_DNS, the VPN endpoint, andCOLLECTOR_DENY_CIDRS(owner’s own ranges, optional). - Host is a name: valid IDNA, at least two labels, not
localhost, not ending in.local,.localhost,.internal,.lan,.home.arpa,.arpa, and not one ofAPI_ALLOWED_HOSTS(the deployment’s own names). Single-label names (Docker service names) are refused. - The name is resolved by gluetun inside the tunnel. Its DNS rebinding protection drops private
answers, and the host rule drops anything from
sp-vpnthat is not WireGuard, so a name that points at the LAN or the host cannot be reached even if the validator missed it.
5.4 Fetch limits and client identity
Section titled “5.4 Fetch limits and client identity”| Limit | Default | Variable |
|---|---|---|
| Max body | 16 MiB, counted on raw bytes as they stream; Content-Length is not trusted | COLLECTOR_MAX_BYTES |
| Connect / total time | 10 s / 60 s | COLLECTOR_CONNECT_TIMEOUT_S, COLLECTOR_TIMEOUT_S |
| Redirects | 3, followed by hand, each hop validated | COLLECTOR_MAX_REDIRECTS |
| Concurrency | 2 global, 1 per host | COLLECTOR_CONCURRENCY |
| Per-host spacing | 10 s between requests to one host | COLLECTOR_HOST_INTERVAL_S |
| Global rate | 60 fetches per hour, 500 per day (hard quota in enrichment_quota, provider collector) | COLLECTOR_HOURLY_QUOTA, COLLECTOR_DAILY_QUOTA |
Accept-Encoding: identity, body read withaiter_raw(): httpx never decompresses, so no gzip bomb in the fetcher. A response with aContent-Encodingis stored as received.verify=Falsefor HTTPS: malware hosts often have broken certificates; integrity is not a concern for hostile content, and the request goes through the tunnel either way.- No cookies, no
Referer,Accept: */*.User-Agentchosen from the URL: a file name with an IoT architecture suffix (arm,arm5,arm6,arm7,mips,mpsl,m68k,ppc,sh4,spc,x86,i686) or a/bins/path gets BusyBox’sWget; any other URL getsWget/1.21.4; a.ps1or.exegetsMozilla/5.0 (Windows NT 10.0; Win64; x64) WindowsPowerShell/5.1. The table lives incollector/collector/identity.py. Never the project name, never a contact address. - Every attempt is recorded in
fetch_attemptswith a constant outcome code; response bodies of failures are not stored.
5.5 Quarantine
Section titled “5.5 Quarantine”Volume quarantine, mounted at /quarantine (collector rw, analyzer and api ro).
/quarantine/ .incoming/ # temp files, same filesystem, wiped at start ab/cd/abcd…<64 hex sha256> # two-level fan-out; no extension, ever- Write:
os.open(.incoming/<random>, O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW, 0o600), hash (sha256) while streaming, abort and unlink past the size cap,fsync,fchmod(0o400), thenos.link/renameto the final path. If the final path exists, the temp file is deleted (dedupe). Directories 0o700. No file ever has an execute bit; a static test forbidschmodwith anyxbit and anyexec*/spawn*call taking a quarantine path. noexec: a named volume cannot carry mount flags without a host path. Defence in depth instead: 0400 files, read-only root filesystems, nothing ever passes a quarantine path to an exec call. Optional (owner, host side): bind the volume to a host directory witho: bind,noexec,nosuid,nodev.- Total cap
QUARANTINE_MAX_MB(default 2,048): before a fetch, the collector compares the sum ofsamples.sizewherestoredwith the cap; at 100 % it stops fetching (quarantine_fullin Settings) but keeps recording URLs. - Retention
QUARANTINE_RETENTION_DAYS(default 90) afterlast_seen: the file is deleted, thesamplesrow and its analysis are kept forever withstored = false.fetch_attemptsolder than 90 days are deleted by the same job. - Never committed (
.gitignorealready excludes volumes; the guardrail test also fails on any file under the repo whose name is a 64-hex string).
6. Static analysis (never execute)
Section titled “6. Static analysis (never execute)”6.1 Library: analysis/suckerpot_analysis (new package, pure functions)
Section titled “6.1 Library: analysis/suckerpot_analysis (new package, pure functions)”No network, no DB, no file system writes. Input is bytes, output is a pydantic model.
def analyze(data: bytes, *, rules: CompiledRules, limits: Limits = Limits()) -> AnalysisResultdef compile_rules(directory: Path) -> CompiledRulesdef identify(data: bytes) -> FileInfo # kind, arch, bits, endian, linking, packerdef digests(data: bytes) -> Digests # sha256, sha1, md5, tlsh | Nonedef printable_strings(data: bytes, *, min_len: int = 6, max_count: int = 500) -> list[str]def deobfuscate_shell(text: str, *, max_passes: int = 5, max_out: int = 256 * 1024) -> Deobfuscationdef extract_iocs(texts: Iterable[str]) -> list[Ioc] # urls, ip:port, domains, with a "where" tag| Feature | Choice | Why |
|---|---|---|
| File type and arch | Own header parsers (~200 lines): ELF (class, endianness, e_machine to x86 / x86-64 / ARM / AArch64 / MIPS / PowerPC / SPARC / m68k / SuperH / ARC, static vs dynamic via PT_INTERP, stripped via .symtab), PE (machine, 32/64, DLL), Mach-O, shebang and shell heuristics, archive magics (gzip, zip, tar, xz, 7z, bzip2), UPX marker | libmagic is a large C parser with its own CVE history and a system dependency; we need a few fields, read with struct from bounded offsets. Every offset is checked against len(data) |
| Hashes | hashlib sha256, sha1, md5; TLSH via py-tlsh (pinned) | No cp313 wheel exists for py-tlsh; it builds from source in the builder stage (g++), only the built wheel reaches the runtime image. If the import fails, tlsh is null, nothing else breaks |
| Strings | Own scanner: ASCII and UTF-16LE runs ≥ 6, first 500 unique, 120 chars each, scan the first 16 MiB | Bounded, no dependency |
| YARA | yara-x (1.22.1, cp313 manylinux wheel verified) | Rust engine (memory safe), built-in scan timeout. Rules in collector/rules/yara/*.yar, compiled once at analyzer start; a broken rule fails the start, not a scan |
| Archives | Identified, not extracted in M21 | No decompression bombs. Single-stream gzip extraction with an output cap may come later |
| Packers | UPX detected and reported, never unpacked | Unpacking is execution-adjacent |
YARA rules shipped (written for this repo, MIT, each with meta: author, description, reference):
mirai.yar (table strings, attack vector names, /bin/busybox + ECCHI-style markers), gafgyt.yar,
miners.yar (XMRig pool and config strings, stratum URLs), redis_module.yar (ELF exporting
RedisModule_OnLoad, rogue-server module names), shell_dropper.yar (multi-arch download loops,
chmod 777 + run + rm), kinsing.yar. A hit gives a family and tags. No real malware is
committed for tests: tests use synthetic buffers (crafted ELF headers, strings assembled in the
test) built at test time.
6.2 Shell deobfuscation
Section titled “6.2 Shell deobfuscation”For samples identified as text and ≤ 1 MiB. Pure string rewriting, iterated until stable (max 5 passes, output capped at 256 KiB):
echo|printf <b64> | base64 -d [| sh|bash]andbase64 --decode <<< …: decode, inline the result.printf '\x41\101',echo -e '\x41',$'\x41': decode escapes.VAR=literalthen$VAR/${VAR}: substitute literal values only (quoted or bare words). No command substitution, arithmetic orevalis ever evaluated;eval "…"is unwrapped textually.- Adjacent quoted pieces (
'ht''tp') are joined.
Each rewrite is recorded (steps: [{"kind": "base64", "count": 2}, …]) and the final text is
stored. The UI shows it as plain text only.
6.3 IOC and C2 extraction
Section titled “6.3 IOC and C2 extraction”From deobfuscated scripts: URLs (http, https, tftp, ftp) become next_url links; http(s)
ones go back to fetch_targets within the depth cap. From binary strings: ip:port pairs and
public IPv4 literals, plus host names matching a bounded domain regex with a TLD in a fixed list.
A best-effort Mirai table decoder tries single-byte XOR (keys 0x22, 0x37, 0x54) on runs of
non-printable bytes and keeps results that look like a host name. Every IOC carries a source
(script_url, binary_ip_port, binary_string, mirai_table) and a confidence (high for URLs and
ip:port in a YARA-matched binary, low otherwise).
Feedback into indicators: high IOCs are upserted into indicators (role = 'c2' for addresses,
existing roles kept on conflict, UNIQUE (type, value) already exists) and each sample’s sha256 is
upserted as type = 'hash', trusted = true. The existing intel worker then looks them up through
the proxy like any other indicator. Private, documentation and own addresses are dropped by the
same filter ingest uses (INDICATORS_IGNORE, API_ALLOWED_HOSTS).
6.4 Parser safety
Section titled “6.4 Parser safety”The analyzer reads the sample (≤ 16 MiB) and hands it to a child process for each sample:
asyncio.create_subprocess_exec(sys.executable, "-I", "-m", "collector.analysis_child", stdin=PIPE, stdout=PIPE, stderr=DEVNULL): fixed argv, no shell, the sample arrives on stdin, so the child never opens a path. This is the only process spawn in the package; the static test allows exactly this call. (CLAUDE.md §5 forbids process spawning in sensors; the analyzer is not a sensor and never executes the sample, it isolates our own parsers.)- First thing in the child:
setrlimitRLIMIT_AS384 MiB,RLIMIT_CPU30 s,RLIMIT_FSIZE0,RLIMIT_NOFILE16,RLIMIT_CORE0. NotRLIMIT_NPROC(counted per uid across the host); the container’spids_limit: 16bounds processes instead. - Parent: wall-clock timeout 60 s then
kill(); stdout capped at 2 MiB; the JSON is validated against theAnalysisResultmodel, so a confused child cannot write arbitrary rows. A crash, timeout or invalid output givesanalysis_status = failed|timeoutwith a constant code; the analyzer moves on. - Inside the child: yara-x timeout 10 s; every parser works on
memoryviewslices with explicit bounds; ELF/PE section and program header counts capped at 256. - Fuzz-style tests: truncated headers, huge counts and offsets, 16 MiB of random bytes, a 16 MiB single line, deeply nested base64 (each pass bigger), a script that expands a variable into itself. Each must return within limits.
7. Data model, roles, API and UI
Section titled “7. Data model, roles, API and UI”7.1 Migration 0018_samples
Section titled “7.1 Migration 0018_samples”ALTER TABLE indicators DROP CONSTRAINT indicators_role_check; -- name per 0009ALTER TABLE indicators ADD CONSTRAINT indicators_role_check CHECK (role IN ('download_host', 'contacted_host', 'hash', 'c2'));
CREATE TABLE fetch_targets ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, url text NOT NULL UNIQUE CHECK (char_length(url) BETWEEN 1 AND 1024), scheme text NOT NULL, host text NOT NULL CHECK (char_length(host) <= 255), indicator_id bigint REFERENCES indicators (id) ON DELETE SET NULL, parent_sha256 text, -- set when found in a fetched script depth smallint NOT NULL DEFAULT 0, status text NOT NULL DEFAULT 'pending' CHECK (status IN ('pending','fetched','failed','gone', 'refused','too_large','unsupported','quarantine_full')), last_error text, attempts smallint NOT NULL DEFAULT 0, fetches smallint NOT NULL DEFAULT 0, first_seen timestamptz NOT NULL, last_attempt_at timestamptz, next_attempt_at timestamptz, last_sha256 text);CREATE INDEX ix_fetch_targets_due ON fetch_targets (next_attempt_at) WHERE status = 'pending';
CREATE TABLE fetch_attempts ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, target_id bigint NOT NULL REFERENCES fetch_targets (id) ON DELETE CASCADE, started_at timestamptz NOT NULL, duration_ms integer, http_status smallint, bytes bigint, redirects smallint, outcome text NOT NULL, sha256 text);
CREATE TABLE samples ( sha256 text PRIMARY KEY CHECK (sha256 ~ '^[0-9a-f]{64}$'), sha1 text NOT NULL, md5 text NOT NULL, tlsh text, size bigint NOT NULL, kind text, arch text, bits smallint, endian text, linking text, packer text, family text, severity text NOT NULL DEFAULT 'notice', first_seen timestamptz NOT NULL, last_seen timestamptz NOT NULL, stored boolean NOT NULL DEFAULT true, analysis_status text NOT NULL DEFAULT 'pending' CHECK (analysis_status IN ('pending','done','failed','timeout')), analysis_version smallint NOT NULL DEFAULT 0);
CREATE TABLE sample_analysis ( sha256 text PRIMARY KEY REFERENCES samples (sha256) ON DELETE CASCADE, analyzed_at timestamptz NOT NULL, elapsed_ms integer, yara jsonb NOT NULL DEFAULT '[]', -- [{rule, namespace, tags, meta}], max 50 strings text[] NOT NULL DEFAULT '{}', -- max 500 x 120 chars deobfuscated text CHECK (deobfuscated IS NULL OR octet_length(deobfuscated) <= 262144), deob_steps jsonb NOT NULL DEFAULT '[]', errors text[] NOT NULL DEFAULT '{}');
CREATE TABLE sample_sources ( -- which URL served which sample sha256 text REFERENCES samples (sha256) ON DELETE CASCADE, target_id bigint REFERENCES fetch_targets (id) ON DELETE CASCADE, first_fetched timestamptz NOT NULL, last_fetched timestamptz NOT NULL, PRIMARY KEY (sha256, target_id));
CREATE TABLE sample_links ( -- chain edges out of a sample sha256 text REFERENCES samples (sha256) ON DELETE CASCADE, kind text NOT NULL CHECK (kind IN ('next_url','c2_ip','c2_domain','c2_url')), value text NOT NULL CHECK (char_length(value) <= 1024), port integer, source text NOT NULL, confidence text NOT NULL, target_id bigint REFERENCES fetch_targets (id) ON DELETE SET NULL, indicator_id bigint REFERENCES indicators (id) ON DELETE SET NULL, PRIMARY KEY (sha256, kind, value));The full chain is: session → indicator_sessions → indicators (url) → fetch_targets →
sample_sources → samples → sample_links (next_url → fetch_targets → … ; c2_* →
indicators). Models go in db/suckerpot_db/models.py. Plain tables, not hypertables: volumes are
small (hundreds of rows a day at most).
Roles (db/suckerpot_db/roles.py, applied by migrate; new secrets
POSTGRES_COLLECTOR_PASSWORD, POSTGRES_ANALYZER_PASSWORD, ≥ 24 chars, pairwise different, added
to the secrets policy):
| Role | Grants |
|---|---|
suckerpot_collector | SELECT on indicators; SELECT, INSERT, UPDATE on fetch_targets, fetch_attempts, samples, sample_sources; DELETE on fetch_attempts; SELECT, INSERT, UPDATE on enrichment_quota, enrichment_status (own providers) |
suckerpot_analyzer | SELECT, UPDATE on samples; SELECT, INSERT, UPDATE on sample_analysis, sample_links; SELECT, INSERT on fetch_targets (next-stage URLs); SELECT, INSERT, UPDATE on indicators; INSERT, UPDATE on enrichment_status |
suckerpot_api | SELECT on all new tables; INSERT on audit_log (exists) |
suckerpot_ingest | unchanged |
7.2 API (api/api/samples.py, api/api/egress.py)
Section titled “7.2 API (api/api/samples.py, api/api/egress.py)”| Method | Path | Who | Notes |
|---|---|---|---|
| GET | /api/v1/samples | any role | Cursor pagination (limit ≤ 200). Filters: kind, arch, family, yara, status, q (hash prefix or URL substring), since. Sort last_seen desc |
| GET | /api/v1/samples/{sha256} | any role | Metadata, analysis (strings capped, deobfuscated text), sources (URLs, first/last fetch, session count, distinct IPs, sensors, latest 20 sessions), links, virustotal_url (a link only, no API call) |
| GET | /api/v1/samples/{sha256}/chain | any role | {nodes, edges}, bounded: depth 3, 100 nodes. Node kinds: session, url, sample, c2 |
| POST | /api/v1/samples/{sha256}/archive | admin | Re-authentication (api/reauth.py: password, CSRF, origin, rate limit), 10 per hour per user. Returns application/zip, AES-256 (pyzipper), password infected, single entry <sha256>.bin, Content-Disposition: attachment, X-Content-Type-Options: nosniff, Cache-Control: no-store. Built in memory (≤ 16 MiB input). Audit sample.download with sha256 and actor; never the file. 404 if stored = false |
| GET | /api/v1/egress | any role | Gate status of ingest and collector from enrichment_status: open/closed, reason code, last check, exit IP in allow-list yes/no, collector quota and quarantine use |
| GET | /api/v1/malware, /api/v1/malware/detail | any role | Existing; sha256, size, virustotal_url now filled from fetch_targets + samples, plus fetch_status |
The API mounts quarantine read-only and only ever reads /quarantine/ab/cd/<sha256> for a
sha256 matching ^[0-9a-f]{64}$; it never streams a raw file. pyzipper is the only new API
dependency.
7.3 UI (follow web/DESIGN.md: no new hue)
Section titled “7.3 UI (follow web/DESIGN.md: no new hue)”- Malware page: each URL row gains a fetch status chip (word + icon: Fetched, Pending, Failed, Refused, Not fetched (TFTP)) and, when fetched, the short hash linking to the sample. A Samples tab lists samples: finding sentence first (“12 new binaries this week, 9 of them Mirai builds for ARM and MIPS”), then rows: family or “Unknown”, kind and arch, size, sources, last seen.
- Sample page
/malware/samples/:sha256: header (family, kind/arch, severity, hashes with copy buttons, VirusTotal link), then Where it came from (URLs and sessions, IP chips, sensors), The chain as a vertical step list (session → script → binary → C2), reusing the campaign story steps, not a new graph library; What it matched (YARA hits asTagBadge); Deobfuscated script (monospace text block, rendered as text only, with the step summary); Strings (virtualised list); admin-only Download as archive button with a dialog that explains the passwordinfectedand asks for the password again. - Settings: an Egress card: “Outbound traffic leaves through the VPN” or “Outbound traffic is paused: the VPN tunnel is down”, last check, collector quota meter, quarantine use against its cap. The exit IP is not displayed by default (one click reveals it).
- Severity: red only for a sample with a malware-family YARA hit or an ELF/PE dropped by a session (malware dropped, §4). Everything else is neutral or notice.
- Mocks in
web/src/mockswith demo-only data (hashes generated, documentation-range IPs).
8. Configuration
Section titled “8. Configuration”New variables (compose default in brackets). .env.example gets placeholders only.
| Variable | Default | Used by |
|---|---|---|
COMPOSE_PROFILES | (unset) | Set to egress to run vpn, collector, analyzer |
VPN_WG_PRIVATE_KEY, VPN_WG_PEER_PUBLIC_KEY, VPN_WG_ADDRESSES, VPN_WG_DNS | empty | vpn (already in the owner’s .env) |
VPN_WG_ENDPOINT_IP, VPN_WG_ENDPOINT_PORT | empty | vpn, gate allow-list derivation, collector deny-list |
EGRESS_PROXY_PASSWORD | empty (≥ 24 chars required with the profile) | vpn, ingest, collector |
EGRESS_PROXY_URL / EGRESS_CONTROL_URL | http://vpn:8888 / http://vpn:8000 | ingest, collector |
EGRESS_ECHO_URLS | https://api.ipify.org,https://icanhazip.com | ingest, collector |
EGRESS_ALLOWED_CIDRS | empty (derived from the endpoint /24) | ingest, collector |
EGRESS_CHECK_INTERVAL_S | 300 | ingest, collector |
COLLECTOR_ENABLED | false (owner sets true) | collector, analyzer |
COLLECTOR_POLL_S, COLLECTOR_MAX_BYTES, COLLECTOR_TIMEOUT_S, COLLECTOR_CONNECT_TIMEOUT_S, COLLECTOR_MAX_REDIRECTS, COLLECTOR_CONCURRENCY, COLLECTOR_HOST_INTERVAL_S, COLLECTOR_HOURLY_QUOTA, COLLECTOR_DAILY_QUOTA, COLLECTOR_REFETCH_HOURS, COLLECTOR_MAX_DEPTH, COLLECTOR_DENY_CIDRS | 15, 16777216, 60, 10, 3, 2, 10, 60, 500, 24, 2, empty | collector |
QUARANTINE_MAX_MB, QUARANTINE_RETENTION_DAYS | 2048, 90 | collector |
ANALYZER_TIMEOUT_S, ANALYZER_MEMORY_MB | 60, 384 | analyzer |
POSTGRES_COLLECTOR_PASSWORD, POSTGRES_ANALYZER_PASSWORD | required with the profile | migrate, collector, analyzer |
Removed: FETCH_PAYLOADS (replaced by COLLECTOR_ENABLED; ingest never fetches payloads).
Behaviour without a VPN: every egress-dependent feature (GeoIP download, reputation, intel, CVE
watch, LLM, alerts, collector) stays idle with reason no_vpn, visible in Settings. There is no
“direct” mode. Already-downloaded GeoIP databases keep working.
9. Task split
Section titled “9. Task split”Wave 1 runs in parallel (separate directories). Wave 2 starts when its inputs are merged. All
commands run through just (containers), never on the host. Every agent prompt must repeat: no
value from ENVIRONMENT.local.md or .env in any file; documentation ranges only.
A. VPN, compose, guardrails, host firewall (sonnet, backend-engineer or main session; security review)
Section titled “A. VPN, compose, guardrails, host firewall (sonnet, backend-engineer or main session; security review)”- Owns:
docker-compose.yml,docker-compose.it.yml,vpn/control-auth.toml,scripts/guardrails/**,scripts/egress-check.sh,scripts/check-vpn-env.sh,scripts/host/firewall.nft,scripts/host/README.md,justfile,.env.example,.github/workflows/ci.yml. - Contracts: §2.2 to §2.4, §4.1, §4.2, §4.4, §5.1 service blocks for
collectorandanalyzer(image built fromcollector/Dockerfile, task D), §8. - Accept: guardrail tests pass on the new compose and fail on each broken snippet;
docker compose configvalid with and withoutCOMPOSE_PROFILES=egress; stack healthy without the profile (CI path);vpnhealthy with the profile on the VM;just egress-checkall PASS (needs B merged for the selftest module). - Verify:
just guardrails,just config,just up,just ps,just egress-check.
B. Egress library and ingest migration (sonnet, backend-engineer)
Section titled “B. Egress library and ingest migration (sonnet, backend-engineer)”- Owns:
egress/**(new packagesuckerpot_egress, incl.selftest),ingest/ingest/**clients listed in §3.1,ingest/ingest/main.py(gate task and status),ingest/Dockerfile,ingest/pyproject.toml,ingest/uv.lock,ingest/tests/**. - Contracts: §3.2 API exactly; §3.4 check order and reason codes;
EgressClosedis retryable and spends no quota. - Accept: unit tests for every gate reason, the transport wrapper,
open_tunnelagainst a fake proxy, SMTP over the tunnel against a fake server;test_egress_static.pypasses and fails on a plantedhttpx.AsyncClient(; no test opens a real outbound connection. - Verify:
just lint,just typecheck,just test; single package as in CLAUDE.md §7 with-w /work/egressand-w /work/ingest.
C. Analysis library and YARA rules (sonnet; opus for the security review)
Section titled “C. Analysis library and YARA rules (sonnet; opus for the security review)”- Owns:
analysis/**(packagesuckerpot_analysis),collector/rules/yara/**. - Contracts: §6.1 signatures, §6.2, §6.3; no I/O; deterministic output; pydantic models shared with D.
- Accept: unit tests per parser with synthetic buffers; every rule compiles with yara-x and matches a synthetic positive and no synthetic negative; fuzz cases of §6.4 return within limits; no real malware in the repo.
- Verify:
just test,just typecheck(package added topackages).
D. Collector and analyzer services (sonnet, backend-engineer; after B and C, can stub them)
Section titled “D. Collector and analyzer services (sonnet, backend-engineer; after B and C, can stub them)”- Owns:
collector/**(package,Dockerfilewithpy-tlshbuilt in the builder stage,identity.py, URL validator, quarantine store, fetch loop, analyzer loop,analysis_child, static test). - Contracts: §5 entirely, §6.4, DB tables and roles of §7.1 (from E).
- Accept: validator tests (every refused form of §5.3, incl. decimal IP, redirect to a
private address, single-label host); store tests (0400, no x bit, dedupe, size cap, temp
cleanup, quarantine full); fetch loop against a fake proxy (caps, retries, quota, gate closed);
child killed on timeout and memory; integration test (
-m integration) fetch → sample → analysis → links against the throwaway stack with a local fake HTTP origin. - Verify:
just lint typecheck test,just test-integration.
E1. Migration and roles (sonnet, backend-engineer; wave 1)
Section titled “E1. Migration and roles (sonnet, backend-engineer; wave 1)”- Owns:
db/migrations/versions/0018_samples.py,db/suckerpot_db/models.py,db/suckerpot_db/roles.py,db/suckerpot_db/entrypoint.py(secrets policy),db/tests/**. - Accept: migration chain test passes up and down; role grants test covers both new roles and
that
suckerpot_apicannot write the new tables. - Verify:
just test,just test-integration.
E2. API (sonnet, backend-engineer; wave 2, after E1)
Section titled “E2. API (sonnet, backend-engineer; wave 2, after E1)”- Owns:
api/api/samples.py,api/api/egress.py,api/api/malware.py,api/api/routes.py,api/pyproject.toml,api/uv.lock,api/tests/**. - Accept: pagination and filters; archive: viewer gets 403, admin without re-auth 401, admin
with re-auth gets a zip that
pyzipperopens withinfectedand whose entry hash equals the sha256; audit row written; invalid sha256 422;stored = false404. - Verify:
just lint typecheck test,just test-integration.
F. Web (sonnet, frontend-engineer; can start in wave 1 on mocks)
Section titled “F. Web (sonnet, frontend-engineer; can start in wave 1 on mocks)”- Owns:
web/src/pages/malware.tsx,web/src/pages/sample-detail.tsx,web/src/pages/settings.tsx(Egress card),web/src/app/router.tsx,web/src/lib/api*,web/src/mocks/**, related components and tests. - Accept: §7.3; vitest green; screenshots 1440/390, dark/light, reviewed by
ui-designer. - Verify:
just web-test,just web-build,just screenshots.
G. Docs (haiku, docs-writer; last)
Section titled “G. Docs (haiku, docs-writer; last)”docs/ARCHITECTURE.md (networks, outbound hosts, services), docs/SECURITY.md,
docs/configuration.md, docs/deployment.md (VPN setup, firewall rule), README.md table. The
orchestrator updates CLAUDE.md §3 and §5 (no more egress network; payloads fetched by
collector, not ingest), with the owner’s approval.
Reviews
Section titled “Reviews”security-reviewer (opus) on A, B and D together (egress, quarantine, parser safety), then the
milestone review. ui-designer on F.
10. Risks and open questions
Section titled “10. Risks and open questions”- Endpoint split: the owner adds
VPN_WG_ENDPOINT_IPandVPN_WG_ENDPOINT_PORTto.env(blocks task A’s live checks, not its code). - Allow-list: does the provider exit on the endpoint’s /24? If not, the owner sets
EGRESS_ALLOWED_CIDRSafter the first run (Settings shows the observed VPN exit on demand). - Capabilities: if gluetun needs
NET_RAWbesidesNET_ADMIN, that is a new approval. - gluetun details to verify on v3.41.1: plain-DNS upstream variable names,
CONNECTto non-443 ports (needed for SMTP 587/465 and odd malware ports), whether the proxy and control ports are accepted fromegress-proxywithoutFIREWALL_INPUT_PORTS, IPv6 address handling. - Attribution: payload hosts see a VPN address fetching soon after an attack on the honeypot. Attackers can correlate and block it; they cannot see the home IP. Accepted.
- Alerts: webhooks lose the address pinning they have today and rely on the official-host list plus gluetun’s rebinding protection. Accepted, reviewed in the security pass.
- TFTP and FTP are recorded, not fetched, in M21.
- CLAUDE.md says payload fetching is done by ingest with
FETCH_PAYLOADS; this design moves it tocollectorwithCOLLECTOR_ENABLED. Needs the owner’s OK to amend.