M30 — Community blocklist feeds

Status: proposal for review. Scope: docs/ROADMAP.md, section M30. Builds on M29 (hub, consensus, spn/1, threat model), M27 (takedown states, through M29 takedown votes), M16 (local blocklists, never-list, exports, token feed) and M22 (STIX renderer). Every address and name here is a placeholder: documentation ranges (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, 2001:db8::/32) and example.org. The hub has no real domain yet: hub.example.org and feeds.example.org are placeholders.

1. Goal

The owner’s “double effort”:

1. Block. Publish free blocklists that any firewall can load (OPNsense, pfSense, nftables, iptables/ipset, MikroTik, MISP). They contain only addresses and names the community has validated as harmful.
2. Report. Get the same targets taken down. M29 already does this with grouped hub reports.

The two halves feed each other. A listed host that is taken down (M27/M29 takedown votes) leaves the list. A host that comes back returns. Public statistics show both: what is blocked and what was removed from the internet.

Hard requirements:

• Feeds are published by the hub, never by a member’s home connection.
• Feeds are built only from M29 consensus targets, plus stricter publication rules (§4). A single member, or a Sybil set that M29 would reject, can never put a value on a list.
• Feeds are public: no account, no token, no cookies, no tracking and no log of downloader addresses.
• Feeds reveal nothing about members: no node, pseudonym, country or per-node time, and no single-node signal (§3.3).
• Every list has a purpose, an expiry and a delisting path.

2. Pieces

Piece	Where	Runs on
Feed format, renderers, minisign, parser	netproto/suckerpot_netproto/feeds/ (pure, no I/O)	hub (renders) and ingest (parses)
Feed generator	hub/hub/feeds/	hub-worker (hourly)
Static feed files	volume ${HUB_VOLUME_PREFIX}_feeds, served by Caddy at https://feeds.example.org/v1/	project infrastructure
Delisting and lookup API	hub/hub/routes/{delist,lookup}.py	hub-api
Node feed consumer	ingest/ingest/network/feed*.py	each node’s ingest, through the VPN egress gate, members and non-members (opt-in)
Node API	api/api/network_feed.py, api/api/feeds.py (local mirror)	node api
Web	Blocklists page, tab “Network”	dashboard
Spec	docs/network-protocol.md, new part “Feeds” (normative copy of §6 and §7)	docs
User guide	docs/feeds.md (consumers: firewalls, verification, licence, delisting)	docs, also the feed site README.txt

hub-db (targets, observations) ──► hub-worker: select → filter → render → sign → publish ──► /srv/feeds (volume)
                                                                                              │ read-only
firewalls, MISP, anyone ◄──(HTTPS GET, static, no logs)── Caddy feeds.example.org ◄───────────┘
node ingest ──(egress gate, VPN)──► same static files (verified, stored, shown, mirrored locally)
delisting requester ──► feeds.example.org/v1/delist (form, PoW) ──► hub-api ──► operator (hub.admin)

3. Threat model additions

M29 §3 holds unchanged. These are the new assets, adversaries and invariants.

3.1 New assets

Asset	Must not be harmed or learned
Accuracy of a public list	An innocent address listed by poisoning, by stale data or by shared infrastructure
Members’ anonymity through the list	Which node saw what, or when, inferred from list content, metadata or change timing
Downloaders’ privacy	Who fetches the lists
Availability of the lists and of the hub API	Feed traffic or a DoS must not take down hub-api
Integrity for consumers	A forged or rolled-back list loaded into thousands of firewalls

3.2 New adversaries

#	Adversary	Capabilities
A8	Competitor or griefer	Wants a third party’s address or domain listed (business sabotage). Controls botnets, can type any command into honeypots, can send spoofed UDP
A9	Listed attacker	Wants their own C2 or payload host off the list: delisting spam, false evidence
A10	Feed DoS	Floods feeds.example.org or the delisting API
A11	Feed forger	MITM, rogue mirror, or a compromised CDN that serves altered or old files

3.3 Mitigations

Attack	Mitigation	Residual
A2/A8 Sybil nodes list a victim	Only M29 confirmed targets are candidates (≥ 5 eligible aged nodes, weight, cohort cap, 2 days, corroboration or 2×k nodes). The high tier adds ≥ 10 nodes, ≥ 3 days, ≥ 3 enrolment weeks and external corroboration (§4.3)	The M29 residual: a patient farm of ≥ 5 aged, well-behaved nodes. The change breaker (§5.6) and delisting bound the damage
A8 drives honest nodes: spoofed UDP packets with the victim’s source address sent to many honeypots	Inbound listing requires TCP-proven activity (§4.2). Evidence only from sip (UDP) never lists. connect and scan never list	None from spoofing. TCP sources cannot be spoofed past the handshake
A8 drives honest nodes: a botnet types wget http://victim.example.org/x into every honeypot	A URL or host seen in commands never lists by itself. A payload host needs a fetched file with a malware family from ≥ 2 nodes, or URLhaus corroboration (§4.2). Shared hosts never list (M16 SHARED_HOSTS)	An A8 who controls the victim’s server can host malware on it. That is no longer an innocent server
A8 plants decoy C2 addresses in malware configs	C2 needs M29 confidence = 'high' plus ≥ 2 distinct samples from ≥ 2 nodes, or ThreatFox corroboration (§4.2)	A real C2 that sits on shared infrastructure: the CDN and allow-list exclusions (§5.3) apply
A1 learns about members from list content	Entries carry no node data. Every published attribute (last_seen, services, tags, families, categories) needs k-support: a value or day appears only when ≥ HUB_MIN_NODES (5) eligible nodes support it (§4.4). No node count beyond the tier, no countries	The M29 group-testing residual (a listing proves the probed set held ≥ 5 nodes). The list makes it cheaper (no enrolment), not stronger
A1 single-target probe: reuses an already-listed address against one suspected sensor, then watches last_seen or services change	k-support (§4.4): one node’s sighting changes nothing public, not even the expiry	None
A1 learns from change timing	The feed is regenerated on a fixed hourly schedule. Additions wait HUB_FEED_HOLD_HOURS (6) after confirmation. Nodes submit days late with a 2 to 20 h random delay (M29)	Hour of confirmation ± hold. No per-node meaning
A3/A4 learn downloaders	Static files. Caddy log discarded, no cookies, no query strings honoured, no JS on list files, no analytics. No CDN by default (§8.2)	Hosting provider’s network logs; a CDN if the operator adds one later (documented decision)
A9 delisting spam	PoW on requests, daily caps, operator review. C2 and payload entries are never auto-suppressed. Inbound auto-suppression is capped per day (§9)	An attacker can hide up to the daily cap of inbound entries for one review cycle
A10	Static files, precompressed, ETag/304, one hourly write. Feed host separate from the API path, host-level connection limits, delisting/lookup rate-limited plus PoW	Volumetric floods are a hosting problem. Lists stay valid for 3 days (valid_until) without the hub
A11	minisign signature (Ed25519) on index.json and every file. The trusted comment binds file name and serial. The node verifies the signature, sha256, monotonic serial and valid_until. Feed key separate from the spn/1 hub key	Firewalls that load the plain URL do not verify. Docs explain, and the node’s verified local mirror is offered (§10.5)
Compromised hub publishes 0.0.0.0/0 or a huge list	The generator emits single addresses only (no prefix shorter than /32 or /128) and caps lines. Renderers and the node parser reject prefixes and over-cap files. The change breaker holds big jumps. Docs tell consumers to put allow rules for their own critical hosts above the block rule	A fully compromised hub with its key can publish arbitrary single addresses up to the caps
Legal: listing an innocent or a remediated host	Strict selection, short TTLs, takedown removal, a never-list for shared infrastructure, a public delisting path with a stated SLA, a factual wording (“observed by ≥ 5 independent sensors”), no owner data (no WHOIS, no reverse DNS) (§11)	Not legal advice. Owner action: a short legal check before turning feeds on

3.4 Feed invariants

#	Invariant	Enforced by
F1	Only targets in M29 state confirmed, reported or taken_down (the latter only for removal bookkeeping) are inputs. candidate, disputed and expired never are	feeds/select.py + tests
F2	Every published value passes the M29 never-list, the feed allow-list, the CDN and Tor exclusions and the reserved-range check at render time (lists may have grown)	feeds/filter.py, renderers re-validate
F3	Entries are single global unicast addresses or exact host names. No prefix, no wildcard, no parent domain	netproto renderers + parser
F4	Every published attribute value and last_seen has k-support (≥ 5 eligible nodes)	feeds/select.py + tests
F5	No node id, fingerprint, pseudonym, country, node count, precise time of sighting or hub target id appears in any feed file	AST/JSON scan test over every rendered file
F6	Feed files are written atomically, signed, and rewritten only when their content changed (stable mtime, so stable ETag and Last-Modified)	feeds/publish.py + tests
F7	The feed site stores and logs no client address; the only module reading the client address is still ratelimit.py (H1)	Caddyfile test, existing AST test
F8	The hub never contacts listed hosts. The only new outbound calls are fetches of CDN range files and the Tor exit list, from hub-worker	Static test on hub/hub/feeds/external.py host allow-list

4. What gets listed

4.1 Lists

List id	Direction	Content	Purpose
inbound	inbound	Attack sources (ip targets) with strong behaviour	Block published services (port forwards, VPN portals, mail, web) from known attackers
inbound-high	inbound	Subset of inbound, high tier	Low-risk default for automatic blocking
outbound	outbound	Malware distribution hosts (payload_ip) and C2 servers (c2_ip)	Stop internal devices from fetching malware or calling home
outbound-high	outbound	Subset of outbound, high tier	Default for firewalls that block without review
outbound-domains	outbound	Payload host names (domain) and C2 names (c2_domain), exact names, revealed by M29	DNS filters (pfBlockerNG DNSBL, Pi-hole, AdGuard Home, Unbound)

Why outbound is the most valuable for firewalls (also in docs/feeds.md):

• It acts after a compromise. Inbound blocking stops some attempts against services you expose, but a patched or unexposed service is not at risk from them anyway. Outbound blocking still works when a device is already infected (an IoT camera, a NAS, a phished laptop): the second-stage download or the C2 check-in fails, and the infection stays harmless.
• False positives cost almost nothing. No legitimate device on your network needs to reach a host that served Mirai to five honeypots. Blocking an inbound residential address can lock out a real customer who now has that address.
• It is small and stable. A few hundred to a few thousand hosts that live for days to weeks. Attack sources are tens of thousands of residential bots that change address daily.
• It produces a signal. A hit on an outbound block rule is a high-quality alert: some device inside is infected. Log it.

4.2 Selection rules (per list)

Inputs: hub targets in F1 states, the per-node observations of the last 14 days (M29 retention), and target_contributors (for the cohort and k-support checks). All rules below are on top of M29 §8.2 confirmation.

List	Kind	Required (all)
inbound	ip	Activity with k-support in STRONG_IN = {login_success, command, download, exploit}, or login with an attempts bucket ≥ 2 (10 to 99) from ≥ k nodes. Those k-supported activities come only from rows whose services are all TCP, i.e. a row that names sip is never evidence, even next to a TCP service (security review H2: spn/1 has one row per address and day with the union of services and activity, so it cannot say which service carried a login; spoofed SIP floods plus one TCP connect would otherwise look like a TCP brute force). Nodes from M30 on send sip sessions as presence only (service and session count, activity connect, no tags, no attempts). Not a research scanner (M29 never-list). connect, scan and http-only never qualify
outbound	payload_ip	A payload with this host_ip was fetched (fetched: true) by ≥ 2 nodes, with a sha256 whose family ∈ FAMILIES_V1 or whose file_type is an executable or script type (elf-*, pe, script-*; never archive, other or none), or the host is corroborated by URLhaus (hub’s optional abuse.ch key)
outbound	c2_ip	M29 confidence = 'high', and ≥ 2 distinct via_sha256 from ≥ 2 nodes, or ThreatFox corroboration. The published port(s) are k-supported
outbound-domains	domain, c2_domain	Revealed (value not null). Same evidence rule as the IP kind. Not in SHARED_HOSTS or under one (suffix match). Exact name only

payload_ip is a new hub-internal target kind. The hub derives it from payloads[].host_ip of M29 batches (no wire change). A payload host is recorded only under payload_ip: it is never folded into the ip row of the same address, so a URL typed into honeypots cannot give an address download / malware-drop evidence for inbound (security review H3). M29 reports payload hosts from payload_ip as malware download locations under the url/domain rule: a file fetched from the host by ≥ 2 eligible nodes that is malware-like (known family or executable or script file type, so an HTML page never counts), and ≥ k eligible nodes in all. A host that was only named in commands is never reported (security re-review H1). It is added to the M29 kind list by h0002. It is not sent in /v1/targets to spn/1 nodes, because nodes reject unknown enum values. A later node release adds it to the accepted enum first; then the hub may send it (M29 §5.8 order).

4.3 Tiers

Tier	Rule
standard	Passes §4.2
high	standard, and ≥ 2 × HUB_MIN_NODES (10) eligible nodes, observed on ≥ 3 distinct days that each have ≥ HUB_MIN_NODES nodes, contributors from ≥ 3 distinct enrolment ISO weeks that each hold ≥ 2 of the nodes (security review M2: one node alone never adds a day or a cohort), and external corroboration (any of the hub’s optional sources). For outbound: also ≥ 2 distinct sha256 (payload) or ThreatFox listing (C2)

*-high files hold the high tier only. The base file holds both tiers. Rich formats carry tier.

4.4 k-support (no single-node signal)

For every entry and every attribute value v (an activity, a service, a tag, a family, a C2 port): v is published only if ≥ HUB_MIN_NODES distinct eligible nodes reported it for this target in the M29 window. last_seen is the latest UTC day D such that ≥ HUB_MIN_NODES distinct eligible nodes observed the target on days D − 2 … D. first_seen is the M29 first_day of the first k-supported window. Expiry (§5.4) runs from last_seen. So one node seeing a listed address again changes nothing public.

The same k-support rule should apply to M29’s /v1/targets fields last_day, services, activity, tags and families (§15, M29 amendment).

4.5 Per-entry metadata (rich formats only)

Field	Content
value	Address or name, canonical (IPv6 compressed lower case, IDNA lower case)
family	4, 6 or dns
list	List id
tier	standard or high
categories	Closed set: brute-force, login-success, command-exec, malware-drop, exploit, payload-host, c2
services	k-supported subset of the M29 services enum (inbound only)
tags	k-supported subset of TAGS_V1
families	k-supported subset of FAMILIES_V1
ports	C2 ports, k-supported (C2 only)
first_seen, last_seen	UTC dates (YYYY-MM-DD)
expires	Date the entry leaves the list without new k-supported sightings
reported	true when the hub has sent an abuse report (M29 reported_day not null); no date

Never published: node counts (beyond what the tier implies), countries, hub target ids, URLs and URL paths (they can identify the victim of a targeted campaign; v1 lists hosts only), sha256 values (they belong to the M22 CERT exports and the hub’s own reporting).

5. Lifecycle

5.1 Generation

hub-worker runs the feed job every HUB_FEED_INTERVAL_MIN (60), after the hourly consensus run:

1. select (§4.2 to 4.4) → candidate entries;
2. hold: a new value enters only HUB_FEED_HOLD_HOURS (6) after its target was first confirmed (targets.confirmed_day + hold, tracked in feed_entries.eligible_at);
3. filter (§5.3);
4. expire and remove (§5.4);
5. cap (§5.5);
6. breaker (§5.6);
7. render, sign, publish (§6, §7).

The serial increases only when some list content changes.

5.2 States of a feed entry

pending (in hold) → listed → removed (expired, taken_down, delisted, excluded, capped). A removed value comes back only with a new k-supported sighting after its removal day, and never while a feed_allow row covers it.

5.3 Exclusions (never listed)

Category	Source	Decision and why
Reserved, private, CGNAT, loopback, link-local, multicast, documentation, bogons	netproto validation (documentation allowed only with HUB_ALLOW_DOCUMENTATION=1, tests)	A list must never block the consumer’s own network
Public DNS resolvers, root servers	NEVER_IPS from M16 (db/suckerpot_db/blocklist_values.py), copied into hub/hub/data/never.yaml. A hub test parses the M16 file with ast (no import) and fails if the hub copy is not a superset	Attackers type them into commands (wget http://8.8.8.8/x); blocking them breaks networks
Shared hosting and SaaS names	SHARED_HOSTS from M16, same sync test, suffix match for domains	Blocking github.com breaks work
Major CDN and cloud front-end ranges	Published range files of large CDNs and cloud front ends, fetched weekly by hub-worker into feed_auto_ranges (source = 'cdn:<name>'). URLs are configurable in hub/hub/data/ranges.yaml, with a size cap and a validation that every prefix is ≥ /8 (v4) / ≥ /16 (v6)	One CDN address serves thousands of sites; listing it blocks innocents. Malware behind a CDN goes to outbound-domains (exact name) and to M29 reports, never into IP lists. Plain cloud compute (VPS providers) is not excluded: that is where malware lives, and short TTLs plus takedown removal handle address reuse
Tor exit relays	Tor Project bulk exit list, fetched daily into feed_auto_ranges (source = 'tor')	Excluded from every list. An exit relays thousands of users; the attacker is not the exit. Lists that claim “validated harmful” must not carry shared relays. Who wants to block Tor already has the authoritative exit list; docs/feeds.md points to it. Counts of excluded Tor addresses are public in stats.json (transparency). Proposed M29 follow-up: no hub abuse reports to Tor exits either (§15)
Research scanners	M29 never-list (ASNs from ingest/rules/scanners.yaml)	They announce themselves and do no harm by our definition; dedicated lists exist
Admin and delisting allow-list	feed_allow (§5.7)	Operator decisions
The hub’s and the feed site’s own addresses, the project’s infrastructure	HUB_FEED_NEVER env (comma separated)	Self-protection

Commercial VPN exits and residential proxy networks cannot be excluded reliably. They are handled by k-support, short inbound TTLs and delisting. This residual is stated in the docs.

5.4 Expiry

Kind	Leaves the list when
ip (inbound)	No k-supported sighting for HUB_FEED_TTL_INBOUND_DAYS (7). Residential bot addresses churn daily to weekly; longer TTLs block the next innocent holder
payload_ip, domain	No k-supported sighting for HUB_FEED_TTL_PAYLOAD_DAYS (14), or taken down: every confirmed url target hosted on it is M29 taken_down (host linkage from payloads[], §6.3 target_hosts). Removal on takedown is immediate. A host ↔ URL link counts only when ≥ HUB_MIN_NODES eligible nodes asserted it on one day (payload_links.nodes, migration h0007; no node id stored), so one member can neither force nor block a takedown removal (security review M3). Cloud addresses are reassigned soon after takedown
c2_ip, c2_domain	No new evidence (a new sample pointing at it, k-supported) for HUB_FEED_TTL_C2_DAYS (30), or M29 taken_down. M27 never contacts C2s, so this is rare; ThreatFox removal also counts as down when the hub has the key
any	Target becomes disputed or expired in M29, the value hits an exclusion, or a delisting is accepted

5.5 Size caps

List	Max entries (env)	When over
inbound	HUB_FEED_MAX_INBOUND (50,000)	Keep high tier first, then latest last_seen. Cut entries are removed(capped)
outbound	HUB_FEED_MAX_OUTBOUND (20,000)	Same
outbound-domains	HUB_FEED_MAX_DOMAINS (20,000)	Same
any file	8 MiB uncompressed	The run fails closed: previous files stay, feed_runs.state = failed

5.6 Change breaker

If a run would add more than HUB_FEED_BREAKER_ADD_PCT (50 %) or remove more than HUB_FEED_BREAKER_REMOVE_PCT (50 %) of a list, and the absolute change is ≥ HUB_FEED_BREAKER_MIN (200), nothing is published. The run is stored held with its diff, and hub_audit gets an entry. The operator runs hub.admin feeds diff|release|discard. This catches poisoning bursts, consensus bugs and data loss. While held, the previous files stay and keep serving until valid_until.

Release modes, HUB_FEED_MODE:

• off: default; no files are written, and the feed site serves README.txt only;
• review: recommended for the first weeks; every run with additions is held for approval;
• auto: only breaker trips are held.

5.7 Allow-list (feed_allow)

Values (an address, a CIDR, or a domain with suffix match) that must not be listed, with reason ∈ owner_request, shared_infra, remediated, policy, false_positive, an optional expires_day (default 90 days for remediated and owner_request, none for the others) and a note. Set by the admin CLI or by an accepted delisting. CIDRs are allowed here (allowing is safe), never in lists.

6. Feed site and formats (normative; copied to docs/network-protocol.md, part “Feeds”)

6.1 URLs

Base: https://feeds.example.org/v1/ (HUB_FEEDS_DOMAIN). /v1/ is the format version. A breaking format change is /v2/, served next to /v1/ for ≥ 6 months. Data versions are the serial.

Path	Content
index.json, index.json.minisig	Manifest (§6.4), signed
<list>.txt	One value per line, IPv4 then IPv6, sorted, \n line ends, no comments, no blank lines (M16 alias rules). OPNsense / pfSense URL Table (IPs), pfBlockerNG, generic
<list>.v4.txt, <list>.v6.txt	Same, one address family (pfBlockerNG IPv4 and IPv6 tabs, tools without dual-stack parsing)
<list>.nft	nftables file for nft -f (§6.2)
<list>.ipset	ipset restore file with an atomic swap (§6.2)
<list>.rsc	MikroTik RouterOS v7 script (§6.2)
<list>.csv	Metadata, RFC 4180, CRLF, header, spreadsheet-safe cells (M16 csv_cell rules)
<list>.json	Metadata (§6.3)
outbound.stix.json, outbound-high.stix.json	STIX 2.1 bundle (M22 renderer, TLP:CLEAR, no identity, no created_by_ref)
outbound-domains.txt, .csv, .json	Domains: one exact name per line
stats.json	Public statistics (§12)
removed.json	Values removed in the last 30 days with removed date and reason expired or taken_down only. Delisted and excluded values are counted, never listed (do not repeat an innocent’s address)
README.txt	Policy, accuracy statement, delisting, licence (links to the official CC0 1.0 legal code)
delist.html	Static delisting form (§9.1)
<any file>.minisig	minisign signature of that file

<list> ∈ inbound, inbound-high, outbound, outbound-high. Domains have no .nft, .ipset, .rsc or .v4/.v6.

HTTP (Caddy):

• GET and HEAD only; no query string honoured.
• ETag and Last-Modified from the file; If-None-Match and If-Modified-Since answer 304.
• Content-Encoding: gzip from precompressed .gz sidecars, only when the client sends Accept-Encoding: gzip.
• Cache-Control: public, max-age=900.
• Access-Control-Allow-Origin: * on stats.json and index.json only (for the project site).
• X-Content-Type-Options: nosniff, HSTS, Content-Type: text/plain; charset=utf-8 for .txt/.nft/.ipset/.rsc/.minisig.

Consumers should poll at most once an hour. The docs recommend every 1 to 6 h.

6.2 Firewall formats (exact shapes)

.nft (the table is owned by the file; the user’s chains live in the same table, §13.3):

# selfhostr outbound serial 4812 generated 2026-11-01T13:00:00Z licence CC0-1.0
table inet selfhostr {
  set outbound_v4 { type ipv4_addr; flags interval; auto-merge; }
  set outbound_v6 { type ipv6_addr; flags interval; auto-merge; }
}
flush set inet selfhostr outbound_v4
flush set inet selfhostr outbound_v6
add element inet selfhostr outbound_v4 { 198.51.100.9, 203.0.113.50 }
add element inet selfhostr outbound_v6 { 2001:db8::50 }

• The set name is <list> with - replaced by _, plus _v4 or _v6.
• An add element line is omitted when its set is empty (empty braces are a syntax error).
• nft -f applies the whole file as one transaction.

.ipset (atomic swap, hash:net so later CIDR support needs no format change):

create selfhostr-outbound-v4 hash:net family inet hashsize 4096 maxelem 65536 -exist
create selfhostr-outbound-v4-new hash:net family inet hashsize 4096 maxelem 65536 -exist
flush selfhostr-outbound-v4-new
add selfhostr-outbound-v4-new 198.51.100.9
swap selfhostr-outbound-v4-new selfhostr-outbound-v4
destroy selfhostr-outbound-v4-new
create selfhostr-outbound-v6 hash:net family inet6 hashsize 1024 maxelem 65536 -exist
...

• maxelem is the list cap rounded up to a power of two.

.rsc (RouterOS v7; dynamic entries with a timeout, so the list refreshes without a window with no entries):

# selfhostr outbound serial 4812
:local t 1d6h
/ip firewall address-list
:do { add list=selfhostr-outbound address=198.51.100.9 timeout=$t comment=selfhostr } on-error={ set [find list=selfhostr-outbound address=198.51.100.9] timeout=$t }
/ipv6 firewall address-list
:do { add list=selfhostr-outbound address=2001:db8::50/128 timeout=$t comment=selfhostr } on-error={ set [find list=selfhostr-outbound address=2001:db8::50/128] timeout=$t }

• The timeout is longer than the recommended daily refresh, so removed entries lapse on their own.
• The docs recommend outbound or the high lists for MikroTik: importing 50,000 lines is slow on small routers.

Every renderer re-validates each value (F2, F3). A value that fails is dropped and counted in feed_runs.counts.dropped_render.

6.3 <list>.json

{"schema": "selfhostr-feed/1", "list": "outbound", "direction": "outbound",
 "serial": 4812, "generated_at": "2026-11-01T13:00:00Z", "valid_until": "2026-11-04T13:00:00Z",
 "licence": "CC0-1.0", "policy": "https://feeds.example.org/v1/README.txt",
 "entries": [
   {"value": "198.51.100.9", "family": 4, "tier": "high", "categories": ["payload-host"],
    "tags": ["mirai"], "families": ["mirai"], "ports": [],
    "first_seen": "2026-10-26", "last_seen": "2026-10-31", "expires": "2026-11-14", "reported": true},
   {"value": "203.0.113.50", "family": 4, "tier": "standard", "categories": ["c2"],
    "tags": [], "families": ["mirai"], "ports": [23],
    "first_seen": "2026-10-28", "last_seen": "2026-10-30", "expires": "2026-11-29", "reported": false}
 ]}

• Inbound entries add services.
• Consumers must ignore unknown fields; feeds differ from spn/1 here, because they are public and long-lived.
• Producers add fields only additively within /v1/.

6.4 index.json

{"schema": "selfhostr-feeds/1", "serial": 4812,
 "generated_at": "2026-11-01T13:00:00Z", "valid_until": "2026-11-04T13:00:00Z",
 "licence": "CC0-1.0", "key_id": "8C1D2E3F4A5B6C7D", "next_key_id": null,
 "lists": [{"id": "outbound", "direction": "outbound", "tier": "standard", "entries": 812, "v4": 790, "v6": 22,
            "files": [{"name": "outbound.txt", "format": "txt", "bytes": 12034,
                       "sha256": "<64 hex>", "modified": "2026-11-01T13:00:00Z"}]}],
 "other": [{"name": "stats.json", "bytes": 4210, "sha256": "<64 hex>"}]}

valid_until = generated_at + HUB_FEED_VALID_DAYS (3). A file that did not change keeps its modified.

6.5 Signatures (minisign)

• Key. A dedicated Ed25519 feed key, not the spn/1 hub key (separation: a feed-key leak cannot impersonate the hub to nodes). Made by python -m hub.admin gen-feed-key. The private key is a Docker secret (HUB_FEED_SIGNING_KEY_FILE). The public key goes out as a standard minisign public key (RW...): published in docs/feeds.md, in README.txt, and configured on nodes as NETWORK_FEED_KEY.
• Format. minisign, algorithm ED (Ed25519 over BLAKE2b-512 of the file), implemented in netproto/feeds/minisign.py with cryptography and hashlib.blake2b.
• Trusted comment. timestamp:<unix> file:<name> serial:<serial>. Verifiers check that file matches the fetched name and that serial matches index.json. Golden vectors in netproto/tests/feed_vectors/; CI also verifies generated files with the upstream minisign binary (pinned container).
• Rotation. next_key_id is announced in index.json ≥ 30 days before the switch. During the overlap, files are signed by both keys (<file>.minisig and <file>.next.minisig).
• Consumer check: minisign -Vm outbound.txt -P <public key>; documented in docs/feeds.md with a 10-line fetch-and-verify shell script for mirrors (scripts/feeds/fetch-verify.sh).

6.6 Publish (atomic, cache-friendly)

feeds/publish.py renders every file into a temporary directory on the same volume. For each file whose sha256 differs from the published one, it writes <name>, <name>.gz (gzip level 9, mtime = 0) and <name>.minisig. Unchanged files are not touched, so their mtime, and with it Caddy’s ETag and Last-Modified, stays stable. index.json and its signature are renamed last: a consumer that reads the index never sees an older file than it lists. Old temporary directories are removed.

7. Hub changes

7.1 Layout (additions)

hub/
  Caddyfile                      # + site {$HUB_FEEDS_DOMAIN}: file_server precompressed gzip, /v1/delist* and /v1/lookup → hub-api
  hub/feeds/
    select.py                    # §4.2–4.4, pure over loaded rows
    filter.py                    # §5.3 exclusions, feed_allow
    external.py                  # CDN range and Tor exit fetches (hub-worker only, fixed host allow-list)
    lifecycle.py                 # hold, expiry, removal, caps, breaker
    publish.py                   # render (netproto), sign, atomic write
    stats.py                     # feed_daily, stats.json
    job.py                       # run_once(), called by worker.py
  hub/routes/delist.py lookup.py
  hub/data/never.yaml ranges.yaml
  hub/static/delist.html README.txt.tmpl   # copied to the feeds volume by the generator
  migrations/versions/h0002_feeds.py

7.2 Compose

• New external volume ${HUB_VOLUME_PREFIX:?}_feeds: mounted read-write at /srv/feeds in hub-worker, read-only in caddy.
• Caddy:
  • new site {$HUB_FEEDS_DOMAIN}, same log discard and header limits;
  • request_body max_size 64KB;
  • only GET/HEAD on files and POST on /v1/delist*;
  • /v1/delist* and /v1/lookup proxied to hub-api.
• hub-api gains no write access to the volume.
• hub/README.md documents host-level per-source connection limits (nftables ct count, no logging) for the feed host.

7.3 Migration h0002_feeds (hub database)

ALTER TABLE targets ... -- kind check gains 'payload_ip'
target_hosts(url_target_id text REFERENCES targets ON DELETE CASCADE, host_target_id text REFERENCES targets ON DELETE CASCADE,
      PRIMARY KEY(url_target_id, host_target_id))           -- from payloads[] (url_b ↔ host_ip / host_b), for takedown removal
feed_entries(list text, value text, family smallint, tier text, state text,  -- pending|listed|removed
      categories text[], services text[], tags text[], families text[], ports int[],
      first_seen date, last_seen date, expires date, reported bool,
      eligible_at timestamptz, listed_day date, removed_day date, removed_reason text,
      target_id text REFERENCES targets ON DELETE SET NULL, PRIMARY KEY(list, value))
feed_runs(serial bigint PK, run_at timestamptz, state text,          -- published|held|discarded|failed|unchanged
      counts jsonb, diff jsonb, error_code text)                     -- diff kept while held, then nulled
feed_daily(day date, list text, entries int, added int, removed_expired int, removed_taken_down int,
      removed_delisted int, removed_excluded int, PRIMARY KEY(day, list))
feed_allow(value text PK, kind text, reason text, note text, added_day date, expires_day date NULL, source text) -- admin|delist
feed_auto_ranges(source text, prefix text, fetched_day date, PRIMARY KEY(source, prefix))
delist_challenges(challenge bytea PK, difficulty int, expires_at timestamptz)
delist_requests(id text PK,                -- 'd_' + 8 random bytes hex
      token_hash bytea NOT NULL,           -- sha256 of a 32-byte status token shown once
      value text NOT NULL, list text NULL, category text NOT NULL, evidence text NULL CHECK (length(evidence) <= 2000),
      contact text NULL CHECK (length(contact) <= 254),
      state text NOT NULL,                 -- received|suppressed|accepted|rejected
      outcome text NULL,                   -- false_positive|remediated|policy|rejected
      public_note text NULL, created_day date NOT NULL, decided_day date NULL)

• No inet columns: values are text, like M29, so the M29 privacy test stays strict.
• delist_requests holds no client address, only days.
• Roles: hub_api gets INSERT/SELECT on delist_challenges and delist_requests, and SELECT on feed_entries (lookup). hub_worker gets the rest.

7.4 Retention (added to retention.py)

Data	Kept
feed_entries in state removed	90 days after removed_day
feed_runs	30 days (diffs nulled when released)
feed_daily	Forever (aggregates, needed for stats.json)
delist_requests.evidence, .contact	Nulled 30 days after decided_day
delist_requests rows	Deleted 180 days after creation once decided; an open request is kept until its decision, its evidence and contact nulled after 180 days
delist_challenges	10 min
feed_allow	Until expires_day
feed_auto_ranges	Replaced on every successful fetch. A failed fetch keeps the last good copy; a copy older than 30 days stops every feed run (ranges_stale), fail closed

7.5 Admin CLI (python -m hub.admin, additions; each writes hub_audit)

• gen-feed-key;
• feeds status|diff|release|discard|pause|resume|rebuild;
• allow add <value> --reason --expires --note, allow remove|list;
• delist list|show <id>, delist accept <id> --as false_positive|remediated|policy --note, delist reject <id> --note;
• ranges refresh.

7.6 Configuration (hub .env.example)

Variable	Default
HUB_FEED_MODE	off (review | auto)
HUB_FEEDS_DOMAIN	feeds.example.org
HUB_FEED_SIGNING_KEY_FILE	/run/secrets/feed_key
HUB_FEED_INTERVAL_MIN / _HOLD_HOURS / _VALID_DAYS	60 / 6 / 3
HUB_FEED_TTL_INBOUND_DAYS / _PAYLOAD_DAYS / _C2_DAYS	7 / 14 / 30
HUB_FEED_MAX_INBOUND / _OUTBOUND / _DOMAINS	50000 / 20000 / 20000
HUB_FEED_BREAKER_ADD_PCT / _REMOVE_PCT / _MIN	50 / 50 / 200
HUB_FEED_NEVER	empty
HUB_EXTERNAL_RANGES_ENABLED	true (CDN and Tor lists; with false, feeds stay off)
HUB_DELIST_POW_DIFFICULTY	20
HUB_DELIST_DAILY_CAP / HUB_DELIST_AUTO_SUPPRESS_DAILY	200 / 50

8. Who can download

8.1 Access

• Public, no account, no token, no API key, no cookies, no referrer checks.
• Same files for everyone, members included. Members get no hidden or earlier list (§10.6).

8.2 Privacy of downloaders

• Caddy logs nothing, as in H1.
• No query parameters, so no per-user URLs.
• No JS on list files; the delisting page has inline JS only, no third-party resource.
• No CDN at launch. Static files with 304s are cheap to serve (a 50,000-line list is about 750 KiB, about 250 KiB gzipped).
• If traffic later needs a CDN, it is an owner decision recorded in hub/README.md, and README.txt must then say “requests pass through , which may log addresses”.
• Third-party mirrors are welcome: signatures make them safe.

8.3 Licence: CC0 1.0 (recommended)

	CC0 1.0	CC BY 4.0
Use in firewalls, products, other lists	No conditions	Attribution required wherever the data is redistributed; firewall vendors and list aggregators often cannot comply cleanly
Enforcement	None needed	Needs an identifiable licensor to enforce; the project is deliberately anonymous
EU database right (sui generis)	Waived	Licensed, with attribution conditions
Precedent	abuse.ch URLhaus and ThreatFox data are CC0	Some lists
Warranty disclaimer	Included (§4 of CC0)	Included

Recommendation: CC0 1.0.

• README.txt on the feed site points to the official CC0 1.0 legal code (https://creativecommons.org/publicdomain/zero/1.0/legalcode); no copy of the legal text is published.
• README.txt adds a non-binding request: “please credit the SuckerPot network if convenient”.
• The “no warranty” statement and the accuracy statement (§11) also go in README.txt.
• The code licence of the repo is unchanged.

9. Delisting

9.1 Requests

• Form. https://feeds.example.org/v1/delist.html: a static page with vanilla JS. It fetches a challenge, solves the PoW in a Web Worker (difficulty 20, about a second), then posts the request.

• API (documented for curl):

  Method	Path	Body → answer
  POST	/v1/delist/challenge	{} → {challenge, difficulty, expires_in: 600}
  POST	/v1/delist	{challenge, pow_nonce, value, list?, category, evidence?, contact?} → {id, status_token, state, listed: bool, sla_hours: 72}
  GET	/v1/delist/{id}	Header Authorization: Bearer <status_token> → {id, state, outcome, public_note, created_day, decided_day}
  GET	/v1/lookup?value=198.51.100.9	→ {value, lists: [{list, tier, categories, first_seen, last_seen, expires}], allowed: bool}. The same data as the public files, rate-limited

• Fields:

  • category ∈ i_operate_this, shared_infrastructure, cleaned_up, address_reassigned, other;
  • evidence is free text, ≤ 2,000 characters, stored and never rendered as HTML;
  • contact is an optional e-mail address, used only to answer and deleted 30 days after the decision.

• Pseudo-anonymous status: the status_token (shown once) is the only way to check a request.

• Limits: M29 ratelimit.py (10/min, 100/day per HMAC’d address), HUB_DELIST_DAILY_CAP globally, and one open request per value. A request for an unlisted value answers listed: false and stores nothing.

• Mail: requests may also go to the project mailbox (M29), handled with the same CLI.

9.2 Interim effect

List of the value	Effect at once
inbound, inbound-high	suppressed: removed from the next run until decision, within HUB_DELIST_AUTO_SUPPRESS_DAILY (50/day). Rationale: a wrongly suppressed bot address costs little for 72 h; an innocent listed address costs a lot
outbound, outbound-domains	None. These lists are the ones attackers most want to escape. Requests are reviewed first (fast-tracked when the category is address_reassigned and there was no sighting for 3 days)

Scope (security review H1, migration h0006): suppression is decided from every list that holds the value, whatever list the request names, and a suppressed value is hidden from inbound only. Each request stores a scope: the list it named, or inbound when that was the only list holding the value, or NULL (every list) otherwise. An accepted request writes its scope into feed_allow.list, so an inbound decision never removes the same address from outbound.

9.3 Decisions and SLA

• Target: a decision within 72 h (README.txt states it). A suppressed inbound value stays suppressed until the decision, even past the SLA (fail toward delisting).

• Outcomes:

  Outcome	Effect on the list	Effect on node reputation
  false_positive (the observations were wrong: a spoof path we missed, shared infra, a bug)	Removed, feed_allow (false_positive, no expiry), M29 target → disputed	M29 §8.3: −0.1 for each contributor still linked (target_contributors live ≤ 30 days, M29 H3). Not extended: keeping links longer would weaken M29’s seizure protection (A4). After 30 days, no penalty; stated
  remediated (cleaned up, or the address was reassigned)	Removed, feed_allow (remediated, 90 days); the target is not disputed	None: the observations were true
  policy (should never be listed: a CDN or resolver missing from the lists)	Removed, feed_allow (policy, no expiry), added to the M29 never-list	None
  rejected	Stays; public_note explains (for example “malware served on 2026-10-30, still listed by URLhaus”)	None

• Nodes are never told which target cost them reputation. standing stays coarse (M29).

10. Node side

10.1 Configuration

Variable	Default	Meaning
NETWORK_FEED_ENABLED	false	Fetch the public feed. Independent of membership: non-members can turn it on
NETWORK_FEED_URL	empty	https://feeds.example.org/v1 placeholder. Empty = not configured (until the project identity exists, as in M29 §12.1)
NETWORK_FEED_KEY	empty	minisign public key (RW...)
NETWORK_FEED_INTERVAL_S	3600	Cycle
NETWORK_FEED_LISTS	inbound,outbound,outbound-domains	Lists stored. The high lists are subsets; the tier is in each row

Passed to ingest (all) and api (NETWORK_FEED_ENABLED, and the URL, because the UI shows public URLs to copy; the URL is public).

10.2 Invariants (extend M29 N1–N10)

#	Invariant
N11	No feed request unless NETWORK_FEED_ENABLED=true and URL and key are set.
N12	Every feed request goes through suckerpot_egress (gate open, VPN proxy), GET only, only to the host of NETWORK_FEED_URL, HTTPS, no redirects, no cookies, Accept-Encoding: identity, generic User-Agent: spn-feed/1. Nothing about the node is sent; no query string.
N13	Data is accepted only if index.json verifies with NETWORK_FEED_KEY, the trusted comment names index.json, the serial is ≥ the stored one (rollback refused, feed_rollback), valid_until > now, and each list file’s sha256 matches the signed index.
N14	Parsed strictly on known fields (unknown fields ignored): single global unicast addresses or valid host names only (no prefix), enums, dates, ≤ 16 MiB per file, ≤ the list caps in §5.5. One invalid entry drops that entry; > 1 % invalid rejects the file (feed_invalid). Nothing pulled is executed or contacted.
N15	After valid_until with no fresh index, the stored entries are kept for display but marked stale, and the local mirror serves empty lists (fail toward not blocking with stale data).

10.3 Migration 0029_network_feed

network_feed_state(id smallint PK CHECK (id=1), serial bigint, generated_at timestamptz, valid_until timestamptz,
  key_id text, index_sha256 text, files jsonb,          -- {name: {sha256, etag}}
  last_fetch_at timestamptz, last_ok_at timestamptz, error_code text, updated_at timestamptz NOT NULL DEFAULT now())
network_feed_entries(list text, value text, family smallint, tier text, categories text[], services text[],
  tags text[], families text[], ports int[], first_seen date, last_seen date, expires date, reported bool,
  seen_here bool NOT NULL DEFAULT false, PRIMARY KEY(list, value))
CREATE INDEX ON network_feed_entries (value);

Grants:

• suckerpot_ingest: all on both tables.
• suckerpot_api: SELECT on both.

Retention: entries are replaced on each fetch. When NETWORK_FEED_ENABLED becomes false, the worker deletes both tables’ rows on its next idle cycle (feed_disabled_wiped).

Leaving the M29 network does not delete the feed (it is public); the M29 N8 wipe stays limited to network_* membership tables, and the network_feed_* tables are explicitly excluded.

10.4 Worker (ingest/ingest/network/feed.py, feed_client.py)

A separate FeedWorker (IntelWorker shape), wired in ingest/ingest/main.py, with status row network_feed in enrichment_status. Each cycle:

1. Check N11 and the gate (egress_closed).
2. Conditional GET index.json (If-None-Match), then GET index.json.minisig. Verify (N13).
3. For each configured list whose .json sha256 changed: GET <list>.json, verify the sha256, parse (N14), and replace that list’s rows in one transaction.
4. seen_here: true when the value is a src_ip of a non-demo, non-benign session in the last 30 days, or a host in indicators, fetch_targets or sample_links in the last 30 days. Local SQL only.
5. Update network_feed_state.

Errors: not_configured, egress_closed, bad_signature, feed_rollback, feed_expired, feed_invalid, too_large, http_<code>.

M29 §9.5 (network-confirmed +2 in local scores) is unchanged. The feed does not change local M16 scores: the local list stays “what my sensor saw”.

10.5 Node API

Method	Path	Who	Answer
GET	/api/v1/network/feed/status	any user	{enabled_env, configured, state: "disabled"|"never"|"ok"|"stale"|"error", error_code, serial, generated_at, valid_until, last_ok_at, signature: "verified"|null, member: bool, public_base_url, lists: [{id, direction, entries, high, seen_here, helped|null, files: [{name, format, url}]}], local_mirror: {feeds_enabled}}. files and url are built from the spec’s fixed file names, so they exist even when fetching is off. helped = entries whose value is in network_targets with mine (members only)
GET	/api/v1/network/feed/entries?list=&tier=&q=&seen_here=&cursor=&limit=	any user	Paginated rows, cursor as in api/api/cursor.py
GET	/feeds/{token}/network/{list}.{txt|csv}, /feeds/network/{list}.{txt|csv}	M16 feed token whose kinds include network	Verified local mirror: the stored entries of {list} (-high = tier filter), minus the M16 always-allow overrides (an override protects your own hosts from the network list), validated again with normalize_value. Empty when stale. Same rate limits and 404 rules as M16

feed_tokens.kinds accepts the new value network (validated in api/api/feed_tokens.py; there is no DB constraint to change). setup_status.py gets an item network_feed: off, ok, or warning with the error code.

10.6 Web: Blocklists page, tab “Network”

Tabs. /defend/blocklists gets Tabs “This sensor” (the existing page, unchanged) and “Network” (/defend/blocklists?tab=network), following the Settings ?tab= pattern.

Feed off or not configured. An EmptyState explains three things:

• what the lists are;
• that firewalls can fetch them directly from the public URLs, without this dashboard (shown with copy buttons when a URL is configured);
• how to let the dashboard fetch them too (just set NETWORK_FEED_ENABLED=true).

No hub contact happens until the flag is on.

Feed on.

• Sentence first: “The SuckerPot network lists 812 malware and C2 hosts and 14,230 attack sources, each confirmed by at least 5 independent sensors. 41 of them also reached this sensor.” Then a status line: “Updated 13:00, signature verified” (or a warning chip for stale, bad_signature and similar).
• One card per list (inbound, outbound, outbound domains):
  • icon and direction label (“Blocks traffic leaving your network”), never colour alone;
  • entries and high-tier counts (StatTile, tabular numbers);
  • a one-line purpose; the outbound card says “Start here”;
  • a “Copy URL” menu per format (plain, IPv4, IPv6, nftables, ipset, MikroTik, CSV, JSON, STIX);
  • “Use the verified copy from this dashboard” (the M16 token flow, admin only) when FEEDS_ENABLED.
• Entries table (virtualised, filters in the URL: list, tier, seen_here, q):
  • IPChip (opens the local attacker page when seen_here);
  • categories and families (TagBadge);
  • first seen, last seen, expires;
  • “Seen here” badge; “You helped” badge (members only).
• “How to use” drawer: short OPNsense and pfSense steps with a link to docs/feeds.md.
• “Why join the network?” card, honest wording: “These lists are free and public. Every firewall gets the same lists, whether or not you join. Joining adds your sensor to the ones that confirm entries, so the lists get faster and more accurate, and you see which entries your sensor helped confirm.” It links to /network. No exclusive perks are promised.

Rules. CLAUDE.md §4 applies:

• no new hue (outbound uses the existing severity token only for the malware category badge, as today);
• WCAG AA, keyboard;
• mocks in web/src/mocks/network-feed.ts with documentation IPs;
• help.ts entries; glossary terms “inbound list”, “outbound list”, “high tier”, “k-support”.

11. Legal and accuracy

• Wording. In README.txt and every rich format’s policy: “Each entry is an address or host name from which at least five independent sensors observed the listed behaviour between first_seen and last_seen. A listing is a factual observation, not an accusation against the holder of the address; many listed devices are themselves compromised. Data is provided as is, with no warranty.”
• Minimisation. No reverse DNS, no WHOIS holder, no URLs, no sample content, no free text in lists.
• Personal data. IP addresses can be personal data (EU GDPR; Swiss revFADP). Basis: legitimate interest in network and information security (GDPR Art. 6(1)(f), Recital 49). Retention is short (TTLs, 30-day removed.json, delisting data deleted on the §7.4 schedule). Objection is handled through delisting. Lookups are possible (§9.1).
• SLA. Acknowledgement at once (ticket), decision target 72 h, interim suppression for inbound lists. The SLA, the outcome rules and the monthly delisting counts are public (stats.json).
• Owner action before HUB_FEED_MODE leaves off: a short legal review of the wording, operator jurisdiction and the anonymous operator identity (the M29 §12.1 identity question also covers the delisting mailbox). This design is not legal advice.

12. Public statistics: stats.json

Read by the project site later (CORS *), regenerated with each run, signed like every file.

{"schema": "selfhostr-feed-stats/1", "generated_at": "2026-11-01T13:00:00Z", "serial": 4812, "licence": "CC0-1.0",
 "lists": [{"id": "inbound", "direction": "inbound", "entries": 14230, "high": 3120, "v4": 14002, "v6": 228},
           {"id": "outbound", "direction": "outbound", "entries": 812, "high": 301, "v4": 790, "v6": 22,
            "payload_hosts": 655, "c2": 157},
           {"id": "outbound-domains", "direction": "outbound", "entries": 96, "high": 40}],
 "daily": [{"day": "2026-10-31", "list": "outbound", "entries": 812, "added": 37,
            "removed": {"expired": 12, "taken_down": 9, "delisted": 0, "excluded": 1}}],
 "takedown": {"outbound_removed_taken_down_30d": 214, "median_days_listed_until_takedown": 5},
 "delisting": {"requests_30d": 6, "accepted_30d": 2, "rejected_30d": 3, "open": 1,
               "median_hours_to_decision": 20},
 "exclusions": {"tor": 312, "cdn": 44, "never_list": 18, "allow_list": 7},
 "network": {"members": 37, "active_7d": 29, "countries": 9}}

Rules:

• daily covers 90 days per list.
• network copies the M29 network_daily aggregates and the M29 k-rule (countries only counts countries with ≥ 3 members, and is null below 3).
• No value appears in stats.json.

13. Consumer documentation (docs/feeds.md, exact steps)

Menu names follow OPNsense 25.x and pfSense CE 2.7/Plus 24.x; the guide says to check the version. URLs use the placeholder base.

13.1 OPNsense

1. Firewall → Aliases → +.
   • Name selfhostr_outbound;
   • Type URL Table (IPs);
   • Refresh Frequency: 0 days 6 hours;
   • Content https://feeds.example.org/v1/outbound.txt;
   • Description “SuckerPot outbound (CC0)”.
   • Save, Apply.
2. Repeat for selfhostr_inbound with inbound-high.txt (start with the high tier).
3. Check Firewall → Settings → Advanced → Firewall Maximum Table Entries: it must exceed the sum of all tables (the lists are ≤ 50,000 entries each).
4. Outbound rule. Firewall → Rules → LAN (repeat per internal interface, or one Floating rule):
   • Action Block (or Reject, which ends faster for clients);
   • Direction in; TCP/IP Version IPv4+IPv6; Protocol any;
   • Source “LAN net”; Destination alias selfhostr_outbound;
   • Log ✓ (a hit means an infected device);
   • Description.
   • Move it above the default allow rule. Save, Apply.
5. Inbound rule (only if you publish services). Firewall → Rules → WAN:
   • Block, Source selfhostr_inbound, Destination any;
   • placed above the rules created by port forwards.
6. Check. Firewall → Diagnostics → Aliases → select the alias: the entry count matches index.json.
7. Put pass rules for your own critical hosts (backup target, VPN peers) above the block rules.

13.2 pfSense

1. Firewall → Aliases → URLs → Add.
   • Type URL Table (IPs);
   • Name selfhostr_outbound;
   • URL https://feeds.example.org/v1/outbound.txt;
   • Update frequency 1 (day).
   • Save, Apply Changes.
2. System → Advanced → Firewall & NAT → Firewall Maximum Table Entries: large enough.
3. Firewall → Rules → LAN → Add (top).
   • Action Block; Address Family IPv4+IPv6; Protocol any;
   • Source LAN subnets; Destination “Single host or alias” selfhostr_outbound;
   • Log.
   • Save, Apply.
4. Inbound: a WAN rule with Source selfhostr_inbound, above the NAT-associated pass rules.
5. Faster refresh with pfBlockerNG (package):
   • Firewall → pfBlockerNG → IP → IPv4 → Add;
   • Name Selfhostr_outbound;
   • Source Definitions: Format Auto, State ON, Source https://feeds.example.org/v1/outbound.v4.txt, Header selfhostr_outbound;
   • Action Deny Outbound; Update Frequency Every 1 hour;
   • Save.
   • Same in IPv6 with outbound.v6.txt.
   • Inbound lists use Action Deny Inbound.
   • Domains: DNSBL → DNSBL Groups → Add, source outbound-domains.txt.
   • Run Firewall → pfBlockerNG → Update → Force Update.

13.3 Others

• nftables. A systemd timer runs curl -fsS -o /etc/nftables.d/selfhostr-outbound.nft …, the minisign verification, then nft -f. An example chain lives in the same inet selfhostr table:

  table inet selfhostr {
    chain fwd {
      type filter hook forward priority -5;
      ip daddr @outbound_v4 counter reject
      ip6 daddr @outbound_v6 counter reject
    }
  }

  Sets are table-scoped, so rules in another table cannot use them.

• ipset/iptables. Run ipset restore -f outbound.ipset, then add iptables -I FORWARD -m set --match-set selfhostr-outbound-v4 dst -j REJECT and the ip6tables equivalent.

• MikroTik. /tool fetch url=… dst-path=selfhostr.rsc followed by /import selfhostr.rsc, run every 6 h by a /system scheduler, plus a forward-chain drop rule on dst-address-list=selfhostr-outbound.

• MISP. Sync Actions → Feeds → Add:

  • format Freetext, URL outbound.txt (or Simple CSV with outbound.csv, value column 1);
  • or import outbound.stix.json.

• DNS filters. Pi-hole and AdGuard Home take outbound-domains.txt as a blocklist URL.

• Verifying. Use minisign -Vm <file> -P <key> and scripts/feeds/fetch-verify.sh, which keeps the last good copy when verification fails (for a local mirror that firewalls then fetch).

14. Tests

netproto/tests/test_feeds*.py:

• Golden files for every renderer (txt, v4/v6, nft with an empty v6 set, ipset, rsc, csv with formula cells, json, STIX validated with stix2-validator).
• Values that must be refused: a prefix, 0.0.0.0/0, private, documentation without the flag, wildcard and parent domains.
• minisign sign/verify vectors (ED, trusted comment).
• Parser: unknown fields ignored, over-cap, more than 1 % invalid.

hub/tests/test_feeds_select.py:

Case	Expected
Confirmed ip with login_success from 5 nodes	inbound standard
Same, 10 nodes, 3 days, 3 cohorts, corroborated	inbound-high too
Confirmed ip, scan/connect only	not listed
Confirmed ip, evidence only from sip	not listed
Payload host fetched by 1 node with a family	not listed; by 2 nodes: outbound
URL typed in commands by 50 sources, never fetched with a family, no URLhaus	not listed
C2 from one sample	not listed; 2 samples from 2 nodes: listed
Address in a CDN range, Tor exit, NEVER_IPS, feed_allow	never listed, counted in exclusions
One node re-sights a listed address on day D	last_seen, services, expires unchanged (k-support)
Inbound TTL passes with no k-supported sighting	removed(expired)
All URLs of a payload host taken_down	removed(taken_down) at the next run
Removed value re-sighted before its removal day	stays removed
Over cap	high tier kept first

hub/tests/test_feeds_publish.py:

• Atomic order (index last);
• unchanged files keep their mtime;
• .gz sidecars deterministic;
• breaker holds at +60 % and releases through the CLI;
• an 8 MiB overflow fails closed;
• HUB_FEED_MODE=off writes nothing.

hub/tests/test_feeds_privacy.py:

• F5 scan: no rendered file contains any node id, fingerprint, pseudonym, target id, country or timestamp other than generated_at/valid_until;
• the Caddyfile feeds site has output discard and no log sink;
• no inet column;
• external.py only contacts hosts from ranges.yaml.

hub/tests/test_delist.py:

• PoW, caps, one open request per value, listed: false stores nothing;
• interim suppression for inbound only, capped per day;
• each outcome’s effect on feed_allow, target state and reputation (penalty only for false_positive, only for live contributor links);
• evidence and contact nulled after 30 days;
• status token required.

hub/tests/e2e (extends M29):

• after the 7-node scenario, run the feed job and fetch through Caddy (no TLS in the test overlay): curl twice gives 304 the second time, Accept-Encoding: gzip returns the precompressed body;
• minisign -V in a pinned container verifies index.json and outbound.txt;
• a delisting round-trip works.

ingest/tests/network/test_feed*.py:

• N11 idle without the flag (a transport that fails on any call);
• N12 gate closed means no call, the host must equal the feed host, no redirects, identity encoding;
• N13 bad signature, rollback, wrong file name in the trusted comment, sha256 mismatch, expired index → old data kept and an error code;
• N14 prefixes and private values dropped, over 1 % invalid rejected, 16 MiB cap;
• N15 stale handling;
• seen_here excludes demo and benign;
• disabling wipes the rows;
• an M29 leave does not wipe the feed.

api/tests/test_network_feed.py:

• Roles;
• status URLs built without fetching;
• local mirror: token kind network required, allow overrides removed, empty when stale, 404 rules and rate limits as in M16.

Web: vitest for the tab states (off, never, ok, stale, bad signature), the copy menu, the filters in the URL and the honesty card; Playwright screenshots of /defend/blocklists?tab=network (off and on), desktop and mobile, dark and light.

15. M29 amendments (needed by M30, owner to confirm)

1. k-support for member-visible target fields. /v1/targets last_day, services, activity, tags and families should use the §4.4 rule. Today one node’s sighting of an already-confirmed target moves them, and an attacker can probe one suspected address with an already-listed source. Same fix, same code (feeds/select.py exposes k_supported()).
2. payload_ip kind (hub-internal in h0002; added to the node enum before the hub sends it).
3. Tor exits: no hub abuse reports to Tor exit relays (same feed_auto_ranges data).
4. N8 scope: the M29 leave wipe covers membership tables only, not network_feed_*.

16. Task split

Wave 0 publishes the contract (§6, §10.5 JSON). Wave 1 and 2 tasks run in parallel in worktrees or on the strictly separate files listed. Placeholders only: documentation IPs and example.org.

Task	Agent / model	Owns	Depends on	Wave
F0. Feed format library and spec	backend-engineer, sonnet (opus review of minisign)	netproto/suckerpot_netproto/feeds/** (models.py, render.py, minisign.py, parse.py), netproto/tests/test_feeds*.py, netproto/tests/feed_vectors/**, docs/network-protocol.md (part “Feeds”)	M29 T0 merged	0
F1a. Hub generator and publication	backend-engineer, sonnet	hub/hub/feeds/{select,filter,lifecycle,publish,stats,job}.py, hub/hub/data/never.yaml, hub/migrations/versions/h0002_feeds.py, hub/hub/models.py (feed models only), hub/hub/worker.py (one call to feeds.job.run_once), hub/hub/retention.py (feed rows), hub/tests/test_feeds_{select,publish,privacy}.py, hub/tests/test_never_sync.py	F0	1
F1b. Hub external ranges, compose, Caddy	backend-engineer, sonnet	hub/hub/feeds/external.py, hub/hub/data/ranges.yaml, hub/Caddyfile (feeds site), hub/docker-compose.yml, hub/docker-compose.test.yml (feeds volume, minisign verify container), hub/.env.example (feed section), hub/tests/e2e/test_feeds_e2e.py	F1a DDL	1
F1c. Delisting, lookup, admin CLI	backend-engineer, sonnet	hub/hub/routes/{delist,lookup}.py, hub/hub/main.py (router include), hub/hub/admin.py (feeds, allow, delist, ranges, gen-feed-key subcommands), hub/static/delist.html, hub/static/README.txt.tmpl, hub/tests/test_delist.py	F1a DDL	1
F2. Node migration	backend-engineer, sonnet	db/migrations/versions/0029_network_feed.py, db/suckerpot_db/models.py (two models), db/suckerpot_db/roles.py	M29 0028	1
F3. Node feed worker	backend-engineer, sonnet	ingest/ingest/network/feed.py, ingest/ingest/network/feed_client.py, ingest/ingest/main.py (wiring), ingest/ingest/features.py (settings), ingest/tests/network/test_feed*.py, docker-compose.yml (ingest/api env only), .env.example (feed section)	F0, F2	2
F4. Node API and local mirror	backend-engineer, sonnet	api/api/network_feed.py, api/api/routes.py, api/api/feeds.py (network routes), api/api/feed_tokens.py (kind network), api/api/setup_status.py (item), api/tests/test_network_feed.py	F2	2
F5. Web tab	frontend-engineer, sonnet	web/src/pages/blocklists.tsx (tabs only), web/src/components/defend/network-feed/**, web/src/lib/network-feed.ts, web/src/mocks/network-feed.ts, web/src/lib/help.ts, web/src/lib/glossary.ts	§10.5 JSON (mocks first)	1 (mocks), 2 (wire)
F6. Docs	docs-writer, haiku	docs/feeds.md (new: lists, formats, OPNsense, pfSense, pfBlockerNG, nftables, ipset, MikroTik, MISP, DNS filters, verification, licence, delisting, privacy), scripts/feeds/fetch-verify.sh, docs/defend.md (Network tab), docs/network.md (feeds and the honest incentive), docs/SECURITY.md (§3 additions), docs/ARCHITECTURE.md (feed host in the outbound hosts table), docs/configuration.md, docs/api.md, docs/ui.md, hub/README.md (operator: feed key, modes, breaker, delisting SLA, CDN decision, connection limits), CHANGELOG.md	all	3

Contracts that must not drift:

• §6 (file names and formats, F0 owns);
• §7.3 DDL (F1a);
• §10.3 DDL (F2);
• §10.5 JSON (F4 owns; F5 mocks match it, checked by a vitest against the type in lib/network-feed.ts).

Reviews:

• security-reviewer (opus) on F0, F1a, F1b, F1c, F3, F4, against §3 (F1–F8, N11–N15), poisoning, signatures, privacy of members and downloaders;
• ui-designer on F5;
• qa-runner: just check, just guardrails, just test-integration, the hub integration job including the minisign verify.

17. Risks and open questions

1. Project identity (M29 §12.1): the feed domain, the feed key and the delisting mailbox ship empty until the owner sets them up; HUB_FEED_MODE=off.
2. Bootstrap: until 5 eligible members exist, every list is empty. README.txt and the UI say so with the current member count.
3. CDN range sources change format sometimes. The fetcher fails closed (§7.4) rather than publish without exclusions.
4. Large consumers may want a CDN. That is an owner decision with a privacy statement change (§8.2).
5. IPv6 inbound: single /128 entries are weak against attackers who rotate within a /64. Prefix listing is deliberately out of v1 (innocent neighbours); revisit with data.
6. Not in v1: URL and sha256 lists, RPZ and hosts formats, a MISP native feed (manifest format). Candidates for later, after the delisting process has run for a while.
7. Legal review before going live (§11).