Lua Internals

This page collects what the comments in ids/reverse_proxy_enhanced/lua/ say about how the modules behave at run time: where state is kept, which timers run, the tuned constants and the bugs behind them. The module table on the Architecture page lists the pure/adapter pairs; the routing stages are in Request Pipeline.

Nginx Phases

Phase	File	What runs
init_by_lua_file (master, once)	init.lua	Builds _G.config, _G.utils and _G.redis_pool, seeds the shared dicts, and generates the session signing key before the workers fork, so every worker shares it.
init_worker_by_lua_file (each worker)	init_worker.lua	Seeds the random generator, loads the ready-pool list, warms backend connections after 3 s, and — in worker 0 only — starts the background timers.
access_by_lua_block (each request)	nginx.conf	Maintenance gate, then honeytoken detection (a hit sets the score to 100), then pattern_analyzer.analyze_request() → session_handler.get_session() → router.decide_route() → pool_router.get_or_assign_pool() → session_handler.update_session(), followed by sophistication_analyzer.classify_session(). The result sets $backend_upstream.

Globals and Shared Dicts

• _G.config — every tunable parameter, mostly overridable from environment variables set in Compose. Passwords must come from the environment.
• _G.utils — IP whitelist check (backed by ip_rules.lua, which handles IPv4, IPv6 and IPv4-mapped IPv6), UUIDs, risk-score sum, security-event logging.
• _G.redis_pool — get_connection() / close_connection() over a per-worker keepalive pool to session_store, with one reconnect retry.

Shared dict	Size	Holds
sessions	10 MB	Worker-local cache of session JSON, saving Redis round-trips.
threat_intel	10 MB	Flagged IPs and user agents, backend health, the ready-pool list (pool_list), mirrored replication and maintenance flags, the WordPress install state.
rate_limit	10 MB	Request-rate counters for automation detection.
honeypot_routes	1 MB	Cached session → pool assignments.

The hot path never waits on Redis for flags: anything an external process sets in Redis (replication, maintenance, database-layer pool health) is copied into threat_intel by a timer, and requests read only the shared dict.

Configuration Notes

Setting	Value	Why
redis.timeout / read_timeout	2000 / 5000 ms	Raised from 1000 / 3000. Sporadic read timeouts were host scheduling stalls under load (empty Redis slowlog), not slow Redis; failing a whole background cycle over a 1 s hiccup was worse than waiting.
session.cookie_name	SERVERID	Looks like HAProxy's stickiness cookie. The old HONEYPOT_SESSION gave the deception away in the blind pentest. The security log format in nginx.conf hardcodes $cookie_SERVERID.
session.max_idle_time	3600 s	Matches typical WooCommerce checkout session length.
session.signing_key	SESSION_SIGNING_KEY	Without it a random key is made at start-up: old cookies stop verifying after a restart (sessions are recovered by fingerprint) and a bad MAC is not scored as tampering.
session.recovery_window	900 s	How long after a session's last request a cookie-less client with the same fingerprint is recovered into it (SESSION_RECOVERY_WINDOW).
session.tamper_score	50	Added to a request whose cookie was edited or forged.
gdpr.timestamp_window / binding_ttl	300 / 3600 s	A replayed token stops matching once it ages out; a binding is prolonged on every match, so only an abandoned one expires.
threat.honeypot_threshold	80	Raised from 50 to stop false positives on busy legitimate REST clients (WooCommerce mobile app, Stripe webhooks).
threat.ip_reputation_cap	10	Address reputation alone can never divert a session; nothing at all is added once shared_ip_sessions (2) sessions used the address within shared_ip_window (3600 s).
threat.score_decay_half_life_seconds	300 s	Added after one clean request was found to un-stick a session that had just matched a CVE. THREAT_DECAY_ENABLED=false sets it to 0, which disables decay.
decay_offense_slowdown / permaflag_offenses	0.75 / 5	Each recorded offense multiplies the half-life by 1.75; at 5 offenses, or a peak at permaflag_score, decay stops (decay_policy.lua).

The operator whitelist (loopback, the Tailscale ranges and 10.0.0.0/8) gives a −20 reputation and is never diverted.

Background Timers

All periodic tasks run in worker 0 only. The health-check timer once ran in every worker; their checks fired at the same instant, so one moment of contention counted as several consecutive failures and marked a healthy backend UNHEALTHY. Moving it under the worker-0 guard fixed both the extra load and the false positive.

Every	Task
5 s	Refresh the ready-pool list from honeypot_pool:ready into the shared dict (first run after 1 s).
5 s	Mirror replication, maintenance and (database layer) pool-health flags from Redis into threat_intel.
15 s	Health-check production_backend and every ready pool.
30 s	Probe /wp-admin/install.php on production for the install state.
30 s	Read new Suricata eve.json alerts into threat_ips.
60 s	Clean up rate-limit counters.
300 s	Evict expired sessions from the local dict (session.cleanup_interval).
blacklist_refresh_hours	AbuseIPDB blacklist pull (first run after 15 s), only when ABUSEIPDB_BLACKLIST_ENABLED. The pull is cached in Redis, so restarts don't spend the daily quota.

Redis Key Schema

pool_router_rules.lua defines the keys pool_manager and the router share; the rest belong to the session and threat modules.

Key	Type	Meaning
honeypot_pool:ready	SET	Pool numbers provisioned and healthy.
honeypot_pool:free	ZSET	Ready pools with no owner (score = pool number).
honeypot_pool:owner:<N>	SET	Sessions assigned to pool N.
honeypot_pool:counter	INT	Round-robin counter for borrowing and sharing.
honeypot_pool:provision	LIST	"A session was just assigned" wake-ups for pool_manager.
honeypot_pool:capped	STR (TTL)	Set by pool_manager while no further pool may start (budget, POOL_MAX, host load).
honeypot_pool:waiting	ZSET	Sessions borrowing a pool until theirs is built (score = when they started waiting); at most 200, oldest dropped.
honeypot_pool:waiting_seen	HASH	Last request time of each waiting session; quiet ones are skipped at hand-off.
honeypot_pool:last_seen	HASH	Last time a request was routed to each pool, written at most every 10 s; drives the idle scale-down.
honeypot_pool:health	HASH	Database layer only: each pool database's container state, published by pool_manager.
honeypot_pool_session:<SID>	STR (24 h)	A session's pool assignment.
honeypot_pool_replicating:<N>	STR	Set by the content-replication script while pool N is being synced.
honeypot_maintenance	STR (TTL)	Set by the dashboard during a WordPress update.
fp_session:<fingerprint>	STR	Passive fingerprint → session, for recovery after cookies are cleared.
session_cookie_rejections	LIST	The last 1000 rejected (edited or forged) cookies.
threat_ips	STR (JSON)	Per-IP reputation; read-modify-write through threat_intel.lua, which also mirrors it into the shared dict.
honeytoken:<id>	STR (30 days)	Injected honeytokens and their use counts.

Pools 1–3 are declared in Compose with an nginx upstream honeypot_backend_N block; higher numbers (up to a sanity ceiling of 64) are created by pool_manager and reached by container DNS name. The names honeypot_eshop_N, honeypot_database_N and honeypot_backend_N must match Compose exactly.

To inspect these keys on a running stack, pass the password from .env (the session_store container does not have it in its environment):

cd ids
REDIS_PW="$(grep ^REDIS_PASSWORD= .env | cut -d= -f2-)"
rcli() { docker compose exec -T session_store redis-cli -a "$REDIS_PW" --no-auth-warning "$@"; }

rcli SMEMBERS honeypot_pool:ready              # healthy pools
rcli ZRANGE honeypot_pool:free 0 -1            # unowned spares
rcli ZRANGE honeypot_pool:waiting 0 -1 WITHSCORES
rcli GET honeypot_pool_session:<SID>           # one session's pool
rcli GET threat_ips                            # per-IP reputation (JSON)

Pool Assignment

ASSIGN_SCRIPT runs inside Redis with EVAL, so two nginx workers can never hand one free pool to two sessions. It returns a pool number and a mode:

1. existing / pending — the session already has a pool; its TTL is refreshed.
2. exclusive — a free ready pool becomes the session's own, and a wake-up asks pool_manager to build the next spare.
3. pending — no free pool but growth is allowed: borrow a ready pool round-robin and join the waiting queue (case A).
4. shared — growth is capped: share a ready pool round-robin for good (case B).
5. none — no ready pool at all.

The free-pool and fallback part of the script (pool_router_rules.lua); KEYS[2] is the free set, KEYS[3] the ready set, KEYS[4] the counter, KEYS[6] the cap flag and KEYS[7] the waiting queue:

while true do
  local popped = redis.call('ZPOPMIN', KEYS[2])
  if not popped or #popped == 0 then break end
  if redis.call('SISMEMBER', KEYS[3], popped[1]) == 1 then
    pool = tonumber(popped[1])
    mode = 'exclusive'
    break
  end
end
if not pool then
  local ready = redis.call('SMEMBERS', KEYS[3])
  if #ready == 0 then
    return {0, 'none'}
  end
  local nums = {}
  for i, v in ipairs(ready) do nums[i] = tonumber(v) end
  table.sort(nums)
  local c = redis.call('INCR', KEYS[4])
  pool = nums[((c - 1) % #nums) + 1]
  if redis.call('EXISTS', KEYS[6]) == 1 then
    mode = 'shared'
  else
    mode = 'pending'
    redis.call('ZADD', KEYS[7], tonumber(ARGV[4]) or c, ARGV[1])
    -- ... trimmed to max_waiting (200), oldest dropped
  end
end

Assignments are cached in honeypot_routes for 300 s, but a pending one for only 3 s, so the move to a session's own pool takes effect quickly. A cached pool that is no longer in the ready list is a cache miss (this fixed the 502s after scale-down). If the assigned pool is unhealthy or being replicated, the request goes to the next healthy pool without changing the stored assignment; if none is healthy it goes to the assigned pool and nginx returns the error. Missing or malformed health data counts as healthy.

Health Checks

• Probe /robots.txt every 15 s with an 8 s timeout; 5 failures mark a backend unhealthy, 2 successes healthy again. Previously 10 s / 5 s / 3: a post-start load burst flipped all three pools unhealthy on slow renders while Docker still reported them healthy. A truly dead backend is still caught within about 75 s.
• The timeout constant used to be declared but ignored (a hardcoded 2000 ms), so tuning it did nothing; a heavy page with 30+ concurrent assets once tripped it.
• Every internal probe sends the User-Agent honeypot-internal-healthprobe, which Suricata rules sid:1000056 and sid:1000060 exclude so the probes don't raise "High Rate HTTP Requests" or "Rapid Vulnerability Scanning". The string is mirrored in ids/suricata_rules/local.rules; change both. It is deliberately not the public resty.http default, which an adversary could spoof.
• Production's health is for monitoring only; nginx's own max_fails / fail_timeout handle it. Log lines name the containers behind an upstream, not just the upstream alias.

Suricata Ingestion

• Reads eve.json, not fast.log: the old regex had 8 captures into 7 variables, so src_ip received the classification text and alerts never reached the real IP.
• Resume position: a Redis error skips the cycle (instead of rescanning a 600 MB+ file inside worker 0's event loop); a missing key starts at the end of the file; a position past the end (truncated file) restarts at 0. One cycle reads at most 8 MB, and a half-written last line is left for the next cycle.
• A cheap substring check skips the many flow/netflow events before any JSON decode.
• Score per alert by Suricata severity: 1 → 50, 2 → 25, otherwise 10, capped at 100. The stored score only rises; decay is applied when it is read. The reason carries the matched signature through to the routing reason.
• Private and loopback src_ip values are container-to-container hops and are not written to threat_ips; the raw alert is still kept (last 1000).
• A missing or unreadable eve.json is logged at most once per 10 minutes rather than looking like "no alerts".

Session Identity

• Session ids are 16 bytes from OpenSSL's CSPRNG (previously a guessable SHA-1 of time, math.random and the worker PID). The cookie is <id>.<HMAC>, compared in constant time; a cookie with a bad MAC is never looked up.
• A cookie-less request is matched through fp_session:<fingerprint>: the TLS ClientHello (GREASE stripped), HTTP version, User-Agent, Accept-Language/-Encoding, the low-entropy client hints and, by default, the IP. The pointer is refreshed on every session update and lives recovery_window after the last request, so a neighbour behind the same NAT with an identical browser is merged only while the session is active.
• The GDPR signature runs alongside, never instead. It fails open: a missing key, a forged token or a Redis error just means no GDPR signal for that request. See GDPR Signature.

Maintenance and Install Gates

Maintenance (maintenance.lua): during a WordPress update the dashboard sets honeypot_maintenance with a TTL, the mirror timer copies it, and the first thing in the access phase serves a 503 page with Retry-After: 120 to production and honeypot alike. Only an explicit "1" counts as on, and the TTL brings the site back on its own if the dashboard dies mid-update. See WordPress Management.

The gate at the top of the access phase in nginx.conf:

local threat_intel = ngx.shared.threat_intel
local maintenance = require "maintenance"
if threat_intel and maintenance.should_serve_maintenance(threat_intel:get("maintenance")) then
    ngx.status = 503
    ngx.header["Content-Type"] = "text/html; charset=utf-8"
    ngx.header["Retry-After"] = maintenance.RETRY_AFTER
    ngx.header["Cache-Control"] = "no-store"
    ngx.say(maintenance.page_html(maintenance.RETRY_AFTER))
    return ngx.exit(503)
end

Install state (wp_install_state.lua): install.php is normally scored hard, but it is legitimate before the first-run wizard. Nginx has no MySQL driver, so the probe looks for WordPress's own "already installed" text. Only production is probed, since pool databases are copies of it. Any probe error keeps the previous value, and unknown or stale state counts as installed (strict), so an outage can never open the fast path.

Honeytokens

honeytoken_handler.lua plants fake high-value artefacts in honeypot responses and treats their later use as confirmed compromise:

Token	Planted in	Detected when
HT-APIKEY	WooCommerce REST /wp-json/wc/v3/ responses	Used in an Authorization header
HT-DBCREDS	A "leaked" wp-config.php on config probes (non-existent internal DB host)	Host, user or password appear in a POST body
HT-ADMINPW	An admin-note page	The hash or plaintext appears in a login POST
HT-S3URL	Media library responses	The URL or its access key id is requested
HT-JWTKEY	Plugin-settings REST response	JWTs signed with it appear as Bearer tokens

Values are SHA-1 derivations of a per-deployment salt, so they are the same across workers and restarts and need no per-variant storage. Detection runs before routing; a hit sets the request's score to 100, logs to ELK and Redis, and records time-to-use from the injection timestamp.

In the access phase, a honeytoken hit overrides whatever the pattern analyser scored:

local ht_result = honeytoken_handler.analyze_request_for_tokens()
-- ...
local pattern_result = pattern_analyzer.analyze_request(ngx.var.request_uri, request_headers, ngx.var.remote_addr)
if ht_result then
    pattern_result.score = 100
    pattern_result.suspicious = true
    table.insert(pattern_result.patterns_matched, "honeytoken_reuse:" .. ht_result.token_id)
end

Content Classifiers

Prompt-injection filter: not wired to any LLM yet (the LLM response feature is planned), but pattern_analyzer runs detect() on every request because injection text is anomalous on a storefront anyway; half its risk score is added. Patterns cover five categories — instruction override, role hijack, prompt exfiltration, context break (code fences, punctuation floods) and base64-encoded payloads. sanitize() exists for the future LLM path.

Sophistication analyzer: a rule-based, auditable classifier (scripted / AI-assisted / manual / unknown) from inter-request timing regularity (gaps over 5 minutes ignored; sub-second pacing is scripted), header completeness, User-Agent category and payload precision. It runs after routing, does no I/O, and returns every contributing signal so the label can be checked in the logs.

Database-Layer Variant

ids/db_proxy/reverse_proxy_enhanced/lua/ overrides only init_worker.lua and health_check.lua. With one production_eshop choosing the database per request (via wp-content/db.php), there are no honeypot_eshop_N backends to warm, health-check or mirror replication flags for — probing them is what used to log endless "honeypot_backend_N marked as UNHEALTHY". Instead the worker copies honeypot_pool:health into the health cache, so a session whose database is down is steered to another until it returns. See Honeypot Layers.

Conventions

• Pure/adapter split. Decisions live in *_rules.lua with no ngx.* or _G.*; adapters do the I/O. router.decide_route() stays in the adapter on purpose: its stages interleave session writes with Redis I/O, and it is the most safety-critical function in the codebase.
• One implementation per helper. url_decode and escape_pattern moved to lua_pattern_utils.lua once there were three copies; the two is_static_asset copies disagreed on query strings (style.css?v=123) and the router now uses the stripping one.
• ngx.null is not nil. A missing Redis key returns the ngx.null sentinel, which once crashed cjson.decode on a fresh Redis; threat_intel.load() handles it once for every caller.
• Fail safe, in the right direction. Maintenance, GDPR and health data fail open (site stays up); the install-state gate fails strict. Optional modules are loaded with pcall(require, …) and disable themselves with a log line.
• No dead code. A module ships only with a live call path — hence the prompt-injection filter feeding scores today, and upload analysis folded into the session score instead of its own location, where detections never outlived the request.

Because the rules modules are pure, a test is a plain Lua file with no OpenResty. Each suite sets package.path = "../?.lua;", so it runs from the tests/ directory:

-- tests/test_decay_policy.lua
check("1 offense @0.75 -> 1.75x", approx(dp.half_life_multiplier(1, 0.75), 1.75))
check("one half-life halves it (no offenses)", approx(dp.decay(100, HL, HL, 0, {}), 50))
check("maxed score never decays (permaflag)", dp.decay(100, 10 * HL, HL, 1, cfg) == 100)

# every suite (exits non-zero if any fails)
ids/reverse_proxy_enhanced/lua/tests/run_all.sh

# one suite
cd ids/reverse_proxy_enhanced/lua/tests && lua test_decay_policy.lua