Home › Docs › Observability

Observability

How the honeypot system provides observability into adversary activities

Observability has one job here: turn an ephemeral request decision into evidence that survives the session. Every scoring result, routing decision and network alert can leave the edge as structured data, and the SIEM side is designed so a missing Kibana never affects detection.

Observability Layers

Log Ingestion: Structured logs from the reverse proxy, Lua engines, and Suricata are centralized for SIEM consumption via Nginx error/security logs and Vector forwarding — over mutual TLS — to Elasticsearch/Kibana. See Transport & mTLS.
SIEM Dashboards: High-level operational views of IDS alerts, threat overview, and web traffic.
Kibana Visualizations: Interactive dashboards and pattern report timelines.
Alerting & Reporting: AbuseIPDB reporting and reputation-based alerting.

The whole SIEM side is fail-open with respect to detection: it turns an ephemeral request decision into durable evidence, but nothing in the request path waits for it, so a SIEM outage never affects routing or scoring.

Telemetry Sources

Nginx error log Security events Structured JSON lines written by log_security_event(), read by Vector and forwarded to Elasticsearch. SECURITY_EVENT prefix eve.json Suricata alerts Parsed every 30 seconds by init_worker.lua for IP reputation and stored in the threat_intel shared dictionary. 30 s interval Decision trace Routing traces Flat decision strings recorded per request so an analyst can reconstruct why a session was diverted. stage-by-stage Kibana Dashboards Six saved dashboards plus a dedicated pattern-report timeline used for per-pattern visual evidence. NDJSON managed

Security Event Logging

_G.utils.log_security_event(event_type, details) appends one structured JSON line per event to the Nginx error log, prefixed with SECURITY_EVENT. Vector reads these lines and forwards them to Elasticsearch; the rules they must follow (JSON lines, empty tables encoded as [], retries never blocking the edge) are the Pipeline Contracts. The shared dictionaries that cache state between requests are listed in Lua Internals.

The logger (init.lua) and a typical call from router.lua:

local log_entry = {
    timestamp = ngx.time(),
    event_type = event_type,
    details = details,
    server_time = os.date("%Y-%m-%d %H:%M:%S")
}
ngx.log(ngx.WARN, "SECURITY_EVENT: ", cjson.encode(log_entry))

_G.utils.log_security_event("routing_to_honeypot", {
    reason = "high_risk_score",
    score = pattern_result.score,
    ip = remote_ip,
    pool = routing_decision.session_data.honeypot_pool,
    session_id = session_data.id
})

To watch them live: docker compose logs -f reverse_proxy | grep SECURITY_EVENT.

Suricata Log Ingestion

init_worker.lua reads new eve.json alerts every 30 seconds, drops private and loopback sources, and stores a severity-graded score per IP in threat_ips, decayed when read. The scoring functions are on Suricata Network IDS, and the reader's resume and size limits on Lua Internals.

Honest Assessment

Strengths: Multi-layer observability, structured security events, empty-table fix, Suricata integration, offline dashboard generation.

Limitations: Dashboard drift, timeline capture fragility, no automated health monitoring, Vector complexity, same-host testing gap.