Skip to main content
One agent is enough for most production Laravel apps. When it isn’t, you have two levers: more drain workers inside one agent, and more agents, on one host or several, all writing to the same PostgreSQL database. See Agent throughput for measured numbers.

When you need more than one instance

  • Agent Health shows the buffer climbing during peak hours and not coming back down.
  • One agent can’t keep ingest and drain aligned even after adding drain workers.
  • You want redundancy: if one agent crashes, the others keep accepting telemetry while it restarts.
If the bottleneck is the drain (writing to PostgreSQL) rather than ingest (accepting payloads), start with NIGHTOWL_DRAIN_WORKERS before adding agents.

Scaling drain workers first

Each agent runs one ingest loop plus a number of drain worker processes. Set the number in the .env the agent reads, then restart the agent:
It takes 1 to 16. A higher value is lowered to 16 with a line in the agent log, and 0 or anything that isn’t a number means 1. The agent prints the number it runs with at startup (Drain workers: 4). With more than one worker, each worker claims its rows in the buffer before writing them, so two workers never write the same row. Rows claimed but not yet written still count as pending, in back-pressure (NIGHTOWL_MAX_PENDING_ROWS) and in the pending rows on Agent Health and /status. Each worker holds its own PostgreSQL connection. Add workers while the drain rate on Agent Health keeps rising, and stop when it doesn’t: past that point the database is the limit.

Running multiple agents on one host (Linux)

The ingest socket always sets SO_REUSEPORT, so several agents can listen on the same port, and on Linux the kernel spreads new connections across them. There is nothing to turn on. Each agent does need two things of its own:
  • Its own buffer, with --sqlite-path or NIGHTOWL_AGENT_SQLITE_PATH. A second agent pointed at a buffer another agent is using waits up to NIGHTOWL_BUFFER_LOCK_WAIT seconds (30 by default) for it to exit, and refuses to start if it’s still running: NightOwl agent will not start: another agent (pid N) is still using this buffer after 30s. Two agents draining one buffer could write the same rows to PostgreSQL twice.
  • Its own health port, with NIGHTOWL_HEALTH_PORT, or no health API at all with NIGHTOWL_HEALTH_ENABLED=false. Otherwise the second agent stops at startup: Could not start the agent: health port 127.0.0.1:2409 is already in use, often by another nightowl:agent on this host.
A systemd template unit gives each instance both, named after its health port:
All instances write to the same PostgreSQL database.
Cached config. After php artisan config:cache, Laravel no longer reads environment variables, so the Environment= line above does nothing and every instance would try the same health port. Command-line options still apply: --sqlite-path (and --host, --port) keep working. With a cached config, drop the Environment= line and set NIGHTOWL_HEALTH_ENABLED=false in .env before caching. Agent Health in the dashboard doesn’t use the local health API, so it keeps working.
SO_REUSEPORT spreads connections across agents on Linux only. On macOS every new connection goes to one listener, so several agents on one port won’t share the load. Use different ports during local development.

Running multiple agents across hosts

For horizontal scaling across machines, put a TCP load balancer (HAProxy, the nginx stream module, or a cloud load balancer) in front of the agent pool. Point your application at the load balancer with NIGHTOWL_INGEST_URI, the host:port your app sends telemetry to:
NIGHTOWL_INGEST_URI (app side) and NIGHTOWL_AGENT_HOST (agent side) are two different settings. NIGHTOWL_AGENT_HOST only controls the address the agent binds to: set it to 0.0.0.0 (or a specific interface) so the agent accepts connections from other hosts. NIGHTOWL_INGEST_URI controls where the app sends. Both default to loopback, which is why a co-located single-host install needs neither. With NIGHTOWL_AGENT_HOST=0.0.0.0 the health port (2409) listens on every interface too, so firewall it along with 2407.
Use a least-connections balancing policy. Any agent can accept any payload, so there’s no sticky-session concern.

PostgreSQL connections

Every drain worker holds one long-lived PostgreSQL connection, so the drain needs agents × workers connections: three agents with four workers each hold twelve. On top of that, each agent keeps one connection from its startup checks and opens short-lived ones for the hourly table statistics and for health alerts, and the NightOwl dashboard connects to read your data. Keep the total under the database’s max_connections. Point the agent at the database’s own port or a session-mode pooler, not a transaction-mode pooler. The quickstart explains why.

Sizing

Measured on a single instance: In the load test the drain was the limit. It ran with one drain worker; we haven’t published a measurement with more. Size from your own Agent Health numbers: if the drain rate sits below your peak ingest rate for long enough to fill the buffer, add workers first, then agents. See Agent throughput for how the load test was run.

Verifying the fan-out

Open Agent Health after starting the second agent. Each agent reports its own row in the Instances table, named hostname:pid, with its own Ingest/s, Drain/s and Write Queue. The sum of the Ingest/s column should match your application’s outgoing traffic. If one agent on a host takes all the traffic, check that the host is Linux, and that each agent really started: an agent that refused to start over a shared buffer or a busy health port leaves only the first one running.