NightOwl is a bring-your-own-database product. Your telemetry — every request, query, exception, job attempt, and log line — lives in a PostgreSQL instance you control. That gives you full ownership of retention, backups, and compliance, and it keeps storage costs decoupled from per-seat dashboard pricing.
Storage model
Every connected app points at one PostgreSQL database. The agent writes through SQLite’s WAL buffer into ~20 nightowl_* tables using COPY for high-volume event tables and INSERT … ON CONFLICT for upsert tables (nightowl_exceptions, nightowl_users).
Because the data lives in your PostgreSQL, you can:
- Connect your own BI tools (Metabase, Grafana, Superset) directly to the
nightowl schema.
- Run arbitrary SQL for ad-hoc analysis without hitting an API rate limit.
- Control exactly where the data is geographically stored, for GDPR or data-residency reasons.
- Back it up alongside the rest of your database with tools you already trust.
Raw telemetry vs. rollups
Every nightowl_* table falls into one of three groups, and knowing which is the whole trick to managing storage.
| Group | Tables | What it is |
|---|
| Raw telemetry | nightowl_requests, nightowl_queries, nightowl_jobs, nightowl_logs, … | One row per event. Almost all of your disk. |
| Rollups | nightowl_request_rollups, nightowl_query_rollups, nightowl_job_rollups, … | Per-minute aggregates. Tiny. What the dashboard charts actually read. |
| Triage & config | nightowl_issues, nightowl_issue_comments, nightowl_settings, nightowl_alert_channels, nightowl_users | Never deleted by retention or by Data Management. |
Rollup coverage per table
| Raw table | Coverage | Deleting it loses |
|---|
nightowl_requests | Fully aggregated | Individual traces, per-route recent-request lists |
nightowl_queries | Fully aggregated | Individual query executions |
nightowl_jobs | Fully aggregated | Individual job attempts and their exception payloads |
nightowl_outgoing_requests | Fully aggregated | Individual outgoing calls |
nightowl_cache_events | Fully aggregated | Individual cache events |
nightowl_mail | Fully aggregated | Individual sent messages |
nightowl_notifications | Fully aggregated | Individual notifications |
nightowl_commands | Fully aggregated | Individual command runs — their output, arguments, and per-run child-event counts |
nightowl_scheduled_tasks | Fully aggregated | Individual scheduled-task runs and their per-run child-event counts |
nightowl_exceptions | Partly aggregated | Stack traces (the issue itself — status, assignee, comments, activity — is kept) |
nightowl_logs | Not aggregated | Everything. Permanently. |
Views filtered by ?user= fall back to raw telemetry because the section rollups carry no user dimension. Those views will be empty over a window you’ve cleared, even for fully-aggregated tables.
Retention
Telemetry is high-volume and low-shelf-life. The agent prunes raw telemetry and rollups on separate windows, so trends outlive the events that produced them:
NIGHTOWL_RETENTION_DAYS=14 # raw telemetry
NIGHTOWL_ROLLUP_RETENTION_DAYS=90 # rollups — tiny, so keep them far longer
Nothing is deleted until you schedule the command. Setting NIGHTOWL_RETENTION_DAYS alone has no effect — it only supplies the default window for nightowl:prune.
Pruning
The nightowl:prune artisan command deletes raw rows older than NIGHTOWL_RETENTION_DAYS and rollup buckets older than NIGHTOWL_ROLLUP_RETENTION_DAYS. Override either for a single run with --days=N / --rollup-days=N, or use --hours=N for sub-day raw retention. Schedule it in the app you’re monitoring:
// routes/console.php
Schedule::command('nightowl:prune')->daily();
DELETE … WHERE created_at < ? per table. On PostgreSQL a DELETE marks rows dead rather than handing disk back: autovacuum makes that space reusable by new rows, so a table under steady ingest settles at a high-water mark instead of growing without bound. Plain VACUUM (ANALYZE) does not shrink the data file — only VACUUM FULL (which takes an exclusive lock) or pg_repack does. On a managed volume such as AWS RDS, allocated storage never shrinks back either, so size the disk for your retention window rather than expecting it to fall after a prune.
Manual cleanup and resets
Selective delete (Data Management)
The Data Management page in the sidebar shows your storage split across raw / rollups / triage, reports how far back your data actually goes, and lets you delete raw telemetry for a specific date range, narrowed by table and filter. Use it to reclaim disk, or to clear noise from a staging incident or a runaway log level without waiting for retention to catch up.
-
Choose data types. Each one shows its rollup coverage and exactly what deleting it keeps and loses.
-
Pick a date range. The most recent data is protected, and how much depends on what you selected:
| Selection includes | Protected window |
|---|
| Only fully-aggregated tables | 24 hours |
| Exceptions or logs | 7 days |
If this app’s rollup tables are missing or were never backfilled, nothing is aggregated and every table is treated as permanent: the page shows a 7-day floor and a warning instead of the usual coverage badge. Run php artisan nightowl:migrate then php artisan nightowl:backfill-rollups in the monitored app before clearing anything.
- Filter by route path, status code, duration, user, job status, log level, exception class, or cache event type. Filters only apply to tables where the column exists.
- Preview shows counts per table, each tagged with its coverage, before you commit. Deletion is then chunked in 10,000-row batches.
Rollup tables are never touched by this flow, so your charts survive. Neither are settings, alert channels, or user identity.
Deleting nightowl_exceptions loses only the per-occurrence stack traces — old occurrences can no longer be opened to see where they were thrown. The issue itself (status, assignee, comments, activity) is kept: an issue is triage state, not telemetry, so clearing exception telemetry never touches it.
Delete app (Danger Zone)
From Settings → Danger Zone, Delete app removes the app from NightOwl along with its agent token and alert routing. Does not drop or truncate your nightowl_* tables — that’s your database, and we won’t touch it. If you want the telemetry gone too, use Data Management for a targeted range or run php artisan nightowl:clear in the customer app for a full truncate.
Data Management deletions are irreversible — there’s no soft-delete. Take a PostgreSQL backup first if the data has any archival value.
Backups
NightOwl doesn’t manage backups — that’s your PostgreSQL provider’s job. The nightowl_* tables back up the same way the rest of your database does. If you use pg_dump, include the schema you configured (public by default) and you’ll capture everything. If you use managed snapshots (RDS, Cloud SQL), they already cover it.
For point-in-time-restore strategies, treat the nightowl_* tables like any other high-write workload — their WAL volume scales with telemetry throughput, so budget your WAL storage accordingly.
Sizing expectations
As a rough baseline for capacity planning:
| Workload | Daily rows | Daily storage (compressed) |
|---|
| Small app (~100 req/s) | ~8.6M | ~500 MB |
| Medium app (~1,000 req/s) | ~86M | ~5 GB |
| Large app (~10,000 req/s) | ~860M | ~50 GB |
Exporting data
For compliance exports, legal holds, or moving to another tool, use direct PostgreSQL access:
pg_dump -h <host> -U <user> -d <db> \
--table='nightowl_*' \
--format=custom \
-f nightowl-export.dump
