Insights, updates, and best practices for database monitoring
NiallUptime says the server is up. Your data observability budget says no. In between sits the failure that pages nobody: the app is healthy and the database quietly stopped being right. Here are the SQL patterns that catch it — and where the DIY version stops scaling.
NiallThe DIY freshness cron is the respectable incumbent — it works until it doesn't. Here's the six things it's missing, each with a concrete failure, and better SQL you can keep even if you never sign up.
NiallA partial OOM in a streaming ingestion pipeline stopped one feed writing rows while the app stayed up and the dashboards stayed cached-green. Here's what caught it, and how to build the same check yourself.
NiallA MERGE that upserted nothing, a stream that fell behind, a column schema evolution dropped — none error, and here are the Spark SQL checks that catch each one.
NiallIt failed three runs, Snowflake paused it to save credits, and the table it fed still answers every query with frozen rows — here's the freshness check that would have caught it the next hour.
NiallStale views, unflushed async inserts, and uncollapsed ReplacingMergeTree duplicates throw no error; here's the SQL that catches each one.
NiallA continuous data check is only as good as how it handles expected absence — here's the tuning that separates a check that pages on the real failure from one your team mutes.
NiallGreat Expectations is free and powerful if you'll host and configure it; here are the alternatives when you want schema-aware SQL checks on production Postgres or ClickHouse running in minutes.
NiallMonte Carlo is built for warehouse estates; when your problem is a single Postgres or ClickHouse database, you're paying for automation you'll never use.
NiallA data quality check can run in three places — as a row is written, as the pipeline transforms it, or continuously against production. Where you put it decides which failures it can see, and most teams cover only the first two.
NiallEight data quality tools — from cron scripts and dbt tests to Monte Carlo — compared on how cost grows, what you keep if you leave, and whether you can read the query behind an alert.
NiallA Stripe webhook stopped recording charges for six hours and every check passed. Here's the one continuous check that would have caught it, built up from a naive count to a baseline that survives quiet nights.
