Profiles / DuckDB
DuckDB
SQLite for analytics: a library that runs inside your Python, R or Node process and queries local files at remarkable speed. On ClickBench it has the best cold-run score in the self-hosted group and loads data quickly, but it is a single-process engine, not a server, so it doesn't take part in the concurrency test.
Best single-user, in-process engineMIT
Strengths
- +Nothing to run: an embedded library with no server to operate.
- +Best cold-run score of the self-hosted group (1.31) and the fastest bulk load (126 s).
- +Reads Parquet, CSV and JSON in place, locally or on object storage.
- +Excellent SQL ergonomics for analysts.
Weaknesses
- −One process, one writer: not built to serve many users or applications at once.
- −About 2× ClickHouse's hot-run score on identical hardware.
- −Took 20.5 GB on disk for the dataset ClickHouse stores in 9.4 GB.
- −No continuous ingestion; data arrives in batches.
Capabilities
| Real-time ingestion | Batch loads; single writer process |
|---|---|
| Columnar storage | Columnar row groups |
| Open source | MIT |
| Managed cloud | Third-party (MotherDuck) |
| Query speed at scale | 2.03 self-hosted |
| High concurrency | Single process; not tested |
| Cost model | Free; you pay for the machine it runs on |
DuckDB vs ClickHouse
On the same c6a.4xlarge, DuckDB scores 2.03 on hot runs to ClickHouse’s 1.03, and stores the data in 20.5 GB to ClickHouse’s 9.4 GB. For notebooks, local data exploration, data apps over Parquet files, CI pipelines, DuckDB is a reasonable choice; for real-time analytics, we recommend ClickHouse.