Every Time-Series Database Benchmark Ever. 2024
Updated February 11, 2024
This page will gather ongoing time-series database benchmarks into one location.
- Top Benchmarks - Those that are open, comprehensive and thorough
- Benchmarks by Date - Sorted Chronologically
The top benchmarks section will focus on time-series databases that are fast, provide time-series joins, time window aggregations and data compression, rather than pull in more general solutions.
Top Time-series Benchmarks
To be considered a top benchmark, a combination of factors including repeatability, open-source, reputation, comprehensiveness, thoroughness were considered. This narrowed the top benchmarks to:
- benchmarking 4 queries on 1.1 Billion NYC Taxi and Uber Trips - by Mark Litwintschik's. Uses different hardware but thorough writeup of each database features.
- Clickbench - 30+ Databases with 42 queries benchmarked. Only includes open-source but highly reproducible.
- STAC-M3 - Non-open costly commercial benchmark
- H2O.ai - reproducible benchmarking of database-like operations in single-node environment.
Updated Feb 11, 2024
Updated Oct 4, 2023
A number of vendors have presented short demo videos for our Modern Databases for Finance series, you may find this useful to quickly see what different databases offer.
Updated Aug 14, 2023
The latest updates were all vendor customized benchmarks for ingestion, each of the vendors benchmarks showed themselves as the fastest at ingestion: questdb, GridDB, TDEngine.
Taxi Ride Benchmarks
Results reproduced from Mark Litwintschik's excellent article.
Mark benchmarked 4 queries against a 1.1 Billion NYC Taxi Trips data set. It had 51 columns was 500 GB in size when in uncompressed CSV format
| Setup | Total Query Time (lower = better) |
|---|---|
| kdb+/q & 4 Intel Xeon Phi 7210 CPUs | 1.0 |
| ClickHouse, 3 x c5d.9xlarge cluster | 4.1 |
| OmniSci & a 16" MacBook Pro | 4.3 |
| Clickhouse on DoubleCloud, s1-c32-m128 | 5.8 |
| BigQuery | 8.0 |
| Redshift, 6-node ds2.8xlarge cluster | 8.0 |
| Amazon Athena | 25.3 |
| Presto 0.214 & 21 x m3.xlarge HDFS cluster | 29.4 |
| Spark 2.4 & 21 x m3.xlarge HDFS cluster | 30.4 |
| Vertica, Intel Core i5 4670K | 147.3 |
| PostgreSQL 9.5 & cstore_fdw | 930.0 |
| Spark 1.6, 5-node m3.xlarge cluster w/ S3 | 2158.0 |
| Elasticsearch (heavily tuned) | 26.3 |
Clickbench
Clickbench is a large suite of benchmarks produced by clickhouse themselves. The focus is not time-series but a wide range of queries. They are being very transparent and open, i.e. on some queries they are beaten but the benchmarks only include open source choices.
| System & Machine | Relative time (lower is better) | Note |
|---|---|---|
| ClickHouse (c6a.metal, 500gb gp2): | ×1.59 | |
| SelectDB (c6a.metal, 500gb gp2): | ×1.88 | |
| ClickHouse (m5d.24xlarge): | ×2.15 | |
| StarRocks (c6a.metal, 500gb gp2): | ×2.16 | |
| Redshift (4×ra3.16xlarge): | ×2.20 | |
| DuckDB (c6a.metal, 500gb gp2): | ×2.74 | |
| QuestDB (partitioned) (c6a.metal, 500gb gp2)†: | ×24.24 | |
| MariaDB ColumnStore (c6a.4xlarge, 500gb gp2)†: | ×59.27 | |
| TimescaleDB (compression) (c6a.4xlarge, 500gb gp2): | ×86.91 | |
| Druid (c6a.4xlarge, 500gb gp2)†: | ×150.50 | |
| PostgreSQL (c6a.4xlarge, 500gb gp2): | ×883.89 | NOT column oriented |
STAC-M3
STAC-M3 are highly commercial and results are not published or reproducible. Typically each vendor pays to perform a STAC-M3 benchmark and at the time they use the fastest available hardware humanly possible and a highly optimized configuration. making results extremely difficult to compare. They are however as far as we know, extremely thorough.
Below is a rough overview of the winners doing press releases by year:
| System & Machine | Relative time (lower is better) |
|---|---|
| 2022 - May | kdb+ + Google Cloud improves up to 18x in latest STAC benchmark |
| 2022 - February | DDN and Shakti Announce Record Breaking Results on the STAC-M3 Benchmark for Financial Trading Applications |
| 2020 | INFOWeka and KDB 3.6 claim 'Record-Breaking Results on 17 STAC-M3 “Tick Analytics” Benchmarks' |
| 2014 | McObject's eXtremeDB McObject and Lucera Set Records for Market Data Analysis |
| 2012 | McObject's eXtremeDB Financial Edition Sets Records in STAC-M3 Benchmark of Market Data Analysis |
h20.ai Reproducible Benchmarks
Aims to benchmark various database-like tools popular in open-source data science. It runs regularly against very latest versions of these packages and automatically updates. They provide this as a service to both developers of these packages and to users. You can find out more about the project in Efficiency in data processing slides and talk made by Matt Dowle on H2OWorld 2019 NYC conference.
All Other Time-series Benchmarks by Date
Rows marked 
in the below table mean they are produced by a vendor themselves and
that you should assume they have chosen a benchmark to highlight their best possible performance.
If you want a benchmark added please contact us.
| Date | Benchmark | Summary | Databases |
|---|---|---|---|
| 2023-05-16 | Vendor TSBS Benchmark | 1M rows per sec loading on customized TSBS benchmark by QuestDB.
Vendor chosen customized benchmark.
|
GridDB, QuestDB, TimescaleDB, ClickHouse |
| 2023-05-02 | Vendor TSBS Benchmark | 300K rows per sec loading on customized TSBS benchmark by GridDB.
Vendor chosen customized benchmark.
|
GridDB, QuestDB, TimescaleDB |
| 2023-04-14 | H2O.ai Database-like Ops Benchmark | DuckDB updates existing H20.ai benchmark.
Vendor chosen customized benchmark.
|
DuckDB vs Pandas vs Clickhouse |
| 2023-02-20 | Vendor TSBS Benchmark | Ingestion and query performance using customized TSBS Benchmark by TDEngine.
Vendor chosen customized benchmark.
|
InfluxDB vs TimescaleDB vs. TDengine |
| 2023-02-15 | Academic Research | benchmarking specialized databases for high-frequency data
kdb  Paper says benchmark was to be open source but can't find it. Source requested from authors.
|
ClickHouse, InfluxDB, kdb+, TimeScaleDB |
| 2023-01-25 | ClickHouse vs PostgreSQL |  Analyse billions of youtube video metrics.
Clickhouse 5-10x faster than postgres for analytic queries |
ClickHouse, PostgreSQL |
| 2023-01-17 | SigNoz | SigNoz is an observability platform based on ClickHouse. Here they perform 500GB log analysis mostly using aggregation queries. Note:
SigNoz / Clickhouse won but this is a vendor self supplied benchmark.
|
ClickHouse, ElasticSearch, MinIO |
| 2022-06-08 | Academic Paper | SciTS: A Benchmark for Time-Series Databases in Scientific Experiments and IoTs
"Click-House supports very high ingestion rates up to 1.3 million records...ClickHouse significantly outperforms other evaluated databases in the speed of data queries and shows reasonably low deviation in query latency."
|
ClickHouse, InfluxDB, TimescaleDB, and PostgreSQL |
| 2022-05-26 | QuestDB Blog | 4Bn rows/sec query benchmark: Clickhouse vs QuestDB vs Timescale
Vendor chosen benchmark to emphasise a very particular scenario.
|
Clickhouse, QuestDB, Timescale |
| 2022-04-05 | KX Blog | KX v DBOps Benchmark Results
Vendor ran benchmark. Claims to have ran fastest 18/19 queries but no speed numbers published.
|
kdb+ |
| 2022-04-02 | data-sleek blog | SingleStore vs. ClickHouse Benchmarks
"SingleStore offers a much stronger solution than ClickHouse. The performance of the queries when joining tables is obvious — queries were 3-186x faster." |
ClickHouse, SingleStore |
| 2022-04-01 | Masters Thesis | Evaluating ClickHouse as a Big Data Processing Solution for IoT-Telemetry | ClickHouse, ElasticSearch, Loki |
| 2022-03-14 | Altinity Blog | Evaluating Altinity ClickHouse vs Singlestore for loading 100b rows
Vendor self-benchmark
|
ClickHouse, altinity, singlestore |
| 2022-02-15 | Gitlab CH Eval | Gitlab APM ClickHouse evaluation - to evaluate ClickHouse as a horizontally scalable datastore for o11y data (metrics, logs, traces).
Chose ClickHouse |
Clickhouse, CrateDB, MongoDB and TimescaleDB |
| 2022-01-18 | Apache Doris | Apache Doris self-published TPC-H benchmarks - Vendor-ran benchmark
|
Apache-Doris |
| 2022-01-18 | DB - Clickhouse vs kdb+ | - Internal Deutsche Bank benchmark.
Clickhouse fastest
|
kdb, clickhouse |
| 2021-08-27 | duckdb |
Database Sorting Benchmark
Duckdb wins their own chosen benchmark, clickhouse close.
|
duckdb, clickhouse, hyper, pandas |
| 2021-07-08 | alibabacloud | ClickHouse vs. Elasticsearch
Clickhouse. "Elasticsearch is excellent in search scenarios where only a few records are filtered out by the WHERE clause. However, in analysis scenarios of large-scale data ... ClickHouse will have better concurrency performance due to its excellent column-storage mode and vectorized computing..." |
ClickHouse, CrateDB, Druid, Hyper, MonetDB, PostgreSQL, SparkSQL, TimescaleDB |
| 2021-04 | brandonharris | ClickHouse vs. Redshift - Financial analysis
Clickhouse faster. |
ClickHouse, Redshift |
| 2021-04 | TSBS | Open Source Time Series Benchmark Suite - Started by TimeScaleDB
Frustratingly they never seem to have published public results. |
Akumuli, Cassandra, ClickHouse, CrateDB, InfluxDB, MongoDB, QuestDB, SiriDB, TimescaleDB, Timestream, VictoriaMetrics |
| 2021 | InfluxDB | Vendor selected benchmarks by influxDB themselves.
Influx have chosen weak databases to compare against. Frustratingly their results whitepapers require email signup to see. They do claim to be 5x faster than MongoDB. |
Elasticsearch, Cassandra, MongoDB, OpenTSDB, TimescaleDB |
| 2020 | mgbench | Open source query benchmarks for machine-generated log data by Andrew Crotty - Assistant Professor of Computer Science at Northwestern University.
Clickhouse |
ClickHouse, CrateDB, Druid, Hyper, MonetDB, PostgreSQL, SparkSQL, TimescaleDB |
| 2020-09-08 | Clickhouse vs Redshift | Performance for FinTech Risk Management.
|
ClickHouse, Redshift |
| 2019-04-10 | Measuring vertical scalability | Measuring vertical scalability for time series databases in Google Cloud by CEO of VictoriaMetrics.
Expected result by vendor: "VictoriaMetrics provides the best vertical scalability for both data ingestion and querying. " |
InfluxDB, TimescaleDB and VictoriaMetrics |
| XXXX | db-benchmarks.com |
They claim to be publishing fair tests but the entire site and github repo is produced by manticore
a vendor. I find it particularly distasteful to say "Fair" but not mention they work directly for one of the companies.
Hence why I'm not linking to their site.
|
ClickHouse, Elastic, Manticore |