kdb+ vs ClickHouse Compared
| ClickHouse | kdb+ | |
|---|---|---|
| Description | Open Source OLAP Database originating from Yandex. | Older proprietary closed source database only popular in Finance. |
| Architecture | Columnar database - MergeTree Based | Columnar database - Array Based |
| Released | June 15, 2016 | 2003 |
| Scaling | Horizontally Scales to more machines. | Vertically Scales to run on one big machine. |
| Language | Mostly ANSI SQL | Custom qSQL and language q |
| License | Apache 2.0 | Closed Source |
| Use Cases | Real-time analytics, advertising, event logging, monitoring, IoT, data warehousing | High-frequency trading, financial services, market data analysis.
See who uses kdb+. |
Technical Features Compared
| ClickHouse | kdb+ | |
|---|---|---|
| Tickers = Low Cardinality Strings | Enum | Symbols |
| Nanosecond Support | Yes. DateTime64 | Yes - timespan/timestamp |
| Asof Joins | ASOF JOIN | Aj |
| Window Joins | No. | wj |
| Latest Value on Time | last_value with Window | last |
| Previous / Next with offset | No. Alternative is ROWS BETWEEN | prev next |
| Moving Averages / Running Functions | Window Functions | mavg |
| Time Bars | toStartOfMinute | xbar |
| Generated Queries | No. | Functional Select |
| Python Integration | Client Only | PyKX customized pip + license |
About ClickHouse and History
ClickHouse is an open-source column-oriented database management system (DBMS) developed by the Russian company Yandex. It is designed for analytical processing of large volumes of data, particularly for OLAP (Online Analytical Processing) workloads. ClickHouse is optimized for high-performance queries on data sets that range from gigabytes to petabytes in size.
ClickHouse Beginnings:
- Origin: ClickHouse originated within Yandex, one of the largest internet companies in Russia. Yandex needed a powerful and scalable database solution to handle the vast amounts of data generated by its various services, including search, advertising, and analytics.
- Development: ClickHouse was developed internally at Yandex to meet their specific requirements for fast and efficient analytics on large datasets.
- Open Sourcing: Recognizing the potential value of ClickHouse beyond its internal use, Yandex decided to release it as an open-source project in 2016.
- Community Growth: Since its open-source release, ClickHouse has gained popularity among companies seeking a high-performance and scalable database solution for analytics.
About kdb+ and History
Kdb+ is a high-performance, in-memory, column-oriented database developed by mostly one man: Arthur Whitney and based on APL.
- Foundation: Kx Systems, founded in 1993 by Arthur Whitney and Janet Lustgarten, created the kdb+ database system. Initially, it was developed to meet the demands of financial institutions for handling and analyzing large volumes of time-series data in real-time.
- Early Development: In the late 1990s and early 2000s, kdb+ gained popularity in the financial industry due to its speed and efficiency in processing and analyzing vast amounts of tick data and other financial data types. Its column-oriented architecture and integrated query language (q) were particularly well-suited for time-series analytics.
- See also full history of kdb+
What's next
