Kdb+ Architecture and Administration - Training Course
Learn industry best practices for architecting and maintaining an enterprise wide Kdb+ data solution. You will learn how to
- Implement security
- Load balance queries across multiple machines
- Reduce storage costs using compression.
- Recover from a crash (logging / checkpointing)
- Speed up existing queries and best practices for structuring data
Contact us now to request Training
Course Content
Day 1
| Functions | Recap of Functions, Dictionaries, Lists etc
Best practices for writing modular code for large scale kdb systems. |
|---|---|
| Tables | Defining, updating, deleting, adding columns, sorting.
Creating relationships between tables using enumeration and Foreign Keys |
| Inter-Process Communication | Communication between servers, sending data (a)synchronously, buffering
Security / Stream processing - by overriding event handlers Load Balancing - distributing real time and historical queries over multiple machines Stitching together real-time / historical data using a Gateway |
| qSQL | Standard SQL queries: "select col from table where ..", what's unique to qSQL |
Day 2
| Joins | Covers the standard SQL joins left-join, inner-join, union-join.
Specialized kdb joins: plus-join, equi-join, asof-join Optimizing joins and queries using attributes |
|---|---|
| Multithreading | built-in threading, Parallel processing.
( Parallel screencast available) |
| Disk Storage | Text Output and Standard serialization - for saving small files
Splaying / Partitioning - For storing large amounts of historical data Segments - For storing extremely large data sets and processing them efficiently in parallel |
| Advanced Analytics | Building on our knowledge of qSQL and disk storage
Efficiently constructing analytics e.g. VWAP / TWAP |
Day 3
| Importing Libraries | Creating C DLL's and calling that code from kdb |
|---|---|
| C / Java Interfaces | Calling kdb from c/Java
(Java API screencast available) |
| Logging / Check pointing | Crash recovery, Transactions and rolling back |
| Memory Management | kdb's memory model, garbage collection
( Memory tutorial available) |
| Compression | How can data be compressed, when to use it and which data to use it on. |
| Extremely large Databases with many cores | What is happening below the covers,
Best practices for constructing map-reduce queries and edge cases to be careful of. |
Prerequisites
This class is a good fit if you have some experience in kdb+ before and are looking to increase your skills to the level where you could be responsible for administering complex kdb+ deployments.
Contact us now to request TrainingOther courses we offer include:
- Introduction to Kdb+ for those new to kdb
- Kdb+ Data Manipulation - for those wanting to perform analytics of data in kdb+