TimeStored Blog

New SQL Docs

We just launched a new sql documentation website: sqldock.com
to allow integration with Pulse / qStudio and docs more easily.
More updates on this integration will be announced shortly. 🙂

Support for 30+ databases added to qStudio and Pulse.

Support for 30+ databases has now been added to both qStudio and Pulse.
Clickhouse, Redis, MongoDB, Timescale, DuckDB, TDEngine and the full list shown below are all now supported.

// Supports Every Popular Database

Pulse is being used successfully to deliver data apps including TCA, algo controls,  trade blotters and various other financial analytics. Our users wanted to see all of their data in one place without the cost of duplication. Today we released support for 30+ databases.

“My market data travels over ZeroMQ, is cached in Reddit and stored into QuestDB. While static security data is in SQL server. With this change to Pulse I can view all my data easily in one place.” – Mark – Platform Lead at Crypto Algo Trading Firm.

// Highlighted Partners

In particular we have worked closely with chosen vendors to ensure compatibility.
A number of vendors have tested the system and documented setup on their own websites:

• TDEngine –  Open-source time series database purpose-built for the IoT (Internet of Things).
• QuestDB –  Open source time-series database with a similar architecture to kdb+ that supports last-by and asof joins. See our crypto Pulse demo.
• TimeScale – PostgreSQL++ for time series and events, engineered for speed.
• ClickHouse and DuckDB – Were tested by members of their community and a number of improvements made.

// The Big Picture

Download Pulse 1.36

Download qStudio 2.52

qStudio – 2.05 – Dark Theme and High DPI Support

qStudio release 2.05 added:

1. Mac / Intellij / Flat / Material Theme support
2. Inlcuding 20+ Intellij and 20+ Material Themes builtin
3. Scaling font size in settings increases code size and all font sizes throughout the UI
4. Jetbrains Mono is now bundled as the default Font for development

Notice Also

• The menu bar is now integrated with the title on platforms with that enabled (Windows 10/11)
• File chooser now includes shortcuts to popular locations
• Native window decorations on Windows 10 – Snapping / Shadows / etc.

qStudio Dark Theme

qStudio Light Mac Theme

Pulse – as a qStudio alternative

qStudio is the number one code editor, server browser and development environment for kdb+.
Today we are launching Pulse, a real-time SQL visualization tool for almost any SQL database.
Within Pulse we have recreated almost all the functionality of qStudio in web form.

If you’ve ever wanted to:

• Share queries and results
• Run analysis from any machine with a browser without a need to install qStudio
• Chart results using a modern charting library

You should consider using Pulse as a shared HTML5 based complement to qStudio..

As you can see below, pulse can be made to look almost the exact same as qStudio. It features the familiar configurable layout with a table/console/charting result panel that allows you to see your result in all formats at once.

Additionally you can

• Bookmark,copy and share links
• Use it with kdb+, postgresql, mysql, oracle and H2 databases
• Try many more chart types including 3D.

Try Pulse Now

qStudio 1.45 Released

qStudio 1.45 Released, we have:

• Bugfix: Ctrl+F Search in source fixed. (Thanks Alex)
• Added Step-Plot Chart display option
• Added Stacked Bar Chart display option
• Added Dot graph render display option (Inspired by Noormo)
• Bugfix: Hidden folders/files regex now works again in file tree and command bar. Target and hidden folders are ignored by default.
• Bugfix: Mac was displaying startup error with java 9

Download

Some example charts:

qStudio adds Step Plots for displaying price Steps.

Our standard time-series graph interpolates between points. When the data you are displaying is price points, it’s not really valid to always interpolate. If the price was 0.40 at 2pm then 0.46 at 3pm, that does not mean it could be interpreted as 0.43 at 2.30pm. Amazingly till now, sqlDashboards had no sensible way to show taht data. Now we do:

For comparison here is the same data as a time-series graph:

The step-plot is usable for time-series and numerical XY data series. The format is detailed on the usual chart format pages.

qStudio now supports Stacked Bar Charts

qStudio has added support for stacked bar charts:

The chart format for this is: The first string columns are used as category labels. Whatever numeric columns appear next are a separate series in the chart. Each row in the data becomes one stacked bar. The table for the data shown above for example is:

You may need to “kdb pivot” your original data to get it in the correct shape.

qStudio Dot Graph Rendering of FIX Order Status

“The Financial Information eXchange (FIX) protocol is an electronic communications protocol initiated in 1992 for international real-time exchange of information related to the securities transactions and markets.”. You can see an example of a FIX message being parsed here.

What we care about is that an order goes through a lifecycle. From newly created to filled or removed. Anything that involves state-transitions or a lifecycle can be visualized as a graph. A graph depicts transitions from one state to another. Often SQL tables record every transition of that state. This can then be summarised into a count of the last state, giving something like the following:

qStudio now automatically converts this result table to DOT format and if you have graphviz“>graphviz installed and on the PATH, will generate the following:

Note I did tweak the table a little to add styling like so:

update style:(`Filled`Removed!("color=green";"color=red")) To,label:(label,'" ",/:cnt) from currentFixStatus

The format is detailed again in our qStudio Chart Data Format page.

This is another even simpler example:

qStudio 1.43 Released – mac save bug fixed

qStudio 1.43 Released. This:

• Adds stack traces to kdb 3.5+
• Fixes the mac bug where the filename wasn’t shown when trying to save a file.
• Fixes a number of multi-threading UI problems

Download it now.

kdb 3.5 Released with Stack Trace

kdb+ 3.5 had a significant number of changes:

• Debugger – At long last we can finally get stack traces when errors occur.
• Concurrent Memory Allocator – Supposedly better performance when returning large results from peach
• Port Reuse – Allow multiple processes to listen on same port. Assuming Linux Support
• Improved Performance – of Sorting and Searching
• Additional ujf function – Similar to uj from v2.x fills from left hand side

kdb Debugger

The feature that most interests us right now is the Debugging functionality. If you are not familiar with how basic errors, exceptions and stack movement is handled in kdb see our first article on kdb debugging here. In this short post we will only look at the new stack trace functionality.

Now when you run a function that causes an error at the terminal you will get the stack trace. Here’s a simple example where the function f fails:

Whatever depth the error occurs at we get the full depth stack trace, showing every function that was called to get there using .Q.bt[]:

The good news is that this same functionality is availabe in qStudio 1.43. Give it a try: qStudio.

Note: the ability to show stack traces relies on qStudio wrapping every query you send to the server with its own code to perform some analysis and return those values. By default wrapping is on as seen in preferences. If you are accessing a kdb server ran by someone else you may have to turn wrapping off as that server may limit which queries are allowed. Unfortunately stack tracing those queries won’t be easily possible.

That’s just the basics, there are other new exposed functions and variables, such as .Q.trp – for trapping calls and accessing traces that we are going to look at in more detail in future.