pragma_storage_info(‘tablename’) returns meta data on the compression for a selected table. e.g.
PIVOT (SELECT row_group_id, column_name, column_id, segment_type,compression,sum(count) AS c
FROM pragma_storage_info('stocks') WHERE segment_type != 'VALIDITY' GROUP BY ALL)
ON compression USING sum(c) AS c GROUP BY column_name,segment_type ORDER BY column_name,segment_type;
Will pivot the compression etc. out to allow easier viewing like so:
column_name segment_type ALP_c BitPacking_c Dictionary_c
close DOUBLE 34182458
date DATE 34182458
high DOUBLE 34182458
low DOUBLE 34182458
open DOUBLE 34182458
pre VARCHAR 34182458
sym VARCHAR 34182458
volume BIGINT 34182458
Both qStudio and Pulse dynamically download drivers on first use.
The locations they store drivers are:
C:\Users\{windows-username}\pulse\libs
C:\Users\{windows-username}\qstudio\libs
./lib
Notice that last ./lib is the current directory that you are running pulse from.
If you have used clickhouse or dolphindb, they should already contain files named:
clickhouse-jdbc-0.4.6.jar
jdbc-1.30.22.5-jar-with-dependencies.jar
To use new versions, you can delete and replace those files with any newer version from a maven repository.
For example from here https://mvnrepository.com/artifact/com.clickhouse/clickhouse-jdbc/0.6.0 the files section has a link to .jar file:
https://repo1.maven.org/maven2/com/clickhouse/clickhouse-jdbc/0.6.0/clickhouse-jdbc-0.6.0.jar
Then restart the application to pick up the new drivers.
We will update the drivers within Pulse and qStudio regularly but it requires manual testing and we won’t be doing this quickly for every version of all 30+ databases we support. So I hope this helps.
Pulse stores all dashboards, database connection details etc. to one single database file pulsedb.mv.db.
- Usually with a .zip install or running on linux/mac the file is called pulsedb.mv.db and in the current directory where you run Pulse.
- On a windows installation, Pulse is installed to C:\Program Files\pulse as is best practice on windows installs, Pulse saves the database to: C:\Users\{{current-username}}\pulse so that each user can have their own settings.
Note sometimes you will see both a pulsedb.trace.db and pulsedb.mv.db file. pulsedb.trace.db is a temporary file written to when Pulse is running to allow faster database operations.
I can’t think of many scenarios where Pulse would save then lose data and it hasn’t been reported by other users.
Perhaps the original pulse.mv.db was created with different user permissions? e.g. You installed and first ran as root and now are trying to run as a limited user?
Or you ran Pulse from a different current directory? Thereby creating different configurations?
Related articles:
Hi Quantific,
Let’s assume you have the kdb+ pivot code (https://stackoverflow.com/questions/30789471/pivot-table-in-kdb-q) and that:
trades_agg:1000#0!trades_agg
Then providing a UI to select columns is simple.
Example video: https://youtu.be/GyTQqKmymSU
Steps:
- Add table trades_agg
- Add a Multi-select component with sql query: ([] c:cols trades_agg)
- Add a Multi-select component with sql query: ([] c:cols trades_agg)
- Add a Dropdown select component with sql query: ([] c:cols trades_agg)
- Add another table with the query:
piv[trades_agg;(),<code>${{key1}}; (),${{key2}}; (),`${{key3}}]
i.e. You want three variables, then to place those 3 variables into the piv function while making sure they are lists of symbols.
// see https://stackoverflow.com/questions/30789471/pivot-table-in-kdb-q
// I then aggregate the data into equal sized buckets
Testing frameworks for kdb+ include:
-
qunit – unit testing similar to junit etc with asserts and integrates well with qStudio
-
k4unit – unit testing driven from a csv file
-
qspec – spec testing framework for kdb+
Kdb doesn’t have a “case when then else”, it has something similar called vector conditional.
This link gives details: http://www.timestored.com/kdb-guides/q-quirks#caseStatement
Basically standard SQL:
`SELECT
CASE
WHEN Date1 >= Date2 THEN Date1
ELSE Date2
END AS MostRecentDate`
Kdb Code:
`q)update mostRecentDate:?[date1>=date2;date1;date2] from t
date1 date2 mostRecentDate
————————————
2001.08.05 2007.09.28 2007.09.28
2011.05.21 2007.07.25 2011.05.21
2005.08.10 2008.07.06 2008.07.06
2003.02.16 2008.10.11 2008.10.11
2007.10.05 2004.03.01 2007.10.05`
The vector conditional takes three args ?[a;b;c]
a is a list of booleans
b and c are a list of same typed values
where a was true, the value is taken from b, otherwise it’s taken from c.
The .z namespace contains functions for accessing the current data and time. e.g.
q).z.t
09:56:06.145
q).z.d
2013.05.21
The pattern is .z.{letter of data type wanted}
q)value each {x!x}`.z.p`.z.P`.z.t`.z.T`.z.d`.z.D`.z.n`.z.z`.z.Z
.z.p| 2013.05.21D09:56:06.145203000
.z.P| 2013.05.21D10:56:06.145203000
.z.t| 09:56:06.145
.z.T| 10:56:06.145
.z.d| 2013.05.21
.z.D| 2013.05.21
.z.n| 0D09:56:06.145203000
.z.z| 2013.05.21T09:56:06.145
.z.Z| 2013.05.21T10:56:06.145
getenv
getenv `LOG_DIR
q)getenv `PATH
"C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;C:\\Windows\\syst..
From the console you can use ctrl+c to break during a long running command:
http://www.timestored.com/kdb-guides/debugging-kdb#interrupt-q
Make sure to exit debug mode so the server answers client queries.
You could import printf like functionality using a C shared library (windows DLL, Linuz .so)
If you start kdb with the argument -T:
q -T 10
This will set a timeout on client queries of 10 seconds. Note not all queries are polite enough to stop when requested and I think commands at the console may ignore this setting.
You should also be able to modify this setting using
\T (seconds)
from within kdb.
