kdb+ HTTP web server

Kdb+ provides a basic implementation of a web server allowing browsing tables that exist on the server, performing queries and downloading data from a web browser. Kdb+ also provides calls to pull data via HTTP. We'll examing how to do this below:

Connect to kdb+ Web Interface
Query kdb+ from a Web Browser
.z.ph HTTP event handler
HTTP Requests - Download data from an internet address into kdb+

Connect to kdb+ Web Interface

To connect to a kdb+ server:

First make sure you know the address of the server you want to connect to.
Here we start a kdb process on our local machine port 5000 using the command q -p 5000
Next open your web browser and enter the host and port of the kdb process. Ours is localhost:5000,

Immediately you will be presented with a window with two panes

the left frame is a list of links to tables that exist
the right hand side will display any of the tables you click.

Query kdb+ from a Web Browser

To send a custom query to the server we use a /? followed by our query as shown:

If you insert a filename with the ending .csv, .xls or .xml this will download a file for that query from the server in either csv/excel format.

.z.ph http event handler

When an HTTP GET request is received, the request is processed by the .z.ph event handler. The argument it receives is a two item list of the form: ( request text ; request header ). Here I have saved it to a variable a:

q)a
"?select%20from%20t%20where%20sym=%60A"
`Host`Connection`Accept`User-Agent`Accept-Encoding`Accept-Language`Accept-Cha..
q)a 0
"favicon.ico"
q)a 1
Host           | "localhost:5000"
Connection     | "keep-alive"
Accept         | "*/*"
User-Agent     | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHT..
Accept-Encoding| "gzip,deflate,sdch"
Accept-Language| "en-US,en;q=0.8"
Accept-Charset | "ISO-8859-1,utf-8;q=0.7,*;q=0.3"

You can override this function to provide your own html interface. Here's an example where we override .z.ph to provide a QStudio like web interface:

Similarly there is a .z.pp event handler for HTTP post events.

HTTP Requests

From kdb+ we can send HTTP requests to webservers. For example we can send a request to yahoo finance servers for a csv file they provide:

	
q)a:`:http://ichart.finance.yahoo.com "GET /table.csv?s=MSFT http/1.0\r\nhost:ichart.finance.yahoo.com\r\n\r\n"
q)80 cut a
"HTTP/1.1 200 OK\r\nDate: Wed, 14 Aug 2013 16:34:30 GMT\r\nP3P: policyref=\"h..
"o.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVD..
" TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT ..
"CNT STA POL HEA PRE LOC GOV\"\r\nVary: Accept-Encoding\r\nConnection: close\..
"Type: text/csv\r\nCache-Control: private\r\n\r\nDate,Open,High,Low,Close,Vol..
"ose\n2013-08-13,32.51,32.55,32.21,32.23,39464100,32.23\n2013-08-12,32.46,32...
..

Let's wrap our http get in a function and then parse our data so we get a table:

	
q)httpGet:{[host;location] (`$":http://",host)"GET ",location," HTTP/1.0\r\nHost:",host,"\r\n\r\n"};
q)b:httpGet[ "ichart.finance.yahoo.com"; "/table.csv?s=MSFT" ]	
q) / remove header
q){(last x ss "Date,") _ x} b
"Date,Open,High,Low,Close,Volume,Adj Close\n2013-08-13,32.51,32.55,32.21,32.2..
q) / parse as table
q){("*********"; enlist ",") 0: (last x ss "Date,") _ x,"\r\n"} b
Date         Open    High    Low     Close   Volume     Adj Close
-----------------------------------------------------------------------
"2013-08-13" "32.51" "32.55" "32.21" "32.23" "39464100" "32.23"   "" ""
"2013-08-12" "32.46" "32.97" "32.46" "32.87" "25493700" "32.64"   "" ""
"2013-08-06" "31.55" "31.67" "31.38" "31.58" "36331500" "31.36"   "" ""
..

This http querying works for any type of page, even plain webpages, and could be used to query REST api's etc.

	
q)b:httpGet[ "www.timestored.com"; "/" ]
q)80 cut b
"HTTP/1.1 200 OK\r\nDate: Wed, 14 Aug 2013 16:45:24 GMT\r\nServer: Apache/2.2..
"ntu)\r\nX-Powered-By: PHP/5.3.2-1ubuntu4.18\r\nVary: Accept-Encoding\r\nConn..
"ose\r\nContent-Type: text/html\r\n\r\n        <!DOCTYPE html PUBLIC \"-//W3C..
" 1.0 Strict//EN\"\n                \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-..
"dtd\">\n        <html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"en\"..
" <head>\n                <meta http-equiv=\"content-type\" content=\"text/ht..
"set=iso-8859-1\" />\n                <title>TimeStored.com - Kdb Training, C..
"ing and Software</title>\n\t\t\t\t<link rel=\"icon\" href=\"/favicon.ico\" t..
..

Contents

Connect to kdb+ Web Interface

Query kdb+ from a Web Browser

.z.ph http event handler

HTTP Requests

See Also

QStudio