Interesting fun talk on finding the highest correlated streams among thousands of streams extremely efficiently by comparing streams against random generated data rather than exhaustively against each other. Followed up by pattern detection over different time windows efficiently which I think is this paper: .
Slides available here.
To those that have ever been forced to write functional selects, the shortcut notation allows a much more readable form.
A topic I find extremely interesting, unfortunately the material was probably too densely packed and I couldn’t follow along well. Hopefully the slides will be released so I can get the details at my own pace. Joe has a great blog http://scalability.org full of high performance articles.
Multithreaded slaves as standard – start kdb with an overridden -s and you can run separate kdb processes on different parts that will each perform part of the query. Standard slaves with par.txt were proving less worthwhile especially for SSD’s. Curious to see how this will compare to mserve.
-23! – optimise reading entire tables by mapping them in all at once. Really I want this to happen automatically (query optimizer please) but I guess we have to be happy with the faster speed.
On TimeStored we monitor visitor statistics, so assuming that most visitors are going to work in kdb this gives us insight into where kdb consultants/developers are located, what OS/browser they use and how long they spent on the site. For example:
kdb locations
If you work in kdb your probably underneath a dot on this map.
 Kdb Developers by Country |
 Kdb Developers by City |
We can see if you work in kdb your probably located in either the UK or US, mostly London or New York. It’s interesting to see Belfast and Newry show up but easily explained as First Derivatives HQ is in Newry and both Citi and First Derivatives have offices in Belfast with kdb staff.
Now that we have seen where kdb developers are located let’s consider what OS/browser they use:
 Kdb Developer Attention Span |
 kdb developer web browsers |
 kdb user operating systems |
UK users spent 1 minute longer on the site than their US counterparts 🙂 Though this could partially be explained by the site being more responsive in the UK as the hosting was based there.
Surprisingly (to me) chrome was the number one browser and someone was using freebsd!