kdb q BadTail Error

Commonly a kdb tick architecture is used to capture data. Part of this is a TP/TickerPlant that recives data, captures it to log and forwards messages on. That log is used for playback and recovery. If it somehow becomes corrupt you will receive a 'badtail error when trying to play it back.

Replaying Tickerplant Log File using -11!

Here we create a log file and replay it successfully using -11!

q)t:([] a:1 2)

q)`:log set () / create an log file and add some entries
`:log
q)h:hopen `:log
q)h enlist (`upd;`t; enlist 6 7) / add log entries
432i
q)h enlist (`upd;`t; enlist 8 9) / add log entries
432i
q)hclose h

q)t
a
-
1
2

q)-11!`:log / playback the log file using -11!
2
q)t
a
-
1
2
6
7
8
9

Badtail Error caused by corrupt log

A common cause of badtail errors is if a disk is full and the log is truncated. I recreated this by trimming the log and trying -11! again:

q)/ I then removed the end of a file using a hex editor
q)-11!`:log /  now I get an error
'badtail

Workaround for badtail Error

To workaround a badtail error we can use a special -11!(-2; logFile) command to find how many chunks are valid and then use -11(chunksNumber; logFile) to play back only up to that point.

q)-11!(-2;`:log)  / use special -11! call to find valid chunks
1 50
q)/ 1 valid chunk until position 50
q)-11!(1;`:log) / playback the one valid chunk
1
q)t
a
-
6
7
q)-11!(1;`:log) / playback the one valid chunk
1
q)t
a
-
6
7
6
7

Replaying Tickerplant Log File using -11!

Badtail Error caused by corrupt log

Workaround for badtail Error

See Also