Documentation

BAR6 Format

partiqon's binary data format for efficient, transparent storage and retrieval of market data. Designed for speed, compact storage, and no vendor lock-in.

Overview

BAR6 is an open, documented binary format for storing historical market data. It supports both bar data (OHLCV) and raw tick data, with bid/ask information. Unlike proprietary formats, BAR6 is fully documented and designed to be read by any tool.

File Structure

Every BAR6 file starts with a 49-byte binary header, followed by fixed-size records:

[Header (49 bytes)]
[Record 1] [Record 2] ... [Record N]

Records are fixed-size and stored in ascending time order. That means record n sits at 49 + n × record_size, so any tool can seek directly to a point in a multi-gigabyte file without parsing what comes before it.

Header (49 bytes)

Little-endian, no padding. Python format string: <IIBIIqq16s

Offset	Size	Type	Field	Description
0	4	uint32	magic	0x42415236 (the ASCII bytes "BAR6")
4	4	uint32	version	2 = current
8	1	uint8	data_type	0 = bar records, 1 = tick records
9	4	uint32	timeframe_minutes	e.g. 15 for M15. 0 for tick files, since tick data has no timeframe
13	4	uint32	record_count	Number of records that follow
17	8	int64	start_ts	First record's timestamp, Unix milliseconds UTC
25	8	int64	end_ts	Last record's timestamp, Unix milliseconds UTC
33	16	char[16]	symbol	Null-padded instrument name, e.g. "DEU.IDX-EUR"

Bar Record (56 bytes each)

Present when data_type = 0. Python format string: <q6d

Offset	Size	Type	Field	Description
0	8	int64	timestamp	Bar OPEN time, Unix milliseconds UTC
8	8	double	open	Opening price
16	8	double	high	Highest price in the bar
24	8	double	low	Lowest price in the bar
32	8	double	close	Closing price
40	8	double	bid	Last bid quote in the bar. Together with ask, the spread a fill pays
48	8	double	ask	Last ask quote in the bar. Together with bid, the spread a fill pays

There is no volume field on a bar record. BAR6 carries bid and ask instead, because a backtest needs to know what a fill would have cost and cannot recover that from a volume figure.

Tick Record (32 bytes each)

Present when data_type = 1. Python format string: <q3d

Offset	Size	Type	Field	Description
0	8	int64	timestamp	Unix milliseconds UTC
8	8	double	bid	Bid price
16	8	double	ask	Ask price
24	8	double	volume	0 when the source did not supply one

Converting Your Data

Use DataConvert to convert your historical data (CSV, MT4 .hst) into BAR6 format. DataConvert runs locally on your machine, so your raw data never leaves your PC.

Input: CSV (timestamp, open, high, low, close, volume) or MT4 .hst
Output: .bin file in BAR6 format
Tool: github.com/hangmann/bar6-dataconvert

Why BAR6?

Compact: 56 bytes per bar, 32 bytes per tick, for efficient storage and fast reads
Documented: Complete specification, no proprietary mystery
Transparent: Read BAR6 files with any language (C++, Python, Java, etc.). Not locked to one platform
Your data, your rules: Convert once, use everywhere

Reading BAR6 Programmatically

BAR6 is simple enough to read in any language. This Python reads either kind of file, and it is executed against real BAR6 files as part of our test suite, so it is known to work rather than merely intended to.

import struct

HEADER_FORMAT = "<IIBIIqq16s"          # 49 bytes
BAR_FORMAT    = "<q6d"                 # 56 bytes
TICK_FORMAT   = "<q3d"                 # 32 bytes

with open("data.bin", "rb") as f:
    header = f.read(struct.calcsize(HEADER_FORMAT))
    (magic, version, data_type, timeframe_minutes,
     record_count, start_ts, end_ts, symbol) = struct.unpack(HEADER_FORMAT, header)

    assert magic == 0x42415236, "not a BAR6 file"
    symbol = symbol.rstrip(b"\x00").decode("utf-8")
    print(symbol, "M" + str(timeframe_minutes), record_count, "records")

    record_format = BAR_FORMAT if data_type == 0 else TICK_FORMAT
    record_size   = struct.calcsize(record_format)

    for _ in range(record_count):
        record = f.read(record_size)
        if len(record) < record_size:
            break                       # truncated file

        if data_type == 0:
            timestamp, o, h, l, c, bid, ask = struct.unpack(record_format, record)
        else:
            timestamp, bid, ask, volume = struct.unpack(record_format, record)
        # Process the record...

The < prefix is load-bearing: it pins little-endian byte order and disables the padding Python would otherwise insert between fields of different sizes. Without it the record sizes come out wrong on some platforms.

Seeking directly to a point in time

Because records are fixed-size and time-ordered, finding a timestamp is a binary search over the file rather than a scan: about 28 reads of 8 bytes each in a file with 170 million records.

def seek_to(f, header_size, record_size, record_count, target_ms):
    """Index of the first record at or after target_ms."""
    lo, hi = 0, record_count
    while lo < hi:
        mid = (lo + hi) // 2
        f.seek(header_size + mid * record_size)
        (timestamp,) = struct.unpack("<q", f.read(8))
        if timestamp < target_ms:
            lo = mid + 1
        else:
            hi = mid
    return lo