What exactly does each column mean? This post documents the Bar Generator output schema end-to-end — precise definitions, units, and formulas — so your research, backtests, and AI features remain deterministic and audit-grade.
Sample header (exact columns)
Gmt time,Ask_first,Ask_max,Ask_min,Ask_last,Bid_first,Bid_max,Bid_min,Bid_last,AskVolume_sum,BidVolume_sum,Spread_mean,Spread_max,Mid_first,Mid_last,Tick_count,Vol_imbalance,Gap_flag,Timestamp_ms,VWAP,HL_range,Mid_std,Price_change_cnt,AskVol_pct,BidVol_pct,Return,HL_ratio,Quote_spread_varReading basics
- Timezone: All timestamps are strict
UTC.Gmt timeis timezone-aware (+00:00). - Price units: All prices/spreads are in price units (not pips). Convert to pips via symbol pip size (e.g., EURUSD pip =
1e-4). - Volumes:
AskVolume_sum/BidVolume_sumaggregate tick volumes (broker/export-specific units). - Determinism: Same inputs + same policy ⇒ bit-for-bit identical outputs.
Column dictionary
| Column | Type | Units | Definition / Formula | Notes |
|---|---|---|---|---|
Gmt time |
datetime (ISO-8601) | UTC | Bar start timestamp, timezone-aware (e.g., 2015-01-01 22:00:00+00:00). |
Strict UTC avoids DST drift and keeps merges consistent. |
Timestamp_ms |
int64 | ms since epoch (UTC) | floor(epoch_ms(Gmt time)). |
Convenient for joins/indexing in data pipelines. |
| — Ask side OHLC within the bar — | ||||
Ask_first | float | price | First ask quote in the bar. | Ask open. |
Ask_max | float | price | Maximum ask quote across ticks in bar. | Ask high. |
Ask_min | float | price | Minimum ask quote across ticks in bar. | Ask low. |
Ask_last | float | price | Last ask quote in the bar. | Ask close. |
| — Bid side OHLC within the bar — | ||||
Bid_first | float | price | First bid quote in the bar. | Bid open. |
Bid_max | float | price | Maximum bid quote across ticks in bar. | Bid high. |
Bid_min | float | price | Minimum bid quote across ticks in bar. | Bid low. |
Bid_last | float | price | Last bid quote in the bar. | Bid close. |
| — Volumes and mix — | ||||
AskVolume_sum | float | volume units | \sum AskVolume_i over ticks in bar. |
Broker/export dependent units. |
BidVolume_sum | float | volume units | \sum BidVolume_i over ticks in bar. |
Broker/export dependent units. |
AskVol_pct | float | ratio (0–1) | AskVolume_sum / (AskVolume_sum + BidVolume_sum) |
Share of ask volume. |
BidVol_pct | float | ratio (0–1) | BidVolume_sum / (AskVolume_sum + BidVolume_sum) |
Share of bid volume. |
Vol_imbalance | float | volume units | AskVolume_sum − BidVolume_sum |
Positive = ask-heavy flow; negative = bid-heavy. |
| — Mid price, spread & variability — | ||||
Mid_first | float | price | (Ask_first + Bid_first) / 2 |
Mid open. |
Mid_last | float | price | (Ask_last + Bid_last) / 2 |
Mid close. |
Spread_mean | float | price | Mean of (Ask_i − Bid_i) across bar. |
Convert to pips via pip size. |
Spread_max | float | price | Max of (Ask_i − Bid_i) across bar. |
Useful for stress screens. |
Quote_spread_var | float | price² | Variance of (Ask_i − Bid_i) across bar. |
Measures intra-bar spread instability. |
Mid_std | float | price | Standard deviation of mid prices within bar. | Micro-volatility proxy. |
HL_range | float | price | max(Mid_i) − min(Mid_i) |
Absolute intra-bar range. |
HL_ratio | float | ratio | HL_range / VWAP |
Scale-free range; ≈ relative range. |
| — Trading activity & returns — | ||||
Tick_count | int | count | Number of ticks captured within the bar interval. | Zero only if interval is empty and policy allows empty bars. |
Price_change_cnt | int | count | Times the mid price changed vs. previous tick inside the bar. | Microstructure churn indicator. |
VWAP | float | price | \sum(Mid_i × Vol_i) / \sum Vol_i, where Vol_i = AskVol_i + BidVol_i |
Falls back to simple mean if volume unavailable. |
Return | float | log-return | By default ln(Mid_last / Mid_first). (Close-to-close mode is configurable and documented in the manifest.) |
Use for bar-level features; deterministic per policy. |
| — Gaps & data quality flags — | ||||
Gap_flag | int (0/1) | flag | 1 if a missing-ticks gap or feed interruption affected the bar; else 0. |
Exact gap policy is documented in the bar policy doc. |
Conversions & examples
- Spread (pips):
Spread_mean / pip_size. For EURUSD (pip =1e-4),0.00024 ≈ 2.4pips. - Relative range (%):
HL_ratio × 100. - Return (bps):
Return × 10,000(approx.).
FAQ
Are spreads stored in pips?
No — spreads are in price units. Convert using the symbol’s pip size (e.g., 1e-4 for EURUSD).
Which return definition do you use?
Default is log return of mid open→close (ln(Mid_last/Mid_first)). Close→close is available as a documented option.
What happens if an interval has no ticks?
Policy can be no bar or empty/marked bar. The chosen rule is explicit and deterministic, and gaps set Gap_flag = 1.
Do you always weight VWAP by volume?
If tick volumes are present, VWAP is volume-weighted; otherwise, it degrades to a simple mid mean — behavior is documented in the manifest.
Try it with your data
Send a sample day of ticks; we’ll return M1–D1 bars + manifest + policy doc, so you can validate definitions and downstream impact.