Generative Limit Order Book Modeling with a Shared Token Vocabulary for Book State and Order Flow
Hao-Han Zhang, Sai Wang, Bo-Kai Cao, Feng-Rui Hua, Lionel M. Ni, Jian Guo
Abstract
Limit order book (LOB) generation has attracted growing interest as a tool for studying market microstructure, constructing market simulators, and producing synthetic order flow. As autoregressive sequence models become increasingly capable of modeling market events, a central design question is how to represent the interaction between evolving book state and order flow within a generative model. We propose a unified autoregressive architecture in which discretized book states and order-flow events occupy a shared token vocabulary. Rather than treating the LOB as a separate conditioning signal or reconstructing it only outside the model, we insert book-state tokens directly into the autoregressive context together with historical order tokens. A decoder-only Transformer is trained with last-position next-order supervision to predict the immediate next event from this joint sequence. During generation, each sampled event is applied to the simulated book, and the resulting state is retokenized before the next prediction. Rather than cross-entropy, we conduct evaluations under free-running generation using distributional distances between generated and real-market microstructure statistics. Experiments on sample Chinese A-share stocks show that the proposed model achieves the strongest aggregate replay fidelity among the configurations considered. The controlled comparisons further suggest that how book state and order flow are combined can materially affect the microstructure statistics produced under autoregressive replay. Taken together, these results support shared-vocabulary modeling of book state and order flow as a straightforward and effective design for closed-loop market simulation, while highlighting joint book–order representation as an important factor in preserving market-dynamic fidelity in generative LOB models.
Source: semanticscholar · PDF
Read the AI summary and key takeaways for traders on WOBR Quant Research.