Towards Financial World Modeling

Market-1T spans all equities traded on US markets from 2008 through 2025. Raw quotes and trades are aggregated into one-second intervals, roughly one trillion observations. The base data is mid-frequency; the same aggregations extend to minute, hourly, or daily scales.

Each one-second snapshot carries 11 features, bucketed into three groups:

• Order book state: best bid/ask prices and sizes
• Price summary: open, high, low, close, and VWAP
• Activity: volume and trade count

Experiments focus on a liquid universe, re-calculated monthly from prior-month activity (Price > $5, ADV ≥ $20M).

The core tasks come from portfolio choice. To pick weights, an investor needs three forecasts: expected return, risk (volatility and covariances), and trading cost. We train models to predict each one, forward return, change in volatility, and change in bid-ask spread, at horizons from 5 minutes to 2 hours. These forecasts are not a trading strategy; they are the inputs a strategy would consume.

• Return (primary evaluation): 5-way classification of 15-minute forward mid-price returns, one bin for zeros, the rest empirical quantiles.
• Volatility change: how much conditional volatility moves, the risk term in the portfolio problem.
• Spread change: how the bid-ask spread moves, a standard proxy for transaction cost.

Supervised models train on these labels directly, one task at a time or all three at once. SSL models see no labels; we read their representations afterward with linear probes on the same targets. We also probe for things finance does not usually label, namely ticker, time of day, and trading day within the month (plus a separate test on SEC-documented manipulation windows). These are nearly uncorrelated with return, volatility, and spread, so a supervised model has no reason to learn them. If a representation predicts them anyway, it is carrying structure beyond the task it was trained for.

Vision SSL leans on strong augmentation, and the workhorse is random scaling and cropping. We port it to markets. Sample a scale s and a random window of the asset-day, then resize to a fixed token length L. The catch is the resize: instead of interpolating pixels, we re-aggregate the way markets already do, last order book state, OHLC, VWAP, summed volume and trade count.

Global views cover broad intraday structure; local views zoom in on shorter horizons. They are sampled independently and may not overlap, so the encoder has to recognize the same day across different windows, resolutions, and sampling rates. Two global views at L = 2048 with s ∈ [0.5, 1.0]; local views at L = 512 with s ∈ [0.05, 0.5]. This augmentation is the default for LeJEPA, DINO, and BYOL.

LeJEPA is our main self-supervised baseline. It learns in representation space: rather than reconstruct raw prices, it predicts the latent of one view of an asset-day from another. The crop makes the views, the encoder embeds each one, and a predictor maps between them, with no return or volatility labels anywhere.

Prediction alone has a trivial solution: map everything to the same point. SIGReg (default λ = 0.01) blocks that collapse by keeping the embedding distribution spread out. The default run is a ViT backbone, learning rate 5×10⁻⁴, six local views, batch size 128. We then freeze the backbone and read it with linear probes on the tasks above, alongside supervised models that did see the labels. On the core tasks the gap is widest at the supervised head and narrows once both are read the same way, by a probe.

We adopt a strict temporal evaluation scheme. All hyperparameter selection and training happen on data prior to calendar time t; final models are evaluated strictly after t. Interleaving training and evaluation days introduces look-ahead bias and is disallowed.

Performance is reported as ΔAUC = AUC(trained) − AUC(random-init), averaging the random baseline over 10 initializations. Even a probe on random projections can beat chance.

Rolling multi-regime evaluation samples 32 training months and evaluates on the immediately following month. The primary task is 5-way classification of 15-minute forward mid-price returns, with one bin reserved for zero returns and the rest empirical quantile bins.

We compare two families. Supervised models learn from labels, predicting future return, volatility change, or spread change from a past window. LeJEPA and the other self-supervised methods learn from views, aligning representations of global and local crops of the same asset-day with no labels.

We evaluate LeJEPA, MAE, CPC, DINO, BYOL, and supervised baselines on the core tasks: return, volatility change, and spread change. Supervised still wins. Across every backbone, the supervised head beats its self-supervised counterpart, and the gap shrinks, but does not close, once we read both with a probe instead of the head.

The tasks also help each other. A model trained on volatility or spread change transfers onto return prediction, and a single backbone with one head per task beats the return-only baseline. The spillover runs one way: into return, not out of it.

A world model should know more than the next price move. It should know where it is: which asset it is watching, what kind of market state it is seeing, where in the calendar it sits. The figure below is a real t-SNE of market-window embeddings, six stocks from three sectors, one global-view crop per ticker-day, colored by ticker.

Under LeJEPA, the points sort into tight per-ticker islands, structure the return label never asked for. Under Supervised, the same six stocks collapse into a far more undifferentiated cloud. The islands are per-ticker, not per-sector: the two semis, two banks, and two oil majors do not merge. That points to each stock's microstructure fingerprint, not its industry.

Prices are not the only thing that moves after news. Sometimes the bigger effect is uncertainty. We condition a frozen LeJEPA encoder on SEC 8-K filings and ask whether disclosure content improves predictions of the post-event market.

A market model that only works in one month is not a market model. The evaluation has to survive different regimes — which is why Market-1T spans 2008 through 2025.

The result is not “self-supervision wins” or “supervised learning is enough.” It is more interesting: Supervised models are better at the targets finance already knows how to name, while LeJEPA preserves structure those targets leave out: asset identity, calendar structure, microstructure fingerprints.

Finance has good labels, but the market holds structure those labels do not name. Closing the gap on the core tasks while keeping that structure is the open problem.

• Not a trading strategy. High AUC does not imply profit after execution, transaction costs, latency, sizing, and risk constraints.
• Not a full market model. The study focuses on single-asset representations; cross-asset structure and portfolio interactions are unexplored.
• Not risk-free to release. Market representations could be misused for manipulation or surveillance evasion. Current data is withheld.