Most investment platforms can show you a chart. Far fewer can reconstruct the portfolio as it actually existed on a given date - with the data, weights, flows, and risk state known at the time. The chart you are looking at is almost certainly not the history of your portfolio. It is a pro-forma backtest of your current book.

Once you accept that, a different set of questions becomes urgent:

• How much did the portfolio make, excluding deposits and withdrawals?

• Did NAV increase because the portfolio made money, or because new capital entered the account?

• Was the strategy beating its benchmark on a true time-weighted basis?

• Was the profit realized, or was it still mark-to-market?

• What was the Sharpe Ratio of the portfolio you actually held - not today’s holdings projected backward?

• Which investment thesis drove the weights at that time, and did it actually generate risk-adjusted return?

And if you manage external capital, the audit questions get harder:

• What NAV did the system show for March 15, 2026 as of 10:00 AM on March 16?

• If a vendor corrected a price after the report was published, can you show the original NAV, the corrected NAV, and the reason for the difference?

• Which pricing policy, FX rate, corporate-action version, market calendar, and code version produced that number?

• Was the NAV live-recorded at EOD, reconstructed later, or manually restated?

• Can you rerun the same risk model on the same data the system had then and get the same result?

Most systems can show a portfolio. Far fewer can answer these questions honestly. This is why a serious investment platform needs a Portfolio Management System (PMS) and, more importantly, an Investment Book of Record (IBOR).

This is the second post in the PMS series. The first one - Designing a PMS. covers TWRR, MWRR, and the broader return-measurement landscape.

Below are screens from QuantJourney PMS - holdings view: positions, weights, prices, P&L, sector mapping, sleeves, and live portfolio state:

Allocation view - sector, industry, country, and asset-class structure preserved as part of the portfolio state, not reconstructed from today’s holdings.

Attribution view - contribution, active weight, sector effects, contributors and detractors tied back to the actual book.

Risk Lab - VaR/CVaR, expected shortfall, Monte Carlo paths, probability of loss, and drawdown computed from the actual portfolio state.

It shows attribution, benchmark regression, sector allocation, and all the usual analytics. But as it’s not enough, we added the IBOR.

A note on vocabulary: IBOR vs ABOR.

Institutional platforms distinguish two books of record, and the distinction matters because they answer different questions and serve different stakeholders.

The Accounting Book of Record (ABOR) is the back-office view. It tracks settled positions, official NAV-of-record for LPs, audited cash and securities balances, and regulatory reporting. It is the source of truth for fund administrators, auditors, and tax. It is intentionally conservative - it reflects what has settled, not what is intended.

The Investment Book of Record (IBOR) is the front-office view. It tracks intended exposures including pending and in-flight trades, reflects the portfolio as the manager sees it for decision-making, and supports real-time P&L, attribution, risk, and pre-trade compliance. It is the source of truth for the CIO and the portfolio manager.

A serious platform needs both, and they need to reconcile. ABOR without IBOR leaves the manager flying blind on intra-day exposure. IBOR without ABOR leaves the LP without an auditable NAV. The two books should agree on settlement date and disagree (in controlled, explainable ways) on trade date.

QuantJourney is built IBOR-first - the manager’s view is the primary object - with a reconciliation layer to the fund administrator’s ABOR (will share a post on reconciliation engine in the future). Bitemporal storage is what makes that reconciliation honest: every difference between IBOR and ABOR is reproducible, dated, and reasoned.

Portfolios are Evolving Narratives

Portfolios are not static baskets. They are evolving expressions of investment theses.

• Last year, the strongest conviction in your portfolio may have been Precious Metals.

• This year, perhaps Oil and Energy contributed more to the bottom line.

• The AI thesis may still be structurally right, but the expression of that thesis has shifted: from semiconductors to data centers, then to power, software, and infrastructure.

If we want to evaluate that evolution twelve months later, a current-holdings screen is not enough.

We need to know what the portfolio actually looked like then - not just what today’s holdings would have done in the past.

Why PMS and IBOR matter

A portfolio analytics screen can show positions, charts, and returns.

A Portfolio Management System has to preserve the operational state of the portfolio: positions, cash, tax lots, transactions, NAV, flows, exposures, and valuations.

An Investment Book of Record goes one level deeper. It becomes the source of truth for what the portfolio was, when it was known, and how each value was computed.

This is the distinction we care about in QuantJourney.

We do not want the portfolio layer to be a chart cache.

We want it to be the investment memory of the platform.

The Common Shortcut: The “Pro-Forma” Trap

Many analytics systems produce portfolio history by using a dangerous shortcut:

current_weights × historical_prices

This produces a beautiful return series. But it answers a different question: “What would today’s portfolio have done historically?”

It does not answer: “What did my actual portfolio do?”

• If I own Oil today, it doesn’t mean I owned it last year.

• If I increased AI exposure in March, that weight didn’t exist in January.

• If I sold Gold after it peaked, today’s portfolio no longer contains the thesis that generated last year’s return.

The resulting chart is not the history of your portfolio. It is a pro-forma backtest of your current book. In QuantJourney, we separate these explicitly.

Why Risk Metrics Become Poisoned

Sharpe Ratio, Volatility, Drawdown, and Beta are all functions of a return series. If the return series is synthetic (pro-forma), the risk metrics are synthetic too.

A Sharpe Ratio calculated on today’s holdings projected backward may be mathematically valid, but it is not the Sharpe Ratio of the portfolio you actually held. If the system doesn’t know what you held during a market crash, the “drawdown” it shows you isn’t your drawdown - it’s the drawdown of a hypothetical object that never existed.

NAV as the Accounting Spine

The right way is to persist the portfolio’s daily state - not just prices, but the portfolio itself. In QuantJourney, this becomes the canonical daily NAV and return spine.

The schema is not the product. The questions it allows us to answer are the product.

ALTER TABLE nav_history ADD COLUMN inflows               NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN outflows              NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN fees                  NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN dividends_received    NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN interest_received     NUMERIC(19,8) DEFAULT 0;

ALTER TABLE nav_history ADD COLUMN twrr_daily            NUMERIC(19,8) DEFAULT NULL;
ALTER TABLE nav_history ADD COLUMN modified_dietz_daily  NUMERIC(19,8) DEFAULT NULL;

ALTER TABLE nav_history ADD COLUMN gross_exposure        NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN net_exposure          NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN leverage              NUMERIC(19,8) DEFAULT 0;

ALTER TABLE nav_history ADD COLUMN realized_pnl_day      NUMERIC(19,8) DEFAULT 0;
ALTER TABLE nav_history ADD COLUMN unrealized_pnl_day    NUMERIC(19,8) DEFAULT 0;

ALTER TABLE nav_history ADD COLUMN valuation_run_id      UUID;
ALTER TABLE nav_history ADD COLUMN input_hash            TEXT;
ALTER TABLE nav_history ADD COLUMN pricing_policy_version TEXT;
ALTER TABLE nav_history ADD COLUMN nav_method            TEXT DEFAULT 'eod_mark_to_market';
ALTER TABLE nav_history ADD COLUMN calendar_mic          TEXT;
ALTER TABLE nav_history ADD COLUMN was_backfilled        BOOLEAN DEFAULT FALSE;

This infrastructure allows us to separate manager performance from capital movement.

Flows are not performance

Suppose the portfolio starts the day at $10 million, and an investor deposits $2 million. The portfolio ends the day at $12.1 million. A naive return calculation says:

(12.1m − 10.0m) / 10.0m = +21%

That is obviously wrong. The portfolio did not make 21%. Most of the move was a cash flow. The correct return needs to remove external flows - but how you remove them is where most platforms get sloppy, and where institutional readers start asking pointed questions.

There are two industry-standard ways to do this, and they answer slightly different questions. We store both.

True Time-Weighted Rate of Return (TWRR) is the GIPS-compliant definition. It removes the effect of cash flows by chaining sub-period returns, where each sub-period is broken at the moment of every external flow:

R_TWRR = Π (1 + r_i) − 1

Each r_i is computed between consecutive flows using the portfolio revaluation at the time of the flow. This requires intra-day NAV revaluation whenever capital enters or leaves the portfolio. It is the cleanest measure of manager skill, because it is fully insensitive to the size and timing of flows.

Modified Dietz is the dollar-weighted approximation. It weights each flow by the fraction of the period it was present in the portfolio:

R_MD = (NAV_t − NAV_{t−1} − F_t) / (NAV_{t−1} + Σ wᵢ · Fᵢ)

where wᵢ = (T − tᵢ) / T is the time-weight of each flow Fᵢ within the period. Modified Dietz does not require intra-day revaluation, which is why fund administrators have used it for decades - it is computable from EOD data alone.

At daily granularity with an end-of-day flow assumption (w = 0, i.e. the flow lands at close), Modified Dietz collapses to the simple form most platforms display and call “TWRR”:

R_t ≈ (NAV_t − NAV_{t−1} − F_t) / NAV_{t−1}

Applied to our example:

(12.1m − 10.0m − 2.0m) / 10.0m = +1%

That is the actual investment return for the day. It separates manager performance from capital movement. Without this correction, Sharpe, volatility, drawdown, benchmark-relative performance, and attribution are all polluted.

Here is the subtlety that matters at the institutional level: when flows occur strictly at EOD, true TWRR and Modified Dietz converge to the same number. When they do not - when a $2m deposit lands at 10:00 AM on a day the market moves 3% - the two diverge, and the spread between them is itself a signal. It is also where reconciliation against the fund administrator’s books either succeeds or fails.

This is why the schema has both twrr_daily and modified_dietz_daily as separate columns. When intra-day flow timestamps are available, twrr_daily is the chained sub-period return. modified_dietz_daily is always the dollar-weighted approximation. Their difference is recorded, not hidden - because in performance reconciliation, hidden differences are how disputes start.

The portfolio needs two clocks: Bitemporality

There is another problem: history changes. Prices get corrected, corporate actions arrive late, and vendors revise data. Most systems simply overwrite the old data, destroying the audit trail.

A serious portfolio system needs two clocks:

business time = the date the value applies to
system time   = when the system knew that value

That is bitemporality.

For NAV history, we store system-time validity:

CREATE EXTENSION IF NOT EXISTS btree_gist;

ALTER TABLE nav_history
  ADD COLUMN system_time_range TSTZRANGE NOT NULL DEFAULT tstzrange(now(), null);

ALTER TABLE nav_history
  ADD CONSTRAINT nav_history_no_system_overlap
  EXCLUDE USING gist (
    node_id WITH =,
    as_of_date WITH =,
    system_time_range WITH &&
  );

CREATE VIEW nav_history_current AS
SELECT *
FROM nav_history
WHERE upper(system_time_range) IS NULL;

If a NAV for a past date is corrected, we do not overwrite the old row.

We close the old system-time range and insert a new version.

Portfolio NAV for 2026-04-01

Version 1:
Known from: 2026-04-01 22:00
Known until: 2026-04-10 09:30
NAV: 10,000,000

Version 2:
Known from: 2026-04-10 09:30
Known until: current
NAV: 10,012,430
Reason: corrected corporate action / vendor restatement

Now we can answer two different questions:

What is the corrected NAV for 2026-04-01 today?

What did the system believe the NAV was on 2026-04-02?

That second question is what most systems cannot answer.

Why this matters for thesis-driven portfolios

This is not only about accounting, it is about investment reasoning. So portfolio evolves through decisions, and while one thesis gets bigger, another gets reduced.

Or you add a hedge, or any position moves from watchlist to active. Or a scenario is confirmed or invalidated and macro regime changes the sizing.

So, let’s say six months later you want to ask:

→ Show me the portfolio as we knew it on 2025-09-30.
→ Show me the weights by thesis.
→ Show me what was allocated to AI infrastructure.
→ Show me what was allocated to oil.
→ Show me what changed after the decision.
→ Show me the actual NAV TWRR after that point.
→ Show me the drawdown and Sharpe using the portfolio we actually held.

Without historical portfolio state, the system cannot answer that honestly. It can only reconstruct. And reconstruction is not the same as memory.

What the API should return - our QuantJourney implementation

The portfolio return series endpoint needs to expose not just values, but coverage and lineage.

Example:

{
  "node_id":"portfolio_XYZ",
  "mode":"actual_nav_twrr",
  "calendar_mic":"XNYS",
  "series": [
    {
      "as_of_date":"2026-04-25",
      "nav":1234567.89,
      "twrr_daily":0.00123,
      "modified_dietz_daily":0.00118,
      "inflows":0,
      "outflows":0,
      "gross_exposure":1.28,
      "net_exposure":0.74,
      "leverage":1.28,
      "drawdown_pct":-0.032,
      "was_backfilled":false
    }
  ],
  "coverage": {
    "trading_days_expected":251,
    "trading_days_available":249,
    "missing_dates": ["2026-01-15","2026-02-03"],
    "has_backfilled_rows":true,
    "backfill_share":0.18
  },
  "lineage": {
    "pricing_policy_version":"official_close_v3",
    "system_time":"2026-04-28T13:42:00Z"
  }
}

That coverage block matters. If part of the history is backfilled, the user should know. And backfilled history can still be useful.

But it should not pretend to be the same as live-recorded history.

Why calendar awareness matters

A 60-day rolling Sharpe should mean 60 trading days. Not 60 calendar days.

That sounds obvious until the portfolio is global. A US portfolio follows a different calendar from a Saudi portfolio. A multi-region portfolio may need a composite calendar.

Exchange holidays change. Some markets have different weekends.

So the NAV series needs to know the portfolio calendar:

calendar_mic = XNYS
calendar_mic = XSAU
calendar_mic = composite_global

Please note: we extend MIC with internal composite codes for multi-region portfolios (within qj-calendar service).

Same applies to e.g. rolling windows, which should be computed over the trading-day-indexed series. Not naive calendar arithmetic.

Final Thoughts: The Memory is the Product

The real question in investing is rarely: “What do we own now?” It is: “What did we believe, how did we size it, what changed, and did it work?”

If your “history” changes when today’s holdings change, it was never history. It was a reconstruction.

If corrected data overwrites the old record, your process is not reproducible.

And if your process is not reproducible, your risk metrics are not fully trustworthy.

At QuantJourney, we treat NAV history as infrastructure, not as a chart cache. The chart is just the output. The memory is the product.

— If you want to see this on your own portfolio or do PoC —

QuantJourney is in development and targeted to private deployment (private cloud) with family offices and emerging managers. If you run a fund, a family office, or a multi-strategy book and the questions at the top of this post are ones you cannot currently answer - replay your portfolio as it actually existed on a given date, separate manager skill from capital flow, reconstruct NAV with full lineage, audit a corrected price against the original - we can show you what this looks like on your own data.

Just reach me,

Jakub

• Contemporaneous correlation tells you how to hedge, not where to position - directional alpha requires a non-zero lag.

• Lead-lag and Granger tests detect those lags, but a statistical lead is not an economic edge until you can explain the transmission mechanism.

• Lead-lag is regime-conditional: a signal stable in one regime can invert in another, so rolling and state-indexed validation is mandatory.

• QuantJourney compiles these as ICDL thesis objects, regime-indexed, live-monitored, and audit-bound - this is the gap between research and tradable infrastructure

In the next posts, we will go deeper into how we are building QuantJourney - a full buy-side investment platform for funds and institutional investors.

As we’re building the QuantJourney portfolio management system (PMS), one issue keeps surfacing: most strategies are built on symmetry, while markets reward asymmetry.

Contemporaneous correlation is a trap. Two assets moving together tells you how to hedge, not where to position. You cannot extract directional alpha from a simultaneous move, by the time both legs have printed, the information is gone.

What you can trade is the structural delay between them: the supply-chain shock that hits the commodity before it hits the downstream equity, the macro print that moves rates before it moves cyclicals, the flow that hits the ETF before it propagates to constituents.

This is where real positioning advantages are built. If you understand the lag between cause and reaction, you are not guessing direction, you are waiting for the math to play out. Today we cover two classical buy-side tools for finding these asymmetries: Lead-Lag Detection and Granger Causality.

The Theory: How We Find Alpha in Time

1. Lead / Lag Detection (Cross-Correlation of Slopes)

When we talk about lead-lag, we are looking for a peak in the cross-correlation function at a non-zero lag. You shouldn’t run it on raw price series as they are non-stationary, and any correlation you compute will be dominated by common trends rather than genuine co-movement.

Instead, we work with log-returns:

Log-returns are (approximately) stationary, which is what makes the cross-correlation meaningful in the first place. We then evaluate:

If $\tau>0$ and the correlation is statistically significant, Asset A leads Asset B. This only matters when the transmission mechanism is economically real, regime-consistent, and not yet arbitraged away.

Let’s test this on a classic macro relationship: Crude Oil (USO) and the Energy Sector ETF (XLE). One caveat before we start - USO is not a clean proxy for spot crude. It holds rolling front-month futures, so contango and roll yield inject their own dynamics into series (I covered this in more depth in How to Trade Oil post). For a production signal we would use front-month WTI directly; USO is used here for reproducibility with free data. Does oil lead the equities, do equities lead the commodity, or is it instantly priced in?

tickers = ['USO', 'XLE']
data = yf.download(
    tickers,
    start='2020-01-01',
    end='2024-01-01',
    auto_adjust=False,
    progress=False,
)['Adj Close']

returns = np.log(data / data.shift(1)).dropna()

def calculate_cross_correlation(series_a, series_b, max_lag=10, min_obs=252):
    """
    Cross-correlation of returns at lags in [-max_lag, +max_lag].
    lag > 0  =>  A leads B by `lag` periods.
    Drops lags where effective sample size < min_obs (one trading year)
    to keep significance thresholds meaningful.
    """
    correlations = {}
    for lag in range(-max_lag, max_lag + 1):
        shifted_a = series_a.shift(lag)
        aligned = pd.concat([shifted_a, series_b], axis=1).dropna()
        if len(aligned) < min_obs:
            correlations[lag] = np.nan
            continue
        correlations[lag] = aligned.iloc[:, 0].corr(aligned.iloc[:, 1])
    return pd.Series(correlations)

cross_corr = calculate_cross_correlation(returns['USO'], returns['XLE'], max_lag=5)

In perfectly efficient markets, the peak correlation occurs exactly at τ = 0. If you see a statistically significant spike at $τ ≠ 0$ , it means yesterday’s oil move may provide a directional edge for today’s energy equities. In daily data, structural lags often stem from supply chain delays, institutional capital rotation, or retail flow catching up to macro news.

But detecting a lag is only the first step. A statistical lead is not yet an economic edge. The real question is: why does the lag exist?

In production, we don’t stop at “USO leads XLE”. We ask what drives the delay:

• earnings revision propagation,

• ETF flow mechanics,

• options hedging,

• sector reallocation,

• or simply a transient common factor move?

If we cannot explain the path by which information travels from Asset A to Asset B, we treat the signal as unstable and likely non-scalable.

Alpha is not in detecting lag - it’s in knowing when the lag is stable, tradable, and not yet arbitraged away. This is where most retail pipelines fail. They detect a non-zero lag and immediately label it alpha. That is premature. A lag without mechanism is often just a temporary statistical artifact. Structural alpha comes from identifying how information diffuses, who is slow to react, and under what market state that delay remains tradable.

More importantly, lead-lag relationships are rarely constant. They are conditional on regime. The same pair can exhibit a clean positive lag in one environment and collapse or invert in another. In our research stack, we model lag as a regime-dependent object:

This is why rolling statistics matter. A global average lag is often misleading. The only lags worth trading are the ones that remain stable within clearly defined states of the world.

A feature that passes a full-sample Granger test (read below) but fails in rolling windows or loses significance out-of-sample is not alpha., it is a backtest artifact. In our pipeline, such features are automatically excluded.

In QuantJourney, a primitive - what you would typically call an alpha factor - is not treated as a single time series but as a stateful intelligence object: trend, acceleration, z-score, percentile extremeness, and its relation graph to other primitives across multiple horizons. This matters because lead-lag rarely propagates through a clean pair. In real markets, information diffuses through a dependency network, not a single edge. Mapping that network is what allows us to move from pairwise statistics to regime-aware signal construction.

Even a statistically valid lag is not a strategy until it is mapped into execution. In practice, that means converting a relationship into a decision rule:

• when does the signal activate,

• how strong must it be,

• how long does it remain valid,

• how is size determined,

• and under what regime does it get disabled.

In other words, the path we use in QuantJourney OMS is as follows:

Signal → Position → Risk → Execution

This is the difference between a research finding and a tradable portfolio process.

def build_signal(feature_x, target_y, regime_state):
    lag_score = compute_lead_lag_score(feature_x, target_y)
    granger_score = rolling_granger_stability(feature_x, target_y)
    ic_score = rolling_information_coefficient(feature_x, target_y)

    if regime_state not in allowed_regimes(feature_x, target_y):
        return 0.0

    signal = lag_score * granger_score * ic_score
    return signal

def size_position(signal, regime_confidence, risk_budget):
    if signal < 1.5:
        return 0.0
    return risk_budget * regime_confidence * signal_decay_adjustment(signal)

2. Granger Causality (Feature Selection)

If lead-lag is a scalpel for timing, Granger Causality is a sieve for features. It asks a strict predictive question:

Does knowing the past values of Feature X improve our ability to predict, compared to predicting using ONLY the past values of Y?

The restricted model (Y explained by its own past):

The unrestricted model (adding lagged X as predictor):

Note that $a_i$ and γ_i are different estimates - they come from different regressions. Adding lagged X re-shuffles the explanatory work across all regressors, so the autoregressive coefficients on Y change as well. The test focuses on the new coefficients:

If the $\beta$ coefficients are jointly significantly different from zero (via an F-test), then X “Granger-causes” Y - meaning its lagged history carries predictive information beyond what Y’s own past already provides.

Granger causality is useful, but it is not causal inference in the economic sense. In production, it is better understood as a predictive inclusion test. It tells us whether the lagged history of X improves forecasting Y, not whether X is the true underlying driver.

This distinction matters because Granger tests can produce false positives in the presence of latent common factors, autocorrelation leakage, or regime shifts. For that reason, we do not rely on a single full-sample result. We evaluate rolling Granger stability, regime-conditional significance, and out-of-sample decay before promoting any feature into a live model.

Let’s test if the VIX (Volatility Index) Granger-causes SPY (S&P 500) returns. Can a spike in volatility today mathematically improve our forecast for tomorrow’s index return?

The Code: Granger Causality

from statsmodels.tsa.stattools import grangercausalitytests

tickers = ['SPY', '^VIX']
data_gc = yf.download(
    tickers,
    start='2022-01-01',
    end='2024-01-01',
    auto_adjust=False,
    progress=False,
)['Adj Close']

returns_gc = np.log(data_gc / data_gc.shift(1)).dropna()

# statsmodels expects a 2D array where:
#   Column 0 = TARGET (the variable we want to predict, here SPY)
#   Column 1 = PREDICTOR (the variable whose lagged values we test, here VIX)
gc_data = returns_gc[['SPY', '^VIX']].values

# maxlag=3: test whether up to 3 days of VIX history improves SPY forecasts
gc_results = grangercausalitytests(gc_data, maxlag=3)

Interpreting the output:

The grangercausalitytests function will output F-tests and Chi-square tests for each lag. You are looking for the p-values.

• If $p<0.05$ at lag 1: past values of VIX contain statistically significant predictive information about next-day SPY returns, beyond what SPY’s own past returns already provide.

• If $p>0.05$ : drop the feature. It may co-move contemporaneously, but it carries no forecasting alpha.

How QuantJourney Operationalizes This

Everything above is classical methodology - lead-lag and Granger tests have been in the buy-side toolkit for decades. What matters in production is not running the test, but building infrastructure that lets you trust the result across regimes, across time, and across audit cycles. This is where QuantJourney diverges from the standard quant-blog pipeline.

Primitives as stateful objects, not time series. In our research stack, a feature like “USO leads XLE by τ days” is not a scalar coefficient and not a single time series. It is a compiled object holding its trend, acceleration, z-score, percentile extremeness, regime-conditional lag distribution, and a relation graph to every other primitive in the system across multiple horizons. When the regime state changes, the object re-evaluates itself against its regime-specific prior — not the full-sample one.

Regime as a joint state, operationally defined. We index every primitive on a four-axis state vector: realized-volatility bucket (low / normal / stressed, rolling 63-day percentiles), liquidity regime (ADV and spread percentiles versus trailing year), macro trend (sign and persistence of the rates / dollar / credit factor), and positioning state (CFTC COT extremes, ETF flow percentiles). A lead-lag coefficient that is stable only in “low-vol, risk-on, long-positioning” is stored as such — and disabled everywhere else.

Hypotheses as compiled, live-monitored objects. The statement “USO leads XLE by 7 days in mid-vol regimes with rising crude term structure” is not a comment in a research notebook. In QuantJourney it is written in ICDL (Investment Concept Description Language), compiled into a live thesis object, and monitored continuously by the Scenario Monitor. When the regime drifts or the lag decays, the thesis is flagged, its conditional risk is re-priced, and the downstream positions are surfaced in the PM Decision Cockpit with a structured decision log. No human has to remember to check it.

Validation is institutional, not ad-hoc. Before any primitive enters the live signal graph, it passes walk-forward validation with CPCV, PBO calculation, and deflated Sharpe. SHA-256 reproducibility hashes are stored in the immutable audit vault so every production decision is traceable to the exact data, parameters, and code that generated the signal. A feature that looks good in-sample but fails CPCV is not quietly downgraded — it is excluded, with the rejection logged.

The Empirical Reality: Cutting Through the Noise

When you run our full QuantJourney pipeline, you generate a series of diagnostic plots. Let’s look at the output from our latest run and discuss what a Portfolio Manager actually sees behind the pretty colors.

1. The Lead-Lag Heatmap: Mapping the Ecosystem

Look at the Optimal Lag matrix. The ±7 values between USO and XLE are not noise - they indicate structural friction. In this dataset, USO movements tend to lead XLE by roughly 7 trading days.

Seven days is long for two liquid, continuously quoted instruments, and you should read it with some suspicion. Part of it is economically real: slow-moving capital rotates into energy equities only after a sustained commodity trend confirms itself, and sell-side earnings revisions for E&P names lag the spot print by a week or two. Part of it is an artifact of USO’s roll mechanics. Before sizing any position on this lag, we validate it against front-month WTI.

Notice the asymmetry. If the relationship were a pure contemporaneous beta, the optimal-lag matrix would be skew-symmetric: lag(A→B) = −lag(B→A). When that symmetry breaks, information is flowing preferentially in one direction, you are not looking at co-movement, you are looking at a transmission channel. If you trade energy stocks off today’s oil price, you are a week early. The alpha window opens after the commodity has already moved.

2. Granger Causality: Feature Selection or Noise?

This chart is the honest feature-selection test. At Lag 1 the p-value sits at 0.016, well below the 0.05 threshold.

VIX does not merely co-move with the market when it drops - its lagged values Granger-cause next-day SPY returns in the statistical sense defined above. For a machine learning model on SPY, VIX is a strong candidate feature: it carries information that SPY’s own price history does not. If all bars were above 0.05, we would drop the feature immediately to avoid overfitting downstream.

Remember the caveat from the theory section: VIX is derived from SPX options, so this is a predictive-inclusion result, not an economic causal claim. The leverage effect and volatility feedback are baked into the relationship.

3. The Strategy Deep-Dive: Rolling IC & Equity Curve

Now look at the Rolling 60d Correlation. This is the common failure mode in retail research. A global correlation of 0.17 gets treated as a constant, as if it were a physical law. It is not. You can see the relationship flip into negative territory during specific market regimes — the same “edge” that is tradable in one state of the world is a money-loser in another.

The backtest of this lag produces a Sharpe of 0.75 versus 0.32 for buy-and-hold. That is a solid improvement in risk-adjusted returns. The Information Coefficient is modest - ICIR of 0.33, which is “weak” by strict standards.

But the point of this signal is not raw return. It is the reduction in max drawdown, from −60% to −35%. That is what lets you stay in the game long enough to compound, and what lets you scale position size without blowing up on the next regime shift. In practice you would not “go long XLE” on this signal alone — you would use the lead-lag as a timing overlay on top of an existing sizing framework.

An IC of 0.04 sounds small, but in systematic trading magnitude matters less than stability. The break-even condition is roughly |IC| × σ_y × √breadth > cost_per_turnover. On large-cap US equities at ~2–3 bps round-trip, a persistent IC in the 0.02–0.04 range clears the bar comfortably. On small-caps or EM names at 15–30 bps, the same IC is an academic curiosity. The edge is not in any single signal - it is in aggregation, regime conditioning, and execution discipline.

Both $corr(slope_A(t), slope_b(t+lag))$ and Granger Causality are essential tools, but they are just the starting point for regime discovery.

• Lead-Lag tells you when to execute.

• Granger Causality tells you what to include in your model.

Run these tests on daily data and the zero-lag correlation will dominate most of the time - markets are highly efficient at daily resolution. The lags live in minute bars, tick data, and alternative sources: corporate-jet movements before M&A announcements, credit-card panel data before earnings, satellite imagery before commodity prints. But the higher you go in frequency, the more the Epps effect contaminates the result - asynchronous trading alone produces spurious lead-lag structure, and nothing tick-level can be trusted without non-synchronous sampling corrections.

And the numbers always have to clear costs. A Granger test flagging Feature X as predictive is table stakes; predictive is not profitable. Every feature we promote to the live model at QuantJourney is evaluated at realistic net-of-cost IC, not gross. Most don’t survive.

This post covered pairwise lead-lag. In reality, information does not diffuse through a single edge - it propagates through a dependency graph across hundreds of instruments, sectors, and macro factors simultaneously. In the future posts, we move from pairwise lags to the full dependency network: how QuantJourney compiles it into a live, regime-aware signal layer that survives regime shifts instead of breaking at them.

One final note on the methodology itself. Granger causality is linear by construction - it asks whether a linear combination of lagged X improves a linear forecast of Y. Real market dependencies are rarely that well-behaved. Tail dependence rises in crises, relationships become asymmetric under stress, and the same pair that looks uncorrelated on average can become highly predictive in the left tail. Transfer entropy - a model-free, information-theoretic measure of directed dependence - captures these nonlinearities without assuming any functional form. In QuantJourney we run both as complementary screens: Granger as the linear baseline, transfer entropy as the nonlinear layer. When they agree, the signal is robust. When transfer entropy is strong but Granger is weak, we are looking at a nonlinear dependency - tradeable, but with different sizing and different risk budgeting than a linear signal. When Granger is strong but transfer entropy is weak, it is usually a linear artifact that will not survive a regime shift. We will go deeper into the nonlinear layer in a future post.

Happy Trading!

QuantJourney

Read more

Read more

Credit is often a better read on underlying market stress. Credit is where macro, liquidity, and corporate bal…

Read more

The VIX measures the market’s expected 30-day volatility for the S&P 500 and is often treated as Wall Street’s fear gauge. But it is not something you can buy directly.

When VIX spikes, many traders …

Read more

Read more

My background is in high-energy physics. Before I started building quant models, I spent years on high-energy physics experiments - analysing particle collisions at TeV energies, measuring spin states, debugging Monte Carlo simulations on HPC clusters.

When I f…

Read more

And Alpha does not live in isolated metrics - it lives in interac…

Read more

As Roger Federer said in his Stanford commencement spe…

Read more

If you’re building a trading bot for Polymarket (or any prediction market)…

Read more

Let’s start.

Every few …

Read more

Portfolio management looks like a UI problem. In practice it’s a truth problem: keeping positions and cash in a state that can be rebuilt day by day, backwards, deterministically.

A real failure mode: a custodian reports transactions with…

Read more

I wish to share with you the fi…

Read more

I suddenly had multiple repositories moving fast: a production API server, a warehouse + ETL, a research we…

Read more

Over the past few weeks we built our Python framework for algorithmic trading on Polymarket. It’s an event-driven toolkit that cl…

Read more

Our latest milestone is the release of the Universe System - a production-hardened, point-in-time (PIT) accurate architecture for tracking major US equity indices (S&P 500, NASDAQ 100, Russell 2000, Do…

Read more

TL;DR

• VWAP/TWAP feels safe because it spreads your order out - but “smooth” can become a fingerprint.

• Regular slicing creates a rhythm in time and size that fast liquidity seekers can measure, not “guess.”

• Once they detect the rhythm, they can lean ahead of your next slice…

Read more

From an institutional quantitative perspective, this approach is fundamental…

Read more

Read more