SMA Crossover Backtest in Depth: Fees, Parameter Sweep, Overfitting Risk

This walkthrough extends the hero script with two additions that every serious backtest needs: a fee sensitivity analysis (how much do costs actually matter?) and a parameter sweep across three window combinations (how stable is the result?). The sweep output is the most important lesson on this page: the same dataset gives results from +106.6% to +134.4% depending purely on which windows you pick. That spread is not a signal. It is the overfitting risk, quantified.

"""AlgoDrill — SMA crossover backtest in depth with vectorbt.

Extends the hero script: fee sensitivity + a 3-combo parameter sweep
to illustrate why a single run proves nothing. Backtest only — no broker.

Versions: vectorbt 1.0.0 · yfinance 1.4.1 · pandas 2.3.3 (run 2026-06-05)
Install:  pip install vectorbt yfinance
"""
import vectorbt as vbt

# 1. Data — same fixed window as the hero script (reproducible)
price = vbt.YFData.download(
    "SPY", start="2015-01-01", end="2024-12-31"
).get("Close")

# 2. Base run: 10/50 SMA crossover, 5 bps fees each side
fast = vbt.MA.run(price, 10)
slow = vbt.MA.run(price, 50)
entries = fast.ma_crossed_above(slow)   # golden cross → buy
exits   = fast.ma_crossed_below(slow)   # death cross  → sell
pf = vbt.Portfolio.from_signals(
    price, entries, exits, init_cash=10_000, fees=0.0005, freq="d"
)
print("=== Base run: fast=10, slow=50, fees=5 bps ===")
print(f"Total Return  {pf.total_return():>8.1%}")
print(f"Sharpe Ratio  {pf.sharpe_ratio():>8.2f}")
print(f"Max Drawdown  {pf.max_drawdown():>8.1%}")
print(f"Trades        {pf.trades.count():>5}")
print(f"Win Rate      {pf.trades.win_rate():>8.1%}")
print()

# 3. Buy-and-hold benchmark (same period, no fees)
bh = vbt.Portfolio.from_holding(price, init_cash=10_000, fees=0.0, freq="d")
print(f"Buy & Hold:   {bh.total_return():.1%} return  (Sharpe {bh.sharpe_ratio():.2f})")
print()

# 4. Fee sensitivity — same parameters, 0 vs 5 vs 20 bps
print("=== Fee sensitivity (fast=10, slow=50) ===")
for bps in (0, 5, 20):
    p = vbt.Portfolio.from_signals(
        price, entries, exits, init_cash=10_000, fees=bps / 10_000, freq="d"
    )
    print(f"  {bps:>2} bps  Return {p.total_return():>7.1%}  Sharpe {p.sharpe_ratio():.2f}  MaxDD {p.max_drawdown():.1%}")
print()

# 5. Parameter sweep — 3 fast/slow combos to illustrate sensitivity
#    WARNING: cherry-picking the best row here is exactly how backtest bias works.
print("=== Parameter sweep (same data, different windows) ===")
combos = [(5, 20), (10, 50), (20, 100)]
for f_w, s_w in combos:
    fa = vbt.MA.run(price, f_w)
    sl = vbt.MA.run(price, s_w)
    en = fa.ma_crossed_above(sl)
    ex = fa.ma_crossed_below(sl)
    p  = vbt.Portfolio.from_signals(
        price, en, ex, init_cash=10_000, fees=0.0005, freq="d"
    )
    print(
        f"  fast={f_w:>2}/slow={s_w:>3}"
        f"  Return {p.total_return():>7.1%}"
        f"  Sharpe {p.sharpe_ratio():.2f}"
        f"  MaxDD {p.max_drawdown():.1%}"
        f"  Trades {p.trades.count():>3}"
    )

Script Output

=== Base run: fast=10, slow=50, fees=5 bps ===
Total Return    106.6%
Sharpe Ratio      0.87
Max Drawdown    -15.1%
Trades           30
Win Rate         46.7%

Buy & Hold:   240.8% return  (Sharpe 0.95)

=== Fee sensitivity (fast=10, slow=50) ===
   0 bps  Return  112.8%  Sharpe 0.90  MaxDD -14.9%
   5 bps  Return  106.6%  Sharpe 0.87  MaxDD -15.1%
  20 bps  Return   89.1%  Sharpe 0.77  MaxDD -15.6%

=== Parameter sweep (same data, different windows) ===
  fast= 5/slow= 20  Return  134.4%  Sharpe 1.03  MaxDD -19.2%  Trades  67
  fast=10/slow= 50  Return  106.6%  Sharpe 0.87  MaxDD -15.1%  Trades  30
  fast=20/slow=100  Return  111.7%  Sharpe 0.81  MaxDD -22.0%  Trades  12

Reading the Results

Base run and buy-and-hold benchmark

The base run (fast=10, slow=50, 5 bps fees) reproduces the hero script: +106.6% return, 0.87 Sharpe, −15.1% max drawdown, 30 trades, 46.7% win rate. The buy-and-hold benchmark is essential context: the same SPY position held from 2015-01-01 to 2024-12-31 with no trading returned +240.8% with a Sharpe of 0.95. The SMA strategy underperforms on return and on risk-adjusted return, but reduces max drawdown.

Fee sensitivity: 0 vs 5 vs 20 bps

Fees are not a cosmetic assumption. Moving from 0 to 20 bps per side drops the return from +112.8% to +89.1% — a 24-point swing — and the Sharpe from 0.90 to 0.77. For the 10/50 combination with 30 trades, the cumulative fee burden over 10 years is manageable. For the 5/20 combination with 67 trades, the same fee increase would be more damaging (roughly 2.2× more trades, 2.2× more cost).

The lesson: always run your backtest at realistic fees, not zero. Zero-fee backtests overstate performance and misrepresent the impact of trading frequency. Retail US equity commissions are now effectively $0 at most brokers, but spread and market impact are not zero, especially for less liquid instruments or larger position sizes. The 5 bps assumption is conservative for liquid ETFs but reasonable as a lower bound.

Parameter sweep: the overfitting risk in numbers

The three-combo sweep — fast=5/slow=20, fast=10/slow=50, fast=20/slow=100 — is not a comprehensive search. It is a minimal illustration of sensitivity. The results range from +106.6% to +134.4% return, from Sharpe 0.81 to 1.03, and from 12 to 67 trades, all on the same historical dataset.

The spread between best and worst is 27.8 percentage points on return and 0.22 on Sharpe. For context: the difference between a strategy you trade and one you don't is often measured in Sharpe differences of 0.1–0.3. A 0.22-Sharpe swing from parameter choice alone means the in-sample rankings are unreliable for selecting which system to trade.

What to Do After the Sweep

The correct use of a parameter sweep is not to pick the best row. It is to:

1. Understand sensitivity: If every combination produces similar results, the strategy is robust to the parameter choice. If results scatter widely (as above), the parameter is a model degree of freedom that the optimizer can fit to noise.
2. Pre-commit before running: Define your hypothesis about which parameter direction should work (longer windows = fewer signals = less whipsaw) before running the sweep. Post-hoc rationalization of results is a form of data snooping.
3. Validate out-of-sample: Reserve a period of data the sweep never sees. Run walk-forward analysis: train on rolling windows, test on the next period, and measure whether performance persists. See Walk-Forward Analysis for the methodology and Backtesting Pitfalls for the seven failure modes this addresses.