Portfolio Optimization Methodology

1. Overview & Philosophy

FolioForecast's portfolio optimization engine applies Modern Portfolio Theory (MPT) and advanced quantitative techniques to find the best-risk-adjusted portfolio weights for your unique goals and constraints. The engine is built on three core principles:

1. Data-driven optimization. Real market data (historical prices, returns, volatilities, correlations) informs all decisions — not opinion or hunches.
2. Multiple perspectives. Instead of forcing one "best" portfolio, we offer 20 distinct optimization methods, each reflecting a different view of what "optimal" means — maximizing risk-adjusted returns, minimizing tail risk, matching risk contributions, or pursuing specific return/volatility targets.
3. Transparent constraints. Real portfolios have real constraints: sector limits, position caps, ESG filters, leverage bounds, minimum allocations. The optimizer respects all user-defined constraints while finding the best weights within those boundaries.

For each optimization method, FolioForecast:

1. Retrieves historical price data for all tickers (up to 10 years by default).
2. Calculates daily returns, annual volatility, and pairwise correlations.
3. Applies the chosen optimization method subject to all constraints.
4. Backtests the resulting portfolio against historical data.
5. Stress-tests it through major historical crises.
6. Runs Monte Carlo simulations to estimate forward-looking risk.

2. Data Sources & Coverage

FolioForecast integrates with EODHD (End-of-Day Historical Data), a premium global financial data provider covering 165,000+ ticker symbols across all major exchanges (US, UK, Canada, Europe, Asia, etc.).

Ticker Universe

• Stocks: All US-listed equities (NYSE, NASDAQ, OTC), plus international stocks from 180+ countries.
• ETFs: US-listed ETFs (iShares, Vanguard, SPDR, Invesco, etc.) and international ETFs.
• Indices: Major market indices (S&P 500, NASDAQ 100, Russell 2000, VIX, sector indices, international indices).
• Currencies: 150+ forex pairs (EUR/USD, GBP/USD, etc.).
• Commodities: Precious metals, energy, agriculture futures proxies.
• Fixed Income: Bond ETFs, government bond indices.

Data Specifications

• Historical depth: Up to 10 years of daily OHLCV (open, high, low, close, volume) data by default. Older data available for longer-term studies.
• Update frequency: Daily, adjusted for corporate actions (splits, dividends). Data arrives by market close + 1 hour.
• Survivorship bias handling: Delisted tickers are retained with their final available price to prevent overstating historical returns.
• Corporate actions: Adjusted prices reflect stock splits, reverse splits, dividend payments, and rights offerings.

Return Calculations

Daily Return = (Close_t − Close_t−1) / Close_t−1

Annual returns are annualized from daily data. Volatility is annualized standard deviation of daily returns (multiplied by √252 trading days). Correlations are computed from the covariance matrix of daily returns.

3. Twenty Optimization Methods

FolioForecast offers 20 distinct portfolio optimization approaches. Each answers a different question about what "optimal" means. Users select the method that best aligns with their investment philosophy and goals.

Mean-Variance Optimization (5 methods)

Based on Markowitz's Modern Portfolio Theory (1952). These methods trade off expected return against volatility.

1. Maximum Sharpe Ratio. Finds the portfolio with the highest risk-adjusted return (excess return per unit of volatility). This is the "best" portfolio for most investors with moderate risk tolerance.

   maximize: (E[R_p] − r_f) / σ_p

2. Minimum Volatility. Finds the portfolio with the lowest volatility — regardless of expected return. Ideal for risk-averse investors or those building a diversified core holding.

   minimize: σ_p = √(w^T Σ w)

3. Efficient Frontier (20 point grid). Generates 20 portfolios across the entire efficient frontier, from minimum-volatility to maximum-return. Users choose the risk/return trade-off that suits them.

   For each target return μ_target: minimize σ_p

4. Target Return. Finds the minimum-volatility portfolio that achieves a user-specified expected return.

   minimize σ_p subject to: E[R_p] = R_target

5. Target Volatility. Finds the maximum-return portfolio that stays within a user-specified volatility (risk) ceiling.

   maximize E[R_p] subject to: σ_p ≤ σ_target

Risk-Based Methods (2 methods)

These methods balance portfolio risk (volatility) rather than dollar allocation. They tend to be more stable through market cycles.

6. Risk Parity. Allocates weights so each asset contributes equally to total portfolio volatility. Requires leverage if low-volatility assets (bonds) are included.

   w_i = (1/σ_i) / Σ(1/σ_j)

7. Equal Risk Contribution (ERC). Similar to Risk Parity but without leverage — adjusts weights so volatility contributions are equal, staying fully invested (w_i ≥ 0, Σw_i = 1).

   minimize: Σ (w_i · σ_i − w_j · σ_j)²

Advanced Tail Risk & Robust Methods (5 methods)

These methods focus on downside risk, tail events, and estimation robustness.

8. Conditional Value at Risk (CVaR) Minimization. Minimizes the expected loss in the worst 5% of outcomes. Particularly useful for risk-averse portfolios and those concerned with tail risk.

   minimize: CVaR_α = E[R_p | R_p ≤ VaR_α]

9. Sortino Ratio Maximization. Like Sharpe Ratio but penalizes only downside volatility (negative returns). Ignores upside volatility.

   maximize: (E[R_p] − r_f) / σ_downside

10. Maximum Drawdown Minimization. Minimizes the maximum peak-to-trough decline in portfolio value over the historical period. Addresses investor fear of large losses.

    minimize: max(peak − trough) / peak

11. Black-Litterman Model. Combines market equilibrium returns (implied by market capitalization weights) with user-specified views on certain assets. Produces more stable allocations than raw optimization by shrinking toward market weights.
12. Hierarchical Risk Parity (HRP). Uses hierarchical clustering of asset correlations and recursively allocates risk down the tree. More robust to estimation error than standard mean-variance optimization.

Baseline & Specialty Methods (8 methods)

13. Equal Weight. Allocates 1/N weight to each asset. Serves as a simple, intuitive baseline. Surprisingly competitive with complex methods in many periods due to reduced estimation error.
14. Market Cap Weight. Weights each asset by its market capitalization relative to the portfolio total. Mimics passive index investing.
15. Dividend Yield Weight. Weights by dividend yield. Favors income-producing assets and is popular with income-focused investors.
16. Inverse Volatility Weight. Allocates inversely to volatility — lower volatility assets get higher weights. Simple, intuitive, and often effective.

    w_i = (1/σ_i) / Σ(1/σ_j)

17. Kelly Criterion. The mathematically optimal growth-focused strategy, derived from information theory. Maximizes expected log wealth. WARNING: Often produces very concentrated portfolios — use with caution.

    w_i = (E[R_i] − r_f) / σ_i²

18. User-Defined Weights. Users can input custom weights manually for comparison. Useful for testing a specific conviction portfolio.
19. Minimum Correlation Portfolio. Finds the portfolio with the minimum average correlation among assets. Emphasizes diversification.
20. Robust Covariance Shrinkage. Uses Ledoit-Wolf shrinkage to improve covariance matrix estimation accuracy, reducing estimation error in the resulting portfolio.

Note: All optimizations are subject to user-defined constraints (sector caps, position limits, margin limits, ESG filters, group constraints, and leverage bounds). If constraints conflict with the optimization objective, the optimizer finds the best feasible solution.

4. Risk Metrics Explained

FolioForecast computes a full suite of risk metrics for each portfolio. Understanding each metric helps you choose the right portfolio for your risk tolerance.

Return & Volatility

• Expected Annual Return. Annualized return from historical data. Computed as the mean of daily returns × 252 trading days.
• Annual Volatility (Std Dev). Annualized standard deviation of daily returns. Measures total variability but doesn't distinguish upside from downside.

Risk-Adjusted Return Metrics

• Sharpe Ratio. Excess return (above risk-free rate) per unit of volatility. Industry standard for risk-adjusted performance.

  Sharpe = (Annual Return − Risk-Free Rate) / Annual Volatility

  Interpretation: Higher is better. Generally, Sharpe > 1.0 is very good, > 0.5 is acceptable.
• Sortino Ratio. Like Sharpe, but only penalizes downside volatility (negative returns). Ignores upside surprises. Useful for portfolios with asymmetric risk.

  Sortino = (Annual Return − Risk-Free Rate) / Downside Volatility

Downside Risk Metrics

• Value at Risk (VaR) at 95%. The loss threshold such that there's a 5% historical chance of experiencing that loss or worse in any given day.

  VaR_95% = 5th percentile of daily returns

  Interpretation: If VaR is −3%, it means 95% of days saw losses of 3% or less; 5% of days saw worse.
• Conditional Value at Risk (CVaR) at 95%. The average loss on the worst 5% of days. Always more severe than VaR. Captures tail risk.

  CVaR_95% = mean of returns ≤ VaR_95%

• Maximum Drawdown. The largest peak-to-trough decline in portfolio value during the historical period. Captures the worst-case loss an investor would have experienced.

  Max Drawdown = min((Trough − Peak) / Peak)

  Interpretation: If Max Drawdown is −40%, you would have lost 40% from peak in the worst historical scenario.

Diversification Metrics

• Herfindahl Index (Concentration). Measures concentration of weights. Ranges 0 to 1. Lower = more diversified.

  Herfindahl = Σ(w_i)²

  Interpretation: Equal-weight portfolio has Herfindahl = 1/N. Concentrated (few large positions) approaches 1.
• Effective Number of Positions. Inverse of Herfindahl. How many equal-weight positions would give the same concentration.

  Effective N = 1 / Herfindahl

• Average Correlation. Mean pairwise correlation among all assets. Ranges −1 to +1. Lower correlations = better diversification.

Rebalancing Metrics

• Portfolio Turnover (Annual). Percentage of portfolio that must be bought/sold to rebalance yearly. Proxies for transaction costs and tax impact.

5. Backtesting Methodology

Every optimized portfolio is backtested against historical data to show how it would have performed if you'd held it from the start of the historical period to today.

Backtesting Process

1. Apply weights to historical daily returns. Each day, the portfolio return is the weighted sum of constituent returns:

   R_portfolio,t = Σ(w_i × R_i,t)

2. Assume annual rebalancing. By default, the portfolio is rebalanced to target weights annually (typically on Jan 1). Intra-year drift is allowed. Users can adjust rebalancing frequency (quarterly, monthly, never).
3. Account for rebalancing costs. Simple transaction cost estimate (0.1% per trade by default, user-configurable) is deducted from returns.
4. Compute cumulative returns. Daily returns are compounded to calculate portfolio value over time.

   Portfolio Value_t = Portfolio Value_t−1 × (1 + R_portfolio,t)

5. Measure backtest statistics. Compute annualized return, volatility, Sharpe ratio, max drawdown, and all other metrics from the backtest period.

Key Assumptions

• No slippage. Backtests assume execution at daily close prices. Real execution incurs slippage (worse prices due to market impact and latency).
• No dividends or cash drag. Dividend payments are reinvested immediately. Cash holdings earn 0% (users can add a cash/money-market asset if desired).
• Constant weights. Unless rebalancing is triggered, weights drift daily. Rebalancing brings them back to target.
• No corporate actions impact. Price data is adjusted for splits and dividends; backtests use adjusted prices.

6. Crisis Stress Testing

A portfolio's optimization was likely done using "normal" market data. But how does it hold up when markets fracture? FolioForecast stress-tests every portfolio against three major historical crises:

Crisis Periods

1. Dot-com Bust (March 2000 – October 2002). NASDAQ fell 78%; technology stocks crashed. Test case: How do defensive/diversified portfolios hold up?
2. Financial Crisis (October 2007 – March 2009). S&P 500 fell 57%; credit markets froze; correlations spiked to 1.0. Test case: Do "safe" diversified portfolios really diversify when you need it most?
3. COVID Crash (February – March 2020). S&P 500 fell 34% in 23 days; volatility spiked; liquidity vanished. Test case: Can the portfolio handle a 30%+ loss in one month?

Stress Test Metrics

For each crisis, FolioForecast reports:

• Period Return. Total return (or loss) during the crisis window.
• Max Drawdown during Crisis. Worst peak-to-trough loss during that period.
• Volatility during Crisis. Volatility in that period (often much higher than normal).
• Recovery Time. Days/months to recover to pre-crisis peak (if it recovered within data window).

Interpretation: A portfolio with 0% loss during the financial crisis would be a huge red flag — either it was all cash (missed upside) or something is wrong with data. Expected results: conservative portfolios might lose 20–35%, aggressive might lose 45–60%. Diversified portfolios should perform better than all-stock baselines.

7. Monte Carlo Simulation

Backtesting shows what happened in the past. But the future won't repeat exactly. Monte Carlo simulation generates thousands of plausible future scenarios to estimate the range of potential outcomes.

Process

1. Estimate expected return and volatility. From historical data, compute the portfolio's expected annual return and volatility.
2. Draw random return sequences. For each of 2,000 simulations, draw 252 random daily returns (one year's worth) from a normal distribution centered on the expected return with standard deviation equal to portfolio volatility.

   R_t ~ N(μ/252, σ/√252)

3. Compound into annual outcomes. Multiply daily returns to get the year's return. Repeat for 10 years (or user-specified horizon).

   Year Return = ∏(1 + R_t) − 1

4. Extract percentiles. For each year, sort the 2,000 simulated returns to find the 5th, 25th, 50th (median), 75th, and 95th percentiles.

Outputs

• Fan chart. Shows the range of outcomes over time — how wide the cone of uncertainty gets.
• Percentile table. Ending portfolio value at 5th, 25th, 50th, 75th, 95th percentiles.
• Worst-case scenario. The 5th percentile outcome — you should be psychologically prepared for this.
• Best-case scenario. The 95th percentile outcome.
• Most likely outcome. The median (50th percentile).

Key Assumptions

• Normal distribution of returns. Monte Carlo assumes returns are normally distributed. Real markets have fat tails (extreme events more common than normal distribution predicts).
• Constant volatility. Volatility is assumed constant over time. Real volatility clusters — high volatility periods tend to follow high volatility.
• No rebalancing in simulation. Monte Carlo reports don't include rebalancing costs. Add a buffer for rebalancing drag.

8. Limitations & Assumptions

FolioForecast is a powerful tool, but it rests on simplifying assumptions. Investors should understand these limitations:

• Past performance ≠ future results. Historical returns, volatilities, and correlations are not guarantees of the future. Market regimes change. Correlation matrices that work in normal times break down during crises.
• Estimation error. Mean-variance optimization is notoriously sensitive to small errors in expected return estimates. A 1% estimation error in a single asset's return can substantially change optimal weights. The engine mitigates this with regularization and alternative methods (HRP, Black-Litterman, etc.), but estimation error is inherent.
• Normal distribution assumption. Most risk metrics and Monte Carlo assume returns are normally distributed. Real returns have fat tails — extreme events occur more often than the normal distribution predicts. This means VaR and Sharpe ratio underestimate tail risk.
• Static correlations. Correlations computed from historical data are treated as fixed. In reality, correlations spike during crises (bonds and stocks both fall together during a financial panic). Portfolio diversification is less effective when you need it most.
• No market microstructure effects. Backtests assume frictionless execution at daily close prices. Real trading incurs:
  • Bid-ask spread (worse for illiquid assets).
  • Market impact (your large order moves the price against you).
  • Slippage (execution worse than expected price).
  • Commissions and fees.
  Expected real costs: 0.05%–0.5% per trade depending on asset liquidity and broker.
• No taxes or transaction costs in optimization. The optimization engine minimizes before-tax, before-fee returns. Actual after-tax returns depend on your tax bracket and holding periods (long-term vs. short-term capital gains). Tax-loss harvesting and wash-sale rules are not modeled.
• No rebalancing timing. Backtests assume annual rebalancing on a fixed date. Optimal rebalancing (rebalance when a position drifts >X%) is not modeled. Drift can be significant in trending markets.
• Survivorship bias. Data includes delisted tickers, but bankruptcies and extreme losers introduce a slight upward bias. This is minimized but not eliminated.
• Data quality. While EODHD is high-quality, some tickers (especially illiquid small-caps, penny stocks, and international exchanges) may have gaps, stale prices, or adjusted-price errors. Users should spot-check holdings.
• Constraints may conflict with objective. If a user-defined constraint (e.g., sector limits) prevents the optimizer from achieving its goal (e.g., max Sharpe), the optimizer will find the best feasible solution, which may be suboptimal.
• Leverage and short selling. Some optimization methods (e.g., Risk Parity) naturally lead to leverage. Margin costs, margin calls, and forced liquidations are not modeled. Shorting is allowed but not recommended without proper risk controls.
• Currency risk. International stocks and bonds introduce currency exposure. Unhedged currency movements can be as volatile as the asset itself. The engine does not automatically hedge currency risk.
• Corporate events. Mergers, spin-offs, and restructurings are treated as price movements. The engine does not separately model announcement risk or deal risk.
• Black swan events. Some tail events (war, pandemics, terrorist attacks) are difficult to model statistically. Monte Carlo simulations are not calibrated for such extreme, rare events.

9. Sources & References

Academic Foundations

• Markowitz, H. (1952). "Portfolio Selection." Journal of Finance, 7(1), 77–91. — Foundational work on mean-variance optimization.
• Sharpe, W. F. (1966). "Mutual Fund Performance." Journal of Business, 39(1), 119–138. — Introduces the Sharpe ratio.
• Sortino, F., & Price, L. (1994). "Performance Measurement in a Downside Risk Framework." Journal of Investing, 3(3), 59–65. — Sortino ratio for downside risk.
• Black, F., & Litterman, R. (1992). "Global Portfolio Optimization." Financial Analysts Journal, 48(5), 28–43. — Black-Litterman model for view-adjusted optimization.
• Rockafellar, R. T., & Uryasev, S. (2000). "Optimization of Conditional Value-at-Risk." Journal of Risk, 2(3), 21–41. — CVaR optimization methods.
• López de Prado, M. (2016). Building Diversified Portfolios that Outperform Rebalancing. — Hierarchical Risk Parity method.
• Kelly, J. L. (1956). "A New Interpretation of Information Rate." Bell System Technical Journal, 35(4), 917–926. — Kelly criterion for growth optimization.
• Ledoit, O., & Wolf, M. (2004). "Honey, I Shrunk the Sample Covariance Matrix." Journal of Portfolio Management, 30(4), 110–119. — Shrinkage for covariance estimation.

Data Sources

• EODHD (End-of-Day Historical Data). Premium financial data provider. 165,000+ tickers across 180+ exchanges. Daily OHLCV data with corporate action adjustments. eodhd.com
• Risk-free rate. US 10-year Treasury yield (or user-specified). Updated daily.
• Crisis period data. Public historical data from CRSP, Yahoo Finance, and Bloomberg archives.

Industry References

• Ibbotson, R. G., & Kaplan, P. D. (2000). "Does Asset Allocation Policy Explain 40%, 90%, or 100% of Performance?" Financial Analysts Journal, 56(1), 26–33. — Asset allocation's role in returns.
• Arnott, R. D., Beck, S. L., Kalesnik, V., & West, J. (2016). "How Can 'Smart Beta' Go Horribly Wrong?" Research Affiliates Publications. — Practical caveats for quantitative optimization.
• Bender, J., Sun, X., Thomas, R., & Zdorovtsov, V. (2018). "The Promises and Pitfalls of Factor Timing." Journal of Portfolio Management, 44(4), 429–446. — Factor rotation and timing risks.