PORTFOLIO CASE STUDY

Can the unemployment rate, combined with recent market behavior, improve forecasts of next-month S&P 500 direction?

1 Executive summary

55.7%Out-of-sample accuracy
68.9%Baseline accuracy
0.49Out-of-sample ROC AUC
122Test observations

This study uses monthly observations from March 2000 through April 2026. A logistic regression is trained through December 2015 and evaluated only on later observations.

Bottom line. The model did not improve on the historical up-rate baseline; its ROC AUC was 0.49. The result is useful precisely because the benchmark is difficult to beat: U.S. equity months are positive more often than they are negative, so headline accuracy without a baseline can be misleading.

2 Research design

The unit of analysis is one calendar month. At the close of month \(t\), the model uses:

  • the latest unemployment rate that would have been published by then;
  • the latest published monthly change in unemployment;
  • the S&P 500 return during month \(t\); and
  • annualized daily volatility during month \(t\).

It predicts whether the S&P 500 return in month \(t+1\) will be positive.

Component Choice Reason
Target Next-month S&P 500 direction Clear binary outcome with an investable forecast horizon
Macro timing Unemployment lagged one month Avoids using a value before its public release
Validation Chronological holdout Preserves ordering and prevents random-split leakage
Baseline Training-period up rate Measures whether the model beats the market’s class imbalance
Metrics Accuracy, balanced accuracy, Brier score, ROC AUC Covers classification and probability quality

2.1 Why publication timing matters

The unemployment rate for a given month is generally released during the following month. Treating month \(t\)’s unemployment rate as known during month \(t\) creates look-ahead bias. This analysis conservatively maps each unemployment observation to the next calendar month, when it is available to a forecaster.

3 Data and preparation

The repository includes versioned raw-data snapshots from two public sources:

  • Yahoo Finance for daily adjusted S&P 500 prices (^GSPC);
  • FRED for the monthly U.S. unemployment rate (UNRATE).

By default, the report renders from those snapshots so the analysis remains available when an upstream service is unavailable. Setting REFRESH_DATA=true attempts a live refresh and updates the snapshots when successful. This render used Versioned Yahoo Finance snapshot and Versioned FRED snapshot.

The current, incomplete market month is excluded. Monthly returns are compounded from daily log returns, while volatility is the standard deviation of daily log returns annualized with \(\sqrt{252}\).

Series Start End Observations
S&P 500 daily prices 2000-01-03 2026-05-29 6641
U.S. unemployment rate 2000-01 2026-05 317
Release-aware model sample 2000-03 2026-04 312 
sp500_daily <- sp500_raw %>%
  arrange(date) %>%
  mutate(daily_log_return = log(adjusted / lag(adjusted))) %>%
  filter(!is.na(daily_log_return))

sp500_monthly <- sp500_daily %>%
  mutate(month_date = floor_date(date, "month")) %>%
  group_by(month_date) %>%
  summarise(
    monthly_log_return = sum(daily_log_return),
    monthly_return = exp(monthly_log_return) - 1,
    annualized_volatility = sd(daily_log_return) * sqrt(252),
    n_trading_days = n(),
    .groups = "drop"
  )

3.1 Market history

3.2 Descriptive relationships

These plots are descriptive, not causal. They show the contemporaneous relationship between unemployment conditions and market behavior without claiming that unemployment alone drives returns.

4 Modeling

4.1 Feature engineering and split

All transformations are calculated in chronological order. Predictor standardization is fit on the training sample and then applied unchanged to the test sample.

feature_names <- c(
  "available_unemployment",
  "available_unemp_change",
  "monthly_return",
  "annualized_volatility"
)

train_data <- model_data %>%
  filter(month_date <= floor_date(as.Date(params$train_end), "month"))

test_data <- model_data %>%
  filter(month_date > floor_date(as.Date(params$train_end), "month"))

feature_center <- vapply(train_data[feature_names], mean, numeric(1))
feature_scale <- vapply(train_data[feature_names], sd, numeric(1))

train_model <- standardize_features(train_data, feature_center, feature_scale)
test_model <- standardize_features(test_data, feature_center, feature_scale)

logit_fit <- glm(
  next_month_up ~ available_unemployment + available_unemp_change +
    monthly_return + annualized_volatility,
  data = train_model,
  family = binomial()
)

4.2 Coefficient interpretation

Because predictors are standardized, each coefficient represents a one-standard-deviation increase in that feature while the others are held constant. Confidence intervals that cross an odds ratio of 1 indicate weak evidence of a stable directional relationship.

5 Out-of-sample evaluation

The model is evaluated from January 2016 through April 2026. The baseline assigns every test observation the positive-month probability observed in the training period.

metric Logistic regression Historical up-rate baseline
Accuracy 55.7% 68.9%
Balanced accuracy 43.4% 50.0%
Brier score 0.250 0.224
ROC AUC 49.2% 50.0%

5.1 Confusion matrix

5.2 Predicted probability through time

6 Conclusions

  1. The evaluation design matters more than the algorithm. A chronological split, release-aware macro features, and an explicit baseline prevent an optimistic result.
  2. Unemployment is more informative about market conditions than short-horizon direction. High-unemployment periods coincide with elevated volatility, but that relationship does not automatically translate into reliable next-month forecasts.
  3. A weak predictive result is still a useful research result. The model provides a transparent benchmark and a foundation for testing richer real-time macro indicators, valuation measures, or walk-forward estimation.

7 Limitations and next steps

  • FRED observations are revised over time; this report uses the latest vintage rather than a real-time vintage database such as ALFRED.
  • The model does not include transaction costs, portfolio sizing, or an investable backtest.
  • One fixed train/test split is transparent but does not measure stability across multiple historical windows.
  • Statistical significance is not economic significance, and this analysis should not be interpreted as investment advice.

A logical extension would use expanding-window validation and real-time macro vintages, then compare logistic regression with regularized models while preserving the same release-aware feature set.

8 Reproducibility

The report was rendered with R 4.5.1. Package versions are recorded in renv.lock; the full analysis pipeline lives in R/analysis.R.

sessionInfo()
## R version 4.5.1 (2025-06-13 ucrt)
## Platform: x86_64-w64-mingw32/x64
## Running under: Windows Server 2022 x64 (build 26100)
## 
## Matrix products: default
##   LAPACK version 3.12.1
## 
## locale:
## [1] LC_COLLATE=English_United States.utf8 
## [2] LC_CTYPE=English_United States.utf8   
## [3] LC_MONETARY=English_United States.utf8
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.utf8    
## 
## time zone: UTC
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices datasets  utils     methods   base     
## 
## other attached packages:
##  [1] PerformanceAnalytics_2.1.0 quantmod_0.4.28           
##  [3] TTR_0.24.4                 xts_0.14.2                
##  [5] zoo_1.8-15                 tidyquant_1.0.12          
##  [7] tidyr_1.3.2                tibble_3.3.1              
##  [9] scales_1.4.0               lubridate_1.9.5           
## [11] ggplot2_4.0.3              dplyr_1.2.1               
## 
## loaded via a namespace (and not attached):
##  [1] RobStatTM_1.0.11   sass_0.4.10        generics_0.1.4     renv_1.2.3        
##  [5] lattice_0.22-9     digest_0.6.39      magrittr_2.0.5     evaluate_1.0.5    
##  [9] grid_4.5.1         timechange_0.4.0   RColorBrewer_1.1-3 fastmap_1.2.0     
## [13] Matrix_1.7-5       jsonlite_2.0.0     mgcv_1.9-3         purrr_1.2.2       
## [17] jquerylib_0.1.4    cli_3.6.6          rlang_1.2.0        splines_4.5.1     
## [21] withr_3.0.2        cachem_1.1.0       yaml_2.3.12        otel_0.2.0        
## [25] tools_4.5.1        curl_7.1.0         vctrs_0.7.3        R6_2.6.1          
## [29] lifecycle_1.0.5    pkgconfig_2.0.3    pillar_1.11.1      bslib_0.11.0      
## [33] gtable_0.3.6       glue_1.8.1         xfun_0.58          tidyselect_1.2.1  
## [37] knitr_1.51         farver_2.1.2       nlme_3.1-169       htmltools_0.5.9   
## [41] rmarkdown_2.31     labeling_0.4.3     compiler_4.5.1     S7_0.2.2          
## [45] quadprog_1.5-8