Open source cross sectional asset pricing

This repo accompanies our paper: Chen and Zimmermann (2022), "Open source cross-sectional asset pricing"

If you use data or code based on our work, please cite the paper:

@article{ChenZimmermann2022,
  title={Open Source Cross Sectional Asset Pricing},
  author={Chen, Andrew Y. and Tom Zimmermann},
  journal={Critical Finance Review},
  year={2022},
  pages={207-264},
  volume={11},
  number={2}
}

If you're only interested in the data, please see:

• The dedicated data website and its FAQ page.
• The Hugging Face dataset OpenSourceAP/CrossSection.

or the download packages:

• Python: openassetpricing (pip install openassetpricing)
• R: OpenSourceAP.DownloadR (install.packages("OpenSourceAP.DownloadR"))

or the SignalDoc browser.

The code is separated into three folders:

1. Signals/pyCode/ Downloads data from WRDS and elsewhere, constructs stock-level signals in Python, and outputs to Signals/pyData/.
2. Portfolios/Code/ Takes in signals and outputs portfolios to Portfolios/Data/. Entirely in R.
3. Shipping/Code/ Used to prepare data for sharing.

Run them in order: Signals/pyCode/master.py builds the signals, then Portfolios/Code/master.R builds the portfolios from them. The portfolio code reads the signals and CRSP data that the signals code writes to Signals/pyData/, so it is not meant to be run on its own. If you only want signals, you can stop after master.py.

More details are below

1. Set up for Creating Signals (Python and R)

• Install Python dependencies:

  cd Signals/pyCode/
  pip install -r requirements.txt

• Install required R packages: googledrive (pulls in gargle), used by Signals/pyCode/. In R: install.packages("googledrive").
• Google Drive login (for 7 OptionMetrics signals). While PATCH_OPTIONM_IV = True in Signals/pyCode/config.py (see #156), CPVolSpread, dCPVolSpread, dVolCall, dVolPut, skew1, SmileSlope and RIVolSpread are downloaded from our 2023 data release with your Google account. Once, from Signals/pyCode/, start R interactively and run source("gdrive_login.R"). Open the printed URL, sign in, tick "See and download all your Google Drive files", and paste the code back into R. The login is cached, so later pipeline runs are headless.
  • Any Google account works, since the files are public. Access is read-only.
  • Skipping the login only loses those seven signals; master.py warns and asks before it starts.
• Copy Signals/pyCode/dotenv.template to Signals/pyCode/.env and add your WRDS and FRED credentials.
  • For FRED credentials, request an API key from FRED

2. (Optional) Generate Prep Data

This is only necessary for a handful of signals

If you have bash:

• from Signals/pyCode/
  • run bash prep1_run_on_wrds.sh to copy the prep scripts to the WRDS Cloud
  • the submission runs four concurrent array tasks of about 1 hour each: OptionMetrics volume; the other OptionMetrics scripts; high-frequency spreads; and 13F plus low-frequency spreads. Scripts within each task run sequentially
  • wait for all four tasks to finish (about 1 hour)
    • use qstat to check whether any task is still running
    • if impatient, check most recent file in ~/temp_prep/log/ on WRDS server.
  • run bash prep2_dl_from_wrds.sh to download the prep data from the WRDS Cloud to Signals/pyData/Prep/

You can alternatively upload to the WRDS Cloud manually, ssh into WRDS, run qsub run_all_prep.sh, and then manually download the prep data.

3. Run the Signals Code

master.py runs the end-to-end Python pipeline. It calls the staged scripts in:

• DataDownloads/ downloads data from WRDS and elsewhere and writes to Signals/pyData/
• SignalMasterTable.py builds the join table used across predictors
• Predictors/ constructs stock-level predictors and outputs to Signals/pyData/Predictors/
• Placebos/ constructs "not predictors" and "indirect evidence" signals and outputs to Signals/pyData/Placebos/

To run:

cd Signals/pyCode/
python master.py

The orchestrator blocks are written to keep running even if a particular download fails (for example due to a missing subscription) so you get as much data as possible. You can track progress in Signals/Logs/.

master.R runs everything. It:

1. Takes in signal data located in Signals/pyData/Predictors/ and Signals/pyData/Placebos/ (or Signals/Data/... for the legacy Stata signals)
2. Takes in CRSP v2 (CIZ) data from Signals/pyData/Intermediate/ (see below)
3. Outputs portfolio data to Portfolios/Data/Portfolios/
4. Outputs exhibits found in the paper to Results/

It also uses SignalDoc.csv as a guide for how to run the portfolios.

To run:

• Option 1 - Command line:

  cd Portfolios/Code/
  Rscript master.R

• Option 2 - RStudio: Open master.R in RStudio and click "Source" or press Ctrl+Shift+S (Cmd+Shift+S on Mac)

Before running: Edit the operational settings in Portfolios/Code/config.R. here::i_am() locates the project root. Start R from the project root or a subfolder, including when sourcing master.R in RStudio. Scientific settings remain in SignalDoc.csv.

Daily portfolios and exhibits are enabled by default. Set run$daily = FALSE to skip daily portfolios, or run$exhibits = FALSE to skip exhibits. A quick run uses run$quick_signals and always skips exhibits. Set paths$output_root to an isolated directory to redirect processed CRSP, portfolio CSVs, summaries and exhibits; upstream signal inputs still come from the project. Relative output roots are resolved against the project root. Dependencies are checked before the build; missing packages must be installed separately.

The portfolio stages run on parallel workers: monthly stages split each configuration's signals across workers, and the daily stage runs its six configurations as separate jobs. Set the worker limit with resources$max_workers in config.R (default 3); the count also drops if RAM is short (see Portfolios/README.md). With 3 workers, the full build for the 2026.10 release took about 3.6 hours on a Mac: about 5 minutes for the baseline portfolios, about 1 hour for the rest of the monthly build, and about 2.6 hours for daily portfolios.

The retired signal-coverage and stock-correlation exhibits are no longer built or shipped; portfolio-return exhibits remain available.

The portfolio code does not download anything itself. It reads the CRSP v2 (CIZ) files that Signals/pyCode/ writes to Signals/pyData/Intermediate/ (monthlyCRSP.parquet, and dailyCRSP.parquet if run$daily = TRUE), plus the three predictors built directly from CRSP (Price, Size, STreversal). master.R stops with a message if any of these files are missing.

For a quick check that the R side works before the full signals run, Signals/pyCode/crsp_only.py downloads just CRSP and builds those three predictors (add --skip-daily if config.R sets run$daily = FALSE). master.R then builds portfolios for Price, Size, and STreversal only.

This code packs the data as parquet, runs quality checks, and uploads the release to Hugging Face (see Shipping/README.md). You shouldn't need to use this but we keep it with the rest of the code for replicability.