Build a Historical Stock Universe for Backtesting

Why the universe definition matters#

A backtest can use accurate prices and still answer the wrong question if its stock list includes only companies that exist today. A historical universe defines which securities were eligible on each date, according to a rule you specify. If companies that later delisted are missing, the tested set may overrepresent firms that survived. That is survivorship bias in the universe: the historical sample has been filtered using information about what happened later. Research on delisting bias discusses why omitted delisted securities can matter to historical-return analysis. https://www.tylergshumway.org/Shumway-DelistingBiasCRSP-1997.pdf

The practical fix is not simply to find an old ticker list. Build dated membership intervals and preserve evidence for each inclusion and exit. A security that later disappeared should still appear for the period when it met your stated universe rule. This guide focuses on that membership record—not on how to run a backtest or choose a trading strategy.

Set the rule before gathering symbols#

Write down the universe rule before collecting securities. For example, “common shares listed on the specified U.S. exchanges on each trading date” is a rule that can be investigated. “Stocks that were good candidates” is not. If you are testing an index-based universe, name the index and the version of its eligibility and maintenance rules; do not substitute today’s constituents for past membership.

Decide which instruments count: common stock, ADRs, preferred shares, funds, or other security types. Specify eligible exchanges, treatment of suspended securities, and whether membership begins on the announcement date or the effective date. For a historical constituent list, use the provider’s dated membership records or change notices where available. Index membership changes over time, and removals can reflect more than delisting, including changes in eligibility or corporate events. Methodology Matters | S&P Dow Jones Indices Record the effective dates and the rule used to interpret them. # Index Policy

These decisions should be explicit because two reasonable universe definitions can produce different lists. The worksheet is not a claim that one definition is universally correct; it is a way to make yours traceable and repeatable.

A useful workflow produces a dated, auditable membership table, not just a folder of symbols. Apply these steps to the full backtest period:

1. Define the eligible security types, exchanges, geography, and any index or screening rules. Note the date range and rule version.
2. Choose a historical source that retains inactive securities. For an index, seek dated constituent files or change records. For a broad exchange-listed universe, check that the security-history dataset includes delisted issues, not only current listings. CRSP documentation describes historical name, listing, and delisting information; check the specific database and fields available to you. CHAPTER 2: DATA DEFINITIONS CRSP 10-YEAR US STOCK DATABASE
3. Assign a stable security identifier where available, and create one row for each continuous membership interval. Keep the ticker as a time-bounded attribute, not as the security’s permanent identity.
4. Enter the start and end dates using a consistent convention, such as effective date included and removal date excluded. Document that convention; verify it against the source’s date definitions.
5. Reconcile every change: ticker or name change, exchange transfer, merger, spin-off, suspension, and delisting. Record the event type and the evidence used. A symbol change alone does not establish whether the underlying security continued.
6. Review unexplained gaps and exits. If a price series ends, determine whether the security left the defined universe, changed identifiers, moved listings, or was delisted. Leave uncertain cases flagged instead of silently dropping them.

Copy this worksheet into a spreadsheet or database. Use one row per uninterrupted period in which a security qualifies. If it exits and later re-enters, create another row rather than stretching one interval across the gap. The table is a recording template, not a substitute for checking the source’s date conventions.

Check ticker changes and delistings#

Ticker symbols are useful labels, but they are not dependable permanent identifiers. A company may change its symbol or name; a listing may move; and a symbol may later be assigned to another security. Where the dataset provides a stable identifier, keep it alongside the ticker and use dated name or identifier history to investigate changes. CRSP’s documentation separates name-history information from delisting history, illustrating why identity changes and exit events should be tracked as distinct records. CHAPTER 2: DATA DEFINITIONS CRSP 10-YEAR US STOCK DATABASE

For each apparent symbol change, compare the issuer name, stable identifier, exchange, and event dates. Ask whether the security continued under a new ticker, whether the original security ended, or whether the evidence is inconclusive. Do not automatically join price histories just because the ticker looks related. Conversely, do not treat every new ticker as a new company without checking the identity record.

For each unexplained end to a price series, look for a delisting or other listing-status event. Record the effective date and reason when supported by evidence. CRSP materials describe delisting information, including dates, codes, and delisting returns; the fields and coverage depend on the dataset. CHAPTER 2: DATA DEFINITIONS A delisting is not necessarily just a final ordinary trading bar, so note whether the source supplies a separate delisting return or event value and how it will be handled in your research. Do not invent a value when it is absent.

If an exchange delisting is relevant, official filings and exchange notices can help verify the event. SEC staff guidance discusses Form 25 in connection with delisting, but a filing should be read in context rather than treated as a complete substitute for a historical membership record. SEC.gov | Exchange Act Forms Keep source references and flag conflicts for review.

Before using the membership table, run this review. “Unknown” is a valid data-quality status; silently omitting a difficult case is not a resolution.

• Every row has a documented membership source and a stated date convention.
• Every eligible interval has a start date; each exit has an end date or a clearly marked unresolved status.
• The universe includes inactive and delisted securities within the chosen scope, not merely today’s active symbols.
• Ticker changes, name changes, exchange transfers, and possible ticker reuse have been checked against security identity evidence.
• Delistings are represented as explicit events where evidence supports them; unexplained price-series endings are not silently treated as ordinary exits.
• Index additions and deletions are dated from historical records, not inferred from a current roster. Index providers publish methodologies and membership maintenance information that can help define what a constituent record means. Methodology Matters | S&P Dow Jones Indices # Index Policy
• A second review has checked date overlaps, accidental gaps, duplicate identifiers, and rows with missing removal reasons.

A membership worksheet improves traceability; it cannot guarantee that the underlying source is complete or that every corporate event has been interpreted correctly. Treat uncertain rows as explicit research tasks.

Frequently asked questions#

Can I use today’s stock list for a historical backtest?

No. It may omit companies that qualified earlier but later delisted or otherwise left the market. Use historical membership records that cover inactive securities, then reconcile their dated intervals.

How should I handle a ticker change?

Prefer a stable security identifier when the source provides one, and retain the ticker as a dated label. Check name, listing, and identifier history before deciding whether two symbols refer to the same continuing security. CHAPTER 2: DATA DEFINITIONS CRSP 10-YEAR US STOCK DATABASE

Should a delisted company disappear from the historical universe?

Record the delisting date and reason when supported, keep the security in the universe for its qualifying interval, and preserve any relevant delisting data provided by your source. Do not assume an ordinary final price bar captures every delisting-related event. CHAPTER 2: DATA DEFINITIONS

How do I reconstruct past index constituents?

Use the provider’s historical constituent records or dated change notices, and retain the effective dates and rule version. A current constituent list does not establish past membership. Methodology Matters | S&P Dow Jones Indices # Index Policy

Make membership auditable#

The core discipline is straightforward: define eligibility, preserve dated evidence, and create one membership interval for each qualifying period. Keep security identity separate from ticker history, investigate every unexplained exit, and record delistings rather than filtering them away. A historical universe built this way makes it clearer which securities were eligible on each date—and which data gaps still need attention.

Sources#

• SEC.gov | Exchange Act Forms — sec.gov
• https://data.nasdaq.com/databases/NDL — data.nasdaq.com
• Methodology Matters | S&P Dow Jones Indices — spglobal.com
• Legal Disclaimers | S&P Dow Jones Indices — spglobal.com
• CHAPTER 2: DATA DEFINITIONS — crsp.org
• CRSP 10-YEAR US STOCK DATABASE — crsp.org
• # Index Policy — spglobal.com
• https://wrds-www.wharton.upenn.edu/documents/400/CRSP_Programmers_Guide.pdf — wrds-www.wharton.upenn.edu
• https://wrds-www.wharton.upenn.edu/documents/399/Data_Descriptions_Guide.pdf — wrds-www.wharton.upenn.edu
• https://datacenter.safefrankfurt.de/datacenter/documents/Data_Description_CRSP_Shumway_20250317.pdf — datacenter.safefrankfurt.de
• https://www.sciencedirect.com/science/article/pii/S1062976996900216 — sciencedirect.com
• https://www.tylergshumway.org/Shumway-DelistingBiasCRSP-1997.pdf — tylergshumway.org
• https://papers.ssrn.com/sol3/Delivery.cfm/5833162.pdf?abstractid=5833162&mirid=1 — papers.ssrn.com
• https://terence-lim.github.io/docs/financial-data-science-notebooks/1.1_stock_prices.html — terence-lim.github.io
• https://github.com/ceta-research/backtests/blob/main/sp500-survivorship/README.md — github.com
• https://studylib.net/doc/8889446/crsp-data-description-guide — studylib.net