We present a methodology for creating enhanced microsimulation datasets that combine the strengths of survey and administrative data sources. The Enhanced CPS dataset demonstrates that careful application of modern statistical methods can substantially improve the data available for policy analysis.
Summary of Contributions¶
Our work makes several key contributions:
Methodological Innovation: The use of Quantile Regression Forests for imputation preserves distributional characteristics while maintaining computational efficiency. The large-scale calibration to 7,000+ targets pushes the boundaries of survey data enhancement.
Practical Tools: We provide open-source implementations that enable researchers to apply, modify, and extend these methods. The modular design facilitates experimentation with alternative approaches.
Validated Dataset: The Enhanced CPS itself serves as a public good for the research community, enabling studies that would otherwise require restricted data access.
Reproducible Research: All code, data, and documentation are publicly available, supporting reproducibility and collaborative improvement.
Key Findings¶
The validation results demonstrate that combining survey and administrative data through principled statistical methods can achieve:
- Improved income distribution representation
- Better alignment with program participation totals
- Maintained demographic and geographic detail
- Suitable accuracy for policy simulation
While no dataset perfectly represents the full population, the Enhanced CPS provides a pragmatic balance of accuracy, detail, and accessibility.
Implications for Policy Analysis¶
Enhanced microdata availability creates immediate implications for policy analysis. More accurate representation of high incomes enables better analysis of progressive tax reforms and revenue estimates. Researchers can now analyze tax and transfer policies jointly rather than in isolation. Geographic identifiers enable subnational policy analysis not possible with administrative tax data alone. Finally, household structure allows examination of policy impacts across family types and income levels.
Broader Implications¶
Beyond the specific dataset, this work demonstrates several broader principles. Combining multiple data sources can overcome individual limitations, showing the value of data integration. Making methods and data publicly available accelerates research progress and demonstrates open science benefits. While perfect data may never exist, pragmatic enhancements can substantially improve analysis capabilities. Furthermore, open-source approaches enable community contributions and continuous improvement.
Limitations and Future Work¶
We acknowledge important limitations including temporal inconsistency between data sources, imputation model assumptions, calibration trade-offs, and validation challenges. Future work should address these through more recent administrative data, enhanced imputation methods, additional validation exercises, and uncertainty quantification.
Call to Action¶
We encourage researchers to apply the Enhanced CPS to policy questions where combined demographic and tax detail adds value, compare findings with other data sources and contribute validation results, leverage the open-source nature to make methodological enhancements, and document use cases, limitations discovered, and suggested improvements.
Final Thoughts¶
The Enhanced CPS represents one approach to a fundamental challenge in microsimulation: the need for comprehensive, accurate microdata. While not perfect, it demonstrates that substantial improvements are possible through careful methodology and open collaboration.
As data availability evolves and methods advance, this work contributes to a future where policy analysis rests on increasingly solid empirical foundations. Our ultimate goal remains better informed policy decisions that improve social welfare.
The enhanced dataset, complete documentation, and all source code are available at https://