Microsimulation¶
The Microsimulation class is the primary tool for population-level policy analysis in PolicyEngine US. It combines representative survey microdata with PolicyEngine’s tax-benefit model to estimate how policies affect the entire US population.
Marital membership and tax roles¶
Dataset person_marital_unit_id and marital_unit_id columns define marital
membership. Loading the dataset and extending its years preserve those IDs.
Microcosm builds these units from the survey’s spouse pointers, and
policyengine-us never rebuilds a dataset’s units.
A household situation’s marital_units mapping also stands as supplied. An
empty mapping gives each person their own unit. A married couple may share a
marital unit while belonging to separate tax units.
When an entity-form situation omits marital_units, the model pairs at most
one couple in each tax unit and household, from the first of these that
applies, and gives everyone else their own unit:
an unambiguous pair of true
is_tax_unit_headandis_tax_unit_spouseinputs for overlapping periods, neither input as a dependent or separated;the head and spouse of a
tax_unit_role_inputsupplied for every member;the two adults of a family whose
is_marriedinput is true, when exactly two of its adults in the tax unit and household are neither input as dependents nor separated;otherwise, the head and spouse that
is_tax_unit_headandis_tax_unit_spouseinfer from age order: the oldest adult, and the next-oldest adult who fails the qualifying child age test against them (under 19, or under 24 if a full-time student, and younger than the head, or permanently and totally disabled). No couple is formed when the head or that spouse is separated (another member’s separation doesn’t matter), when the two live in different households, when a family they share has a falseis_marriedinput, or when either has a falseis_tax_unit_headoris_tax_unit_spouseinput for the role this step gives them. A tax unit whosefiling_statusinput isJOINTskips the age test, because only a married couple files jointly (26 USC 6013(a)).
Only married people file separately
(26 USC 1(d)), so when a
household holds exactly two tax units whose filing_status input is
SEPARATE, their two filers are then paired, unless either is separated, a
family they share has a false is_married input, or either is already in a
couple from the steps above. A unit’s filer is its one true is_tax_unit_head
input, or else its only adult who isn’t input as a dependent; a SEPARATE unit
with no such filer still counts toward the two, so it blocks the pairing. A
marital unit is a married, co-habiting couple, so separate filers in different
households stay apart. A filing_status input that changes between periods
counts as no status.
The fourth step reads the same inputs and parameters as the role formulas, so the units it builds agree with the inferred tax roles in the years the inputs are dated for: a couple sent without marriage information is a couple for SSI and Medicaid as well as for tax, and a parent’s adult child who meets the age test is neither. When anyone in a situation supplies a role, student or age input for a year, Core gives everyone else that variable’s default for the year instead of running its formula, and the fourth step reads the same defaults. The fourth step checks the years the inputs are dated for, so a payload dated for a year other than the simulation’s default input year pairs the same couple. Under a reform, it uses the reform’s dependent age limits.
Membership can’t change between periods. Inputs that can’t settle one static
couple give everyone in that tax unit and household their own unit: two head
or two spouse inputs, a dependent also marked head or spouse in a unit with
both a head and a spouse input, role or is_married inputs that change between
periods, or a fourth-step couple that differs between years. With only a spouse
input, on a person also input as a dependent, the fourth step pairs that person
with the oldest other adult, and the role formulas give the same couple.
Known limitations, which callers avoid by supplying marital_units:
Like the role fallback, the fourth step pairs an adult who fails the age test with the head, such as a parent and their adult child or two unrelated adults in one tax unit.
A tax unit with two couples pairs only the couple the role formulas pick; the other two adults get their own units.
Spouses in two tax units are paired only as the two separate filers of one household. A family’s true
is_marriedinput doesn’t pair them, and neither do separate returns when the household holds more than two.The separate-filer pairing reads no age or relationship, so a parent and an adult child who each file separately from an absent spouse, beside a third tax unit, become a couple.
A
filing_statusinput dated for more than one year counts only when it is the same in every year, so a status that isJOINTin the calculation year and different in another year reads as no status.JOINTandSEPARATEinputs say who is married, butSINGLE,HEAD_OF_HOUSEHOLDandSURVIVING_SPOUSEinputs don’t say who isn’t: two adults in one tax unit with any of them can still be paired by the fourth step, as on main.A lone true
is_tax_unit_spouseinput on the oldest adult makes that person both head and spouse in the role formulas, as on main; the fourth step pairs them with the next adult.A lone true
is_tax_unit_spouseinput skips the fourth step’s separation check: the fourth step pairs that person with the oldest other adult even when that adult is separated.Marital units follow the years the inputs are dated for, while the role formulas run again in any year without a role input. So a calculation for another year can give roles the units don’t match. For example, a grandchild’s undated
is_tax_unit_spouse: falselands in the default input year (2026), so the grandparents get their own units, while a 2025 calculation still makes them head and spouse.
These rules apply to Python household simulations and entity-form YAML tests, including tests with reforms or inline parameters. Variable-only tests keep Core’s own defaults.
Getting started¶
from policyengine_us import Microsimulation
# Create a microsimulation with the default dataset
sim = Microsimulation()
# Calculate a variable for the default year (2024)
household_net_income = sim.calc("household_net_income")
# The result is a weighted MicroSeries
print(f"Total household net income: ${household_net_income.sum():,.0f}")
print(f"Mean household net income: ${household_net_income.mean():,.0f}")
Methods: calc() vs calculate()¶
The Microsimulation class provides two methods for computing variables:
calc() / calculate()¶
These methods are aliases - calc is shorthand for calculate. Both return a MicroSeries object that includes survey weights, enabling proper population-level statistics.
# These are equivalent
income = sim.calc("household_net_income")
income = sim.calculate("household_net_income")
# MicroSeries supports weighted aggregations
income.sum() # Total across population (using weights)
income.mean() # Weighted mean
income.median() # Weighted median
income.gini() # Gini coefficient
Key parameters¶
variable_name(str): The variable to calculateperiod(int or str): The time period (e.g.,2024or"2024")map_to(str): Entity to map results to (see below)use_weights(bool): IfFalse, returns raw unweighted array
# Calculate SNAP benefits for 2024
snap = sim.calc("snap", period=2024)
# Get unweighted array
snap_raw = sim.calc("snap", period=2024, use_weights=False)
Entity mapping with map_to¶
PolicyEngine US models multiple entity levels: person, tax_unit, spm_unit, family, and household. The map_to parameter transforms results between these levels.
Mapping from groups to persons¶
When mapping a group-level variable (like household_net_income) to persons, each person receives their group’s value:
# Get household income at person level (each person gets their household's value)
person_hh_income = sim.calc("household_net_income", map_to="person")
# Useful for person-weighted analysis
print(f"Average income per person: ${person_hh_income.mean():,.0f}")
Mapping from persons to groups¶
When mapping a person-level variable to groups, values are summed by default:
# Sum employment income to household level
hh_earnings = sim.calc("employment_income", map_to="household")
Common use cases¶
# Calculate poverty at person level for person-weighted statistics
person_poverty = sim.calc("in_poverty", map_to="person")
poverty_rate = person_poverty.mean()
# Map tax credits to see household-level impact
hh_eitc = sim.calc("eitc", map_to="household")
Available datasets¶
The default build¶
The default is the certified Microcosm build, pinned by build id and hosted as a HuggingFace dataset repository:
# Default: the certified Microcosm build named in DEFAULT_DATASET.
sim = Microsimulation()
Supplying another population¶
A population file has to satisfy the SPM input contract: it supplies primitive
inputs, including observed county_fips codes as five-digit strings and
source-backed is_spm_independent_minor_role values, and it must not store
formula-owned SPM outputs such as spm_unit_spm_threshold. Any observed Census
measurement is retained under a separate report-only name.
The legacy files under hf://policyengine/policyengine-us-data/ predate that
contract, so SPM measurements are not available over them:
cps_2023.h5storesspm_unit_spm_threshold, so the loader rejects it.enhanced_cps_2024.h5loads and computes tax variables, but carries no SPM independence roles, so 18 of its SPM units - each a lone 15-to-17-year-old - classify no measurement adult, andspm_unit_spm_thresholdraisesSPM_COMPOSITION_REQUIREDover the file. That is whattest_legacy_enhanced_cps_lacks_source_backed_spm_independence_roleschecks. Outputs that no longer reach the threshold, household net income and benefits among them, do compute over the file.
A household simulation that has no county input can select an SPM area explicitly instead:
sim = Microsimulation(dataset=..., spm={"geography_kind": "national"})
Entity-level single-year datasets are extended through the latest available
economic-assumption year by default. If a calculation needs only an earlier
range, set dataset_end_year to its final year. Setting it to the dataset’s own
year loads only that year:
sim = Microsimulation(dataset=single_year_dataset, dataset_end_year=2024)
dataset_end_year also works with the default dataset and entity-level HDFStore
paths. It does not apply to an existing USMultiYearDataset or to the legacy
variable-centric HDF5 format.
Tax-unit roles¶
A population file can supply the tax units its constructor built, not just
their membership, through tax_unit_role_input (person): HEAD, SPOUSE or
DEPENDENT. The certified Microcosm build supplies it. When every member of a
tax unit has a supplied role, is_tax_unit_head and is_tax_unit_spouse
follow it and is_tax_unit_dependent covers everyone else. Explicitly set
is_tax_unit_head, is_tax_unit_spouse or is_tax_unit_dependent inputs
still take precedence.
Without supplied roles, a unit falls back to age ordering. The oldest adult is the head, skipping adults input as tax-unit dependents unless every adult in the unit is one. When two such adults share a two-person marital unit within the tax unit, the older of that couple is the head instead: a married couple can file jointly (26 USC 6013(a)), and a couple who files a joint return can’t be claimed as anyone’s dependents (26 USC 152(b)(2)). The spouse is the head’s partner in their marital unit. A head without one takes the next-oldest such adult, except an adult who meets the qualifying child age test against the head (26 USC 152(c)(3)): under 19, or under 24 if a full-time student, and younger than the head, or permanently and totally disabled at any age. That adult stays a dependent. PolicyEngine infers no spouse when the head or selected spouse candidate is separated. It selects the candidate before checking separation, without promoting a younger adult; member order breaks equal-age ties. A separated member doesn’t count toward a couple. When a situation omits marital units, the model builds them by these same rules (see “Marital membership and tax roles” above), so they give the same roles. A minor living without a parent is left with no head.
This rule has three known limitations. Supply is_tax_unit_head,
is_tax_unit_spouse or tax_unit_role_input for these households:
When a tax unit holds a married couple and an older adult outside their marital unit, the couple heads the unit and the older adult becomes their dependent, whatever that adult’s income. The older adult may file instead, for example a parent who claims a married child and the child’s spouse who don’t file jointly.
An adult outside the head’s marital unit who fails the age test, such as a 45-year-old child with wages, is still paired with the head, and the unit files jointly. Under the law each would file their own return, or the adult could be a qualifying relative, but the pairing keeps the adult’s income on the one return the model computes for the unit.
A permanently and totally disabled adult outside the head’s marital unit becomes the head’s dependent, whatever their income. Examples are a spouse living apart in a care facility, or a self-supporting sibling, whose income then leaves the return.
The Microcosm build carries its constructor’s filing status in a
filing_status_input column. No variable has that name, so the column is
ignored. By default, filing_status is computed from the unit’s members,
including any supplied roles, and the filing rules. Reforms to those rules
apply when the formula computes the status. A direct filing_status input
supplied by a caller still overrides the formula.
Supplied roles fail closed rather than falling back. Each of these raises a
ValueError:
roles supplied for only some members of a unit;
a unit without exactly one
HEAD, or with more than oneSPOUSE.
Two consequences follow from honoring supplied roles. First, a dependent’s
income is left out of the return they are claimed on, because
irs_gross_income excludes tax-unit dependents, and so are their
above-the-line deductions. The model does not yet compute a dependent’s own
return (#9618),
so an adult dependent’s earnings leave the income tax base. Second, a minor can
head a return, which programs that identify minor parents as a tax-unit head or
spouse under 18 will treat as a minor parent
(#9619).
Filtering by geography¶
The microdata includes geographic identifiers that can be used for state-level analysis:
# Calculate for all households, then filter to a specific state
sim = Microsimulation()
in_california = sim.calc("CA") # Boolean for California residents
ca_benefits = sim.calc("household_benefits")[in_california]
print(f"California total benefits: ${ca_benefits.sum():,.0f}")
Each state has a corresponding boolean variable using its two-letter abbreviation (e.g., CA, NY, TX).
Subsampling for faster development¶
For development and testing, use subsample() to work with a smaller representative sample:
sim = Microsimulation()
sim.subsample(1_000) # Sample 1,000 households
# Or use a fraction
sim.subsample(frac=0.1) # 10% of households
# Results are still weighted to represent the full population
total = sim.calc("household_net_income").sum()
The subsample() method:
Samples households (keeping all people within sampled households)
Adjusts weights to maintain population totals
Accepts a
seedparameter for reproducibility
Winners and losers analysis¶
A common use case is comparing baseline policy to a reform:
from policyengine_us import Microsimulation
from policyengine_core.reforms import Reform
# Define a reform (example: double the EITC)
reform = Reform.from_dict(
{"gov.irs.credits.eitc.max[0].amount": {"2024-01-01.2100-12-31": 1246}},
country_id="us",
)
# Create baseline and reformed simulations
baseline = Microsimulation()
reformed = Microsimulation(reform=reform)
# Calculate net income change at person level for proper population weighting
baseline_income = baseline.calc("household_net_income", map_to="person")
reformed_income = reformed.calc("household_net_income", map_to="person")
gain = reformed_income - baseline_income
# Analyze winners and losers
winners = gain > 1 # Gained more than $1
losers = gain < -1 # Lost more than $1
no_change = ~winners & ~losers
print(f"Winners: {winners.mean():.1%}")
print(f"Losers: {losers.mean():.1%}")
print(f"No change: {no_change.mean():.1%}")
# Total cost/benefit
print(f"Net cost: ${gain.sum():,.0f}")
# Average gain among winners
print(f"Average gain for winners: ${gain[winners].mean():,.0f}")
Weight sanity checks¶
When running microsimulations, verify that weights produce sensible population totals:
sim = Microsimulation()
# Check population. The weights are the values here, so read them
# unweighted: a weighted sum would square them.
person_weight = sim.calc("person_weight", map_to="person", use_weights=False)
print(f"Total population: {person_weight.sum():,.0f}")
# Check household count
household_weight = sim.calc("household_weight", use_weights=False)
print(f"Total households: {household_weight.sum():,.0f}")
# Verify key aggregates against published statistics
total_earnings = sim.calc("employment_income").sum()
print(f"Total employment income: ${total_earnings / 1e12:.2f}T")
total_snap = sim.calc("snap").sum()
print(f"Total SNAP benefits: ${total_snap / 1e9:.1f}B")
calc returns a weighted series whose sum() multiplies each value by its
weight, which is what makes the aggregates above population totals. Summing a
weight variable that way squares the weights: with household weights of 2 and
3, sim.calc("household_weight").sum() is 13 rather than 5. Either opt out
with use_weights=False, as above, or use count(), which sums the weights
themselves - sim.calc("age", map_to="person").count() is the same population
total.
Compare these totals against official statistics to validate your analysis:
US population: ~330 million
US households: ~130 million
SNAP expenditure: ~$110 billion annually
Performance tips¶
Subsample during development: Use
sim.subsample(1_000)while iteratingAvoid redundant calculations: Store results in variables rather than recalculating
Use appropriate entities: Calculate at the natural entity level when possible
# Good: Calculate at natural entity level, then map if needed
snap = sim.calc("snap") # spm_unit level
snap_per_person = sim.calc("snap", map_to="person")
# Slower: Unnecessarily mapping multiple times
Complete example¶
from policyengine_us import Microsimulation
from policyengine_core.reforms import Reform
import pandas as pd
# Create simulations
baseline = Microsimulation()
baseline.subsample(10_000, seed=123) # For faster iteration
# Define a reform
reform = Reform.from_dict(
{
"gov.contrib.ubi_center.basic_income.amount": {"2024-01-01.2100-12-31": 1000},
"gov.contrib.ubi_center.basic_income.phase_out.rate": {
"2024-01-01.2100-12-31": 0.1
},
},
country_id="us",
)
reformed = Microsimulation(reform=reform)
reformed.subsample(10_000, seed=123)
# Calculate impacts
baseline_income = baseline.calc("household_net_income", map_to="person")
reformed_income = reformed.calc("household_net_income", map_to="person")
change = reformed_income - baseline_income
# Distributional analysis by income decile
decile = baseline.calc("household_income_decile", map_to="person")
results = []
for d in range(1, 11):
in_decile = decile == d
avg_change = change[in_decile].mean()
results.append({"Decile": d, "Average change": avg_change})
df = pd.DataFrame(results)
print(df)