Running simulations

The Simulation class is the heart of PolicyEngine UK - it calculates tax and benefit outcomes for people and households. This guide walks you through everything from basic calculations to advanced microsimulation analysis.

Creating your first simulation¶

Every simulation starts with a situation that describes the people, benefit units, and households you want to analyse:

from policyengine_uk import Simulation

situation = {
    "people": {"person_1": {"age": {2025: 30}, "employment_income": {2025: 30_000}}},
    "benunits": {"benunit_1": {"members": ["person_1"]}},
    "households": {"household_1": {"members": ["person_1"]}},
}

sim = Simulation(situation=situation)
income_tax = sim.calculate("income_tax", 2025)
print(f"Income tax: £{income_tax[0]:.2f}")

This creates a simulation for one person and calculates their annual income tax liability. The result shows they would pay £3,486 per year in income tax.

Understanding the situation structure¶

The situation dictionary has three main sections:

People define individuals with their characteristics:

"people": {
    "sarah": {
        "age": {2025: 28},
        "employment_income": {2025: 35_000},
        "pension_contributions": {2025: 2_800}
    },
    "tom": {
        "age": {2025: 32},
        "self_employment_income": {2025: 18_000}
    }
}

Benefit units group people for benefit calculations (typically families):

"benunits": {
    "family": {
        "members": ["sarah", "tom"]
    }
}

Households group people who live together:

"households": {
    "home": {
        "members": ["sarah", "tom"],
        "housing_costs": {2025: 11400},  # Annual rent
        "council_tax": {2025: 1_800}   # Annual
    }
}

Calculating different variables¶

PolicyEngine UK can calculate hundreds of variables. Here are some common ones:

# Taxes
income_tax = sim.calculate("income_tax", 2025)
national_insurance = sim.calculate("national_insurance", 2025)
council_tax = sim.calculate("council_tax", 2025)

# Benefits
universal_credit = sim.calculate("universal_credit", 2025)
child_benefit = sim.calculate("child_benefit", 2025)
state_pension = sim.calculate("state_pension", 2025)

# Summary measures
net_income = sim.calculate("household_net_income", 2025)
in_poverty = sim.calculate("in_poverty", 2025)
marginal_tax_rate = sim.calculate("marginal_tax_rate", 2025)

Apprenticeship inputs¶

Apprentice status and entitlement to the apprentice minimum wage use separate annual person inputs:

is_apprentice records apprentice status and continues to determine the apprentice employer National Insurance relief, subject to its age limit.
is_in_first_year_of_apprenticeship records whether the person is within the first 12 months of the current apprenticeship employment. It defaults to False; it does not establish apprentice status on its own.

The apprentice minimum wage applies when is_apprentice is true and the person is either under 19 or has the first-year input set to true. Both minimum_wage and minimum_wage_category use this rule. At age 19 or over, an apprentice with a false or omitted first-year input uses the ordinary age-related rate. A later-year apprentice under 25 can therefore retain employer National Insurance relief while using the ordinary minimum wage.

The first 12 months refer to the current apprenticeship employment, rather than an academic or training-provider year. Employment with a previous employer counts where statutory continuity is preserved. A genuinely separate apprenticeship contract starts a new period, as explained in HMRC’s apprenticeship contract guidance and regulation 5.

For example, the person inputs for a 22-year-old in the first 12 months are:

"parent": {
    "age": {2026: 22},
    "is_apprentice": {2026: True},
    "is_in_first_year_of_apprenticeship": {2026: True},
    "employment_income": {2026: 8_000},
}

Previously, setting only is_apprentice selected the apprentice minimum wage at every age. Existing callers must now supply the first-year input for apprentices aged 19 or over who qualify for that rate. Missing tenure does not establish that someone has completed 12 months; the false default avoids assuming eligibility for the lower rate. The distinction also affects the minimum-income conditions for Tax-Free Childcare, extended childcare entitlement and the Childcare Offer for Wales.

These inputs classify the model’s annual period. The model does not infer an apprenticeship start date, change the classification at a birthday or anniversary within the year, or advance apprenticeship tenure during dataset projection. Callers must supply the classification for each modelled year. Annual minimum-wage calculations retain the existing fiscal-year conversion: the model uses the rate in force on 30 April for the whole labelled year. This change does not introduce a pay-reference-period calculation or alter the dated rate parameters.

Simulating policy scenarios¶

To test policy changes, create a scenario and apply it to your simulation. This example shows what happens when we increase Universal Credit’s standard allowance for single people over 25 from the current rate to £500 per month:

from policyengine_uk import Scenario

# Create a scenario that increases UC standard allowance
scenario = Scenario(
    parameter_changes={
        "gov.dwp.universal_credit.standard_allowance.single.OVER_25": 500
    }
)

# Apply the scenario to create a new simulation
reformed_sim = Simulation(situation=situation, scenario=scenario)

# Compare the average UC payment between baseline and scenario
baseline_uc = sim.calculate("universal_credit", 2025).mean()
reformed_uc = reformed_sim.calculate("universal_credit", 2025).mean()

# Calculate the difference to see the policy impact
difference = reformed_uc - baseline_uc
print(f"Average UC increase: £{difference:.2f} per month")

Working with families and children¶

Family situations require more detailed setup:

family_with_children = {
    "people": {
        "parent": {"age": {2025: 35}, "employment_income": {2025: 28_000}},
        "partner": {"age": {2025: 33}, "employment_income": {2025: 12_000}},
        "child_1": {"age": {2025: 7}},
        "child_2": {"age": {2025: 4}},
    },
    "benunits": {"family": {"members": ["parent", "partner", "child_1", "child_2"]}},
    "households": {
        "home": {
            "members": ["parent", "partner", "child_1", "child_2"],
            "housing_costs": {2025: 9600},  # Annual housing costs
            "childcare_costs": {2025: 7200},  # Annual childcare costs
        }
    },
}

family_sim = Simulation(situation=family_with_children)

# Calculate family-specific benefits - these are per benefit unit (family)
child_benefit = family_sim.calculate("child_benefit", 2025).mean()
childcare_support = family_sim.calculate("universal_credit_childcare", 2025).mean()

print(f"Child benefit: £{child_benefit:.2f} per month")
print(f"Childcare support: £{childcare_support:.2f} per month")

Using different data sources¶

From pandas DataFrames¶

A DataFrame holds one row per person and one column per input and year, named variable_name__year (note the double underscore __). This is the format Simulation.to_input_dataframe() exports, so an exported frame loads back.

import pandas as pd

from policyengine_uk import Simulation

df = pd.DataFrame(
    {
        "person_id__2025": [1, 2, 3, 4],
        "benunit_id__2025": [1, 1, 1, 2],
        "household_id__2025": [1, 1, 1, 2],
        "age__2025": [38, 36, 7, 70],
        "employment_income__2025": [32_000, 18_000, 0, 0],
        # A household value repeats on the row of each member.
        "region__2025": ["LONDON", "LONDON", "LONDON", "WALES"],
        "council_tax__2025": [1_800, 1_800, 1_800, 1_400],
    }
)

sim = Simulation(dataset=df)
print(sim.calculate("income_tax", 2025))  # One value per person
print(sim.calculate("council_tax", 2025))  # One value per household

benunit_id__<year> and household_id__<year> name each person’s benefit unit and household. Without them, person_benunit_id__<year> and person_household_id__<year> name them; with neither, each person is a benefit unit and household of their own.
A benefit unit or household variable takes the value on its members’ rows, which must all hold the same value.
Identifiers are whole numbers, and each person’s person_id is their own.
The simulation’s dataset is the UKMultiYearDataset the frame describes, with a year for each year the frame gives values for (identifier columns hold for every year, so they add none). Values for a month or another period that is not one year are set on the simulation directly. A variable given for both months and the year is set column by column in the frame’s order, since a year’s value is spread over the months not already set.
to_input_dataframe() gives the identifier columns for every year that one of its columns gives a whole year’s value for, since a simulation built from the frame holds the identifiers in each of those years. So an exported frame loads back to a simulation that exports the same frame, even when the simulation was given inputs for years outside its data, such as the Universal Credit health element status the rebalancing records for 2026 to 2029.
Inputs carry forward to later years but not back to earlier ones. So when set_input gives a simulation an input for a year before every year it holds the identifiers for, it also gives the identifiers for that year. Formulas that read identifiers, such as the draws that place each person in their year of age, then read the same people and groups there as in the simulation its exported frame loads back to.

From entity tables with `UKSingleYearDataset` and `UKMultiYearDataset`¶

If your data is already split into person-, benefit-unit-, and household-level tables, use the UK dataset classes directly:

import pandas as pd

from policyengine_uk import Simulation
from policyengine_uk.data import UKMultiYearDataset, UKSingleYearDataset

person_2025 = pd.DataFrame(
    {
        "person_id": [1, 2],
        "person_benunit_id": [1, 1],
        "person_household_id": [1, 1],
        "age": [35, 7],
        "employment_income": [32_000, 0],
    }
)

benunit_2025 = pd.DataFrame(
    {
        "benunit_id": [1],
    }
)

household_2025 = pd.DataFrame(
    {
        "household_id": [1],
        "region": ["LONDON"],
        "council_tax": [1_800],
    }
)

dataset_2025 = UKSingleYearDataset(
    person=person_2025,
    benunit=benunit_2025,
    household=household_2025,
    fiscal_year=2025,
)

sim = Simulation(dataset=dataset_2025)
print(sim.calculate("income_tax", 2025))

Each table should contain the ID columns needed to link entities together:

person: person_id, person_benunit_id, person_household_id, plus any person-level variables
benunit: benunit_id, plus any benefit-unit variables
household: household_id, plus any household-level variables

Whatever the data source, keep identifiers to whole numbers from -2,147,483,648 to 2,147,483,647. A simulation stores them as 32-bit integers, so within that range the identifiers it returns (for example sim.calculate("person_household_id", 2025)) are the ones in your tables. Outside it, the simulation gives an EntityIdWarning: the identifiers it returns differ from yours, and so may anything rebuilt from them.

Benefit-unit and household rows can be in any order. The loader aligns each year’s group inputs by its ID column to the simulation’s sorted group IDs; person rows keep their supplied order. This alignment does not reorder the caller’s tables. Multi-year tables must still describe the same population.

Use UKSingleYearDataset when you have one cross-section and want PolicyEngine UK to extend it forward using its standard uprating assumptions.

Use UKMultiYearDataset when you already have explicit tables for multiple years and want those year-by-year values loaded as-is:

dataset = UKMultiYearDataset(
    datasets=[
        dataset_2025,
        UKSingleYearDataset(
            person=person_2026,
            benunit=benunit_2026,
            household=household_2026,
            fiscal_year=2026,
        ),
    ]
)

sim = Simulation(dataset=dataset)
print(sim.calculate("household_net_income", 2026))

Simulation and Microsimulation can also load H5 files from local paths, Hugging Face URLs, or Google Cloud Storage URLs:

Files written by either UKSingleYearDataset.save(path) or UKMultiYearDataset.save(path) can be loaded with Simulation(dataset=path). The loader detects single-year tables and multi-year groups from the H5 layout.

sim = Microsimulation(
    dataset="gs://policyengine-uk-data-private/enhanced_frs_2023_24.h5@1.55.10"
)

For gs:// URLs, a numeric suffix after @ pins an exact GCS generation. A non-numeric suffix pins the PolicyEngine data version stored in the object’s GCS metadata.

From survey datasets¶

For population-level analysis, use survey data:

from policyengine_uk import Microsimulation

# Load the Enhanced Family Resources Survey
sim = Microsimulation(
    dataset="hf://policyengine/policyengine-uk-data/enhanced_frs_2022_23.h5"
)

# Calculate population totals using survey weights
# This gives us an estimate for the entire UK population
total_income_tax = sim.calculate("income_tax", 2025).sum() / 1e9
print(f"Total UK income tax revenue: £{total_income_tax:.1f}bn")

Microsimulation analysis¶

The Microsimulation class extends Simulation with survey weights for population estimates:

from policyengine_uk import Microsimulation

# Compare baseline and reform
baseline = Microsimulation(
    dataset="hf://policyengine/policyengine-uk-data/enhanced_frs_2022_23.h5"
)

# Create a scenario that increases the basic rate of income tax to 25%
scenario = Scenario(parameter_changes={"gov.hmrc.income_tax.rates.uk[0].rate": 0.25})
reformed = Microsimulation(
    dataset="hf://policyengine/policyengine-uk-data/enhanced_frs_2022_23.h5",
    scenario=scenario,
)

# Calculate revenue impact by comparing household net incomes
# When net income falls, government revenue increases
baseline_net = baseline.calculate("household_net_income", 2025)
reformed_net = reformed.calculate("household_net_income", 2025)

# The negative of the income change gives us the revenue change
revenue_change = -(reformed_net - baseline_net).sum() / 1e9
print(f"Additional government revenue: £{revenue_change:.1f}bn")

# Analyse distributional impact by looking at poverty rates
# in_poverty is a boolean variable, so mean() gives us the poverty rate
poverty_baseline = baseline.calculate("in_poverty", 2025).mean()
poverty_reformed = reformed.calculate("in_poverty", 2025).mean()

poverty_change = poverty_reformed - poverty_baseline
print(f"Poverty rate change: {poverty_change:.1%}")

Advanced techniques¶

Understanding caching behavior¶

# Variables are cached once you calculate them
income_tax = sim.calculate("income_tax", 2025)  # Cached after first calculation
# Subsequent calls return the cached value unless inputs change

# To insert new values, use set_input with arrays
current_income = sim.calculate("employment_income", 2025)
new_income = current_income * 0 + 35_000  # Create array with new values
sim.set_input(
    "employment_income", 2025, new_income
)  # This will invalidate related caches

# Parameters are also cached when requested
parameters = sim.tax_benefit_system.parameters(2025)  # Cached for 2025

# If you have a reform that changes parameters after requesting them:
scenario = Scenario(parameter_changes={"gov.hmrc.income_tax.rates.uk[0].rate": 0.25})
# The parameter change won't take effect unless you reset the cache:
sim.tax_benefit_system.reset_parameter_caches()

Calculating multiple variables efficiently¶

Instead of calculating variables one by one, you can get multiple results at once using the built-in calculate_dataframe method:

# Get multiple variables in a single DataFrame
variables = [
    "employment_income",
    "income_tax",
    "universal_credit",
    "household_net_income",
]
results_df = sim.calculate_dataframe(variables, 2025)

# This gives you a DataFrame with columns for each variable
print(results_df.head())
print(f"Average net income: £{results_df['household_net_income'].mean():.2f}")

Working with different time periods¶

# Multi-year analysis to see how income changes over time
years = [2023, 2024, 2025]
income_by_year = {}

for year in years:
    income_by_year[year] = sim.calculate("household_net_income", year)

# Compare average household income across years
for year in years:
    avg_income = income_by_year[year].mean()
    print(f"{year}: £{avg_income / 12:.2f} per month")

Mapping variables to different entities¶

Some variables can be calculated at different levels:

# Calculate employment income at different entity levels
# Individual level - one value per person
person_income = sim.calculate("employment_income", 2025)
print(f"Average personal employment income: £{person_income.mean():.2f}")

# Household level - sums all employment income within each household
household_income = sim.calculate("employment_income", 2025, map_to="household")
print(f"Average household employment income: £{household_income.mean():.2f}")

# Benefit unit level - sums employment income within each benefit unit (family)
benunit_income = sim.calculate("employment_income", 2025, map_to="benunit")
print(f"Average benefit unit employment income: £{benunit_income.mean():.2f}")

Using custom weights¶

For microsimulation with custom weights:

# Access and modify survey weights for sensitivity analysis
weights = sim.get_weights("income_tax", 2025)
modified_weights = weights * 1.1  # Increase all weights by 10%

# Calculate population total with modified weights
unweighted_values = sim.calculate("income_tax", 2025, use_weights=False)
weighted_total = (unweighted_values * modified_weights).sum()
print(f"Total with 10% higher weights: £{weighted_total / 1e9:.1f}bn")

Debugging simulations¶

When simulations don’t behave as expected:

# Check individual components to understand the tax calculation
# These help you trace through how income tax is calculated step by step
employment_income = sim.calculate("employment_income", 2025).mean()
personal_allowance = sim.calculate("personal_allowance", 2025).mean()
taxable_income = sim.calculate("adjusted_net_income", 2025).mean()
income_tax = sim.calculate("income_tax", 2025).mean()

print(f"Employment income: £{employment_income:.2f}")
print(f"Personal allowance: £{personal_allowance:.2f}")
print(f"Taxable income: £{taxable_income:.2f}")
print(f"Income tax: £{income_tax:.2f}")

# Verify the household composition was interpreted correctly
avg_age = sim.calculate("age", 2025).mean()
children_per_household = sim.calculate("household_count_children", 2025).mean()
print(f"Average age: {avg_age:.1f} years")
print(f"Average children per household: {children_per_household:.1f}")

Comparing simulations and analysing dependencies¶

PolicyEngine UK provides utilities for comparing simulations and understanding variable relationships:

Fetching variable dependencies¶

The get_variable_dependencies method helps you understand what variables feed into a calculation:

from policyengine_uk import Microsimulation

sim = Microsimulation()

# Get direct dependencies of income tax
deps = sim.get_variable_dependencies("income_tax")
# Returns: ['earned_income_tax', 'savings_income_tax', 'dividend_income_tax', 'CB_HITC', ...]

# Get dependencies up to 3 levels deep to see the full calculation tree
deep_deps = sim.get_variable_dependencies("income_tax", depth=3)
# Returns expanded list including dependencies of dependencies

This is particularly useful for debugging unexpected results or understanding the full impact of policy changes.

Comparing baseline and reformed simulations¶

The compare method creates a DataFrame showing differences between simulations:

baseline = Microsimulation()
reformed = Microsimulation()  # Apply your reforms here

# Compare specific variables between simulations
comparison_df = baseline.compare(
    reformed, baseline.get_variable_dependencies("income_tax", depth=1), 2029
)

# The DataFrame has columns for each variable:
# - variable_self: baseline value
# - variable_other: reformed value
# - variable_change: difference between reformed and baseline

This makes it easy to trace through exactly how a reform affects different components of the tax and benefit system.

Performance tips¶

The simulation system is designed to be flexible and powerful. Start with simple examples and gradually build up to more complex analyses as you become familiar with the structure and capabilities.

Creating your first simulation¶

Understanding the situation structure¶

Calculating different variables¶

Apprenticeship inputs¶

Simulating policy scenarios¶

Working with families and children¶

Using different data sources¶

From pandas DataFrames¶

From entity tables with UKSingleYearDataset and UKMultiYearDataset¶

From survey datasets¶

Microsimulation analysis¶

Advanced techniques¶

Understanding caching behavior¶

Calculating multiple variables efficiently¶

Working with different time periods¶

Mapping variables to different entities¶

Using custom weights¶

Debugging simulations¶

Comparing simulations and analysing dependencies¶

Fetching variable dependencies¶

Comparing baseline and reformed simulations¶

Performance tips¶

From entity tables with `UKSingleYearDataset` and `UKMultiYearDataset`¶