Running simulations

The Simulation class is the heart of PolicyEngine UK - it calculates tax and benefit outcomes for people and households. This guide walks you through everything from basic calculations to advanced microsimulation analysis.

Creating your first simulation¶

Every simulation starts with a situation that describes the people, benefit units, and households you want to analyse:

from policyengine_uk import Simulation

situation = {
    "people": {"person_1": {"age": {2025: 30}, "employment_income": {2025: 30_000}}},
    "benunits": {"benunit_1": {"members": ["person_1"]}},
    "households": {"household_1": {"members": ["person_1"]}},
}

sim = Simulation(situation=situation)
income_tax = sim.calculate("income_tax", 2025)
print(f"Income tax: £{income_tax[0]:.2f}")

This creates a simulation for one person and calculates their annual income tax liability. The result shows they would pay £3,486 per year in income tax.

Understanding the situation structure¶

The situation dictionary has three main sections:

People define individuals with their characteristics:

"people": {
    "sarah": {
        "age": {2025: 28},
        "employment_income": {2025: 35_000},
        "pension_contributions": {2025: 2_800}
    },
    "tom": {
        "age": {2025: 32},
        "self_employment_income": {2025: 18_000}
    }
}

Benefit units group people for benefit calculations (typically families):

"benunits": {
    "family": {
        "members": ["sarah", "tom"]
    }
}

Households group people who live together:

"households": {
    "home": {
        "members": ["sarah", "tom"],
        "housing_costs": {2025: 11400},  # Annual rent
        "council_tax": {2025: 1_800}   # Annual
    }
}

Calculating different variables¶

PolicyEngine UK can calculate hundreds of variables. Here are some common ones:

# Taxes
income_tax = sim.calculate("income_tax", 2025)
national_insurance = sim.calculate("national_insurance", 2025)
council_tax = sim.calculate("council_tax", 2025)

# Benefits
universal_credit = sim.calculate("universal_credit", 2025)
child_benefit = sim.calculate("child_benefit", 2025)
state_pension = sim.calculate("state_pension", 2025)

# Summary measures
net_income = sim.calculate("household_net_income", 2025)
in_poverty = sim.calculate("in_poverty", 2025)
marginal_tax_rate = sim.calculate("marginal_tax_rate", 2025)

Simulating policy scenarios¶

To test policy changes, create a scenario and apply it to your simulation. This example shows what happens when we increase Universal Credit’s standard allowance for single people over 25 from the current rate to £500 per month:

from policyengine_uk import Scenario

# Create a scenario that increases UC standard allowance
scenario = Scenario(
    parameter_changes={
        "gov.dwp.universal_credit.standard_allowance.single.OVER_25": 500
    }
)

# Apply the scenario to create a new simulation
reformed_sim = Simulation(situation=situation, scenario=scenario)

# Compare the average UC payment between baseline and scenario
baseline_uc = sim.calculate("universal_credit", 2025).mean()
reformed_uc = reformed_sim.calculate("universal_credit", 2025).mean()

# Calculate the difference to see the policy impact
difference = reformed_uc - baseline_uc
print(f"Average UC increase: £{difference:.2f} per month")

Working with families and children¶

Family situations require more detailed setup:

family_with_children = {
    "people": {
        "parent": {"age": {2025: 35}, "employment_income": {2025: 28_000}},
        "partner": {"age": {2025: 33}, "employment_income": {2025: 12_000}},
        "child_1": {"age": {2025: 7}},
        "child_2": {"age": {2025: 4}},
    },
    "benunits": {"family": {"members": ["parent", "partner", "child_1", "child_2"]}},
    "households": {
        "home": {
            "members": ["parent", "partner", "child_1", "child_2"],
            "housing_costs": {2025: 9600},  # Annual housing costs
            "childcare_costs": {2025: 7200},  # Annual childcare costs
        }
    },
}

family_sim = Simulation(situation=family_with_children)

# Calculate family-specific benefits - these are per benefit unit (family)
child_benefit = family_sim.calculate("child_benefit", 2025).mean()
childcare_support = family_sim.calculate("universal_credit_childcare", 2025).mean()

print(f"Child benefit: £{child_benefit:.2f} per month")
print(f"Childcare support: £{childcare_support:.2f} per month")

Using different data sources¶

From pandas DataFrames¶

A DataFrame holds one row per person and one column per input and year, named variable_name__year (note the double underscore __). This is the format Simulation.to_input_dataframe() exports, so an exported frame loads back.

import pandas as pd

from policyengine_uk import Simulation

df = pd.DataFrame(
    {
        "person_id__2025": [1, 2, 3, 4],
        "benunit_id__2025": [1, 1, 1, 2],
        "household_id__2025": [1, 1, 1, 2],
        "age__2025": [38, 36, 7, 70],
        "employment_income__2025": [32_000, 18_000, 0, 0],
        # A household value repeats on the row of each member.
        "region__2025": ["LONDON", "LONDON", "LONDON", "WALES"],
        "council_tax__2025": [1_800, 1_800, 1_800, 1_400],
    }
)

sim = Simulation(dataset=df)
print(sim.calculate("income_tax", 2025))  # One value per person
print(sim.calculate("council_tax", 2025))  # One value per household

benunit_id__<year> and household_id__<year> name each person’s benefit unit and household. Without them, person_benunit_id__<year> and person_household_id__<year> name them; with neither, each person is a benefit unit and household of their own.
A benefit unit or household variable takes the value on its members’ rows, which must all hold the same value.
Identifiers are whole numbers, and each person’s person_id is their own.
The simulation’s dataset is the UKMultiYearDataset the frame describes, with a year for each year the frame gives values for (identifier columns hold for every year, so they add none). Values for a month or another period that is not one year are set on the simulation directly. A variable given for both months and the year is set column by column in the frame’s order, since a year’s value is spread over the months not already set.
to_input_dataframe() gives the identifier columns for every year that one of its columns gives a whole year’s value for, since a simulation built from the frame holds the identifiers in each of those years. So an exported frame loads back to a simulation that exports the same frame, even when the simulation was given inputs for years outside its data, such as the Universal Credit health element status the rebalancing records for 2026 to 2029.

From entity tables with `UKSingleYearDataset` and `UKMultiYearDataset`¶

If your data is already split into person-, benefit-unit-, and household-level tables, use the UK dataset classes directly:

import pandas as pd

from policyengine_uk import Simulation
from policyengine_uk.data import UKMultiYearDataset, UKSingleYearDataset

person_2025 = pd.DataFrame(
    {
        "person_id": [1, 2],
        "person_benunit_id": [1, 1],
        "person_household_id": [1, 1],
        "age": [35, 7],
        "employment_income": [32_000, 0],
    }
)

benunit_2025 = pd.DataFrame(
    {
        "benunit_id": [1],
    }
)

household_2025 = pd.DataFrame(
    {
        "household_id": [1],
        "region": ["LONDON"],
        "council_tax": [1_800],
    }
)

dataset_2025 = UKSingleYearDataset(
    person=person_2025,
    benunit=benunit_2025,
    household=household_2025,
    fiscal_year=2025,
)

sim = Simulation(dataset=dataset_2025)
print(sim.calculate("income_tax", 2025))

Each table should contain the ID columns needed to link entities together:

person: person_id, person_benunit_id, person_household_id, plus any person-level variables
benunit: benunit_id, plus any benefit-unit variables
household: household_id, plus any household-level variables

Use UKSingleYearDataset when you have one cross-section and want PolicyEngine UK to extend it forward using its standard uprating assumptions.

Use UKMultiYearDataset when you already have explicit tables for multiple years and want those year-by-year values loaded as-is:

dataset = UKMultiYearDataset(
    datasets=[
        dataset_2025,
        UKSingleYearDataset(
            person=person_2026,
            benunit=benunit_2026,
            household=household_2026,
            fiscal_year=2026,
        ),
    ]
)

sim = Simulation(dataset=dataset)
print(sim.calculate("household_net_income", 2026))

Simulation and Microsimulation can also load H5 files from local paths, Hugging Face URLs, or Google Cloud Storage URLs:

Files written by either UKSingleYearDataset.save(path) or UKMultiYearDataset.save(path) can be loaded with Simulation(dataset=path). The loader detects single-year tables and multi-year groups from the H5 layout.

sim = Microsimulation(
    dataset="gs://policyengine-uk-data-private/enhanced_frs_2023_24.h5@1.55.10"
)

For gs:// URLs, a numeric suffix after @ pins an exact GCS generation. A non-numeric suffix pins the PolicyEngine data version stored in the object’s GCS metadata.

From survey datasets¶

For population-level analysis, use survey data:

from policyengine_uk import Microsimulation

# Load the Enhanced Family Resources Survey
sim = Microsimulation(
    dataset="hf://policyengine/policyengine-uk-data/enhanced_frs_2022_23.h5"
)

# Calculate population totals using survey weights
# This gives us an estimate for the entire UK population
total_income_tax = sim.calculate("income_tax", 2025).sum() / 1e9
print(f"Total UK income tax revenue: £{total_income_tax:.1f}bn")

Microsimulation analysis¶

The Microsimulation class extends Simulation with survey weights for population estimates:

from policyengine_uk import Microsimulation

# Compare baseline and reform
baseline = Microsimulation(
    dataset="hf://policyengine/policyengine-uk-data/enhanced_frs_2022_23.h5"
)

# Create a scenario that increases the basic rate of income tax to 25%
scenario = Scenario(parameter_changes={"gov.hmrc.income_tax.rates.uk[0].rate": 0.25})
reformed = Microsimulation(
    dataset="hf://policyengine/policyengine-uk-data/enhanced_frs_2022_23.h5",
    scenario=scenario,
)

# Calculate revenue impact by comparing household net incomes
# When net income falls, government revenue increases
baseline_net = baseline.calculate("household_net_income", 2025)
reformed_net = reformed.calculate("household_net_income", 2025)

# The negative of the income change gives us the revenue change
revenue_change = -(reformed_net - baseline_net).sum() / 1e9
print(f"Additional government revenue: £{revenue_change:.1f}bn")

# Analyse distributional impact by looking at poverty rates
# in_poverty is a boolean variable, so mean() gives us the poverty rate
poverty_baseline = baseline.calculate("in_poverty", 2025).mean()
poverty_reformed = reformed.calculate("in_poverty", 2025).mean()

poverty_change = poverty_reformed - poverty_baseline
print(f"Poverty rate change: {poverty_change:.1%}")

Advanced techniques¶

Understanding caching behavior¶

# Variables are cached once you calculate them
income_tax = sim.calculate("income_tax", 2025)  # Cached after first calculation
# Subsequent calls return the cached value unless inputs change

# To insert new values, use set_input with arrays
current_income = sim.calculate("employment_income", 2025)
new_income = current_income * 0 + 35_000  # Create array with new values
sim.set_input(
    "employment_income", 2025, new_income
)  # This will invalidate related caches

# Parameters are also cached when requested
parameters = sim.tax_benefit_system.parameters(2025)  # Cached for 2025

# If you have a reform that changes parameters after requesting them:
scenario = Scenario(parameter_changes={"gov.hmrc.income_tax.rates.uk[0].rate": 0.25})
# The parameter change won't take effect unless you reset the cache:
sim.tax_benefit_system.reset_parameter_caches()

Calculating multiple variables efficiently¶

Instead of calculating variables one by one, you can get multiple results at once using the built-in calculate_dataframe method:

# Get multiple variables in a single DataFrame
variables = [
    "employment_income",
    "income_tax",
    "universal_credit",
    "household_net_income",
]
results_df = sim.calculate_dataframe(variables, 2025)

# This gives you a DataFrame with columns for each variable
print(results_df.head())
print(f"Average net income: £{results_df['household_net_income'].mean():.2f}")

Working with different time periods¶

# Multi-year analysis to see how income changes over time
years = [2023, 2024, 2025]
income_by_year = {}

for year in years:
    income_by_year[year] = sim.calculate("household_net_income", year)

# Compare average household income across years
for year in years:
    avg_income = income_by_year[year].mean()
    print(f"{year}: £{avg_income / 12:.2f} per month")

Mapping variables to different entities¶

Some variables can be calculated at different levels:

# Calculate employment income at different entity levels
# Individual level - one value per person
person_income = sim.calculate("employment_income", 2025)
print(f"Average personal employment income: £{person_income.mean():.2f}")

# Household level - sums all employment income within each household
household_income = sim.calculate("employment_income", 2025, map_to="household")
print(f"Average household employment income: £{household_income.mean():.2f}")

# Benefit unit level - sums employment income within each benefit unit (family)
benunit_income = sim.calculate("employment_income", 2025, map_to="benunit")
print(f"Average benefit unit employment income: £{benunit_income.mean():.2f}")

Using custom weights¶

For microsimulation with custom weights:

# Access and modify survey weights for sensitivity analysis
weights = sim.get_weights("income_tax", 2025)
modified_weights = weights * 1.1  # Increase all weights by 10%

# Calculate population total with modified weights
unweighted_values = sim.calculate("income_tax", 2025, use_weights=False)
weighted_total = (unweighted_values * modified_weights).sum()
print(f"Total with 10% higher weights: £{weighted_total / 1e9:.1f}bn")

Debugging simulations¶

When simulations don’t behave as expected:

# Check individual components to understand the tax calculation
# These help you trace through how income tax is calculated step by step
employment_income = sim.calculate("employment_income", 2025).mean()
personal_allowance = sim.calculate("personal_allowance", 2025).mean()
taxable_income = sim.calculate("adjusted_net_income", 2025).mean()
income_tax = sim.calculate("income_tax", 2025).mean()

print(f"Employment income: £{employment_income:.2f}")
print(f"Personal allowance: £{personal_allowance:.2f}")
print(f"Taxable income: £{taxable_income:.2f}")
print(f"Income tax: £{income_tax:.2f}")

# Verify the household composition was interpreted correctly
avg_age = sim.calculate("age", 2025).mean()
children_per_household = sim.calculate("household_count_children", 2025).mean()
print(f"Average age: {avg_age:.1f} years")
print(f"Average children per household: {children_per_household:.1f}")

Comparing simulations and analysing dependencies¶

PolicyEngine UK provides utilities for comparing simulations and understanding variable relationships:

Fetching variable dependencies¶

The get_variable_dependencies method helps you understand what variables feed into a calculation:

from policyengine_uk import Microsimulation

sim = Microsimulation()

# Get direct dependencies of income tax
deps = sim.get_variable_dependencies("income_tax")
# Returns: ['earned_income_tax', 'savings_income_tax', 'dividend_income_tax', 'CB_HITC', ...]

# Get dependencies up to 3 levels deep to see the full calculation tree
deep_deps = sim.get_variable_dependencies("income_tax", depth=3)
# Returns expanded list including dependencies of dependencies

This is particularly useful for debugging unexpected results or understanding the full impact of policy changes.

Comparing baseline and reformed simulations¶

The compare method creates a DataFrame showing differences between simulations:

baseline = Microsimulation()
reformed = Microsimulation()  # Apply your reforms here

# Compare specific variables between simulations
comparison_df = baseline.compare(
    reformed, baseline.get_variable_dependencies("income_tax", depth=1), 2029
)

# The DataFrame has columns for each variable:
# - variable_self: baseline value
# - variable_other: reformed value
# - variable_change: difference between reformed and baseline

This makes it easy to trace through exactly how a reform affects different components of the tax and benefit system.

Performance tips¶

The simulation system is designed to be flexible and powerful. Start with simple examples and gradually build up to more complex analyses as you become familiar with the structure and capabilities.

Creating your first simulation¶

Understanding the situation structure¶

Calculating different variables¶

Simulating policy scenarios¶

Working with families and children¶

Using different data sources¶

From pandas DataFrames¶

From entity tables with UKSingleYearDataset and UKMultiYearDataset¶

From survey datasets¶

Microsimulation analysis¶

Advanced techniques¶

Understanding caching behavior¶

Calculating multiple variables efficiently¶

Working with different time periods¶

Mapping variables to different entities¶

Using custom weights¶

Debugging simulations¶

Comparing simulations and analysing dependencies¶

Fetching variable dependencies¶

Comparing baseline and reformed simulations¶

Performance tips¶

From entity tables with `UKSingleYearDataset` and `UKMultiYearDataset`¶