Student loan repayment validation - PolicyEngine UK documentation

This notebook compares PolicyEngine UK’s calculated student loan repayments against reported repayments from the Family Resources Survey (FRS) microdata. Understanding the alignment between modelled and reported values helps assess model accuracy and identify areas for improvement.

Background¶

A borrower repays a percentage of income above a threshold, and the threshold depends on the loan’s plan. The figures below are for 2024-25, the year the code on this page runs (year = 2024; the model reads a year as the fiscal year that starts on 6 April).

Plan 1: loans for courses begun on or before 31 August 2012 under England’s and Wales’s student support regulations (SI 2009/470 reg. 3A), and every Student Finance Northern Ireland loan (GOV.UK). 9% of income above £24,990.
Plan 2: Student Finance England loans for courses begun from 1 September 2012 to 31 July 2023, and Student Finance Wales loans for courses begun on or after 1 September 2012 (GOV.UK). 9% of income above £27,295.
Plan 4: loans under Scottish student loan regulations from 1998 onwards whose repayments the Scottish Ministers have HMRC collect (SI 2009/470 reg. 3D). GOV.UK puts every Student Awards Agency Scotland loan on Plan 4. 9% of income above £31,395.
Plan 5: Student Finance England loans for courses begun on or after 1 August 2023 (GOV.UK). Its threshold is £25,000, but SI 2009/470 reg. 15(2BA), which SI 2022/1335 reg. 15(9) inserted, requires no repayment of a Plan 5 loan on or before 5 April 2026, so Plan 5 borrowers owe nothing in 2024-25 or 2025-26. The model doesn’t yet apply reg. 15(2BA): it charges Plan 5 borrowers 9% of income above £25,000 in those years too.
Postgraduate: 6% of income above £21,000.

The FRS captures reported student loan repayments, while PolicyEngine calculates repayments based on income and loan plan type.

from policyengine_uk import Microsimulation
import numpy as np
import pandas as pd

sim = Microsimulation()
year = 2024

# Get student loan data
reported = sim.calculate("student_loan_repayments", year).values
modelled = sim.calculate("student_loan_repayment", year).values
plan = sim.calculate("student_loan_plan", year).values
income = sim.calculate("adjusted_net_income", year).values
weight = sim.calculate("person_weight", year).values

Student loan plan distribution¶

First, let’s examine the distribution of student loan plans in the weighted population:

# Plan distribution (weighted)
plan_names = {
    0: "None",
    1: "Plan 1",
    2: "Plan 2",
    3: "Postgraduate",
    4: "Plan 4",
    5: "Plan 5",
}
for plan_id, name in plan_names.items():
    count = weight[plan == plan_id].sum() / 1e6
    print(f"{name}: {count:.2f}m people")

Aggregate comparison¶

Comparing total reported vs modelled repayments:

total_reported = (reported * weight).sum() / 1e9
total_modelled = (modelled * weight).sum() / 1e9

print(f"Total reported repayments: £{total_reported:.2f}bn")
print(f"Total modelled repayments: £{total_modelled:.2f}bn")
print(f"Ratio (modelled/reported): {total_modelled / total_reported:.2f}")

Individual-level alignment¶

For people who report making student loan repayments, how well do our calculations align?

# Filter to people with reported repayments > 0
has_reported = reported > 0

if has_reported.sum() > 0:
    # Correlation
    correlation = np.corrcoef(reported[has_reported], modelled[has_reported])[0, 1]
    print(f"Correlation (people with reported > 0): {correlation:.3f}")

    # Match rate
    both_positive = (reported > 0) & (modelled > 0)
    match_rate = both_positive.sum() / has_reported.sum() * 100
    print(f"People with both reported & modelled > 0: {match_rate:.1f}% of reporters")

    # Mean values
    print(f"\nMean reported (reporters): £{reported[has_reported].mean():,.0f}")
    print(f"Mean modelled (reporters): £{modelled[has_reported].mean():,.0f}")
    print(f"Mean income (reporters): £{income[has_reported].mean():,.0f}")

What can make the two measures differ¶

The code compares two variables. student_loan_repayments holds the repayments a dataset reports; where a dataset supplies none, its formula returns student_loan_repayment, and the two match exactly. student_loan_repayment applies the plan’s rate to income above the plan’s threshold, and caps the result at student_loan_balance where the balance is known. They can differ for these reasons:

Annual income: the model applies the threshold to a year’s income (student_loan_income, net income less benefits in kind), not to each pay period, so it doesn’t reflect how income varies within the year.
Voluntary repayments: the formula holds no voluntary repayments, so it leaves out any that a reported figure includes.
One plan per person: student_loan_plan gives each person one plan, so the model can’t charge someone who holds loans on two plans for both.
Missing plans: a person with no plan (NONE) gets no modelled repayment, whatever they report. #1764 reports a run on enhanced_frs_2023_24.h5 with policyengine-uk 2.88.43 for 2026 in which 5,570 people had a reported repayment and none had a modelled one, because no person had a plan.
Plan 5 before April 2026: the model charges Plan 5 borrowers repayments that the law doesn’t require until 6 April 2026 (see the plan list above).

Results¶

This page stores no output. The code reports the weighted totals of the two measures and their ratio and, for people with a reported repayment, the correlation between the measures, the share who also have a modelled repayment, and the mean of each measure and of adjusted net income. It draws no conclusion about how closely they agree: that needs a run with a stated date, model version and dataset.