Skip to content

Add nonfiler income calibration targets#994

Draft
MaxGhenis wants to merge 1 commit into
mainfrom
codex/nonfiler-income-targets
Draft

Add nonfiler income calibration targets#994
MaxGhenis wants to merge 1 commit into
mainfrom
codex/nonfiler-income-targets

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

Summary

  • add CBO AGI-by-source filer-only calibration targets for wages, self-employment, taxable pensions, taxable Social Security, qualified dividends, net capital gains, and taxable interest plus ordinary dividends
  • add BEA/NIPA all-population macro targets for wages, proprietors' income, personal interest income, and personal dividend income
  • support additive target expressions in the unified calibration matrix so targets like taxable_interest_income+dividend_income can stay linear in survey weights
  • add IRS SOI wages (00200) as irs_employment_income and tests for the new target extraction/matrix filtering behavior

Local Evaluation

Used policyengine-us==1.691.13 with the public calibration stratified_extended_cps.h5 and published calibration DB as the base. Private source impute/PUF clone was skipped because the raw private inputs were not locally available.

New-target local fit:

  • matrix: 40,365 targets x 119,990 columns, 9.3M nonzeros
  • diagnostics: mean absolute relative error 68.7%, median 48.6%, <10% = 13.9%, <25% = 29.4%

Old-target control fit under the same settings:

  • matrix: 40,348 targets x 119,990 columns, 8.5M nonzeros
  • diagnostics: mean absolute relative error 68.5%, median 48.2%, <10% = 13.9%, <25% = 29.7%

Selected old -> new aggregate estimates:

  • NIPA wages: $4.128T -> $4.061T vs $12.388T target
  • CBO filer employment income: $3.792T -> $3.728T vs $10.833T target
  • NIPA proprietors' income: $1.078T -> $1.068T vs $2.023T target
  • CBO filer self-employment income: $249.5B -> $251.7B vs $1.916T target
  • NIPA dividends: $581.9B -> $577.5B vs $2.219T target

Baseline SPM under the local weights:

  • SPM poverty: 59.69% -> 59.99%
  • deep SPM poverty: 41.08% -> 41.27%

Interpretation: the new targets are included and build locally, but this default full-target local fit barely moves the aggregates. The next calibration improvement is likely target weighting/grouping rather than only adding rows.

Tests

  • uv run --frozen --with policyengine-us==1.691.13 pytest policyengine_us_data/tests/test_etl_national_targets.py policyengine_us_data/tests/test_calibration/test_unified_matrix_builder_constrained_values.py -q
  • uv run --frozen --with policyengine-us==1.691.13 python -m py_compile policyengine_us_data/db/etl_national_targets.py policyengine_us_data/utils/loss.py policyengine_us_data/calibration/unified_matrix_builder.py policyengine_us_data/db/validate_database.py policyengine_us_data/utils/target_variables.py
  • git diff --check

@MaxGhenis MaxGhenis force-pushed the codex/nonfiler-income-targets branch 8 times, most recently from d4d84de to 41faed8 Compare May 17, 2026 20:45
@MaxGhenis MaxGhenis force-pushed the codex/nonfiler-income-targets branch from 41faed8 to 5c24de0 Compare May 17, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant