A single, governed master dataset of 48,790 U.S. adults — built once, used everywhere — feeding both a Power BI analytics layer for human exploration and four tuned ML classifiers for automated scoring. The goal wasn't just to predict who earns more than $50K; it was to make the why legible to a non-technical decision-maker.
The starting brief called for two things that are easy to build separately and dangerous to build separately: a Power BI dashboard for exploring workforce demographics, and ML models predicting income bracket. The tempting shortcut is two datasets — one imputed and encoded for modeling, one left more readable for BI. I deliberately avoided that. Two datasets drift: a fix applied to one never reaches the other, and by the second refresh cycle "the data" means two different things depending on who you ask.
Instead, everything here traces back to one governed pipeline (build_master_dataset.py)
that produces a single master table, adult_master.csv / .parquet — 48,790
rows, 26 columns. Power BI reads the human-readable income_bracket and categorical
columns directly; the ML pipeline reads the numeric income_target flag and the same
underlying predictors. One source of truth, two consumers.
Unknown and paired with a boolean *_was_missing flag, so the fact that a
value was missing stays fully auditable instead of being silently smoothed over.
The Power BI layer isn't a KPI wall — it's built around one reusable DAX pattern,
High Income Rate vs Overall, that can drop onto any breakdown (occupation, education,
region, work-hours category) and immediately show whether that segment sits above or below the
population baseline. Twenty measures in total, plus three sort-key columns so categories like
"Part-time (<20)" through "Overtime (60+)" render in logical rather than alphabetical order.
Rather than trust a single model's built-in feature importance, I computed permutation importance across all four tuned models on a held-out sample and looked for agreement. The signal was consistent: relationship status and marital status dominate, followed by capital gains, education, age, occupation, and hours worked.
Relative ranking from permutation importance averaged across logistic regression, random forest, HistGradientBoosting, and XGBoost — not a single model's opinion.
Two findings were worth calling out explicitly to a non-technical reader rather than leaving buried
in a model file. Married individuals — specifically the "Husband" and "Wife" relationship
categories — show a high-income rate near 45–47%, against roughly 1.5% for adults still
classified as someone's child, a gap that reflects age and career-stage confounding as much as
marital status itself. And fnlwgt, a U.S. Census sampling weight included in the raw
data, has effectively zero correlation with income (Pearson r = −0.006) — it was excluded from
modeling entirely rather than left in to add noise.
Four classifiers were tuned via RandomizedSearchCV with stratified cross-validation,
using the dataset's original train/test split (32,537 / 16,253 rows) to stay comparable with the
historical UCI benchmark. Categorical features were handled natively where the model supported it
(HistGradientBoosting, XGBoost) rather than blown out with one-hot encoding.
| Model | Accuracy | Precision | Recall | F1 | ROC-AUC |
|---|---|---|---|---|---|
| Logistic Regression | 81.1% | 0.567 | 0.845 | 0.679 | 0.907 |
| Random Forest | 81.7% | 0.575 | 0.863 | 0.690 | 0.917 |
| HistGradientBoosting | 87.3% | 0.776 | 0.652 | 0.708 | 0.927 |
| XGBoost | 83.4% | 0.604 | 0.865 | 0.712 | 0.928 |
On the recommendation: XGBoost edges out HistGradientBoosting on ROC-AUC (0.928 vs
0.927) — but that's the wrong reason to call it the clear winner. HistGB actually has better
accuracy, precision, and error rate; XGBoost's class-rebalancing (scale_pos_weight)
trades precision for recall, catching more true high earners at the cost of more false positives.
Which one is "better" depends on whether the downstream decision punishes missed opportunities or
wasted outreach more — a business question, not a modeling one, and I said so rather than picking a
winner by a 0.001 margin.