Machine Learning Basics · 14 min · 160 XP
Features that carry signal
Turn raw columns into features a model can use, without leaking the answer into them.
Models only see numbers, and the way you turn raw columns into numbers usually matters more than which algorithm you choose. Three jobs come up every time: encoding categories, handling missing values, and making sure no feature secretly contains the label.
Categories. Turning city into Oslo = 1, Dublin = 2, Milan = 3 tells a linear model that Milan is "three times" Oslo, which is nonsense. One-hot encoding gives each category its own 0/1 column instead. Learn the list of categories from the training data, and decide what an unseen category does in production. The usual answer is all zeros, meaning "none of the ones I know".
Missing values. Fill them in (with the training mean, for example) and add a column that says the value was missing. Missingness is often signal in its own right: a customer who never entered a phone number behaves differently from one who did.
Target leakage is a feature that wouldn't exist at prediction time, or that exists only because of the outcome. To predict whether a customer will churn next month, cancellation_reason is a perfect predictor, and it's useless, because you only have it after they've churned. For every feature, ask: would I know this at the moment I need to make the prediction?
from sklearn.preprocessing import OneHotEncoder from sklearn.impute import SimpleImputer encoder = OneHotEncoder(handle_unknown="ignore").fit(train[["city"]]) # unseen → all zeros imputer = SimpleImputer(strategy="mean", add_indicator=True).fit(train[["age"]])
Loading your workspace…