Cleaning Without Hiding Problems

Make every repair explicit

pd.to_numeric(series, errors="coerce") converts valid numbers and turns invalid text into missing values. This is safer than silently keeping a mixed-type column. After conversion, investigate why values are missing and choose a policy.

Imputation means replacing a missing value with an estimated one. The median - the middle sorted value - is often more resistant to extreme values than the mean. Fit any production imputation rule on training data only; learning a median from the test set leaks information across the evaluation boundary.

Models usually need numbers. Categories such as basic, pro, and enterprise can be encoded. One-hot encoding creates one zero-or-one column per category. Do not assign arbitrary ranks unless the categories truly have an order.

Feature engineering creates useful inputs from raw fields, such as account age from a signup date. Every feature must be available at prediction time, have a clear definition, and be computed the same way in training and production.

Warning: fillna(0) is not universal cleaning. Zero may mean a real age, price, or count. A blanket replacement can erase the distinction between unknown and genuinely zero.