1

We find that empirical evaluations of heuristic privacy defenses can be highly misleading, and propose a new evaluation protocol that is reliable and efficient.

We show that the strength of a model’s inductive bias determines whether interpolation of noisy data is harmless or harmful.

We reveal unexpected benefits of regularization even in the overparameterized regime by proving that for both linear regression and classification, avoiding interpolation significantly improves generalization.