What does it mean to perturb a black hole?