The Dangers of a Little Impurity