Doing Right for the Wrong Reasons