Families 'devastated' city could tear down homes