Taxpayers want city manager out of office during investigations