Miller: "If you refuse to come back to the office, you're going to be laid off.