"One of my roles is to be the umpire" says Judge Dorow. "Then you gotta be fair, your honor. You're a public servant" says Brooks