Rep. Dan Goldman, who rarely falls out of line with Democrats, says he is “very concerned”