Refugees may have the hardest time with Travel Ban