The Pelosi issue has raised concerns