Are We Really That Polarized ?