Emma Roberts is being 'very strict' about who visits her baby