Father Joe insists he is not destitute