Family says elderly man was neglected by caretakers