It's never been about civil rights