Well, to start, the Supreme Court could have ruled differently in Plessy v. Ferguson.
In Plessy, the Supreme Court condoned the practice of "separate but equal" making it permissible for the white south (and north) to create discriminatory practices.