Segregation was not just against Foundational Black Americans