Social Darwinism stated that the rich should not give to the poor because they needed to build up themselves that the strongest would survive and the weakest would be weened out
The Fifteenth Amendment had a significant loophole: it did not grant suffrage to all men, but only prohibited discrimination on the basis of race and former slave status. States could require voters to pass literacy tests or pay poll taxes - difficult tasks for the formerly enslaved, who had little education or money.
I think that you just answered your own question in the question you asked.