Americans were the white settlers.
In this similie Adams is referring to the memory of Lincoln.
African Americans did not have a good education opportunity. Segregation was big and many African Americans were rejected from getting a better education.