I think that what Frederick Douglass meant by the "wail of millions" is that while the 4th of July is a great day to be celebrated by non-African American people as it meant the true independence of the 13 colonies from Great Britain and freedom for many of its citizens, in fact slavery is still intact at the time of his speech in 1852 and so July 4th cannot be celebrated by the vast majority of the black people in the US who are still under the terrible yoke of slavery.
