I feel like he's saying, when slaves came to America, they weren't welcome to all the holidays, they were whipped instead of celebrated (on this day particularly), and then the day finally comes when they get to celebrate the Fourth of July but they can't really celebrate it because too much bad stuff happened that day. He's been saying why should anyone celebrate it when they know slavery was going on at that time. Now Douglass was an African American and this is his perspective. If you ask a Caucasian what they did that day, they will have a different story. That's what Douglass means when he says, "The story of most nations is difficult to catalogue." I hope this helps! :)
