Slaves were most commonly worked as cotton/Crop pickers, The more north you go up the less crops and plants can be grown due to the long winter months so this would be a cause as less slaves as they were mostly used as house workers/ nanny's.
Slavery was less in other smaller colonies because most colonies had a hunter gathers system and they could supply themselves with enough food for a while, but with a slave or another person working for you that would be another person that you would have to feed and that was not needed but if you lived in a greater city that had a large amount of people and a small government it would be perfect because you would not be a nomad traveling around.
