Answer:
Yes. But only on the land that wasn't pre-occupied by the native.
Explanation:
The vast majority of the land in North America at the time was inhabited. Coming to this territory to form farms and towns would not bother any particularly group of people.
The creation for farms and towns itself wasn't immoral.
But, doing so while taking the ancestral land of the locals and forced them to moved away or kill them is considered as 'immoral'. There were plenty of space that hey can occupied without doing so.
The abolitionist movement was against people being dis-empowered and enslaved because of their colour. The feminist movement grew from the same point, particularly in the USA and England, for women realized they had been dis-empowered because of their gender. Neither gender or colour was choice.