People of color is more used as if someone might be mulatto, or might have colour. People of colour is more used for anyone that isn't white/in a minority group, including Asians. The both can have the same meaning, someone who haw a lot a melanin that usually isn't white, and coloured people is more someone that is not white. I hope that makes since.
The social interaction can change because when you think of the word, coloured people, that is something more associated with things like lynching, racial discrimination, slavery, The Trail of Tears, etc., this topic can make certain people uncomfortable changing the atmosphere of the conversation. If you say people of colour, it applies on a broader stand than just the races that were discriminated in America (African-Americans, Latinos,Japanese people, people from the Middle East, Native Americans, the list goes on and on and on, like, never ending...) but to a broader stance of people of colour around the world, which can be more interesting, with culture, wanting to visit, and sometimes cultural appropriation and topics like that, but still!
