Because the loyalists didn’t have to pay taxes
The spread of Islam into the Maghreb is an example of Arab invasion of North Africa. The Arabs send Muslim army to Maghreb (modern morocco) to battle against Byzantines and allies, and finally conquer most of the North Africa were many Berbers (locals of Maghreb) join the Muslim army. After Arabs gain more power in the region, the Berbers are now ready to adopt and convert to Islam as well as the Arabic culture.
It was known as The Holocaust, which a Holocaust is where one group of people try to eliminate a certain group of people like how the Nazis tried to get rid of the Jewish people.