I think it's the culture/genes they were brought/born into. Because it can be genetically made into your DNA or you might be raised in that certain household/culture that you're used to.
The way their tongue moves is indeed the simple explanation. For example, in France certain parts of their mouth sound certain letters differently than how it is pronounced in English. Being trained to speak one way, then having to do it the opposite way, is tough, to say the least!
The original word or phrase is known as the subject of the anagram. Any word or phrase that exactly reproduces the letters in another order is an anagram.