An outlier can affect the mean by a lot. For example, if my neighbor moves out and Bill Gates moves in (he would be a <u>super</u> outlier!), the mean annual income would increase enormously.
The median, on the other hand, would not change at all. The income in the middle is still the income in the middle; Gates' income would not budge the median.