In Act 3, Scene 2:
CALIBAN
(to TRINCULO) Thou liest, thou jesting monkey, thou! I would my valiant master would destroy thee. I do not lie. " (The Tempest, Act 3, Scene 2)
Caliban calls Ariel "a jesting monkey" and a liar, who is in turn, speaking as Trínculo. Ariel is imitating Trínculo´s voice and is invisible, thus the confussion. It is worth mentioning that "jester" means fool.
This might be the case due to the lack of beauty they possess. Bayous and swamps typically are dirty, and are filled with insects. Many people would find that as a low standard of beauty.