I've heard this phrased as "I plead the fifth" but it typically means that they are invoking the 5th amendment. Whether in jest or an actual legal situation, by saying this they invoke their right to not incriminate themselves.
As the 5th amendment means that someone can't be punished without evidence. Whatever you say may be considered evidence.
