If it's getting annoying or is in any way inappropriate, cut it off now. However, if he is not completely mean, wait. You don't want to be unkind. Funniest thing to do would be to say "You're like a brother to me," But that would be mean and awkward, so it's best to just wait it out...
