He thinks Romeo is there to insult his family because of the ongoing feud.
Explanation:
Tybalt is an explosive and easily angered character. Moreover, he cares deeply about his family's honor. Therefore, he is immediately upset when he realizes Romeo is at the party. He knows that his family and Romeo's family are enemies, and therefore, he assumes that Romeo is there in order to insult the family. However, he is unable to attack Romeo because of the intervention of Capulet.