I believe he may want to eat at the table because back then in the segregated times African Americans were not allowed to eat with the other guests they had to eat in the kitchen. So basically he was saying he was an American as well and his color made no difference to whether or not he could eat with the rest of the guests.