J. Patrick Lewis is speaking about 9/11, a terrorist attack which destroyed the World Trade Center Towers, more commonly known as the Twin Towers, and killed many people with it. In this part of the poem, the Empire State Building is the 'narrator'. He's saying that he's lonely because his 'big twin brothers' have died.
Hope this is what you're looking for!