The only Islamic empire that controlled territory in Europe in the early modern era was the Ottoman Empire. This empire started to develop and expand in the 15th century. The Ottomans managed to use the decline of the surrounding empires so they managed to conquer them with relative ease, thus creating a very large empire on excellent strategic location.
Part of the Ottoman Empire was located in Europe, or more specifically it controlled the Balkan Peninsula. The aspirations of the empire initially were to expand into Central Europe and possibly Western Europe, but the forces of these regions allied and didn't allowed the Ottomans to expand beyond the Balkans. The empire existed until the early 20th century, when it lost most of its territory and was reformed to create the modern day country of Turkey.