The Bill of Rights was the compromise that eventually got the Anti-Federalists to agree to ratify the United States Constitution because they had been worried about the Federal Government being too powerful.
We can see from here however, that there was opposition to the Bill of Rights.
The opponents argued that if particular rights were outlined specifically, the rights that were not outlined in like manner may be abused by the government which in effect would lead to the Bill of Rights limiting the rights of the very people it is meant to empower.
James Madison however, believed that the risk of this could be mitigated.