He was one of the authors of the Constitution, so he supported the federal government...which was created in the Constitution. (It would be pretty weird to not support the thing you helped write.) If you can remember the fact that he helped write the Constitution and the Federalist Papers, their title and purpose should help you remember he supported the federal government. The Constitution created the federal government. The Federalist Papers were a series of essays published in NY newspapers that tried to convince people to support the ratification of the US Constitution.
Before the Constitution, the US was governed by the Articles of Confederation, which was just a weak alliance of the 13 states. ("confederation" = "an organization that consists of a number of parties or groups united in an alliance or league") There was only a congress in this alliance (no President, no Supreme Court) and that congress was structured so that each state had one vote. And this congress was only entrusted with a very few powers; most powers were State governments' decisions to make. The result was a United States government that didn't govern much and wasn't that united.
Enter the Constitution: a plan to create a stronger government for the whole country. But many people were not in favor of this, so there was a major debate throughout the country, and in particular, in certain states...like New York.
So the people who helped write the Constitution, James Madison being the person credited with authoring it most primarily, also wrote essays to inform citizens of the decision making and theories behind the Constitution. People who were in favor of the Constitution were Federalists. Those essays were the Federalist Papers. (And, as the Constitution helped form a federal government, this hopefully makes sense.)
