You can build a cylinder in two ways: either you draw a circle on a plane, and "raise" it perpendicularly to the plane itself, or you start with a rectangle, and rotate it around one of its sides.
For these reasons, if you cut the cylinder with a plane parallel to the base you have a circular cross section, while if you cut the cylinder with a plane perpendicular to the base you have a rectangular cross section.