A pyramid is a polyhedron that has a base, which can be any polygon, and three or more triangular faces that meet at a point. These triangular sides are sometimes called the lateral faces to distinguish them from the base.
The name of a pyramid is often determined by the shape of the base.
Hence a rectangular pyramid has a rectangular shape, such that a cross section will produce a rectangular shape.