No rounding is necessary to answer the question.
We can subtract the (linear) density of the second box from that of the first to see which box is heavier per unit height.
This value is obviously less than zero, so ...
... the box of magazines has the greater mass per unit height.
_____
The question didn't ask for the mass per height, and we didn't compute it. All we did was make a conversion to comparable units. The units we ended up with are mixed English and metric units, but that doesn't matter for the purpose of comparison.
Since all we're really interested in is <em>the sign of the difference</em> of mass/height, we don't even need to actually compute that difference. We just need to do enough computation to be able to tell whether the sign is positive or negative.