The simile implies that the two men are poorly kept and dirty from the use of "ragged." "Rags" is defined as "a torn piece of cloth worn from use" usually used to describe the state of someone who is not wealthy who doesn't have expensive or even decent clothes to wear.
