Savants is a term which describes individuals who are extremely deficient in some areas of intelligence but have one area where they are extremely gifted.
This ultimately implies that, savant refers to any individual who is an expert in a particular field such as science or literature i.e he or she is an erudite person with several achievements in their career but is considered to be deficient in other areas.
Basically, savants may have extreme abilities in either music, calculations, map making (cartography) or artwork etc.
The metaphor in this statement is "the great millstone" it kind of qualification word it express how big the time was. But in reality, millstone doesn't even have a relationship with the time. So "the great millstone" is a metaphor.