I believe the speaker thinks her body is aging. As time goes by, she grows older and older, and her body is not as strong as it used to be when she was young. Her body is frail, in pain, and cannot do the things she was once able to do, which is evident in those lines you wrote above.
Considering the context of the poem, these lines reveal that the speaker’s body is aging. She suffers from a number of ailments that come with age and therefore she feels her body is rotting away. This feeling is expressed in the line "And my clay house mold’ring away." These ailments have also left her body very weak. The line ""Nor grinding pains my body frail" corroborates this view. Based on these two lines the speaker perhaps feels the physical body is a vessel that people possess in their mortal form. This vessel endures the hardships that time brings. Only upon death is a person free from the physical body.
