You're given that φ is an angle that terminates in the third quadrant (III). This means that both cos(φ) and sin(φ), and thus sec(φ) and csc(φ), are negative.
Recall the Pythagorean identity,
cos²(φ) + sin²(φ) = 1
Multiply the equation uniformly by 1/cos²(φ),
cos²(φ)/cos²(φ) + sin²(φ)/cos²(φ) = 1/cos²(φ)
1 + tan²(φ) = sec²(φ)
Solve for sec(φ) :
sec(φ) = - √(1 + tan²(φ))
Given that cot(φ) = 1/4, we have tan(φ) = 1/cot(φ) = 1/(1/4) = 4. Then
sec(φ) = - √(1 + 4²) = -√17
