First, recall that
cos(A - B) = cos(A) cos(B) + sin(A) sin(B)
so you just need to find cos(A) and sin(B).
Since both A and B end in the second quadrant, you know that
• cos(A) and cos(B) are both negative
• sin(A) and sin(B) are both positive
Then from the Pythagorean identity, you get
cos²(A) + sin²(A) = 1 ==> cos(A) = -√(1 - sin²(A)) = -2√10/7
cos²(B) + sin²(B) = 1 ==> sin(B) = +√(1 - cos²(B)) = √21/5
You'll end up with
cos(A - B) = (-2√10/7) (-2/5) + (3/7) (√21/5)
… = (4√10 + 3√21)/35
(which makes the last sentence in the question kind of confusing, because this expression doesn't get much simpler and it's certainly not a rational number)
