Lagrangian:
where the function we want to minimize is actually
, but it's easy to see that
and
have critical points at the same vector
.
Derivatives of the Lagrangian set equal to zero:
Substituting the first three equations into the fourth gives
Solving for
, we get a single critical point at
, which in turn gives the least distance between the plane and (0, 2, 5) of
.