First, note that for any differentiable functions
![g(x),G(x)](https://tex.z-dn.net/?f=g%28x%29%2CG%28x%29)
, we have
![G(g(x))](https://tex.z-dn.net/?f=G%28g%28x%29%29)
attaining its relative extrema at the same points as
![g(x)](https://tex.z-dn.net/?f=g%28x%29)
. This follows from the fact that
![\bigg(G(g(x))\bigg)'=G'(g(x))g'(x)=0](https://tex.z-dn.net/?f=%5Cbigg%28G%28g%28x%29%29%5Cbigg%29%27%3DG%27%28g%28x%29%29g%27%28x%29%3D0)
and so
![G(g(x))](https://tex.z-dn.net/?f=G%28g%28x%29%29)
has critical points at the same values of
![x](https://tex.z-dn.net/?f=x)
as does
![g(x)](https://tex.z-dn.net/?f=g%28x%29)
.
Now recall that the distance between any point in
![\mathbb R^3](https://tex.z-dn.net/?f=%5Cmathbb%20R%5E3)
and
![(10,11,0)](https://tex.z-dn.net/?f=%2810%2C11%2C0%29)
is
![d(x,y,z)=\sqrt{(x-10)^2+(y-11)^2+z^2}](https://tex.z-dn.net/?f=d%28x%2Cy%2Cz%29%3D%5Csqrt%7B%28x-10%29%5E2%2B%28y-11%29%5E2%2Bz%5E2%7D)
By the fact mentioned above, we know that the minimum distance between any point on the surface
![z^2=xy+1](https://tex.z-dn.net/?f=z%5E2%3Dxy%2B1)
will occur at the same point
![(x,y,z)](https://tex.z-dn.net/?f=%28x%2Cy%2Cz%29)
that minimizes the squared distance; that is,
![\hat d(x,y,z)=(x-10)^2+(y-11)^2+z^2](https://tex.z-dn.net/?f=%5Chat%20d%28x%2Cy%2Cz%29%3D%28x-10%29%5E2%2B%28y-11%29%5E2%2Bz%5E2)
We can then use the following function as the Lagrangian:
![L(x,y,z,\lambda)=(x-10)^2+(y-11)^2+z^2+\lambda(z^2-xy-1)](https://tex.z-dn.net/?f=L%28x%2Cy%2Cz%2C%5Clambda%29%3D%28x-10%29%5E2%2B%28y-11%29%5E2%2Bz%5E2%2B%5Clambda%28z%5E2-xy-1%29)
which has partial derivatives
![\begin{cases}L_x=2(x-10)-\lambda y\\L_y=2(y-11)-\lambda x\\L_z=2z+2\lambda z\\L_\lambda=z^2-xy-1\end{cases}](https://tex.z-dn.net/?f=%5Cbegin%7Bcases%7DL_x%3D2%28x-10%29-%5Clambda%20y%5C%5CL_y%3D2%28y-11%29-%5Clambda%20x%5C%5CL_z%3D2z%2B2%5Clambda%20z%5C%5CL_%5Clambda%3Dz%5E2-xy-1%5Cend%7Bcases%7D)
Set each partial derivative equal to 0. The third equation yields
![2z+2\lambda z=2z(1+\lambda)=0\implies z=0\text{ or }\lambda=-1](https://tex.z-dn.net/?f=2z%2B2%5Clambda%20z%3D2z%281%2B%5Clambda%29%3D0%5Cimplies%20z%3D0%5Ctext%7B%20or%20%7D%5Clambda%3D-1)
In the case of
![z=0](https://tex.z-dn.net/?f=z%3D0)
, we have
![xy+1=0](https://tex.z-dn.net/?f=xy%2B1%3D0)
, or
![y=-\dfrac1x](https://tex.z-dn.net/?f=y%3D-%5Cdfrac1x)
. Note that it must be the case that
![x\neq0](https://tex.z-dn.net/?f=x%5Cneq0)
. However, substituting this into the first two equations gives
![\begin{cases}2(x-10)+\frac\lambda x=0\\2\left(-\frac1x-11)-\lambda x=0\end{cases}\implies\begin{cases}2x^2-20x+\lambda=0\\-\lambda x^2-22x-2=0\end{cases}](https://tex.z-dn.net/?f=%5Cbegin%7Bcases%7D2%28x-10%29%2B%5Cfrac%5Clambda%20x%3D0%5C%5C2%5Cleft%28-%5Cfrac1x-11%29-%5Clambda%20x%3D0%5Cend%7Bcases%7D%5Cimplies%5Cbegin%7Bcases%7D2x%5E2-20x%2B%5Clambda%3D0%5C%5C-%5Clambda%20x%5E2-22x-2%3D0%5Cend%7Bcases%7D)
![\implies (2-\lambda)x^2-42x+\lambda-2=0](https://tex.z-dn.net/?f=%5Cimplies%20%282-%5Clambda%29x%5E2-42x%2B%5Clambda-2%3D0)
from which it follows necessarily that
![x=0](https://tex.z-dn.net/?f=x%3D0)
and
![\lambda=2](https://tex.z-dn.net/?f=%5Clambda%3D2)
. So we've arrived at a contradiction.
This means we must have
![\lambda=-1](https://tex.z-dn.net/?f=%5Clambda%3D-1)
From this, our first two equations become
![\begin{cases}2(x-10)+y=0\\2(y-11)+x=0\end{cases}\implies\begin{cases}2x+y=20\\x+2y=22\end{cases}](https://tex.z-dn.net/?f=%5Cbegin%7Bcases%7D2%28x-10%29%2By%3D0%5C%5C2%28y-11%29%2Bx%3D0%5Cend%7Bcases%7D%5Cimplies%5Cbegin%7Bcases%7D2x%2By%3D20%5C%5Cx%2B2y%3D22%5Cend%7Bcases%7D)
![\implies x=6,y=8](https://tex.z-dn.net/?f=%5Cimplies%20x%3D6%2Cy%3D8)
which in turn yield
![z^2=48+1=49\implies z=\pm\sqrt{49}=\pm7](https://tex.z-dn.net/?f=z%5E2%3D48%2B1%3D49%5Cimplies%20z%3D%5Cpm%5Csqrt%7B49%7D%3D%5Cpm7)
Therefore we have two points on
![z^2=xy+1](https://tex.z-dn.net/?f=z%5E2%3Dxy%2B1)
which are closest to (10, 11, 0), namely
![(6,8,\pm7)](https://tex.z-dn.net/?f=%286%2C8%2C%5Cpm7%29)
.