Make a change of coordinates:


The Jacobian for this transformation is

and has a determinant of

Note that we need to use the Jacobian in the other direction; that is, we've computed

but we need the Jacobian determinant for the reverse transformation (from

to

. To do this, notice that

we need to take the reciprocal of the Jacobian above.
The integral then changes to

Answer:
It is the third one because of formula I² + 2I*II + II²
In your case I = 4x and II = 3
It can either be equivalent to 11 or 18.