I'm assuming all of (x^2+9) is in the denominator. If that assumption is correct, then,
One possible answer is 
Another possible answer is 
There are many ways to do this. The idea is that when we have f( g(x) ), we basically replace every x in f(x) with g(x)
So in the first example above, we would have

In that third step, g(x) was replaced with x^2+9 since g(x) = x^2+9.
Similar steps will happen with the second example as well (when g(x) = x^2)
5(2x - 8) + 15 = -15
-15 -15 subtract 15 from each side
5(2x - 8) = -30
÷5 ÷5 divide both sides by 5
2x - 8 = -6
+8 +8 add 8 to each side
2x=2
÷2 ÷2 divide both sides by 3
x = 1
Checking:
5(2(1)-8) + 15 = -15
5(-6) + 15 = -15
-30 + 15 = -15
-15 = -15 Correct! x=1
Let the side of each side of photo is a inch.
So area of photo = (side)^2 =

But as given area of photo is 9 square inch.
So


So side of photo is 3 inch.