Question

Let y be a random variable with a known distribution, and consider the square loss function `(a; y) = (a????y)2. We want to find the action a that has minimal risk, namely, to find a = arg mina E(a ???? y)2, where the expectation is with respect to y. Show that a = Ey, and the Bayes risk (i.e. the risk of a) is Var(y). In other words, if you want to try to predict the value of a random variable, the best you can do (for minimizing expected square loss) is to predict the mean of the distribution. Your expected loss for predicting the mean will be the variance of the distribution. You should use the fact that Var(y) = Ey2 ???? (Ey)2.

Answer 1

Answer/ Explanation:

Since X is exponentially distributed, its expected value is given by E[X]=1/λ=2.

Therefore,  E[Y]=E[1−2X]=E[1]+E[−2X]=E[1]−2E[X]=1−2E[X]=1−2⋅2=−3.

Hence,

We define the moment-generating function of Y as MY(t). It is given by

MY(t)=E[etY]=E[et(1−2X)]=E[ete−2tX]=E[et]E[e−2tX].

If I give you the hint that E[g(Y)]=∫∞0g(y)fY(y)dy, where fY(y) is the probability density function of Y, can you also solve for the moment generating function of Y?

We have E[X2]=2/λ2=2/(0.5)2=8. Thus,

E[Y2]=E[(1−2X)2]=E[1−4X+4X2]=E[1]−4E[X]+4E[X2]=1−4⋅2+4⋅8=25.

So,

Var(Y)=E[Y2]−E[Y]2=25−(−3)2=16.

Continuing for the moment-generating function:

MY(t)=E[et]E[e−2tX]=etE[e−2tX]=et∫∞x=0e−2txfX(x)dx,

where fX(x) is the probability density function of X and thus satisfies fX(x)=λe−λx. Substituting yields

MY(t)=et∫∞x=0e−2txλe−λxdx=λet∫∞x=0e−x(2t+λ)dx=λet2t+λ.

It is also good to note that

If you are after expectation, variance or moment generating function of Y then it is not needed to find the PDF of Y (see the answer of Ritz).

This is not an answer on the question in the title, but one on the question in the body.

FY(y)=P(Y≤y)=P(1−2X≤y)=P(X≥0.5−0.5y)=1−FX(0.5−0.5y)

Note that the last equality demands that FX is continuous.

Differentating on both sides gives fY on LHS and an expression in fX on RHS.