3 The correlation coefficient and its properties interpretat

3. The correlation coefficient and its properties/ interpretation Let X, Y be two random variables. The correlation coefficient between X and Y is defined as (a) For any a, b E R, show that Cov(aX + b, cY + d) = a c Cov(X, Y) and Corr(aX + b, cY + d) = Corr(X, Y), i.e., the correlation coefficient is invariant to linear transformations. (b) Recall that E[Y] is the \'\'best\'\' constant that approximates Y, in the sense that it minimizes K(a) = E[(Y - a)^2] (Ex.7 in HW 5). Suppose now we want to approximate (predict) Y by a linear function of X, aX + b. Show that the \'\'best\'\' values for a and b are given by a = E[Y] - b E[X], b = Corr(X, Y) in the sense that they minimize K(a,b) = E[(Y - a - bX)^2]. Hint: Set the partial derivatives of K(a, b) with respect to a and b equal to 0. (c) Show that min K(a,b) = Var[Y] . (1 - Corr^2(X,Y)) and deduce that -1

Solution

= E((ax)+b-E(ax)-b)*E((cy)+d-E(cy)-d)

= E(a(X-E(x))*E(c(Y-E(Y))

= ac E(X-E(X))E(Y-E(Y))

= ac cov(X,Y)

Let us consider Y=aX+b-----1

            E(Y) =aE(X)+b-------2

1-2 =Y-E(X) = aX+b-aE(X)-b =a(X-E(X)

Squaring and taking expectations on both sides

We get

E(Y-E(Y))² = a² E(X-E(X))²

Var(y) = a² Var(X)

Var(aX+b) = a^{2 2}_X

Similarly Var(cY+d) = c^{2 2}_y

Corr(ax+b,cy+d) = ac * cov(x,y)/ a^{2 2}_X c^{2 2}_y

                                    = cov(x,y)/ _{X y}

                                    = corr(x,y)

E(y/x) = a+bx-------1

By def E(y/x) = y.f(y/x)dy = a+bx

= 1/f(x) y.f(x,y)dy = a+bx-----2

Multiplying both sides with f(x) and integrating with respect to x we get

y.f(x,y)dydx = a f(x)dx+b x f(x)dx

yf(x,y)dy = a+bE(x)

E(y) = a+bE(x)-------3

Multiplying both sides of 2by x* f(x) and integrating with respect to x we get

x y.f(x,y)dydx = a xf(x)dx+b x² f(x)dx

E(xy)= aE(x)+bE(x²)

Cov(xy) + X*Y =aX+b(²_X +X²)--------4

By solving equation 3 and 4 we get

b = cov(xy)/ ²_X and a= Y- cov(xy)/ ²_X (X)

b = r(xy) _{X y} / (_X)^2

b= r(xy) _y / (_X)

r(xy) = cov(xy)/_{X y}

by taking summasions and squaring on both sides we get

Using Schwartz inequality we have

(aibi)^2<=(ai)^2* (bi)^2

(aibi)^2/(ai)^2* (bi)^2 <=1

d) if the correlation coefficient r(xy) =1 the perfect positive correlation

if the correlation coefficient r(xy) =-1 the perfect negative correlation

if the correlation coefficient r(xy) =0 the perfect no correlation