Problem 1

The linear regression model can be written in matrix notation as y = X + . Create the table shown below and describe what each term (y,X,,) represents (interpretation), specify the dimension of each term (size), indicate whether we model the term as random or non-random, and whether the term is observed or unobserved.

Problem 2

Assuming a simple linear regression model, derive the ordinary least squares estimators of ₀ and ₁. Do not use matrix notation in deriving your solution.

Problem 3

Let H = X(X^TX)¹X^T is the hat matrix. Prove that IH is a projection matrix (Symmetric + Idempotent).

Problem 3

While proving that ₁ is an unbiased estimator of ₁, we represented the OLS estimate as ₁ = ^Pk_iY_i, where k_i = _P^X_(Xⁱi^X_X)2. Use it with the properties of k_i, that we already have proved, to derive the variance of ₁. (Hint: in linear regression framework we assume V ar(_i) = ² and Cov(_i,_j) = 0 when i 6= j)

Problem 4

P(yiyi)2

Under simple linear regression model, the Mean Squared Error (MSE) is defined as _n2 where n is the number of observations. MSE is an unbiased estimator of ², where ² is the variance of _i. What is an unbiased estimator of the variance of ₁?

Problem 5

The square root of the variance of an estimator is the standard error (SE). You can derive the SE () from problem 4. According to theory,

1

1 1

t_n2

SE(₁)

where t_n2 represents a Students t distribution with n 2 degrees of freedom. Find an expression for 95%

confidence interval of ₁.

Problem 6

If ₁ = 2, SE(₁) = 0.02, and n = 50 calculate 95% confidence interval for ₁.

Problem 7

Based on the confidence interval on problem 6, perform the hypothesis test,

H₀ : ₁ = 0 Vs. H₁ : ₁ 6= 0

Problem 8

Use the simu_hw1.txt data and fit a multiple linear regression model with response as the response variable and pred1, pred2, and pred3 as predictors. Write down the equation of the fitted line (fitted model).