[SOLVED] COMP3670 / COMP6670 Introduction to Machine Learning Semester 2 2025 Prolog

COMP3670 / COMP6670: Introduction to Machine Learning

Semester 2, 2025

Tutorial Week 2

In the lecture slides, the following table of derivatives of common functions are given.

The following useful rules are given, assuming g, h are differentiable functions.

• If f(x) = g(x) + h(x), then f'(x) = g'(x) + h'(x).

• If f(x) = g(x) − h(x), then f'(x) = g'(x) − h'(x).

• If f(x) = c · g(x) where c is a constant, then f'(x) = c · g'(x).

• (product rule) If f(x) = g(x) · h(x), then

f'(x) = g(x) · h'(x) + g'(x) · h(x).

• (quotient rule) If f(x) = h(x)/g(x), when h(x) ≠ 0, we have:

• (chain rule) If f(x) = g(h(x)), then

f'(x) = g'(h(x)) · h'(x).

Question 1      Making sense of parameters of a hypothesis class

In a supervised learning task, each datapoint represents a second-hand tablet. The features are the phone’s capacity (x1) and age (x2), while the label is its price. A linear model is used:

y = θ0 + θ1×1 + θ2×2 .

What signs do you expect for θ0, θ1 and θ2? Justify your answer.

Question 2      Formal definition of derivative

Compute the derivative of f : R → R, f(x) = x3 by using the formal limit definition of the derivative.

Question 3      Differentiation using table and rules

By using the table and rules provided at the beginning of this document, compute the derivatives of the following functions:

(a) f(x) = e2x

(b) f(x) = (2x + 1)e2x

(c) f(x) = 1+x3/x2

Question 4      Local minima, local maxima, or neither?

Let f(x) = 2×3 − 21×2 + 60x + 4.

(a) Compute f'(x), and verify that f'(5) = 0.

(b) Factorize f'(x).

(c) Without plotting the graph of f, determine whether x = 5 is a local minima of f, a local maxima of f, or neither.

Question 5      Computing the gradient of a multi-dimensional function

Let f(x1, x2, x3) = (x1)2e2x2
lnx3. Compute ∇f(x1, x2, x3).

Question 6      Gradient descent and step sizes

In the lecture, we gave an example of gradient descent on the function f(x) = x2.

(a) For any step size λ, write down the explicit gradient descent update rule.

(b) Using part (a), explain what will happen when

(i) 0 < λ < 1

(ii) λ = 1

(iii) λ > 1