COMP3670 / COMP6670: Introduction to Machine Learning
Semester 2, 2025
Tutorial Week 2
In the lecture slides, the following table of derivatives of common functions are given.
The following useful rules are given, assuming g, h are differentiable functions.
• If f(x) = g(x) + h(x), then f'(x) = g'(x) + h'(x).
• If f(x) = g(x) − h(x), then f'(x) = g'(x) − h'(x).
• If f(x) = c · g(x) where c is a constant, then f'(x) = c · g'(x).
• (product rule) If f(x) = g(x) · h(x), then
f'(x) = g(x) · h'(x) + g'(x) · h(x).
• (quotient rule) If f(x) = h(x)/g(x), when h(x) ≠ 0, we have:
• (chain rule) If f(x) = g(h(x)), then
f'(x) = g'(h(x)) · h'(x).
Question 1 Making sense of parameters of a hypothesis class
In a supervised learning task, each datapoint represents a second-hand tablet. The features are the phone’s capacity (x1) and age (x2), while the label is its price. A linear model is used:
y = θ0 + θ1x1 + θ2x2 .
What signs do you expect for θ0, θ1 and θ2? Justify your answer.
Question 2 Formal definition of derivative
Compute the derivative of f : R → R, f(x) = x3 by using the formal limit definition of the derivative.
Question 3 Differentiation using table and rules
By using the table and rules provided at the beginning of this document, compute the derivatives of the following functions:
(a) f(x) = e2x
(b) f(x) = (2x + 1)e2x
(c) f(x) = 1+x3/x2
Question 4 Local minima, local maxima, or neither?
Let f(x) = 2x3 − 21x2 + 60x + 4.
(a) Compute f'(x), and verify that f'(5) = 0.
(b) Factorize f'(x).
(c) Without plotting the graph of f, determine whether x = 5 is a local minima of f, a local maxima of f, or neither.
Question 5 Computing the gradient of a multi-dimensional function
Let f(x1, x2, x3) = (x1)2e2x2
lnx3. Compute ∇f(x1, x2, x3).
Question 6 Gradient descent and step sizes
In the lecture, we gave an example of gradient descent on the function f(x) = x2.
(a) For any step size λ, write down the explicit gradient descent update rule.
(b) Using part (a), explain what will happen when
(i) 0 < λ < 1
(ii) λ = 1
(iii) λ > 1