[SOLVED] EE5904/ME5404 Neural Networks: Homework #3

Homework3_2023

EE5904/ME5404 Neural Networks: Homework #3
Important note: the due date is 19/03/2023. Please submit the softcopy of your report to the submission folder in CANVAS. Late submission is not allowed unless it is well justified. Please supply the MATLAB code in your answer if computer experiment is involved.
Please note that the MATLAB toolboxes for RBFN and SOM are not well developed. Please write your own codes to implement RBFN and SOM instead of using the MATLAB toolbox.
Q1. Function Approximation with RBFN (10 Marks)
Consider using RBFN to approximate the following function:
, for ]1,1[x.
The training set is constructed by dividing the range [-1, 1] using a uniform step length 0.05, while the test set is constructed by dividing the range [-1, 1] using a uniform step length 0.01. Assume that the observed outputs in the training set are corrupted by random noise as follows.
where the random noise n(i) is Gaussian noise with zero mean and stand deviation of one, which can be generated by MATLAB command randn. Note that the test set is not corrupted by noises. Do the following computer experiments:
a). Use the exact interpolation method (as described on pages 16-25 in the slides of lecture five), and determine the weights of the RBFN. Assume the RBF is Gaussian function with standard deviation of 0.1. Evaluate the approximation performance of the resulting RBFN using the test set.
(3 Marks)
b). Follow the strategy of Fixed Centers Selected at Random (as described on page 37 in the slides of lecture five), randomly select 15 centers among the sampling points. Determine the weights of the RBFN. Evaluate the approximation performance of the resulting RBFN using the test set. Compare it to the result of part a).
(4 Marks)
c). Use the same centers and widths as those determined in part a), and apply the regularization method as described on pages 42-45 in the slides for lecture five. Vary the value of the regularization factor and study its effect on the performance of the RBFN.
(3 Marks)
2
Q2. Handwritten Digits Classification using RBFN (20 Marks)
In this task, you will build a handwritten digits classifier using RBFN. The training data is provided in MNIST_database.mat. Each binary image is of size 2828. There are 10 classes in MNIST_database.mat; please select two classes according to the last two different digits of your matric number (e.g. A0642311, choose classes 3 and 1; A1234567, choose classes 6 and 7) Make sure you have selected the correct 2 classes for both training and testing. There will be some mark deduction for wrong classes selected. Please state your handwritten digit classes for both training and testing.
In MATLAB, the following code can be used to load the training and testing data:
——————————————————————————————————-
load MNIST_database.mat;
% train_data training data, 784×1000 matrix
% train_classlabel the labels of the training data, 1×1000 vector
% test_data test data, 784×250 matrix
% train_classlabel the labels of the test data, 1×250 vector
——————————————————————————————————-
After loading the data, you may view them using the code below:
——————————————————————————————————-
tmp=reshape(train_data(:,column_no),28,28);
imshow(double(tmp));
——————————————————————————————————-
To select a few classes for training, you may refer to the following code:
——————————————————————————————————-
trainIdx = find(train_classlabel==0 | train_classlabel==1 | train_classlabel==2); % find the location of classes 0, 1, 2
Train_ClassLabel = train_classlabel(trainIdx);
Train_Data = train_data(:,trainIdx);
——————————————————————————————————-
Please use the following code to evaluate:
——————————————————————————————————-
TrAcc = zeros(1,1000);
TeAcc = zeros(1,1000);
thr = zeros(1,1000);
TrN = length(TrLabel);
TeN = length(TeLabel);
for i = 1:1000
t = (max(TrPred)-min(TrPred)) * (i-1)/1000 + min(TrPred);
thr(i) = t;
TrAcc(i) = (sum(TrLabel(TrPred<t)==0) + sum(TrLabel(TrPred>=t)==1)) / TrN;
TeAcc(i) = (sum(TeLabel(TePred<t)==0) + sum(TeLabel(TePred>=t)==1)) / TeN;
end
plot(thr,TrAcc,’.- ‘,thr,TeAcc,’^-‘);legend(‘tr’,’te’);
——————————————————————————————————-
3
TrPred and TePred are determined by TrPred(j)=(TrData(:,j))=0 and TePred(j)=(TeData(:,j))=0 where is the number of hidden neurons. TrData and TeData are the training and testing data selected based on your matric number. TrLabel and TeLabel are the ground-truth label information (Convert to {0,1} before use!).
You are required to complete the following tasks:
a) Use Exact Interpolation Method (as described in pages 16-25 of lecture five) and apply regularization (as described in pages 42-45 of lecture five). Assume the RBF is Gaussian function with standard deviation of 100. Firstly, determine the weights of RBFN without regularization and evaluate its performance; then vary the value of regularization factor and study its effect on the resulting RBFNs performance.
(6 Marks)
b) Follow the strategy of Fixed Centers Selected at Random (as described in page 37 of lecture five). Randomly select 100 centers among the training samples. Firstly, determine the weights of RBFN with widths fixed at an appropriate size and compare its performance to the result of a); then vary the value of width from 0.1 to 10000 and study its effect on the resulting RBFNs performance.
(8 Marks)
c) Try classical K-Mean Clustering (as described in pages 38-39 of lecture five) with 2 centers. Firstly, determine the weights of RBFN and evaluate its performance; then visualize the obtained centers and compare them to the mean of training images of each class. State your findings.
(6 Marks)
Q3. Self-Organizing Map (SOM) (20 Marks)
a) Write your own code to implement a SOM that maps a 1-dimensional output layer of 25 neurons to a heart curve. Display the trained weights of each output neuron as points in a 2D plane, and plot lines to connect every topological adjacent neurons (e.g. the 2nd neuron is connected to the 1st and 3rd neuron by lines). The training points sampled from the heart curve can be obtained and illustrated by the following code:
——————————————————————————————————-
t = linspace(-pi,pi,200);
trainX = [t.*sin(pi*sin(t)./t); 1-abs(t).*cos(pi*sin(t)./t)]; 2×200 matrix, column-wise points
plot(trainX(1,:),trainX(2,:),’+r’);
——————————————————————————————————-
(3 Marks)
b) Write your own code to implement a SOM that maps a 2-dimensional output layer of 25 (i.e. 55) neurons to a square. Display the trained weights of each output neuron as points in a 2D plane, and plot lines to connect every topological adjacent neurons
4
(e.g. neuron (2,2) is connected to neuron (1,2) (3,2) (2,1) (2,3) by lines). The training points sampled from the square can be obtained by the following code:
——————————————————————————————————-
trainX = rands(2,500); 2×500 matrix, column-wise points
plot(trainX(1,:),trainX(2,:),’+r’);
——————————————————————————————————-
(4 Marks)
c) Write your own code to implement a SOM that clusters and classifies handwritten digits. The training data is provided in MNIST_database.mat (as introduced in Q2). Please omit two classes according to the last two different digits of your matric number (e.g. A0642311, ignore classes 3 and 1; A1234567, ignore classes 6 and 7.) Make sure you have selected the correct 8 classes for both training and testing. There will be some mark deduction for wrong classes selected. Please state your handwritten digit classes for both training and testing.
After loading the data, complete the following tasks:
c-1) Print out corresponding conceptual/semantic map of the trained SOM (as described in page 24 of lecture six) and visualize the trained weights of each output neuron on a 1010 map (a simple way could be to reshape the weights of a neuron into a 2828 matrix, i.e. dimension of the inputs, and display it as an image). Make comments on them, if any.
(8 Marks)
c-2) Apply the trained SOM to classify the test images (in test_data). The classification can be done in the following fashion: input a test image to SOM, and find out the winner neuron; then label the test image with the winner neurons label (note: labels of all the output neurons have already been determined in c-1). Calculate the classification accuracy on the whole test set and discuss your findings.
(5 Marks)
The recommended values of design parameters are:
1. The size of the SOM is 125 for a), 55 for b), 1010 for c).
2. The total iteration number N is set to be 500 for a) & b), 1000 for c). Only the first (self-organizing) phase of learning is used in this experiment.
3. The learning rate () is set as:
()=0exp2,=0,1,2,
where 0 is the initial learning rate and is set to be 0.1, 2 is the time constant and is set to be N.
4. The time-varying neighborhood function is:
,()()=exp,22()2,=0,1,2,
5
where , is the distance between neuron j and winner i, () is the effective width and satisfies: ()=0exp1,=0,1,2,
where 0 is the initial effective width and is set according to the size of output layers lattice, 1 is the time constant and is chosen as =log (0).
Again, please feel free to experiment with other design parameters which may be different from the given ones.