Congratulations! You made it through the first weeks content! The first week is the hardest because you have to learn about many new concepts. The first week is also the most math heavy. So it is all downhill from here. You deserve to kick your feed up and enjoy a glass of wine. But first, let's solve this weeks challenge.
The goal of the challenge is to deepen your understanding of how neural networks work. You will implement a multi class classifier in Excel.
You got a call from Italy. In a famous wine region, a disaster has happened. The labeling machine has mixed up the labels of three wine cultivars. Now there are thousands of bottles of which nobody knows who made them. Your first offer to distinguish the three wine makers by taste and then spend a couple of months drinking wine and labeling bottles as been refused. Instead, you are to build a classifier which recognizes the wine maker from 13 attributes of the wine.
The wine makers had 178 bottles left in their cellars. So we will use these bottles as our training data.
Go to the 'Week 1 Challenge' Excel file. In the first sheet you find the data. The original data contains the 13 measurements for all 178 bottles. Because this is a multi class problem, the output $y$ has already been converted to one hot matrix. To make training easier, the data has also been normalized already. You will learn about normalization next week, but the basic goal is to ensure that all features of the data have the same mean and standard deviation. This makes it easier to deal with the data. The normalized data and one hot encodings have been copied over to the logistic regression and 2 Layer Wine net sheet.
Your first task it to implement logistic regression in Excel. To make the sheet easier to handle, Forward pass, Backward pass and weight update are ordered horizontally, not vertically. So scroll to the right to see the full sheet. The network has an input size of 13 and an output of 3. It uses a softmax activation for the output layer. To update parameters, copy over the New W1 and New b1 over into W1 and b1on the left of the sheet. Make sure to use Paste values only when you copy over the weights! Try to experiment with the learning rate while you train and get the loss as low as possible.
In the next sheet, you will implement a 2 layer neural network. The hidden unit has a size of 5. The activation function of the hidden layer is tanh, the activation function for the output layer is softmax again. You can train this network the same way as the logistic regression network. Note that you have to copy over two sets of weights and biases this time.
The excel function for matrix multiplication is called MMULT. To multiply two matrices, select the output area where you want the output to be, enter the formula and hit CONTROL + SHIFT + ENTER. See this youtube tutorial if you have trouble.
Sometimes you might encounter a warning 'You cannot change part of an array.' In this case, either hit CONTROL + SHIFT + ENTER to apply the formula for the entire area or exit with ESC.
To transpose a matrix in excel you can use the TRANSPOSE function. Not that if you use transpose you always need to use CONTROL + SHIFT + ENTER, otherwise it will not do anything.
You can calculate $e^x$ using the EXP function. To calculate the exponent of multiple values element wise, like you have to do for softmax, you enter EXP and then the range of cells. For this you will have to hit CONTROL + SHIFT + ENTER again.
You will have to enter the formula for softmax element wise. That is, you enter the formula for one cell and can then expand it for the other cells. Note that softmax needs all cells of the example. So you need to make the reference to those fixed. Say you have a sample with 3 values you want to compute softmax for.
| 0 | A | B | C |
|---|---|---|---|
| 1 | 0.3 | 0.4 | 0.1 |
| $A_1$ 1 | $A_1$ 2 | $A_1$ 3 |
|---|---|---|
| EXP(A1)/SUM(EXP(\$A1:\$C1)) | EXP(B1)/SUM(EXP(\$A1:\$C1)) | EXP(C1)/SUM(EXP(\$A1:\$C1)) |
You can create random numbers in Excel using the RAND function. Note however, that the rand function create a new random number every time excel refreshes. So to create random numbers once and then freeze them, use RAND for all cells first, then copy it and paste the values using paste values onlyon the same cells.
This weeks challenge is not a competition for the most accurate prediction. You pass, if your team can submit a full implementation of both the logistic regression and 2 layer net. If you submit a 3 layer network, too, you will be invited for a winners interview and might feature in the global version of this course.