Deep Learning http://www.deeplearningbook.org
Neural Networks and Deep Learning http://neuralnetworksanddeeplearning.com/index.html
So you ask your friends who have bought houses in that same neighborhoods, and you end up with three data points:
| Area (sq ft) (x) | Price (y) |
|---|---|
| 2,104 | 399,900 |
| 1,600 | 329,900 |
| 2,400 | 369,000 |
A simple predictive model (“regression model”)
Model Evaluation
Loss Function (also, cost function)
Gradient Descent
In [128]:
def softmax(s):
return np.exp(s) / np.sum(np.exp(s), axis=0)
softmax([1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0])
Out[128]:
That is, prior to applying softmax, some vector components could be negative, or greater than one; and might not sum to 1; but after applying softmax, each component will be in the interval (0,1), and the components will add up to 1, so that they can be interpreted as probabilities.
Furthermore, the larger input components will correspond to larger probabilities.
Softmax is often used in neural networks, to map the non-normalized output of a network to a probability distribution over predicted output classes.
In [12]:
import numpy as np
def sigmoid(x):
return 1/(1 + np.exp(-x))
sigmoid(0.5)
Out[12]:
In [13]:
# Naive scalar relu implementation.
# In the real world, most calculations are done on vectors
def relu(x):
if x < 0:
return 0
else:
return x
relu(0.5)
Out[13]:
In [1]:
%matplotlib inline
import matplotlib.pyplot as plt
import torch
from torch import nn, optim
from torch.autograd import Variable
import numpy as np
In [2]:
x_train = np.array([[2104],[1600],[2400]], dtype=np.float32)
y_train = np.array([[399.900], [329.900], [369.000]], dtype=np.float32)
# x_train = np.array([[3.3], [4.4], [5.5], [6.71], [6.93], [4.168],
# [9.779], [6.182], [7.59], [2.167], [7.042],
# [10.791], [5.313], [7.997], [3.1]], dtype=np.float32)
# y_train = np.array([[1.7], [2.76], [2.09], [3.19], [1.694], [1.573],
# [3.366], [2.596], [2.53], [1.221], [2.827],
# [3.465], [1.65], [2.904], [1.3]], dtype=np.float32)
plt.plot(x_train, y_train, 'r.')
plt.show()
In [3]:
x_train = torch.from_numpy(x_train)
y_train = torch.from_numpy(y_train)
nn.Linear
Applies a linear transformation to the incoming data: $y = xA^T + b$
In [4]:
# Linear Regression Model
class LinearRegression(nn.Module):
def __init__(self):
super(LinearRegression, self).__init__()
self.linear = nn.Linear(1, 1) # input and output is 1 dimension
def forward(self, x):
out = self.linear(x)
return out
model = LinearRegression()
In [5]:
# Define Loss and Optimizatioin function
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=1e-9)#1e-4)
In [6]:
num_epochs = 1000
for epoch in range(num_epochs):
inputs = Variable(x_train)
target = Variable(y_train)
# forward
out = model(inputs)
loss = criterion(out, target)
# backward
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch+1) % 50 == 0:
print('Epoch[{}/{}], loss: {:.6f}'
.format(epoch+1, num_epochs, loss.data.item()))
where :`N` is the batch size.
In [109]:
model.eval()
Out[109]:
In [7]:
predict = model(Variable(x_train))
predict = predict.data.numpy()
plt.plot(x_train.numpy(), y_train.numpy(), 'ro', label='Original data')
plt.plot(x_train.numpy(), predict, 'b-s', label='Fitting Line')
plt.xlabel('X', fontsize= 20)
plt.ylabel('y', fontsize= 20)
plt.legend()
plt.show()
使用pytorch建立卷积神经网络并处理MNIST数据。 https://computational-communication.com/pytorch-mnist/