use nn.Linear() The neural network layer is constructed more simply

The multi category problem before

It's handwritten on each layer

In fact, we can use it nn.Linear, You don't have to write

The first parameter here is in, The second parameter is out, It's in line with our normal thinking habits

 

With the activation function

 

 

If we want to implement a network structure of our own

No need to implement backward(),nn.Module It will be provided automatically ,pytorch Of autograd The package will automatically implement the function of backward derivation
class MLP(nn.Module): def __init__(self): super(MLP,self).__init__()
self.model = nn.Sequential( nn.Linear(784,200), nn.ReLU(inplace=True),
nn.Linear(200,200), nn.ReLU(inplace=True), nn.Linear(200,10),
nn.ReLU(inplace=True), ) def forward(self,x): x = self.model(x) return x
In the training data section
net = MLP() optimizer = optim.SGD(net.parameters(), lr=learning_rate) criteon
= nn.CrossEntropyLoss() for epoch in range(epochs): for batch_idx, (data,
target) in enumerate(train_loader): data = data.view(-1, 28*28) logits =
net(data) loss = criteon(logits, target) optimizer.zero_grad() loss.backward()
# print(w1.grad.norm(), w2.grad.norm()) optimizer.step()
We were in Deep learning and neural network ( Four ) This is what the optimizer wrote in the actual combat

It's a w,b Write a parameter of list

Now our net Inherited from nn.module, Will be able to w,b The parameter of is added automatically nn.parameters inside

 

Rewrite the previous multi classification problem
import torch import torch.nn as nn import torch.nn.functional as F import
torch.optim as optim from torchvision import datasets, transforms
batch_size=200 learning_rate=0.01 epochs=10 train_loader =
torch.utils.data.DataLoader( datasets.MNIST('dataset', train=True,
download=True, transform=transforms.Compose([ transforms.ToTensor(),
transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=batch_size,
shuffle=True) test_loader = torch.utils.data.DataLoader(
datasets.MNIST('dataset', train=False, transform=transforms.Compose([
transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])),
batch_size=batch_size, shuffle=True) class MLP(nn.Module): def __init__(self):
super(MLP, self).__init__() self.model = nn.Sequential( nn.Linear(784, 200),
nn.ReLU(inplace=True), nn.Linear(200, 200), nn.ReLU(inplace=True),
nn.Linear(200, 10), nn.ReLU(inplace=True), ) def forward(self, x): x =
self.model(x) return x net = MLP() optimizer = optim.SGD(net.parameters(),
lr=learning_rate) criteon = nn.CrossEntropyLoss() for epoch in range(epochs):
for batch_idx, (data, target) in enumerate(train_loader): data = data.view(-1,
28*28) logits = net(data) loss = criteon(logits, target) optimizer.zero_grad()
loss.backward() # print(w1.grad.norm(), w2.grad.norm()) optimizer.step() if
batch_idx % 100 == 0: print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss:
{:.6f}'.format( epoch, batch_idx * len(data), len(train_loader.dataset), 100. *
batch_idx / len(train_loader), loss.item())) test_loss = 0 correct = 0 for
data, target in test_loader: data = data.view(-1, 28 * 28) logits = net(data)
test_loss += criteon(logits, target).item() pred = logits.data.max(1)[1]
correct += pred.eq(target.data).sum() test_loss /= len(test_loader.dataset)
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
test_loss, correct, len(test_loader.dataset), 100. * correct /
len(test_loader.dataset)))

train The effect is basically the same

 

In the previous method, we will have initialization problems , Not here

Here we are w and b The parameter of has been returned to nn.Linear() Managed , Not exposed to us , We can't initialize it directly

Then, when we use its interface, it has its own initialization method , We don't have to worry about it

 

Layers and structure of fully connected networks

Fully connected networks are also called linear layers

Calculate the number of layers , The input layer is not computed , Output layer to be calculated

So this network has 4 layer

And if you ask how many hidden layers there are 3 Yes

For a certain floor , We generally mean that the weight of this layer and the output of this layer are added together to call a layer

For the second layer, it means this

 

 

This network is used to process very simple data sets ——MNIST Of the image dataset

MNIST Each image in the dataset has 28*28 Pixels , So the input is 28*28;MNIST There are altogether 10 class , So the output is 10 Points

That is, the input is 28*28 Matrix of , In order to facilitate the processing of full connection layer , We'll make it even 784 Vector of layers , And then the intermediate nodes are all 256 Points

Let's calculate for such a network , How many parameters are needed

The parameter quantity of neural network is how many lines there are

784*256+256*256+256*256+256*10 = 390K

So the parameter is 390K

And then each parameter uses one 4 A floating-point number of bytes , therefore 390k*4 = 1.6MB

So we need to 1.6M Memory or video memory ( If you use GPU The words of )

 

That number now looks small
however MNIST It's in 80 It was born in the age , At that time, it was still 386 When
At that time, the processors were probably only tens to hundreds KB
For such a simple network , It doesn't fit into memory
 

 

 

Technology
©2020 ioDraw All rights reserved
error: (-215:Assertion failed) Solution python Simple record of network programming mac solve Enter passphrase for key Enter the password each time Huawei Hongmeng system learning notes 9- Ecological construction of developers Apple iPhone 12 Price leakage : The official highest price is close to 1 Ten thousand yuan Docker Import of containers and mirrors , export Cross analysis of data analysis Notes on core principles of reverse engineering ( One )——Hello World-1Python Student information management system ( Lite ) Ant old employees leave early and miss millions of wealth , Some people say they don't regret it