RuntimeError: mat1 and mat2 shapes cannot be multiplied (5760x6 and 128x4)

created at 09-08-2022 views: 10

problem

When using the pytorch framework to define a subclassed network structure, this problem of mismatching the shapes of mat1 and mat2 may sometimes occur. As follows, a 7-layer cnn network is defined:

class CNN(nn.Module):
    def __init__(self):
        super(CNN,self).__init__()
        self.conv1 = nn.Sequential(
            nn.Conv2d(
                in_channels=1,
                out_channels=16,
                kernel_size=3,
                stride=1,
                padding=1
            ),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2),
        )
        self.conv2 = nn.Sequential(
            nn.Conv2d(16,32,3,1,1),
            nn.ReLU(),
            nn.MaxPool2d(2),
        )
        self.conv3 = nn.Sequential(
            nn.Conv2d(32,64,2,1,1),
            nn.ReLU(),
            nn.MaxPool2d(2),
        )
        self.out = nn.Linear(128,4)
    def forward(self,x):
        x = self.conv1(x)
        x = self.conv2(x)
        output = self.out(x)
        return output

The following error will appear:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (5760x6 and 128x4)

reason & solution

This problem stems from the fact that the shape of the output of the last pooling layer defined is not the same as the shape of the input of the fully connected layer. We know by printing the output shape of the pooling layer in the forward pass function:

def forward(self,x):
    x = self.conv1(x)
    x = self.conv2(x)
    print(x.shape)
    output = self.out(x)
    return output


>> torch.Size([30, 32, 6, 6])  

([30, 32, 6, 6]) where 30 is the set batch_size, the latter three-dimensional is its real shape, and the input of the fully connected layer is a one-dimensional feature, so you need to add a flatten layer performs a flattening operation. After flattening as follows:

torch.Size([30, 1152])

In view of the characteristics of the pytorch framework, it is necessary to add another fully connected layer to connect the flattening layer and the last fully connected layer. The input shape is 1152 and the output is 128. (i.e. encapsulate one more layer in the above code conv3 and out):

    def __init__(self):
        super(CNN,self).__init__()
      ...... 
         self.conv3 = nn.Sequential(
            nn.Conv2d(32,64,2,1,1),
            nn.ReLU(),    
            nn.MaxPool2d(2),
        )
        self.dense = nn.Sequential(
            nn.Flatten(),
            nn.Linear(1152,128),
            nn.Linear(128,4),
        )

    def forward(self,x):
        x = self.conv1(x)
        x = self.conv2(x)
        output = self.out(x)
        return output

The code no longer reports an error, and the training network is successful.

created at:09-08-2022
edited at: 09-08-2022: