RuntimeError: one of the variables needed for gradient computation has been modified by an inplace

created at 09-08-2022 views: 5

problem

The error encountered when adding your own ideas to the pytorch deep learning model. The error report shows that there is a problem with the backpropagation of the network, and a variable that needs to calculate the gradient is modified by the inplace operation (such as a+=b).

reason

After code inspection, I found that one of the steps is to store the sum of the image features image_features extracted in each batch as sum, and in the next batch, add the sum of sum.

There is a problem at this step. The original idea is to sum across batch, so that the features are less offset. But since sum will retain the gradient from the previous batch, this will cause the gradient of the previous batch to interfere with the gradient of this batch. Therefore, modify the code to only find the sum or average of image features within each batch.

The error is still due to network structure and calculation problems.

created at:09-08-2022
edited at: 09-08-2022: