caught runtimeerror in replica 0 on device 0

created at 07-03-2022 views: 60

Sometimes we get an error when running the python|deep learning bert code:

caught runtimeerror in replica 0 on device 0

After consulting a lot of solutions, the final one that works is the following:

solution

Change the data type dtype=next(self.parameters()).dtype to dtype=torch.float32 in the Bert model code file

like:

modeling.py file

extended_attention_mask = extended_attention_mask.to(dtype=next(self.parameters()).dtype) # fp16 compatibility

change to:

extended_attention_mask = extended_attention_mask.to(dtype=torch.float32) # fp16 compatibility

Sometimes you can modify the modeling.py file, sometimes you need to modify the modeling_bert.py, just try to debug it yourself! !

created at:07-03-2022
edited at: 07-03-2022: