Python: how to continue tasks after capturing exceptions

created at 06-24-2021 views: 4

Recently written scraping scripts, some bad data will always occur in the acquired URL link, that is, after access, Requests will throw an abnormality, causing the crawler interruption. So I want to find the method to skip an abnormal URL to continue executing the program after capturing anomalies.  Methods as below: 

while True:
   try:
       r=requests.get(url,timeout=5)
       with open(path+'/'+name+".txt",'w') as myfile:
           myfile.write(r.content)
           myfile.close()
   except Exception as ex:
       template = "An exception of type {0} occurred. Arguments:\n{1!r}"
       message = template.format(type(ex).__name__, ex.args)
       print('\n'+message)
       break
   finally:
       break

First, one of the dead cycles, then capture exceptions through try ... except

After the capture is complete, then the loop is jumped through the Return or Break and proceeds. Pay attention to the last finally, meaning whether or not a abnormality is captured, the following code will be executed.

Please log in to leave a comment.