如何在Python中实现优雅错误处理?以Selenium爬LinkedIn为例
问题:Selenium爬取LinkedIn的优雅错误处理方式
我正在使用Selenium爬取LinkedIn,这项任务稳定性极差,经常抛出异常。我希望找到一种优雅的错误处理方式,网上常见的try-catch写法过于繁琐。现有代码如下:
try: WebDriverWait(driver, 20).until(EC.element_to_be_clickable(job)) job_title = job.find_element(By.CLASS_NAME, "base-search-card__title").text company = job.find_element(By.CLASS_NAME, "base-search-card__subtitle").text location = job.find_element(By.CLASS_NAME, "job-search-card__location").text except : print("Boom Boom")
若任意一个find_element方法抛出异常,整个try块就会停止执行并进入except分支。我希望实现单个元素提取失败时仅返回空字符串,而不触发全局的except。我尝试过将每个提取逻辑封装成函数,示例如下:
def extract_job_title(job): try: return job.find_element(By.CLASS_NAME, "base-search-card__title").text except: return ""
然后通过job_title = extract_job_title(job)调用,但这种方式依然繁琐。我想要类似Swift中的简洁写法,例如:
let job_title = try? job.find_element(By.CLASS_NAME, "base-search-card__title").text ?? ""
请问Python中是否有类似的语法?如果没有,有没有其他更优雅的实现方式?
解决方案
Python没有Swift那种try?+空合运算符的原生语法,但可以通过以下几种简洁方式实现需求:
1. 通用提取函数
不用为每个元素单独写函数,封装一个通用的提取逻辑:
def extract_text(element, by, value, default=""): try: return element.find_element(by, value).text except Exception: return default
调用时只需传入定位参数,代码简洁直观:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable(job)) job_title = extract_text(job, By.CLASS_NAME, "base-search-card__title") company = extract_text(job, By.CLASS_NAME, "base-search-card__subtitle") location = extract_text(job, By.CLASS_NAME, "job-search-card__location")
2. 利用find_elements特性
find_elements找不到元素时返回空列表,不会抛出异常,结合三元表达式可以简化代码:
job_title = job.find_elements(By.CLASS_NAME, "base-search-card__title")[0].text if job.find_elements(By.CLASS_NAME, "base-search-card__title") else "" company = job.find_elements(By.CLASS_NAME, "base-search-card__subtitle")[0].text if job.find_elements(By.CLASS_NAME, "base-search-card__subtitle") else "" location = job.find_elements(By.CLASS_NAME, "job-search-card__location")[0].text if job.find_elements(By.CLASS_NAME, "job-search-card__location") else ""
嫌三元表达式长的话,可以封装成lambda:
get_text = lambda elem, by, val: elem.find_elements(by, val)[0].text if elem.find_elements(by, val) else "" job_title = get_text(job, By.CLASS_NAME, "base-search-card__title") company = get_text(job, By.CLASS_NAME, "base-search-card__subtitle")
3. 海象运算符优化(Python 3.8+)
用海象运算符避免重复调用find_elements,提升效率:
job_title = elems[0].text if (elems := job.find_elements(By.CLASS_NAME, "base-search-card__title")) else "" company = elems[0].text if (elems := job.find_elements(By.CLASS_NAME, "base-search-card__subtitle")) else "" location = elems[0].text if (elems := job.find_elements(By.CLASS_NAME, "job-search-card__location")) else ""
4. 装饰器实现通用捕获(进阶)
如果需要更灵活的默认值配置,可以写一个装饰器:
def catch_default(default=""): def decorator(func): def wrapper(*args, **kwargs): try: return func(*args, **kwargs) except Exception: return default return wrapper return decorator
使用方式:
# 定义提取函数 @catch_default() def get_job_title(job): return job.find_element(By.CLASS_NAME, "base-search-card__title").text job_title = get_job_title(job) # 或者直接包裹lambda company = catch_default()(lambda job: job.find_element(By.CLASS_NAME, "base-search-card__subtitle").text)(job)
内容的提问来源于stack exchange,提问作者user19231332
相关产品推荐
相关产品推荐

