如何编写可复用Python自定义函数捕获BeautifulSoup元素缺失错误？

Hey there! I feel your pain—repeating those try-except blocks every time you want to extract text from a potentially missing element is such a drag. Let’s build a clean, reusable solution that eliminates all that boilerplate code.

First, a quick note on a common pitfall: if you try to pass soup.div.text directly to a function, Python will evaluate that expression before the function runs, which means the AttributeError will throw before your error catcher even gets a chance to work. Instead, we need to pass a delayed-execution operation (like a lambda) so the function can handle the error internally.

Solution: A safe_get Function #

Here’s a simple, flexible function that wraps the risky operation and returns a default value if anything goes wrong:

def safe_get(func, default='N/A'):
    try:
        # Execute the passed-in operation (e.g., accessing soup element text)
        return func()
    except (AttributeError, TypeError):
        # Catch cases where:
        # - The element doesn't exist (soup.div is None, so .text throws AttributeError)
        # - The element exists but doesn't have a .text attribute (rare in BeautifulSoup, but safe to handle)
        return default

How to Use It #

Every time you need to extract text from a potentially missing element, wrap the extraction logic in a lambda and pass it to safe_get:

from bs4 import BeautifulSoup

# Example soup object
html = "<html><h1>Hello World</h1></html>"
soup = BeautifulSoup(html, 'html.parser')

# Get existing element text
title = safe_get(lambda: soup.h1.text)
# Output: "Hello World"

# Get missing element text (returns default)
missing_div_text = safe_get(lambda: soup.div.text)
# Output: "N/A"

# Custom default value
author = safe_get(lambda: soup.find('span', class_='author').text, default='Unknown Author')
# Output: "Unknown Author"

Bonus: Decorator Version (For Repeated Use Cases) #

If you find yourself extracting the same element type multiple times, you can use a decorator version to make it even cleaner:

def safe_extractor(default='N/A'):
    def decorator(func):
        def wrapper(*args, **kwargs):
            try:
                return func(*args, **kwargs)
            except (AttributeError, TypeError):
                return default
        return wrapper
    return decorator

# Define reusable extractor functions
@safe_extractor(default='Unknown Title')
def get_page_title(soup):
    return soup.h1.text

@safe_extractor(default='No Price Listed')
def get_product_price(soup):
    return soup.find('div', class_='price').text

# Use them like regular functions
title = get_page_title(soup)
price = get_product_price(soup)

This keeps your code DRY (Don’t Repeat Yourself) and makes it easy to update default values or error handling logic in one place.

内容的提问来源于stack exchange，提问作者Dmitry Rastorguev