R网页爬取技术问询：计算梅森男篮胜负场次场均得分

Hey there! Let's walk through how to build a solution to scrape that schedule page and compute the average points George Mason scores in wins vs. losses. Since you already have your CSS selector from SelectorGadget, we can focus on turning that into working code with Python's go-to scraping tools.

Step 1: Install Required Libraries #

First, make sure you have the necessary packages installed. Open your terminal and run:

pip install requests beautifulsoup4

Step 2: Full Scraping & Calculation Code #

Here's a complete, commented script that will fetch the page, extract the scores, and crunch the numbers. Just replace YOUR_CSS_SELECTOR_HERE with the selector you got from SelectorGadget:

import requests
from bs4 import BeautifulSoup

# Target schedule URL
url = "http://gomason.com/schedule.aspx?path=mbball"

# Optional: Add a user-agent header to avoid being blocked (mimics a browser)
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
}

try:
    # Fetch the page content
    response = requests.get(url, headers=headers)
    response.raise_for_status()  # Throw an error if the request fails (e.g., 404, 403)

    # Parse the HTML content with BeautifulSoup
    soup = BeautifulSoup(response.text, "html.parser")

    # Replace this with your actual CSS selector from SelectorGadget
    score_selector = "YOUR_CSS_SELECTOR_HERE"
    score_elements = soup.select(score_selector)

    # Initialize counters for our stats
    total_win_points = 0
    win_count = 0
    total_loss_points = 0
    loss_count = 0

    # Loop through each extracted score element
    for element in score_elements:
        score_text = element.get_text(strip=True)
        
        # Skip any entries that don't follow the "X-Y" format
        if "-" not in score_text:
            print(f"Skipping non-score entry: {score_text}")
            continue
        
        # Split the score into Mason's points and the opponent's points
        mason_score_str, opp_score_str = score_text.split("-", 1)
        
        # Convert strings to integers (handle cases where scores might have extra text)
        try:
            mason_score = int(mason_score_str)
            opp_score = int(opp_score_str)
        except ValueError:
            print(f"Skipping invalid score format: {score_text}")
            continue
        
        # Update stats based on win/loss
        if mason_score > opp_score:
            total_win_points += mason_score
            win_count += 1
        elif mason_score < opp_score:
            total_loss_points += mason_score
            loss_count += 1
        # Ignore ties (rare in college basketball, but just in case)

    # Calculate averages (avoid division by zero if there are no wins/losses)
    avg_win_points = round(total_win_points / win_count, 1) if win_count > 0 else 0
    avg_loss_points = round(total_loss_points / loss_count, 1) if loss_count > 0 else 0

    # Print the final stats
    print("\nGeorge Mason Men's Basketball Season Stats:")
    print(f"Winning Games: {win_count} | Average Points Scored: {avg_win_points}")
    print(f"Losing Games: {loss_count} | Average Points Scored: {avg_loss_points}")

except requests.exceptions.RequestException as e:
    print(f"Error loading the schedule page: {e}")
except Exception as e:
    print(f"Unexpected error during processing: {e}")

Key Notes & Troubleshooting Tips #

• Selector Validation: Double-check your CSS selector by testing it in your browser's DevTools (right-click → Inspect → Console, run document.querySelectorAll("YOUR_SELECTOR") to see if it only picks up the score elements). Sometimes SelectorGadget might grab extra elements, so you may need to refine it (e.g., add a parent class like .sidearm-schedule-game to narrow it down).
• Anti-Scraping Measures: If you get a 403 Forbidden error, the site might be blocking automated requests. Adding the User-Agent header (as shown in the code) usually fixes this. For more persistent blocks, you could add a small delay between requests with time.sleep(1).
• Page Structure Changes: College sports sites often update their layouts during the offseason. If your script stops working later, re-run SelectorGadget to get a new selector.

内容的提问来源于stack exchange，提问作者Reeza