Facebook爬虫开发问题：如何将指定帖子的评论及评论日期导出至CSV文件

Hey there! Let's work through your Facebook crawler issues step by step. I'll use the facebook-scraper library here—it's a handy tool for scraping public Facebook posts and comments without needing the official Graph API (just remember to always follow Facebook's Terms of Service when scraping content).

Step 1: Install the Required Library #

First, install the facebook-scraper package using pip:

pip install facebook-scraper

We'll also use Python's built-in csv module for handling the CSV export, so no extra installation is needed for that.

Step 2: Full Python Code Implementation #

Here's a complete script that scrapes comments from a specified Facebook post, includes the comment date, and exports everything to a CSV file:

from facebook_scraper import get_posts
import csv
from datetime import datetime

def scrape_facebook_comments(post_id, output_csv):
    # Define the fields we want to save in CSV
    fields = ['comment_id', 'comment_text', 'commenter_name', 'comment_date']
    
    # Open CSV file for writing (utf-8 encoding handles special characters)
    with open(output_csv, 'w', newline='', encoding='utf-8') as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=fields)
        writer.writeheader()  # Write the header row
        
        # Scrape the post and its comments
        for post in get_posts(
            post_urls=[post_id],
            comments=True,  # Enable comment scraping
            extra_info=True,  # Fetch additional details like comment dates
            options={"comments_limit": 1000}  # Adjust the number of comments to scrape as needed
        ):
            # Iterate through each full comment object
            for comment in post['comments_full']:
                # Convert the datetime object to a readable string format
                formatted_date = comment['time'].strftime('%Y-%m-%d %H:%M:%S')
                
                # Structure the comment data to match our CSV fields
                comment_data = {
                    'comment_id': comment['comment_id'],
                    'comment_text': comment['text'],
                    'commenter_name': comment['commenter_name'],
                    'comment_date': formatted_date
                }
                
                # Write the comment row to CSV
                writer.writerow(comment_data)
    
    print(f"Successfully exported comments to {output_csv}!")

# Example usage
if __name__ == "__main__":
    # Replace with your target Facebook post ID
    # Get the post ID from the post URL: e.g., in https://www.facebook.com/PageName/posts/123456789/, the ID is 123456789
    target_post_id = "123456789"
    output_file = "facebook_comments.csv"
    
    scrape_facebook_comments(target_post_id, output_file)

Key Details & Notes #

• Finding the Post ID: Navigate to the Facebook post, check the URL—you'll see a numeric string like 123456789 after /posts/ or /story.php?story_fbid=. That's your post ID.
• Handling Private/Restricted Posts: If the post isn't public, you'll need to pass your Facebook cookies to the scraper. Export cookies from your browser (using an extension like "Get Cookies.txt") and add the cookies parameter to get_posts():

  get_posts(post_urls=[post_id], comments=True, extra_info=True, cookies="path/to/your/cookies.txt")
  

• Customizing Date Format: The strftime('%Y-%m-%d %H:%M:%S') converts the datetime object to a standard string. Adjust the format (e.g., '%m/%d/%Y' for MM/DD/YYYY) based on your needs.
• Avoiding Blocks: Facebook may restrict your IP if you scrape too aggressively. Consider adding small delays between requests with time.sleep() and avoid scraping massive volumes of data in a short window.

内容的提问来源于stack exchange，提问作者Gxz