使用Tweepy流式获取推特数据时遇401错误及好友列表超50用户报错问题

Hey there, let’s break down why you’re hitting that Tweepy Error 401 only when dealing with users who have over 50 friends, and fix it step by step.

Error 401 (Unauthorized) in this scenario almost always ties to either API permission gaps or missing pagination handling for large friend lists. Twitter’s API has strict limits on how much data you can fetch in one go, and when you exceed implicit limits without proper handling, it can trigger an auth-related error even if your tokens look correct.

1. Verify Your API Token Permissions #

First, double-check your credentials in the Twitter Developer Portal:

• Make sure your access token has the read:follower permission enabled (for v1.1 APIs). If you created the token before enabling this permission, regenerate a new access token/secret pair.
• Confirm your consumer key/secret and access token/secret are all linked to the same project, and none have expired.

2. Add Proper Pagination Handling #

Twitter’s v1.1 friends/list endpoint only returns up to 200 users per request by default, and it won’t automatically fetch all friends for users with large lists. When you try to pull more than the default limit without pagination, it can trigger auth errors. Use Tweepy’s Cursor to handle this seamlessly:

# Replace your friend-fetching code with this
for friend in tweepy.Cursor(api.get_friends, count=200).items():
    # Process each friend here
    print(friend.screen_name)

The Cursor will automatically handle the next_cursor parameter to fetch all pages of friends until there are none left.

3. Enable Rate Limiting Handling #

Twitter’s API enforces rate limits (15 requests every 15 minutes for friends/list in free tiers). If you hit these limits without waiting, it can manifest as a 401 error. Initialize your Tweepy API with rate limit handling:

api = tweepy.API(
    auth,
    wait_on_rate_limit=True,  # Automatically wait when rate limited
    wait_on_rate_limit_notify=True  # Print notifications when waiting
)

4. Fix the Incomplete Non-BMP Character Mapping #

Your code cuts off at non_bmp_map = dict.fromkeys(r... — this is used to handle emojis/non-standard characters in Twitter data, which can cause encoding errors that indirectly trigger auth issues. Complete it like this:

non_bmp_map = dict.fromkeys(range(0x10000, sys.maxunicode + 1), 0xfffd)
# Use it when processing text, e.g.:
clean_text = tweet.text.translate(non_bmp_map)

import re
import tweepy
import sys
import time

# Fill in your credentials
consumer_key = 'YOUR_CONSUMER_KEY'
consumer_secret = 'YOUR_CONSUMER_SECRET'
access_token = 'YOUR_ACCESS_TOKEN'
access_token_secret = 'YOUR_ACCESS_TOKEN_SECRET'

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

# Initialize API with rate limit handling
api = tweepy.API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)

# Complete non-BMP mapping for emoji/encoding fixes
non_bmp_map = dict.fromkeys(range(0x10000, sys.maxunicode + 1), 0xfffd)

# Example: Fetch and process all friends for a user
def fetch_friends(screen_name):
    try:
        friends = []
        for friend in tweepy.Cursor(api.get_friends, screen_name=screen_name, count=200).items():
            friends.append(friend.screen_name)
            # Optional: Process friend data here
            clean_name = friend.name.translate(non_bmp_map)
            print(f"Friend: {clean_name} (@{friend.screen_name})")
        return friends
    except tweepy.TweepError as e:
        print(f"Error fetching friends: {e}")
        return []

# Test with a user
fetch_friends("target_user_screen_name")

• If you still get 401 errors, try regenerating your access token/secret in the Developer Portal — sometimes old tokens can have permission mismatches.
• Make sure your Twitter account (the one linked to the access token) doesn’t have any restrictions that would block API access.

内容的提问来源于stack exchange，提问作者ashely vindsor