如何在Django Model中创建并保存Tweepy抓取的数量不定的Twitter媒体URL?
解决方案
你当前遇到的核心问题有两个:
- 原Tweets模型的单个
url字段无法存储多个媒体链接 - 现有代码中如果推文没有媒体,
url变量未定义会直接触发报错
以下提供两种可选实现方案:
方案一:单表存储(适合简单场景)
不需要额外建表,直接用JSON类型字段存储媒体URL列表即可。
步骤1:修改Tweets模型
from django.db import models class Tweets(models.Model): tweet_id = models.BigIntegerField(unique=True) handle = models.CharField(max_length=100) location = models.CharField(max_length=200, blank=True) description = models.TextField(blank=True) date = models.DateTimeField() content = models.TextField() # 替换原来的url字段,默认存储空列表 media_urls = models.JSONField(default=list)
步骤2:修改业务代码
user = api.get_user(screen_name=team) timeline = tweepy.Cursor( api.user_timeline, id=team, exclude_replies=True, include_rts=False).items(20) handle = user['screen_name'] location = user['location'] description = user['description'] for tweet in timeline: tweet_id = tweet['id'] date = tweet['created_at'] content = tweet['text'] # 初始化媒体URL列表,无媒体时默认为空 media_urls = [] if 'media' in tweet.entities: for media in tweet.extended_entities.media: media_urls.append(media['media_url']) Tweets.objects.update_or_create( tweet_id=tweet_id, defaults={ 'handle': handle, 'location': location, 'description': description, 'date': date, 'content': content, 'media_urls': media_urls } )
方案二:关联表存储(符合数据库范式,适合复杂场景)
如果后续需要对单个媒体资源做单独查询、统计等操作,建议拆分出单独的媒体表。
步骤1:修改模型
from django.db import models class Tweets(models.Model): tweet_id = models.BigIntegerField(unique=True) handle = models.CharField(max_length=100) location = models.CharField(max_length=200, blank=True) description = models.TextField(blank=True) date = models.DateTimeField() content = models.TextField() # 新增媒体表,和推文表一对多关联 class TweetMedia(models.Model): tweet = models.ForeignKey(Tweets, on_delete=models.CASCADE, related_name='medias') media_url = models.URLField(max_length=500)
步骤2:修改业务代码
user = api.get_user(screen_name=team) timeline = tweepy.Cursor( api.user_timeline, id=team, exclude_replies=True, include_rts=False).items(20) handle = user['screen_name'] location = user['location'] description = user['description'] for tweet in timeline: tweet_id = tweet['id'] date = tweet['created_at'] content = tweet['text'] # 先创建/更新推文基础信息 tweet_obj, created = Tweets.objects.update_or_create( tweet_id=tweet_id, defaults={ 'handle': handle, 'location': location, 'description': description, 'date': date, 'content': content } ) # 先清空该推文原有媒体记录,避免重复 tweet_obj.medias.all().delete() # 批量写入新的媒体记录 media_list = [] if 'media' in tweet.entities: for media in tweet.extended_entities.media: media_list.append(TweetMedia(tweet=tweet_obj, media_url=media['media_url'])) if media_list: TweetMedia.objects.bulk_create(media_list)
内容的提问来源于stack exchange,提问作者Ryan Thomas
相关产品推荐
相关产品推荐

