如何为Python Craigslist爬虫添加指定回溯天数的筛选逻辑

云原生阿岚

2026-8-16

需求说明

我已编写好一个Python脚本，可无错误地爬取目标网页的发布日期与标题。现需为脚本添加逻辑，使其仅抓取days_to_go_back参数指定回溯天数内的内容，例如当参数设为2时，仅抓取10月6日及以后的内容。

原脚本

import requests
from bs4 import BeautifulSoup

link = 'https://atlanta.craigslist.org/search/lab'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}

days_to_go_back = 2

with requests.Session() as s:
    s.headers.update(headers)
    res = s.get(link)
    soup = BeautifulSoup(res.text,"lxml")
    for item in soup.select("[id^='search-results'] li[class*='result']"):
        date = ' '.join(item.select_one("time.result-date").text.split())
        title = item.select_one("h3.result-heading > a").get_text(strip=True)
        print(date,title)

当前输出

Oct 7 General labor crew member needed
Oct 7 General Labor Demo
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 Craftsman Wanted
Oct 7 Home Cleaners Wanted! Make over $145+ for 1bd, $165 for 2bd cleans.
Oct 7 Sign Company In Search of Installers and Fabricators
Oct 7 ***PILE DRIVERS-DRILLERS-TRAINEES***
Oct 7 Warehouse Package Handler
Oct 7 Workers Needed for Local Fair (Cash Paid Daily)
Oct 7 Warehouse Team Member - Multiple Shifts Available
Oct 7 Construction
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Amazon Warehouse Assistant - Immediate Hire
Oct 7 Amazon Picker - Overnight Shifts Available
Oct 7 Warehouse Team Member - With Sign-On Bonus!
Oct 7 Amazon Warehouse Assistant
Oct 7 Need a Crew of 4+
Oct 6 Helper and Construction
Oct 6 Truck Driver/El Camionero
Oct 6 Workers Needed for Local Fair (Cash Paid Daily)
Oct 6 Roofing Crews Needed! ($2,000 Sign On Bonus - Start Today!)
Oct 6 Available Immediately General Labor/Driver with Truck or Van Positions
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 6 $20-25/hr. SIMPLE VALET TRASH COLLECTOR-Camp Creek/East Point 30331
Oct 6 Asphalt workers labors
Oct 6 CDL-A Company Drivers - $5K SIGN-ON BONUS! 💰 🚚
Oct 6 EXPERIENCED HANDYMAN ***2 immediate openings
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR
Oct 5 Georgia Canvasser
Oct 5 💚 CLEANING JOBS AVAILABLE 💚  $22+/hr, Paid Daily 💚
Oct 5 Farm Help Animal Care
Oct 5 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 5 Maintenance/Custodial
Oct 5 Hiring House Keeper For Airbnb
Oct 5 Warehouse Material Handler: $15.92/hr,M-F/Full Time/Day Shift/Benefits
Oct 5 Laborers, Forman, and Drivers
Oct 5 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 5 CANVASS POSITION (MAKE MONEY THIS WEEK)
Oct 5 Hiring full Auto Detailer
Oct 5 Amazon Packer - Part-time Night Shifts Available
Oct 5 Warehouse Team Member Sun - Wed (06:30PM - 05:00AM)
Oct 5 Amazon Warehouse Assistant - Immediate Hire
Oct 5 EC3 Construction Is Now Hiring Framers
Oct 5 Detailer Wanted $16.00hr. plus 250.00 BONUS$$$
Oct 5 FREE $10 CASH FOR EVERYONE GET IT TODAY! SUPPLIES LIMITED HURRY
Oct 5 Now Hiring: Diesel technicians
Oct 5 Warehouse Associate
Oct 5 Delivery Driver Needed
Oct 5 Drivers Helper - Weekly Pay
Oct 5 Landscape Crew Member
Oct 5 Seasonal Laborers
Oct 5 Seasonal Laborers
Oct 5 WINDOW AND DOOR INSTALLERS NEEDED
Oct 5 Commercial Roofers and Helpers
Oct 5 Lyft Driver (part-time / full-time)
Oct 5 Lyft Driver (part-time / full-time)
Oct 5 Warehouse Associate/Driver
Oct 5 Grand Opening! Now Hiring! Immediate work!
Oct 5 C.D.L Drivers wanted
Oct 5 SET UP Workers Needed for Local Fair (Cash Paid Daily)
Oct 5 Workers Needed for Local Fair (Cash Paid Daily)
Oct 4 Distribution Center General Worker Position
Oct 4 Distribution Center General Worker Position
Oct 4 Distribution Center General Worker Position
Oct 4 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 4 Warehouse Worker
Oct 4 House Cleaner Needed
Oct 4 Janitor Job offer
Oct 4 HIRING TIRE TECHNICIANS
Oct 4 CDL-A Company Drivers - $5K SIGN-ON BONUS! 💰 🚚
Oct 4 Part Time Groundskeeper/Cleaner
Oct 4 Driver and helper
Oct 4 Full-Time Community Trash Collector
Oct 4 * Deliver with DoorDash *
Oct 4 Experienced maintenance men
Oct 4 Maintenance Manager for Luxury Apts!
Oct 4 Maintenance Manager for Luxury Apts!
Oct 4 Maintenance Tech II for Luxury Apts!
Oct 4 Maintenance Tech II for Luxury Apts!
Oct 4 Maintenance Tech I for Luxury Apts!
Oct 4 Stucco Repair Help
Oct 4 Lyft Driver (part-time / full-time)
Oct 4 Lyft Driver (part-time / full-time)
Oct 4 Workers Needed for Local Fair (Cash Paid Daily)
Oct 4 Warehouse Team Member - Multiple Shifts Available
Oct 4 🌊 Tidal Wave Lawrenceville ǀ Car Wash Attendant 🌊
Oct 4 🌊 Tidal Wave Kennesaw ǀ Car Wash Attendant 🌊
Oct 4 🌊 Tidal Wave Gainesville ǀ Car Wash Attendant 🌊
Oct 4 🌊 Tidal Wave Dacula ǀ Car Wash Attendant 🌊
Oct 4 🌊 Tidal Wave Canton ǀ Car Wash Attendant 🌊
Oct 4 Heavy equipment operator
Oct 4 **Full Time Building Cleaner (pressure washing/brush scrubbing) Needed**
Oct 4 Amazon Picker - Up to $16.5/hr
Oct 4 Warehouse Laborer - Immediate Hire
Oct 4 Warehouse Team Member - Split Shifts (20hr/week)
Oct 4 Warehouse Team Member - Morning Shifts Available
Oct 4 Warehouse Team Member - Multiple Shifts Available

期望输出（仅保留10月6日及以后内容）

Oct 7 General labor crew member needed
Oct 7 General Labor Demo
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 Craftsman Wanted
Oct 7 Home Cleaners Wanted! Make over $145+ for 1bd, $165 for 2bd cleans.
Oct 7 Sign Company In Search of Installers and Fabricators
Oct 7 ***PILE DRIVERS-DRILLERS-TRAINEES***
Oct 7 Warehouse Package Handler
Oct 7 Workers Needed for Local Fair (Cash Paid Daily)
Oct 7 Warehouse Team Member - Multiple Shifts Available
Oct 7 Construction
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Amazon Warehouse Assistant - Immediate Hire
Oct 7 Amazon Picker - Overnight Shifts Available
Oct 7 Warehouse Team Member - With Sign-On Bonus!
Oct 7 Amazon Warehouse Assistant
Oct 7 Need a Crew of 4+
Oct 6 Helper and Construction
Oct 6 Truck Driver/El Camionero
Oct 6 Workers Needed for Local Fair (Cash Paid Daily)
Oct 6 Roofing Crews Needed! ($2,000 Sign On Bonus - Start Today!)
Oct 6 Available Immediately General Labor/Driver with Truck or Van Positions
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 6 $20-25/hr. SIMPLE VALET TRASH COLLECTOR-Camp Creek/East Point 30331
Oct 6 Asphalt workers labors
Oct 6 CDL-A Company Drivers - $5K SIGN-ON BONUS! 💰 🚚
Oct 6 EXPERIENCED HANDYMAN ***2 immediate openings
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR

修改后的脚本

import requests
from bs4 import BeautifulSoup
from datetime import datetime, timedelta

link = 'https://atlanta.craigslist.org/search/lab'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}

days_to_go_back = 2
# 计算截止日期：当前日期减去指定回溯天数
cutoff_date = datetime.now() - timedelta(days=days_to_go_back)

with requests.Session() as s:
    s.headers.update(headers)
    res = s.get(link)
    soup = BeautifulSoup(res.text,"lxml")
    for item in soup.select("[id^='search-results'] li[class*='result']"):
        # 读取time标签的datetime属性，获取机器可读的发布时间
        time_tag = item.select_one("time.result-date")
        post_datetime = datetime.fromisoformat(time_tag['datetime'])
        # 过滤早于截止日期的内容
        if post_datetime < cutoff_date:
            continue
        date = ' '.join(time_tag.text.split())
        title = item.select_one("h3.result-heading > a").get_text(strip=True)
        print(date,title)

关键修改说明

引入datetime模块：用于处理日期计算和比较，避免解析文本日期的歧义。
计算截止日期：用当前时间减去days_to_go_back天，得到需要保留内容的最早日期。
使用datetime属性解析发布时间：直接读取HTML中time标签的datetime属性，这个属性是标准的ISO格式时间，比解析文本日期更准确可靠。
添加过滤逻辑：如果发布时间早于截止日期，跳过该条内容；否则正常输出。

内容的提问来源于stack exchange，提问作者MITHU

如何为Python Craigslist爬虫添加指定回溯天数的筛选逻辑

原脚本 #

当前输出 #

期望输出（仅保留10月6日及以后内容） #

修改后的脚本 #

关键修改说明 #

原脚本

当前输出

期望输出（仅保留10月6日及以后内容）

修改后的脚本

关键修改说明