如何从Craigslist动态接口抓取职位发布日期与标题？

尝试抓取波士顿Craigslist会计类职位页面的标题与发布日期，页面内容通过动态接口加载。目前已能从JSON响应中解析出标题，但无法获取发布日期。

已尝试代码：

import requests
from pprint import pprint

link = 'https://sapi.craigslist.org/web/v7/postings/search/full?batch=4-0-360-0-0&cc=US&lang=en&searchPath=acc'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
with requests.Session() as s:
    s.headers.update(headers)
    res = s.get(link)
    for item in res.json()['data']['items']:
        print(item)

当前输出：

[12030636, 2611302, 23, -1, '1:1~42.3492~-71.0768', 'Executive Assistant']
[12017824, 2609705, 23, -1, '1:2~42.3943~-71.218', 'Staff Accountant - Accounts Receivable (TEMP)']
[11638522, 2526012, 23, -1, '2:3~42.2093~-70.9963', 'Bookkeeper']
[11626278, 2524450, 23, -1, '1:1~42.3492~-71.0768', 'Top Consulting Company seeking Accounting Associate']
[11353351, 2456092, 23, -1, '1:1~42.3492~-71.0768', 'ID Bookkeeper-Interior Design Bookkeeper/Accountant-Work Remotely']
[11348351, 2455214, 23, -1, '1:4~42.3647~-71.1042', 'Bookeeper needed part-time']

期望输出：

Oct 7 Executive Assistant
Oct 7 Staff Accountant - Accounts Receivable (TEMP)
Oct 6 Bookkeeper
Oct 6 Top Consulting Company seeking Accounting Associate
Oct 5 ID Bookkeeper-Interior Design Bookkeeper/Accountant-Work Remotely
Oct 5 Bookeeper needed part-time

接口返回的items数组中，第二个元素（索引为1的位置）是发布时间的小时级时间戳，将其乘以3600转换为标准Unix时间戳（秒）后，即可转换为可读日期格式。修改后的代码如下：

import requests
import datetime

link = 'https://sapi.craigslist.org/web/v7/postings/search/full?batch=4-0-360-0-0&cc=US&lang=en&searchPath=acc'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
with requests.Session() as s:
    s.headers.update(headers)
    res = s.get(link)
    for item in res.json()['data']['items']:
        # 转换时间戳为日期
        timestamp = item[1] * 3600  # 小时转秒
        post_date = datetime.datetime.fromtimestamp(timestamp).strftime("%b %d")
        title = item[-1]
        print(f"{post_date} {title}")

关键说明 #

• item[1]：接口返回的时间值以小时为单位，乘以3600转换为Unix时间戳（秒）
• strftime("%b %d")：将datetime对象格式化为缩写月份+日期的格式，如"Oct 7"
• item[-1]：直接取数组最后一个元素作为职位标题

内容的提问来源于stack exchange，提问作者robots.txt