如何设置Centris.ca API请求以获取英文响应?
问题:无法让Centris.ca的API返回英文响应
背景
我正在学习一门课程,但课程内容已过时,目标房地产网站的部分内容发生了变更。获取房产数据需要调用两个API接口:
- 接口1:
https://www.centris.ca/property/UpdateQuery - 接口2:
https://www.centris.ca/Property/GetInscriptions
目前遇到的核心问题是:找不到让这两个接口返回英文响应的方法。
已尝试的操作
- 请求头:设置
Accept-Language: en-US,en;q=0.6 - Cookie检查:未发现与语言设置相关的Cookie项
爬虫代码
import scrapy from scrapy.selector import Selector import json class ListingsSpider(scrapy.Spider): name = "listings" allowed_domains = ["www.centris.ca"] position = { "startPosition": 0 } def start_requests(self): query = { "query":{ "UseGeographyShapes":0, "Filters":[ ], "FieldsValues":[ { "fieldId":"Category", "value":"Commercial", "fieldConditionId":"", "valueConditionId":"" }, { "fieldId":"SellingType", "value":"Rent", "fieldConditionId":"", "valueConditionId":"" }, { "fieldId":"RentPrice", "value":0, "fieldConditionId":"ForRent", "valueConditionId":"" }, { "fieldId":"RentPrice", "value":999999999999, "fieldConditionId":"ForRent", "valueConditionId":"" } ] }, "isHomePage": True } yield scrapy.Request( url="https://www.centris.ca/property/UpdateQuery", method="POST", body=json.dumps(query), headers={ 'Content-Type': 'application/json', 'Content-Language': 'en' }, callback=self.update_query ) def update_query(self, response): yield scrapy.Request( url="https://www.centris.ca/Property/GetInscriptions", method="POST", body=json.dumps(self.position), headers={ 'Content-Type': 'application/json', 'accept-language': 'en-US,en;q=0.6', 'referer': 'https://www.centris.ca/en/properties~for-rent?view=Thumbnail', 'cache-control': 'no-cache' }, cookies={'currency': 'USD', 'country': 'UY'}, callback=self.parse ) def parse(self, response): resp_dict = json.loads(response.body) html = resp_dict.get('d').get('Result').get('html') sel = Selector(text=html) listings = sel.xpath("//div[@class='property-thumbnail-item thumbnailItem col-12 col-sm-6 col-md-4 col-lg-3']") for listing in listings: print("not yet implemented")
补充说明
本次爬取的目标是网站的API接口,而非网页页面本身。
内容的提问来源于stack exchange,提问作者gary rizzo
相关产品推荐
相关产品推荐

