写入当前URL至CSV触发TypeError错误的技术求助
修复Selenium脚本中CSV写入的TypeError问题
咱们先把问题拆解开:你碰到的TypeError本质是Python不允许列表和元组直接拼接——你代码里的row是列表类型,而(url1,)是元组类型,两者没法直接用+操作。而且更关键的是,你其实已经在收集数据的时候把url1加到每个row里了,后面写入时重复添加完全是画蛇添足。
核心修复方案
直接去掉写入时的拼接操作就行,因为你在data.append([bp1, ba1, bp3, url1])这一步已经把当前URL放进了每个数据行的列表里。另外,你代码里在with块外面重新赋值url1 = driver.current_url也是多余的,可以调整到循环外统一获取。
修改后的完整代码
import csv import os import time from random import shuffle from selenium import webdriver from selenium.common.exceptions import TimeoutException, NoSuchElementException from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.ui import WebDriverWait as wait driver = webdriver.Chrome() driver.set_window_size(1024, 600) driver.maximize_window() driver.get('https://www.betstar.com.au/sports/soccer/44317884-football-scotland-ladbrokes-league-2/') # XPaths for the data groups = "//*[@class='fullbox']" xp_bp1 = ".//*[@class='row'][1]//td[1]" xp_ba1 = ".//*[@class='row'][2]//td[1]" xp_bp3 = ".//*[@class='row'][1]//span" while True: try: # wait for the data to populate the tables #wait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, (xp_bp1)))) time.sleep(2) data = [] url1 = driver.current_url # 统一获取当前URL,避免重复调用 for elem in driver.find_elements_by_xpath(groups): try: bp1 = elem.find_element_by_xpath(xp_bp1).text except: bp1 = None try: ba1 = elem.find_element_by_xpath(xp_ba1).text except: ba1 = None try: bp3 = elem.find_element_by_xpath(xp_bp3).text except: bp3 = None data.append([bp1, ba1, bp3, url1]) print(data) with open('test.csv', 'a', newline='', encoding="utf-8") as outfile: writer = csv.writer(outfile) for row in data: writer.writerow(row) # 直接写入已包含URL的row列表 except TimeoutException as ex: pass except NoSuchElementException as ex: print(ex) break
额外说明
如果你以后确实需要在写入时动态添加内容(比如收集数据时没加URL),那要保证拼接的两个对象是同类型:
- 要么把元组转成列表:
writer.writerow(row + [url1]) - 要么把列表转成元组:
writer.writerow(tuple(row) + (url1,))
至于你尝试的for row in zip(data)会导致数据挤在同一列,是因为zip(data)会把data里的每个子列表当成单个元素打包成元组,写入CSV时自然会把整个子列表放到一列里,这个方向本身就不对,不用再试啦。
内容的提问来源于stack exchange,提问作者user9155788
相关产品推荐
相关产品推荐

