Python使用range的for循环输出重复 如何调整避免重复打印历史结果
问题根因
打印和DataFrame生成的代码写在for循环内部,每爬取1条数据就会将当前已爬取的所有数据打印一次,导致出现累加重复输出的效果。
调整方法
仅需要将df = pd.DataFrame(song_list)和print(df)两行代码移出for循环,放在循环执行结束后的位置即可,不需要修改range控制循环的逻辑。
另外你代码里的if _ == Exception: break逻辑无效,可以直接删除,_是循环迭代的计数变量,永远不可能等于Exception类。
调整后完整代码
import time from selenium import webdriver import selenium from selenium.webdriver.chrome import service from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import pandas as pd ser= Service("C:\Program Files (x86)\chromedriver.exe") options = webdriver.ChromeOptions() options.add_experimental_option('excludeSwitches', ['enable-logging']) driver = webdriver.Chrome(options=options,service=ser) driver.get('https://soundcloud.com/jujubucks') print(driver.title) wait = WebDriverWait(driver,30) wait.until(EC.element_to_be_clickable((By.ID,"onetrust-accept-btn-handler"))).click() try: song_list = [] i = 1 for _ in range(20): song_contents = driver.find_element(By.XPATH, "//li[@class='soundList__item'][{}]".format(i)) driver.execute_script("arguments[0].scrollIntoView(true);",song_contents) search = song_contents.find_element(By.XPATH, ".//a[contains(@class,'soundTitle__username')]/span").text search_song = song_contents.find_element(By.XPATH, ".//a[contains(@class,'soundTitle__title')]/span").text search_date = song_contents.find_element(By.XPATH, ".//time[contains(@class,'relativeTime')]/span").text search_plays = song_contents.find_element(By.XPATH, ".//span[contains(@class,'sc-ministats-small')]/span").text i+=1 option ={ 'Artist': search, 'Song_title': search_song, 'Date': search_date, 'Streams': search_plays } song_list.append(option) # 循环结束后再生成DataFrame并打印 df = pd.DataFrame(song_list) print(df) except Exception: pass driver.quit()
调整后会等20条数据全部爬取完成后,仅打印一次完整的20条数据列表。
内容的提问来源于stack exchange,提问作者Houston Khanyile
相关产品推荐
相关产品推荐

