我正在嘗試在谷歌趨勢中以地鐵級別抓取給定關鍵字的資料。到目前為止，我已經通過 selenium 將我想要的關鍵字輸入到 googletrends 中，但我只是找不到一種方法來單擊下拉選單，選擇一個并將其下載為 csv。我特別卡在下拉選單部分（也許我無法找到正確的類/ID）。
我有一個限制，這需要作為刮板而不是通過 pyTrends API 來完成。
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://trends.google.com/trends/?geo=US')
element = driver.find_element(By.XPATH,'.//*[@id="input-254"]')
element.send_keys("glass")
element.send_keys(Keys.ENTER)
#till this point I am able to enter the word and arrive at the selection page
dropdown1 =driver.find_element(By.CLASS_NAME,'_md-select-value').click() #this is my attempt at clicking on the drop down. However this doesn't seem to work.
我希望在 subregion 下拉選單下選擇 METRO 并使用 subregion 旁邊的按鈕下載 csv 并將其存盤在特定路徑中。
有人可以幫我選擇和下載給定關鍵字和地區的 csv 作為 Metro 嗎？
謝謝你。
更新：
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://trends.google.com/trends/?geo=US')
element = driver.find_element(By.XPATH,'.//*[@id="input-254"]')
driver.maximize_window()
keywords = ["glass","wolf","cat","dog"]
for key in keywords:
#loop through the keywords
element.send_keys(key)
element.send_keys(Keys.ENTER)
#click on cookie
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".cookieBarButton.cookieBarConsentButton"))).click()
#click on subregion
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[normalize-space(.)='Subregion']"))).click()
#click on metro
elementbtn=WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//md-option[@value='metro']")))
driver.execute_script("arguments[0].click();", elementbtn)
#download file
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//html/body/div[2]/div[2]/div/md-content/div/div/div[2]/trends-widget/ng-include/widget/div/div/div/widget-actions/div/button[1]/i"))).click()
然而，這只會下載關鍵字串列中的第一個元素，然后導致錯誤：Th“訊息：過時的元素參考：元素未附加到頁面檔案”。
uj5u.com熱心網友回復：
您使用的 class_name 在頁面上有 23 個元素并且不是唯一的。請嘗試使用唯一的 xpath。
使用 webdriverwait 并等待元素可點擊。首先單擊 cookie 按鈕，然后單擊相應的下拉選單
代碼：
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://trends.google.com/trends/?geo=US')
element = driver.find_element(By.XPATH,'.//*[@id="input-254"]')
driver.maximize_window()
element.send_keys("glass")
element.send_keys(Keys.ENTER)
#click on cookie
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".cookieBarButton.cookieBarConsentButton"))).click()
#click on subregion
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[normalize-space(.)='Subregion']"))).click()
#click on city
elementbtn=WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//md-option[@value='city']")))
driver.execute_script("arguments[0].click();", elementbtn)
更新
driver = webdriver.Chrome('chromedriver.exe')
keywords = ["glass","wolf","cat","dog"]
for key in keywords:
#loop through the keywords
driver.get('https://trends.google.com/trends/?geo=US')
element = driver.find_element(By.XPATH,'.//*[@id="input-254"]')
driver.maximize_window()
element.send_keys(key)
element.send_keys(Keys.ENTER)
#click on cookie
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".cookieBarButton.cookieBarConsentButton"))).click()
#click on subregion
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[normalize-space(.)='Subregion']"))).click()
#click on metro
elementbtn=WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//md-option[@value='metro']")))
driver.execute_script("arguments[0].click();", elementbtn)
#download file
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//html/body/div[2]/div[2]/div/md-content/div/div/div[2]/trends-widget/ng-include/widget/div/div/div/widget-actions/div/button[1]/i"))).click()
time.sleep(2)#slowdown the loop
轉載請註明出處，本文鏈接：https://www.uj5u.com/gongcheng/516129.html
上一篇：為什么還要阻止不執行腳本化管道？