我試圖廢棄此頁面上的搜索結果元素：https : //shop.bodybuilding.com/search? q = protein bar & selected_tab =Products with selenium 但結果它只給了我 4 個第一個元素。我不知道為什么？它是一個 javascript 頁面？以及如何洗掉此搜索頁面上的所有元素？這是我創建的代碼：
import requests
import numpy as np
import pandas as pd
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome(executable_path='C:/chromedriver')
url = 'https://shop.bodybuilding.com/search?q=protein bar&selected_tab=Products'
driver.get(url)
soup = BeautifulSoup(driver.page_source, 'html.parser')
all_items = soup.find_all('div', {'class': 'ProductTile ProductTile--flat Animate AnimateOnHover Animate--fade-in Animate--animated'})
for i in range(len(all_items)):
prices=all_items[i].find('div', {'class': 'Price ProductTile__price'}).text
names=all_items[i].find('p', {'class': 'ProductTile__title'}).text
images=all_items[i].find('img')['src']
url=all_items[i].find('a', {'class': 'Anchor ProductTile__image'})['href']
print(images)
這是此頁面上名稱的結果，如您所見，它只抓取了前 4 個元素！
BSN Protein Crisp Bars
Optimum Nutrition Protein Wafers
Herbaland Vegan Protein Gummies
Battle Bars Full Battle Rattle (FBR) Protein Bar
價格、影像和 URL 是否相同？
uj5u.com熱心網友回復：
怎么修
您必須滾動，以便加載所有專案：
last_height = driver.execute_script("return document.body.scrollHeight")
while True:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(1)
new_height = driver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_height
soup = BeautifulSoup(driver.page_source, 'html.parser')
all_items = soup.find_all('div', {'class': 'ProductTile ProductTile--flat Animate AnimateOnHover Animate--fade-in Animate--animated'})
for i in all_items:
prices=i.find('div', {'class': 'Price ProductTile__price'}).text if i.find('div', {'class': 'Price ProductTile__price'}) else None
names=i.find('p', {'class': 'ProductTile__title'}).text
images=i.find('img')['src']
url=i.find('a', {'class': 'Anchor ProductTile__image'})['href']
print(images)
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/386238.html
上一篇：我可以在物件中放置多個值嗎？
下一篇：試圖找到正確的xpath