我試圖使用 selenium 多次點擊加載更多按鈕，但是我無法點擊加載更多按鈕（它甚至不是一個按鈕......）
當我嘗試點擊它時，element click intercepted: Element is not clickable at point即使我明確編碼它也會顯示錯誤wait.until(EC.element_to_be_clickable）
我想知道我的代碼有什么問題。我使用的代碼是
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
import time
url = "https://www.architectural-review.com/buildings/house"
options = webdriver.ChromeOptions()
driver = webdriver.Chrome()
driver.get(url)
# accept cookies
wait0 = WebDriverWait(driver, 10)
cookie_button = wait0.until(EC.element_to_be_clickable((By.XPATH,'/html/body/div[1]/div/div[6]/button[1]')))
cookie_button.click()
print("loading more projects ...")
count=5
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.XPATH,'/html/body/div[2]/div/div[3]/section[5]/div/div[5]/a')))
print(element)
print("now element is clickable")
while count>1:
element.click();
count-=1
print(count)
HTML 看起來像（它不是按鈕，不確定這是否重要）
<div class="view-more">
<a href="#" class="cpb-bottom-more-news-link dynamic-loader" data-term-id="869" data-block-id="7" data-offset="18" data-load-number="4" data-resulting-class="fourth-post" data-exclude="MzE4Mzk5LDMxODM2MCwzMTc2MDAsNjUyNjEsNjQzMTMsNjQzMTgsNjQyNzMsNjMyNTc=">Load More </a>
</div>
uj5u.com熱心網友回復：
這有點棘手。
“加載更多”按鈕最初位于此處的螢屏之外，因此您需要滾動頁面才能單擊該按鈕。但是當你滾動這個頁面時，它滾動得比需要的多，所以“加載更多”按鈕再次出現在螢屏之外，所以這次你將不得不向后滾動:)
即使沒有加載，“加載更多”按鈕也會出現在頁面上更多的。在這種情況下，這個按鈕就變得不可見了。因此，要停止滾動，您需要檢查style該元素的屬性。
此外，您需要改進定位器。非常長的絕對 XPath 極易損壞且不可靠。
以下代碼有效！
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)
url = "https://www.architectural-review.com/buildings/house"
driver.get(url)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.cmplz-btn.cmplz-accept'))).click()
while True:
load_more = wait.until(EC.presence_of_element_located((By.CLASS_NAME, "cpb-bottom-more-news-link")))
load_more.location_once_scrolled_into_view
time.sleep(0.4)
driver.execute_script("window.scrollBy(0, arguments[0]);", -300)
time.sleep(0.4)
displayed = load_more.get_attribute("style")
if "none" in displayed:
break
driver.execute_script("arguments[0].click();", load_more)
uj5u.com熱心網友回復：
這是一種無需 Selenium 即可獲取該資訊的方法（Selenium 主要是一種測驗工具，當所有其他方法都失敗時，這應該是網路抓取企業的最后一次呼叫）：
import requests
from bs4 import BeautifulSoup as bs
import base64
from tqdm import tqdm ## if using Jupyter, do `from tqdm.notebook import tqdm`
import pandas as pd
headers = {
'referer': 'https://www.architectural-review.com/buildings/house',
'x-requested-with': 'XMLHttpRequest',
'accept-language': 'en-US,en;q=0.9',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}
s = requests.Session()
s.headers.update(headers)
big_list = []
for x in tqdm(range(0, 50, 4)):
url = f'https://www.architectural-review.com/wp-admin/admin-ajax.php?action=lazy-load&offset={x}&term_id=869&block_id=7&class=fourth-post&load_number=4&exclude=MzE4Mzk5LDMxODM2MCwzMTc2MDAsNjUyNjEsNjQzMTMsNjQzMTgsNjQyNzMsNjMyNTcsMzI2NzE2LDMyNjY5OSwzNDY3MSwzMjg3MzgsMTc5ODY1LDE3OTgzNSwxNzk3NjgsMTc5NTk4LDg3NDgyLDY3Mjk2LDY1NjIwLDYzNjQwLDYzMzgyLDYzMzQzLDYzMjk3LDYyOTMxLDYyOTI1LDYyMTA2LDYwOTE4LDU3MjMyLDU3MTE0LDU3MTAzLDU3MDg4LDU3MDgxLDU3MDc1LDU2OTkzLDU2OTc4'
r = s.get(url)
soup = bs(base64.b64decode(r.json()['body']), 'html.parser')
houses = soup.select('div[]')
for house in houses:
title = house.select_one('h2').get_text(strip=True)
author = house.select_one('span[]').get_text(strip=True)
url = house.select_one('h2').parent.get('href')
big_list.append((title, author, url))
df = pd.DataFrame(big_list, columns=['title', 'author', 'url'])
print(df)
終端結果：
100%
13/13 [00:13<00:00, 1.10it/s]
title author url
0 Hl?euberg artist studio in Skaresstr?nd, Iceland by Studio Bua Olivia Wynne Houck https://www.architectural-review.com/awards/ar-house/hloduberg-artist-studio-in-skardsstrond-iceland-by-studio-bua
1 Weekend house on Salamis Island, Greece by AREA (Architecture Research Athens) Dorette Panagiotopoulou https://www.architectural-review.com/awards/ar-house/weekend-house-on-salamis-island-greece-by-area-architecture-research-athens
2 Beaconsfield House in Perth, Australia by Simon Pendal Architect Rachael Bernstone https://www.architectural-review.com/awards/ar-house/beaconsfield-house-in-perth-australia-by-simon-pendal-architect
3 House in Kanazawa in Ishikawa Prefecture, Japan by Shota Nakanishi Architects Danielle Demetriou https://www.architectural-review.com/awards/ar-house/house-in-kanazawa-in-ishikawa-prefecture-japan-by-shota-nakanishi-architects
4 House on Hamburg? in Bohüslan, Sweden by Manthey Kula Rasmus W?rn https://www.architectural-review.com/awards/ar-house/house-on-hamburgo-in-bohuslan-sweden-by-manthey-kula
5 El Garaje in Madrid, Spain by Nomos Miguel Fernández-Galiano https://www.architectural-review.com/awards/ar-house/el-garaje-in-madrid-spain-by-nomos
6 Water Tower in Castle Acre, UK by Tonkin Liu Nile Bridgeman https://www.architectural-review.com/awards/new-into-old/water-tower-in-castle-acre-uk-by-tonkin-liu
7 BAAN 38 in Bangkok, Thailand by Junsekino Architect and Design Madeleine Weavers https://www.architectural-review.com/awards/new-into-old/baan-38-in-bangkok-thailand-by-junsekino-architect-and-design
8 Yamato House in Nara, Japan by Tadashi Yoshimura Architects Nancy (Yao) Ji https://www.architectural-review.com/places/japan/yamato-house-in-nara-japan-by-tadashi-yoshimura-architects
9 Baltrasna House in Dublin, Ireland by Ryan W Kennihan Architects Lisa Godson https://www.architectural-review.com/places/ireland/baltrasna-house-in-dublin-ireland-by-ryan-w-kennihan-architects
10 Couldrey House in Brisbane, Australia by HNNA and Peter Besley Kirsty Volz https://www.architectural-review.com/awards/ar-house/couldrey-house-in-brisbane-australia-by-hnna-and-peter-besley
11 Big Space, Little Space in Buffalo, United States by Davidson Rafailidis Raymund Ryan https://www.architectural-review.com/awards/ar-house/big-space-little-space-in-buffalo-united-states-by-davidson-rafailidis
12 Home grown: Arigho Larmour Wheeler Architects, Ireland Marwa El Mubark https://www.architectural-review.com/awards/ar-emerging/home-grown-arigho-larmour-wheeler-architects-ireland
13 Polished conversation: Clancy Moore, Ireland Lili Zarzycki https://www.architectural-review.com/awards/ar-emerging/polished-conversation-clancy-moore-ireland
14 Solo House, Matarra?a, Spain, by Pezo von Ellrichshausen AR Editors https://www.architectural-review.com/buildings/solo-house-matarrana-spain-by-pezo-von-ellrichshausen
15 Solo House, Matarra?a, Spain, by OFFICE KGDVS AR Editors https://www.architectural-review.com/buildings/solo-house-matarrana-spain-by-office-kgdvs
16 Holiday pay: vacation homes by Living Architecture in the U.K. and Solo Houses in Matarra?a, Spain Manon Mollard https://www.architectural-review.com/buildings/holiday-pay-vacation-homes-by-living-architecture-in-the-u-k-and-solo-houses-in-matarrana-spain
17 Shadow play: Hawthorn House, Victoria, Australia, by Edition Office Samson Séyí Famusan https://www.architectural-review.com/awards/ar-house/shadow-play-hawthorn-house-victoria-australia-by-edition-office
18 Balancing blocks: Casa St Cugat, Catalonia, Spain, Camps Felip Arquitecturia Siufan Adey https://www.architectural-review.com/awards/ar-house/balancing-blocks-casa-st-cugat-catalonia-spain-camps-felip-arquitecturia
19 Spotlight: house in a garden, London, UK, Gianni Botsford Architects Nile Bridgeman https://www.architectural-review.com/awards/ar-house/spotlight-house-in-a-garden-london-uk-gianni-botsford-architects
20 Light room: photographer’s house, London, UK, Hugh Strange Architects Shukri Sultan https://www.architectural-review.com/awards/ar-house/light-room-photographers-house-london-uk-hugh-strange-architects
21 Ascending order: Loop Terrace, Kobe, Japan, Tomohiro Hata Architect and Associates Thomas Aquilina https://www.architectural-review.com/awards/ar-house/ascending-order-loop-terrace-kobe-japan-tomohiro-hata-architect-and-associates
22 Holding the line: The Longhouse, Daylesford, Australia, by Partners Hill Josh Fenton https://www.architectural-review.com/awards/ar-house/holding-the-line-the-longhouse-daylesford-australia-by-partners-hill
23 Slice of life: the Cut, Koganecho, Japan, by Persimmon Hills Architects Aoi Phillips https://www.architectural-review.com/awards/ar-house/slice-of-life-the-cut-koganecho-japan-by-persimmon-hills-architects
24 Brick house: Casa Volta, Oaxaca, Mexico by Ambrosi Etchegaray Marwa El Mubark https://www.architectural-review.com/awards/ar-house/brick-house-casa-volta-oaxaca-mexico-by-ambrosi-etchegaray
25 Fringe benefits: Song house, Shanghai, China, by AZL Architects Harry den Hartog https://www.architectural-review.com/awards/ar-house/fringe-benefits-song-house-shanghai-china-by-azl-architects
26 Stepping stone: Hamra Studio residence, Gotland, Sweden, by Collectif Encore Lili Zarzycki https://www.architectural-review.com/awards/ar-house/stepping-stone-hamra-studio-residence-gotland-sweden-by-collectif-encore
27 Cut a dash: Hollybrook Grove in Dublin, Ireland, by David Leech Architects Eleanor Beaumont https://www.architectural-review.com/awards/ar-house/cut-a-dash-hollybrook-grove-in-dublin-ireland-by-david-leech-architects
28 Noreile Breen: ‘I am from this place, I have a responsibility to it’ Rupert Bickersteth https://www.architectural-review.com/archive/noreile-breen-i-am-from-this-place-i-have-a-responsibility-to-it
29 Footprints in time: House in Kamitomii, Kurashiki, Japan, by General Design Co Kaz Yoneda https://www.architectural-review.com/awards/ar-house/footprints-in-time-house-in-kamitomii-kurashiki-japan-by-general-design-co
30 Domestic bliss: houses in Dublin by Clancy Moore, TAKA, Ryan W Kennihan Architects and Arigho Larmour Wheeler Architects Eleanor Beaumont https://www.architectural-review.com/buildings/domestic-bliss-houses-in-dublin-by-clancy-moore-taka-ryan-w-kennihan-architects-and-arigho-larmour-wheeler-architects
31 Travel agency: Irene Pérez, TEd'A Arquitectes, Spain Lili Zarzycki https://www.architectural-review.com/buildings/travel-agency-irene-perez-teda-arquitectes-spain
32 Impractical genius: J?rn Utzon's Sydney Opera House and Can Lis, Mallorca Gerard Reinmuth https://www.architectural-review.com/buildings/impractical-genius-jorn-utzons-sydney-opera-house-and-can-lis-mallorca
33 Nature nurtured: Carla Jua?aba Studio, Brazil Manon Mollard https://www.architectural-review.com/today/nature-nurtured-carla-juacaba-studio-brazil
34 Front of house: T O B Architect, Ireland Eleanor Beaumont https://www.architectural-review.com/buildings/front-of-house-t-o-b-architect-ireland
35 H?tel Wolfers: Henry van de Velde's frozen ruin Eleanor Beaumont https://www.architectural-review.com/buildings/hotel-wolfers-henry-van-de-veldes-frozen-ruin
36 Shadow box: House V-D, Ghent, by Marie-José Van Hee Architecten Helen Thomas https://www.architectural-review.com/buildings/shadow-box-house-v-d-ghent-by-marie-jose-van-hee-architecten
37 Inside the Box: Kokoon by Helsinki’s Aalto University Wood Program Students, Finland George Kafka https://www.architectural-review.com/awards/ar-house/inside-the-box-kokoon-by-helsinkis-aalto-university-wood-program-students-finland
38 Pioneer Spirit: Casa Bruma by Fernanda Canales and Claudia Rodríguez in Mexico Lili Carr https://www.architectural-review.com/awards/ar-house/pioneer-spirit-casa-bruma-by-fernanda-canales-and-claudia-rodriguez-in-mexico
39 Overarching Craft: Mesura's house extension in Elche, Spain Rafael Gómez-Moriana https://www.architectural-review.com/awards/ar-house/overarching-craft-mesuras-house-extension-in-elche-spain
40 Planes and plants: Vo Trong Nghia Architects’ Binh House in Ho Chi Minh City Josh Comaroff https://www.architectural-review.com/awards/ar-house/planes-and-plants-vo-trong-nghia-architects-binh-house-in-ho-chi-minh-city
41 Building in potential: house in Nobeoka, Japan by Schemata Architects Jan Vranovsk√? https://www.architectural-review.com/awards/ar-house/building-in-potential-house-in-nobeoka-japan-by-schemata-architects
42 Veiled in Secrecy: The Habitat for Orphan Girls by ZAV Architects, Iran Mehr Shafiei https://www.architectural-review.com/awards/ar-house/veiled-in-secrecy-the-habitat-for-orphan-girls-by-zav-architects-iran
43 Overarching ambition: Two houses, Oropesa, Spain by Paredes Pedrosa Arquitectos Andrew Ayers https://www.architectural-review.com/buildings/overarching-ambition-two-houses-oropesa-spain-by-paredes-pedrosa-arquitectos
44 Winds of change: Casa de Los Vientos in Ca?diz, Spain by Jose? Luis Mun?oz Manon Mollard https://www.architectural-review.com/awards/ar-emerging/winds-of-change-casa-de-los-vientos-in-cadiz-spain-by-jose-luis-muñoz
45 Economy class: Mrs Fan’s Plugin House in Beijing, China by People’s Architecture Office Manon Mollard https://www.architectural-review.com/awards/ar-emerging/economy-class-mrs-fans-plugin-house-in-beijing-china-by-peoples-architecture-office
46 Sense of an ending: Red House in London by 31/44 Architects Eleanor Beaumont https://www.architectural-review.com/awards/ar-emerging/sense-of-an-ending-red-house-in-london-by-31-44-architects
47 Dachaland: the Russian dacha DK-CM https://www.architectural-review.com/buildings/dachaland-the-russian-dacha
48 Iturbide Studio in Mexico City by Taller Mauricio Rocha Gabriela Carrillo Michael Webb https://www.architectural-review.com/buildings/iturbide-studio-in-mexico-city-by-taller-mauricio-rocha-gabriela-carrillo
49 A host of oasts: Caring Wood by James Macdonald Wright with Niall Maxwell Jay Merrick https://www.architectural-review.com/buildings/a-host-of-oasts-caring-wood-by-james-macdonald-wright-with-niall-maxwell
50 Coogee House, Sydney, Australia by Chenchow Little Architects Eleanor Beaumont https://www.architectural-review.com/awards/ar-house/coogee-house-sydney-australia-by-chenchow-little-architects
51 The Captain's House, Fuzhou, China by Vector Architects Eleanor Beaumont https://www.architectural-review.com/awards/ar-house/the-captains-house-fuzhou-china-by-vector-architects
該50限制是任意的，您可以提高并獲得更多屬性。看pandas的相關檔案：https ://pandas.pydata.org/docs/
同樣適用于請求：https ://requests.readthedocs.io/en/latest/
對于 tqdm，請訪問https://pypi.org/project/tqdm/
轉載請註明出處，本文鏈接：https://www.uj5u.com/net/535713.html
上一篇：最大高度不會產生預期的縮放效果
下一篇：如何訪問查詢塊之外的變數？