我正在嘗試向下滾動到網頁右側以下
uj5u.com熱心網友回復：
要向下滾動到網頁右側zillow網站的最底部，您可以在分頁元素可見時scrollIntoView以誘導WebDriverWait為visibility_of_element_located()并且您可以使用以下任一定位器策略：
代碼塊：
driver.get("https://www.zillow.com/clayton-county-ga/houses/3-_beds/2.0-_baths/?searchQueryState={"pagination":{},"usersSearchTerm":"Clayton County, GA","mapBounds":{"west":-85.19662367135751,"east":-83.66952406198251,"south":33.16207210856734,"north":33.76924644337602},"regionSelection":[{"regionId":1622,"regionType":4}],"isMapVisible":true,"filterState":{"beds":{"min":3},"baths":{"min":2},"sqft":{"min":1000,"max":3000},"built":{"min":1965},"con":{"value":false},"apa":{"value":false},"mf":{"value":false},"ah":{"value":true},"sort":{"value":"globalrelevanceex"},"lot":{"max":43560},"land":{"value":false},"gar":{"value":true},"tow":{"value":false},"manu":{"value":false},"apco":{"value":false}},"isListVisible":true,"mapZoom":9}") driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='search-pagination']"))))瀏覽器快照：
uj5u.com熱心網友回復：
互聯網世界中最聰明的網站之一是zillow它還提供API。如果你真的熱愛并想從心里學習網路抓取，為什么要硬編碼，為什么不從 api 中提取資料呢？requests下面是一個示例，如何僅使用模塊從 api 抓取資料作為 json 回應。
腳本：
import requests
headers = {
"User-Agent": "Mozilla/5.0 ",
"Accept": "*/*",
"Accept-Language": "en-US,en;q=0.5",
"Pragma": "no-cache",
"Cache-Control": "no-cache"
}
with requests.Session() as s:
s.headers.update(headers)
s.head('https://www.zillow.com/')
for page in range(1,3):
params = {
"searchQueryState": '{"pagination":{"currentPage":2},"usersSearchTerm":"Clayton County, GA","mapBounds":{"west":-84.85604749948251,"east":-84.01010023385751,"south":33.18506146243053,"north":33.746411533949434},"regionSelection":[{"regionId":1622,"regionType":4}],"isMapVisible":false,"filterState":{"beds":{"min":3},"baths":{"min":2},"sqft":{"min":1000,"max":3000},"built":{"min":1965},"isCondo":{"value":false},"isApartment":{"value":false},"isMultiFamily":{"value":false},"isAllHomes":{"value":true},"sortSelection":{"value":"globalrelevanceex"},"lotSize":{"max":43560},"isLotLand":{"value":false},"hasGarage":{"value":true},"isTownhouse":{"value":false},"isManufactured":{"value":false},"isApartmentOrCondo":{"value":false}},"isListVisible":true}',
"wants": '{"cat1":["listResults"],"cat2":["total"]}'
}
r = s.get('https://www.zillow.com/search/GetSearchPageState.htm',params=params).json()
#print(r)
for card in r['cat1']['searchResults']['listResults']:
price=card['price']
print(price)
輸出：
$235,000
$299,000
$259,000
$290,000
$255,000
$300,000
$295,000
$250,000
$250,000
$259,900
$290,000
$315,000
$249,000
$289,900
... 很快
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/459908.html
標籤：Python 硒 网页抓取 网络驱动程序等待 js-scrollintoview