我不明白為什么 .find_all() 沒有回傳我搜索的所有標簽。
基本上我試圖抓取這個網站并獲取所有“tr”標簽的串列（主要對標簽中的標簽感興趣）但它只回傳嵌套在標簽中的第一個“tr”標簽，我m 假設它永遠不會到達 . 最終目標：我只需要從表格主體的每一行中提取葡萄酒名稱。
def web_parser1():
import requests
from bs4 import BeautifulSoup
import numpy as np
import pandas as pd
response_obj_1 = requests.get("https://www.globalwinescore.com/ranking/?idx=Wine&dFR[country][0]=Usa")
soup = BeautifulSoup(response_obj_1.text, "html.parser")
trtags = soup.find_all("tr")
print(trtags)
uj5u.com熱心網友回復：
實際上，url 是動態的，bs4 不能模仿 javascript，但資料也會生成 api 呼叫 json 回應作為 post 方法，從中可以輕松地requests僅使用模塊獲取資料。
腳本：
import requests
import json
body= {"requests":[{"indexName":"Wine","params":"query=&hitsPerPage=50&maxValuesPerFacet=1000&page=0&facets=["country","regions","color","type","appellation","vintage","classification","labels","score"]&tagFilters=&facetFilters=[["country:Usa"]]"},{"indexName":"Wine","params":"query=&hitsPerPage=1&maxValuesPerFacet=1000&page=0&attributesToRetrieve=[]&attributesToHighlight=[]&attributesToSnippet=[]&tagFilters=&analytics=false&facets=country"}]}
headers= {
'content-type': 'application/json',
}
api_url = "https://heioa8c3j2-1.algolianet.com/1/indexes/*/queries?x-algolia-agent=Algolia for vanilla JavaScript (lite) 3.24.5;instantsearch.js 2.2.3;JS Helper 2.23.0&x-algolia-application-id=HEIOA8C3J2&x-algolia-api-key=02525019312db516b514d0a95aefb1c1"
jsonData = requests.post(api_url, data=json.dumps(body), headers=headers).json()
#print(jsonData)
for item in jsonData['results'][0]['hits']:
name=item['wine']
print(name)
輸出：
Ridge Vineyards, Monte Bello, Santa Cruz Mountains
Kapcsandy Family Winery, State Lane Vineyard Grand-Vin Cabernet Sauvignon, Napa Valley
Abreu Vineyard, Thorevilos Cabernet Sauvignon, Napa Valley
Continuum, Proprietary Red, Oakville
Ridge Vineyards, Monte Bello, Santa Cruz Mountains
Abreu Vineyard, Madrona Ranch Cabernet Sauvignon, Napa Valley
Abreu Vineyard, Thorevilos Cabernet Sauvignon, Napa Valley
Lokoya Winery, Mount Veeder Cabernet Sauvignon, Napa Valley
Abreu Vineyard, Thorevilos Cabernet Sauvignon, Napa Valley
Screaming Eagle, Cabernet Sauvignon, Napa Valley
Verite, La Joie, Sonoma County
Screaming Eagle, Cabernet Sauvignon, Napa Valley
Harlan Estate, Napa Valley
Ridge Vineyards, Monte Bello, Santa Cruz Mountains
Dana Estates, Lotus Vineyard Cabernet Sauvignon, Napa Valley
Verite, La Muse, Sonoma County
Abreu Vineyard, Madrona Ranch Cabernet Sauvignon, Napa Valley
Kongsgaard, The Judge Chardonnay, White, Napa Valley
Screaming Eagle, Cabernet Sauvignon, Napa Valley
Dunn Vineyards, Trailer Vineyard Cabernet Sauvignon, Howell Mountain
Harlan Estate, Napa Valley
Abreu Vineyard, Madrona Ranch Cabernet Sauvignon, Napa Valley
Schrader Cellars, Old Sparky Beckstoffer To Kalon Vineyard Cabernet Sauvignon, Napa Valley
Dominus Estate, Christian Moueix, Napa Valley
Harlan Estate, Napa Valley
Hundred Acre, Few And Far Between Cabernet Sauvignon, Napa Valley
Abreu Vineyard, Madrona Ranch Cabernet Sauvignon, Napa Valley
Harlan Estate, Napa Valley
Abreu Vineyard, Madrona Ranch Cabernet Sauvignon, Napa Valley
Colgin Cellars, Tychson Hill Vineyard Cabernet Sauvignon, Napa Valley
Harlan Estate, Napa Valley
Screaming Eagle, Cabernet Sauvignon, Napa Valley
Venge Vineyards, Bone Ash Vineyard Cabernet Sauvignon, Napa Valley
Peter Michael Winery, Au Paradis Cabernet Sauvignon, Oakville
Colgin Cellars, Tychson Hill Vineyard Cabernet Sauvignon, Napa Valley
Harlan Estate, Napa Valley
Colgin Cellars, Ix Estate Red, Napa Valley
Schrader Cellars, Beckstoffer To Kalon Vineyard Cabernet Sauvignon, Napa Valley
Schrader Cellars, T6 Beckstoffer To Kalon Vineyard Cabernet Sauvignon, Napa Valley
Futo Estate, 5500 Sld - Cabernet Sauvignon, Stags Leap District
Hundred Acre, Fortification Port, Napa Valley
Chappellet, Pritchard Hill Cabernet Sauvignon, Napa Valley
Colgin Cellars, Cariad Red, Napa Valley
Screaming Eagle, Cabernet Sauvignon, Napa Valley
Schrader Cellars, T6 Beckstoffer To Kalon Vineyard Cabernet Sauvignon, Napa Valley
Cayuse Vineyards, En Chamberlin Vineyard Syrah, Walla Walla Valley
Harlan Estate, Napa Valley
Robert Mondavi Winery, To Kalon Vineyard Reserve Cabernet Sauvignon, Oakville
Schrader Cellars, Ccs Beckstoffer To Kalon Vineyard Cabernet Sauvignon, Napa Valley
Screaming Eagle, Cabernet Sauvignon, Napa Valley
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/456911.html