我正在嘗試從以下網頁https://smws.com/all-whisky?sort=priceesc&page=1&per-page=128抓取一些資訊
我試圖抓取的資訊是每個專案的名稱加上產品代碼（下面的代碼還沒有完全存在）。例如，對于第一個專案，這將是“晚安和歡樂”和“53.389”。然后我希望在整個網頁上進行迭代。我的代碼如下：
import requests
import time
import smtplib
from email.message import EmailMessage
import hashlib
from urllib.request import urlopen
from datetime import datetime
from bs4 import BeautifulSoup
from selenium import webdriver
url = 'https://smws.com/all-whisky?sort=pricedesc&page=1&per-page=128'
soup = BeautifulSoup(urlopen(url).read(),features="html.parser")
test = soup.find_all('a.href')
print(test)
然而，這只是回傳
[]
請有人可以幫助我回傳在 a.href 中回傳的資訊，并可能對整個頁面進行迭代。
謝謝！
uj5u.com熱心網友回復：
從每個標簽獲取鏈接[或href屬性]的正確方法a并不find_all('a.href')像您使用的那樣。你可以改為做類似的事情
test = [s for s in soup.find_all('a') if s.get('href')]
#for just the links:
#test = [s.get('href') for s in soup.find_all('a') if s.get('href')]
并獲取 您獲取的 htmla中具有href和可用的標簽。
然后我列印了
print(f'{len(test)} results:\n')
for t in test: print(f'[{t.get_text(strip=True)}]({t.get("href")})')
查看每個標簽中的文本和鏈接。
然而，這仍然不會讓你得到你提到想要的資料
每個專案的名稱加上產品代碼
因為它尚未加載到您獲取的 html 中（您可以使用print(soup.text)...進行檢查，而且，如果您嘗試，print(soup.find('ul', {"id":"product-listing-container"}).prettify())您會看到它ul#product-listing-container充滿了“骨架產品”，而不是像之后頁面上那樣在article標簽中填充屬性產品已加載）。
但我也注意到你的進口中有硒，所以很容易解決（你只需要確認你是成年人，然后等待產品加載然后使用 bs4 或直接呼叫 selenium 的find_elements.. .)
uj5u.com熱心網友回復：
請看下面的代碼，它作業得很好——
url = 'https://smws.com/all-whisky?sort=pricedesc&page=1&per-page=128'
r = requests.get(url)
soup = BeautifulSoup(r.content,features="html.parser")
for link in soup.find_all('a'):
print(link.get('href'))
uj5u.com熱心網友回復：
該頁面中的資訊正在通過 XHR 呼叫通過 javascript 進行補充。如果您想通過請求獲取該資訊，則需要抓取實際的 API 端點。這是一種方法：
import requests
import pandas as pd
pd.set_option('display.max_columns', None)
pd.set_option('display.max_colwidth', None)
r = requests.get('https://api.smws.com/api/v1/bottles?store_id=uk&parent_id=61&page=1&sortBy=pricedesc&minPrice=0&maxPrice=0&perPage=128')
df = pd.json_normalize(r.json()['items'])
print(df)
終端列印的結果：
id sku name url cask_no region age abv cask_type spirit_type list_description price stock release_date distilleddate image product_categories categories
0 3072 004343GB0700631 Analytical allure /analytical-allure/ 4.343 Highland 31.0 53.6% 2nd fill ex-madeira hogshead Malt Whisky None 695.00 24 2022-10-19 13:00:00 1/5/1991 https://cdn11.bigcommerce.com/s-vagfena5nz/products/3072/images/4821/4.343-web__47052.1666104996.386.513.png?c=1 All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Cask Type, All Whisky/Whisky/Outturn, All Whisky/Whisky, Whisky/Outturn/Mid-Month Releases, All Whisky/Whisky by Region/Highland, All Whisky/Whisky/Vaults Collection, All Whisky/Flavour Profiles/Lightly Peated, All Whisky/Whisky by Age/21 Years, All Whisky/Cask Type/Other Fortified Wines & Wine Casks, All Whisky/Whisky/Spirit of the Season [All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Cask Type, All Whisky/Whisky/Outturn, All Whisky/Whisky, Whisky/Outturn/Mid-Month Releases, All Whisky/Whisky by Region/Highland, All Whisky/Whisky/Vaults Collection, All Whisky/Flavour Profiles/Lightly Peated, All Whisky/Whisky by Age/21 Years, All Whisky/Cask Type/Other Fortified Wines & Wine Casks, All Whisky/Whisky/Spirit of the Season]
1 2826 053389GB0700630 Goodnight and joy /goodnight-and-joy/ 53.389 Islay 30.0 55.5% refill ex-sherry butt Malt Whisky None 495.00 15 2022-07-25 08:00:00 21/6/1991 https://cdn11.bigcommerce.com/s-vagfena5nz/products/2826/images/4416/53.389-web__29542.1658745823.386.513.png?c=1 All Whisky/Whisky by Age/21 Years, All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Peated, All Whisky/Whisky by Region, All Whisky/Whisky by Region/Islay, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Cask Type/Sherry Casks, No Menu Link/External Tastings, No Menu Link/External Tastings/Sept22 Tasting Bottles, All Whisky/Cask Type, All Whisky/Whisky/Last Drops [All Whisky/Whisky by Age/21 Years, All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Peated, All Whisky/Whisky by Region, All Whisky/Whisky by Region/Islay, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Cask Type/Sherry Casks, No Menu Link/External Tastings, No Menu Link/External Tastings/Sept22 Tasting Bottles, All Whisky/Cask Type, All Whisky/Whisky/Last Drops]
2 1124 095039GB0700638 Indian summer in a Japanese garden /indian-summer-in-a-japanese-garden/ 95.39 Speyside 38.0 53.0% Refill ex-sherry butt Malt Whisky None 435.00 35 2021-06-17 13:00:00 15/1/1982 https://cdn11.bigcommerce.com/s-vagfena5nz/products/1124/images/1770/95.39-VC-box-web__23227.1623844189.386.513.png?c=1 All Whisky/Whisky by Region/Speyside, All Whisky/Whisky by Age/21 Years, All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Old & Dignified, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Cask Type/Sherry Casks, All Whisky/Cask Type [All Whisky/Whisky by Region/Speyside, All Whisky/Whisky by Age/21 Years, All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Old & Dignified, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Cask Type/Sherry Casks, All Whisky/Cask Type]
3 2111 012064GB0700631 Oh-so-orange /oh-so-orange/ 12.64 Speyside 31.0 43.4% 2nd fill ex-bourbon barrel Malt Whisky None 434.65 11 2022-02-11 09:00:00 7/5/1990 https://cdn11.bigcommerce.com/s-vagfena5nz/products/2111/images/3102/12.64-VC-box-hires__68013.1657010779.386.513.png?c=1 All Whisky/Whisky by Region/Speyside, All Whisky/Whisky by Age/21 Years, Categories Removed, All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Sweet Fruity & Mellow, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Cask Type/Bourbon Casks, All Whisky/Cask Type, Last Drops, All Whisky/Whisky/Last Drops [All Whisky/Whisky by Region/Speyside, All Whisky/Whisky by Age/21 Years, Categories Removed, All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Sweet Fruity & Mellow, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Cask Type/Bourbon Casks, All Whisky/Cask Type, Last Drops, All Whisky/Whisky/Last Drops]
4 2897 012062GB0700631 Heaven-sent /heaven-sent/ 12.62 Speyside 31.0 57.9% 1st fill ex-Pedro Ximenes butt Malt Whisky None 385.00 5 2022-08-22 11:00:00 7/9/1989 https://cdn11.bigcommerce.com/s-vagfena5nz/products/2897/images/4563/12.62-VC-box-web_1__60925.1660742340.386.513.png?c=1 All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Deep Rich & Dried Fruits, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Whisky by Region/Speyside, All Whisky/Whisky by Age/21 Years, All Whisky/Cask Type/Sherry Casks, All Whisky/Cask Type, All Whisky/Whisky/Last Drops [All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Deep Rich & Dried Fruits, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Whisky/Vaults Collection, All Whisky/Whisky by Region/Speyside, All Whisky/Whisky by Age/21 Years, All Whisky/Cask Type/Sherry Casks, All Whisky/Cask Type, All Whisky/Whisky/Last Drops]
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
94 2898 006058GB0700608 Red swirl lollipop in a sidecar /red-swirl-lollipop-in-a-sidecar/ 6.58 Speyside 8.0 65.5% 2nd fill ex-bourbon barrel Malt Whisky None 49.80 14 2022-08-19 08:00:00 18/9/2013 https://cdn11.bigcommerce.com/s-vagfena5nz/products/2898/images/4565/6.58-web__01010.1660745386.386.513.png?c=1 All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Spicy & Sweet, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky by Age/Under 12 Years, All Whisky/Whisky by Region/Speyside, All Whisky/Whisky/Whisky Best Buys, All Whisky/Cask Type/Bourbon Casks, All Whisky/Cask Type [All Whisky/Flavour Profiles, All Whisky/Flavour Profiles/Spicy & Sweet, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky by Age/Under 12 Years, All Whisky/Whisky by Region/Speyside, All Whisky/Whisky/Whisky Best Buys, All Whisky/Cask Type/Bourbon Casks, All Whisky/Cask Type]
95 3025 GN6002GB0700601 Old timey sippin' gin! /old-timey-sippin-gin/ GN6.2 Edinburgh 1.0 49.3% 1st fill ex-bourbon barrel Malt Whisky None 45.75 48 2022-10-07 08:00:00 10/9/2020 https://cdn11.bigcommerce.com/s-vagfena5nz/products/3025/images/4741/GN6.2-web__93878.1664901852.386.513.png?c=1 All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky by Age/Under 12 Years, All Whisky/Cask Type, All Whisky/Cask Type/Bourbon Casks, All Whisky/Spirits, All Whisky/Spirits/Gin, All Whisky/Whisky, All Whisky/Whisky/Outturn [All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Whisky by Age/Under 12 Years, All Whisky/Cask Type, All Whisky/Cask Type/Bourbon Casks, All Whisky/Spirits, All Whisky/Spirits/Gin, All Whisky/Whisky, All Whisky/Whisky/Outturn]
96 942 048111GB0700607 Baking in the potting shed /baking-in-the-potting-shed/ 48.111 Speyside 9.0 61.8% 1st fill ex-bourbon barrel None None 45.60 24 2022-10-07 08:00:00 7/11/2011 https://cdn11.bigcommerce.com/s-vagfena5nz/products/942/images/1391/48.111-bc__44272.1617989677.386.513.png?c=1 All Whisky/Whisky by Age/Under 12 Years, All Whisky/Whisky by Region/Speyside, All Whisky/Flavour Profiles/Spicy & Sweet, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Cask Type/Bourbon Casks, All Whisky/Cask Type, All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky/Whisky, All Whisky/Whisky/Outturn [All Whisky/Whisky by Age/Under 12 Years, All Whisky/Whisky by Region/Speyside, All Whisky/Flavour Profiles/Spicy & Sweet, All Whisky, All Whisky/Single Cask Whisky, All Whisky/Cask Type/Bourbon Casks, All Whisky/Cask Type, All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky/Whisky, All Whisky/Whisky/Outturn]
97 345 SM0001GX0700607 Lemon odyssey /lemon-odyssey SM0.1 Blended Malt 7.0 50.0% 1st & 2nd fill ex-bourbon barrels Malt Whisky None 45.00 50 None 11/6/2012 https://cdn11.bigcommerce.com/s-vagfena5nz/products/345/images/3979/Lemon-Odyssey-web__34434.1651854289.386.513.png?c=1 All Whisky/Whisky by Age/Under 12 Years, All Whisky/Whisky/Whisky Highlights, All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Region/International, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky [All Whisky/Whisky by Age/Under 12 Years, All Whisky/Whisky/Whisky Highlights, All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Region/International, All Whisky/Whisky by Age, All Whisky, All Whisky/Single Cask Whisky]
98 2904 DISC-VMT-GATH22-GB Christmas Selection Box /Gathering-Discovery-Pack-2022 None None NaN None None None Pick up a perfect present for the whisky lover in your life or get in the seasonal spirit yourself – either with a spot of self-exploration or by tuning into our online tasting, where our ambassador will guide you through the selection. 30.00 107 2022-08-22 08:00 None https://cdn11.bigcommerce.com/s-vagfena5nz/products/2904/images/4578/TP_May_2020_Festival_01s__44827.1662024217.386.513.jpg?c=1 All Whisky/Whisky/Whisky Best Buys, All Whisky/Whisky/Outturn, All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Whisky/Cases & Tasting Packs, All Whisky/Whisky/Whisky Gifts for all Occasions, All Whisky/Whisky/Spirit of the Season [All Whisky/Whisky/Whisky Best Buys, All Whisky/Whisky/Outturn, All Whisky/Flavour Profiles, All Whisky/Whisky by Region, All Whisky/Whisky by Age, All Whisky, All Whisky/Whisky/Cases & Tasting Packs, All Whisky/Whisky/Whisky Gifts for all Occasions, All Whisky/Whisky/Spirit of the Season]
99 rows × 18 columns
???
在此處查看熊貓檔案：https ://pandas.pydata.org/docs/
此外，請求檔案：https ://requests.readthedocs.io/en/latest/
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/518197.html