我是第一次進行網路抓取并遇到了問題。我必須獲取某些產品的產品價格(代碼中的 url),但是,當產品有折扣時,它會出錯。這是我現在擁有的代碼(洗掉了中間的幾行,但它的作業原理是這樣的):
import requests
from bs4 import BeautifulSoup
#import csv
#import pandas as pd
links = []
url='https://www.ah.nl/producten/pasta-rijst-en-wereldkeuken?page={page}'
for page in range(1,2):
req = requests.get(url.format(page=page))
soup = BeautifulSoup(req.content, 'html.parser')
for link in soup.select('div[] a'):
abs_url = 'https://www.ah.nl' link.get('href')
#print(abs_url)
#GETTING THE PRICE
p_price = []
req4 = requests.get(abs_url)
soup = BeautifulSoup(req4.content, 'html.parser')
p_price = (soup.find(class_='price-amount_root__37xv2 product-card-hero-price_now__PlF9u')).text
輸出是這樣的:
runcell(0, '/Users/eva/Desktop/MDDD/TestingProduct.py')
0.55
0.35
2.99
0.65
Traceback (most recent call last):
File "/Applications/Spyder.app/Contents/Resources/lib/python3.9/spyder_kernels/py3compat.py", line 356, in compat_exec
exec(code, globals, locals)
File "/Users/eva/Desktop/MDDD/TestingProduct.py", line 22, in <module>
p_price = (soup.find(class_='price-amount_root__37xv2 product-card-hero-price_now__PlF9u')).text
AttributeError: 'NoneType' object has no attribute 'text'
所以它給了我前四個價格,然后是錯誤。我試圖解決它添加這個:
if p_price != AttributeError:continue
但這沒有用。我不介意有折扣的產品不在資料集中。關于如何保持 for 回圈繼續進行的任何提示 - 所以洗掉給出錯誤的價格?
謝謝!
uj5u.com熱心網友回復:
您可以從結果頁面獲取價格,而無需訪問每個單獨的鏈接。此外,通過明智地使用 css :not() 偽類選擇器,您可以排除出現折扣的舊價格,從而消除錯誤:
import requests
from bs4 import BeautifulSoup
url='https://www.ah.nl/producten/pasta-rijst-en-wereldkeuken?page={page}'
for page in range(1, 3):
print(f"Page: {page}")
print()
req = requests.get(url.format(page=page))
soup = BeautifulSoup(req.content, 'html.parser')
for product in soup.select('[data-testhook="product-card"]'):
print(product.select_one('[data-testhook="product-title"]').get_text(strip=True))
print(product.select_one('[data-testhook="price-amount"]:not([class*=price_was])').text)
print()
uj5u.com熱心網友回復:
您收到 NoneType 錯誤,因為所有專案都沒有包含價格并且要擺脫此錯誤,您可以使用if else None statement
import requests
from bs4 import BeautifulSoup
#import csv
#import pandas as pd
links = []
url='https://www.ah.nl/producten/pasta-rijst-en-wereldkeuken?page={page}'
for page in range(1,2):
req = requests.get(url.format(page=page))
soup = BeautifulSoup(req.content, 'html.parser')
for link in soup.select('div[] a'):
abs_url = 'https://www.ah.nl' link.get('href')
#print(abs_url)
#GETTING THE PRICE
p_price = []
req4 = requests.get(abs_url)
soup = BeautifulSoup(req4.content, 'html.parser')
p_price = (soup.find(class_='price-amount_root__37xv2 product-card-hero-price_now__PlF9u'))
p_price = p_price.text if p_price else None
print(p_price)
輸出:
0.55
0.35
2.99
0.65
None
0.49
0.65
0.59
0.92
0.99
0.79
0.55
0.65
2.19
3.00
2.00
0.89
0.89
0.66
0.89
0.89
1.25
1.99
1.19
0.99
0.79
1.79
1.99
1.79
6.29
1.19
1.39
2.19
0.65
1.95
0.79
None
None
None
None
2.00
2.29
0.49
1.29
1.55
1.59
1.39
2.99
2.00
0.99
1.39
1.65
1.19
0.99
0.99
2.29
1.99
2.69
0.49
0.99
0.79
2.19
2.00
3.69
0.89
2.29
0.45
1.85
2.00
2.00
5.99
1.09
2.79
1.19
3.29
0.95
轉載請註明出處,本文鏈接:https://www.uj5u.com/caozuo/483247.html
上一篇:如何用R報廢表類名?
