使用python對多個標簽進行Web抓取-有解無憂

嗨，我正在使用 python 進行網路抓取。這是我的代碼：

from bs4 import BeautifulSoup
import requests

page = requests.get(
    'https://www.indeed.com/viewjob?jk=78fc5cc6a9d2aaa3&q=developer&l=Hammond, LA&tk=1g3udv32opki1801&from=web&advn=2300444857198541&adid=371529140&ad=-6NYlbfkN0C3HlOxE-u7vDWDmHVgHclVijSpnbvDTTioTnwCLVe0OEwH_1p9qQb-3snK62Gml60thtHyOlr-diC2sIty8supkOLuy2apQt4gi355WXBpDDHQbuCkuMyYIfjito5_MzRa3sg8VkVKd5pvUD9rUt1RWPXpPzu2chM4oyLuN4riMCIsCh8gpIyWcPu7RV4Xt1Zp8PdeRuChYB95XZ0TM5bOYVexvf3lCdm4d3RG2TNPX5iZvX0mlZBhUQ2kufKY6TKI_2UZvTMgDAYwVjtFnB0qxEJi9aMmmp2GHECMAyifjTOAZkTUQnyIjUK_mFI7R7siYE6sIQSqPTt0pfEfvT4U-dfQpsmzdA1D0ZYdO-igFhm2rrEIwalOqCYEFwd3_cTBVkXzQBiiVA==&pub=4a1b367933fd867b19b072952f68dceb&vjs=3').text
soup = BeautifulSoup(page, 'lxml')
jobs = soup.find(
    'div', class_='jobsearch-JobComponent-description icl-u-xs-mt--md')
    
    job_desc = jobs.find('p').text.replace('', '')
    print(f"job description:{job_desc}")

在上面的代碼中，我使用了 BeautifulSoup。我還能夠獲得職位描述。但我的問題是我只得到一個行句子，因為div標簽內部job discription有更多p標簽，我只能列印第一個標簽。正如我在圖片中所附的那樣，我怎樣才能將所有職位描述div作為段落？[示例圖片頁面][1] 我也嘗試使用 for 回圈：

job_desc = jobs.find_all('p')
for desc in job_desc:
    job_de = desc.find('p')
    print(f"job description:{job_de}")

作為回應，我能夠得到

job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None
job description:None

here is the link I am trying to scripe: https://www.indeed.com/viewjob?jk=78fc5cc6a9d2aaa3&q=developer&l=Hammond, LA&tk=1g3udv32opki1801&from=web&advn=2300444857198541&adid=371529140&ad=-6NYlbfkN0C3HlOxE-u7vDWDmHVgHclVijSpnbvDTTioTnwCLVe0OEwH_1p9qQb-3snK62Gml60thtHyOlr-diC2sIty8supkOLuy2apQt4gi355WXBpDDHQbuCkuMyYIfjito5_MzRa3sg8VkVKd5pvUD9rUt1RWPXpPzu2chM4oyLuN4riMCIsCh8gpIyWcPu7RV4Xt1Zp8PdeRuChYB95XZ0TM5bOYVexvf3lCdm4d3RG2TNPX5iZvX0mlZBhUQ2kufKY6TKI_2UZvTMgDAYwVjtFnB0qxEJi9aMmmp2GHECMAyifjTOAZkTUQnyIjUK_mFI7R7siYE6sIQSqPTt0pfEfvT4U- dfQpsmzdA1D0ZYdO-igFhm2rrEIwalOqCYEFwd3_cTBVkXzQBiiVA==&pub=4a1b367933fd867b19b072952f68dceb&vjs=3 [1]：https ://i.stack.imgur.com/Dsm8w.png

uj5u.com熱心網友回復：

嘗試這個：

from bs4 import BeautifulSoup
import requests


soup = BeautifulSoup(page, 'lxml')
job_desc = soup.find(
    'div', 
    class_='jobsearch-JobComponent-description icl-u-xs-mt--md'
).get_text()
    
print(f"job description:{job_desc}")

如果您只想要檔案或標簽中的人類可讀文本，您可以使用 get_text() 方法。它以單個 Unicode 字串的形式回傳檔案中或標簽下的所有文本：

美麗的湯檔案中 get_text() 的鏈接：https ://www.crummy.com/software/BeautifulSoup/bs4/doc/#get-text

轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/483241.html

標籤：Python 网页抓取美丽的汤

上一篇：為什么selenium中的命令會單獨作業，但如果放入腳本selenium不會找到元素

下一篇：刮痧問題。未檢索到的資料