所以我是一個 python 初學者，試圖抓取這個網站http://www.edwaittimes.ca/WaitTimes.aspx ，它給出了醫院的等待時間。目前我正在嘗試列印所有醫院的名稱。
如果 .html 檔案位于我正在使用的 python 檔案的檔案夾中，我的代碼就可以作業
from bs4 import BeautifulSoup
import requests
def print_hospitals():
with open('website.html','r') as html_file:
content = html_file.read()
soup = BeautifulSoup(content, 'lxml')
hospital_table = soup.find_all('div',class_="Row")
for hospital in hospital_table:
if hospital.a is not None:
print(hospital.a.text)
但是當我將 requests.get 與 URL 一起使用時。該代碼不列印任何內容。也沒有錯誤訊息。
from bs4 import BeautifulSoup
import requests
def print_hospitals_request():
html_text = requests.get('http://www.edwaittimes.ca/WaitTimes.aspx').text
soup = BeautifulSoup(html_text, 'lxml')
hospital_table = soup.find_all('div',class_="Row")
for hospital in hospital_table:
if hospital.a is not None:
print(hospital.a.text)
誰能幫我解決這個問題
uj5u.com熱心網友回復：
該頁面正在使用 Ajax 從外部 URL 加載資料。所以beautifulsoup什么都看不到。要加載資料，您可以使用下一個示例：
import requests
from bs4 import BeautifulSoup
hospitals_csv = "http://www.edwaittimes.ca/Shared/Images/sites2.csv"
data = [
l.split("|")[:-1]
for l in requests.get(hospitals_csv).text.splitlines()[:-1]
]
all_data = ""
for hospital, city in data:
url = (
"http://www.edwaittimes.ca/Shared/Images/"
hospital
(".html" if city == "Vancouver" else "_gp.html")
)
print(f"Getting {url}")
all_data = requests.get(url).text
soup = BeautifulSoup(all_data, "html.parser")
for row in soup.select(".Row"):
print(row.get_text(strip=True, separator=" "))
印刷：
Lions Gate Hospital Patients of all ages seen 02:28 05:06
North Van Urgent & Primary Care Centre Patients of all ages seen UPCC is for mild to moderate illness 01:38 04:15
Squamish General Hospital Patients of all ages seen 01:39 02:16
Whistler Health Care Centre Patients of all ages seen 00:43 01:52
Pemberton Health Centre Patients of all ages seen No patients seen in the last two hours. 02:05
Sechelt Hospital Patients of all ages seen 02:08 04:52
Richmond Hospital Patients of all ages seen 02:36 05:16
Richmond Urgent and Primary Care Centre Patients of all ages seen (lab offsite) UPCC is for mild to moderate illness 03:46 03:28
Vancouver General Hospital Patients of ages 17 and older seen 02:18 05:40
St. Paul's Hospital Patients of all ages seen 00:34 04:26
Mount Saint Joseph Hospital Patients of all ages seen 01:01 02:35
UBC Hospital (UBCH) Patients of all ages seen UBCH is for mild to moderate illness 01:22 01:22
City Centre Urgent & Primary Care Centre Patients of all ages seen UPCC is for mild to moderate illness 01:49 02:30
REACH Urgent and Primary Care Centre Patients of all ages seen (lab & x-ray offsite) UPCC is for mild to moderate illness Currently open, call (604) 216-3138 for wait time
Northeast Urgent and Primary Care Centre Patients of all ages seen (lab & x-ray offsite) UPCC is for mild to moderate illness 02:50 02:50
Southeast Urgent and Primary Care Centre Patients of all ages seen (lab & x-ray offsite) UPCC is for mild to moderate illness 02:12 01:52
BC Children's Hospital Patients seen up to age 16 02:23 04:39
uj5u.com熱心網友回復：
您正在抓取的網頁上似乎不存在您要查找的課程。我class_="Row"用class_="grid_8"which 替換了網頁上存在的一個類，它起作用了：
from bs4 import BeautifulSoup
import requests
def print_hospitals_request():
html_text = requests.get('http://www.edwaittimes.ca/WaitTimes.aspx').text
soup = BeautifulSoup(html_text, 'lxml')
hospital_table = soup.find_all('div', class_="grid_8")
for hospital in hospital_table:
if hospital.a is not None:
print(hospital.a.text)
print_hospitals_request()
uj5u.com熱心網友回復：
美麗的湯和請求作業正常。你在理論上所做的作業。事情就是這樣，您正在閱讀的 html 是站點本身發出另一個請求然后基于該請求填充表的結果。如果您進入并使用瀏覽器上的開發人員工具，您將看到一個具有特定操作的表單元素。我的猜測是 get 請求會填充用戶看到的初始 html，然后表單請求和一些 javascript 從服務器獲取資料。
沒有錯誤，因為那是 get 請求的結果。我不確定向該表單呼叫發布請求會做什么，并且我不確定該網站的使用條款或條件。
假設您確實有權使用該 API，這不僅僅是閑散的好奇心。你可以走兩條路線之一。您可以嘗試使用 get 而不是 post 來模擬頁面發出的請求。另一種是使用selenium（通過python系結或其他方法）打開瀏覽器，呼叫等待某些元素存在或發生超時，然后使用selenium而不是bs4來抓取頁面。
如果這是為了練習，我在 wikipedia 上使用了 bs4，這是一個很好的開放內容來源，其中包括大量表格并將其全部發送為原始 html。
uj5u.com熱心網友回復：
網址是動態的。所以你可以在 bs4 中使用 selenium
import time
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get('http://www.edwaittimes.ca/WaitTimes.aspx')
time.sleep(2)
soup = BeautifulSoup(driver.page_source,'lxml')
hospital_table = soup.find_all('div',class_="Row")
for hospital in hospital_table:
if hospital.a is not None:
print(hospital.a.text)
輸出
Mount Saint Joseph Hospital
UBC Hospital (UBCH)
City Centre Urgent & Primary Care Centre
Vancouver General Hospital
St. Paul's Hospital
REACH Urgent and Primary Care Centre
Northeast Urgent and Primary Care Centre
Southeast Urgent and Primary Care Centre
BC Children's Hospital
Richmond Hospital
Richmond Urgent and Primary Care Centre
Squamish General Hospital
Whistler Health Care Centre
Lions Gate Hospital
Sechelt Hospital
Pemberton Health Centre
North Van Urgent & Primary Care Centre
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/478744.html
標籤：Python 网页抓取 美丽的汤 python-请求-html 网络抓取语言