我對網路抓取很陌生,遇到了一個我無法解決的奇怪問題。我想從歐盟網站(用于抓取的歐盟網站)上抓取一般資訊。我想用 BeautifulSoup 提取有關開放和關閉日期等的資訊。
問題是當我通過 InspectElement 比較 HTML 代碼時,我能夠顯示 span-tags 和 div-tags。
是否可以繞過“app-root”和“script-elements”并將它們轉換為我可以從中決議資訊的常規html代碼?
#URL
eu_url = ""
#Opening connection grabbing the page
uClient = uReq(eu_url)
#Read page and save as a variable
page = uClient.read()
#Close connection
uClient.close()
#HTML Parsing with BeautifulSoup
page_soup = soup(page, "html.parser").prettify()
這輸出:
<body>
<app-root>
<div id="loader-wrapper">
<div id="loader">
</div>
</div>
</app-root>
<!-- Sitemap Generator -->
<!-- <script type="text/javascript">
var _0xaea9 = ["sitemapgenerator:: ", "innerHTML", "body", "document", "stringify", "*", "postMessage", "setTimeout"]; window[_0xaea9[7]](function () { parent[_0xaea9[6]](_0xaea9[0] JSON[_0xaea9[4]](window[_0xaea9[3]][_0xaea9[2]][_0xaea9[1]]), _0xaea9[5]) }, 3000);
</script> -->
<noscript>
<!-- to the list of topics and faqs -->
<a href="/info/funding-tenders/opportunities/data/topic-list.html">
Topic List
</a>
</noscript>
<script src="runtime.626faac5e81de98cbf62.js" type="text/javascript">
</script>
<script src="polyfills.7351ea247f50946be3ed.js" type="text/javascript">
</script>
<script src="scripts.89d4984a80cecb9fb6d4.js" type="text/javascript">
</script>
<script src="main.4ceb2dd3c58718a9975b.js" type="text/javascript">
</script>
</body>
該腳本應該打開整個代碼,看起來像這樣: 檢查元素代碼
有人對此有解決方案嗎?
uj5u.com熱心網友回復:
您可以查詢一個 API 來獲取所需的資料。
嘗試這個:
import requests
api_url = "https://ec.europa.eu/info/funding-tenders/opportunities/data/topicDetails/erasmus-edu-2022-net-edu-migrants.json?lang=en"
response = requests.get(api_url).json()["TopicDetails"]["actions"][0]
print(f"{response['plannedOpeningDate']} - {response['deadlineDates'][0]}")
輸出:
07 June 2022 - 12 October 2022
轉載請註明出處,本文鏈接:https://www.uj5u.com/qukuanlian/508449.html
