除了最后的 id 之外，我有一堆完全相同的鏈接。我要做的就是遍歷每個鏈接，然后使用下載為 PDF 按鈕將論文下載為 PDF。在理想情況下，檔案名將是論文的標題，但如果這不可能，我可以稍后重命名它們。讓它們全部下載更重要。我有 200 個鏈接，但我將在這里提供 5 個作為示例。
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3860262
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2521007
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3146924
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2488552
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3330134
我想做的事可能嗎？我對通過 URL 回圈來抓取表格有一定的了解，但我從未嘗試過使用下載按鈕做任何事情。
我沒有示例代碼，因為我不知道從哪里開始。但是像
for url in urls:
(go to each link)
(download as pdf via the "download this paper" button)
(save file as title of paper)
uj5u.com熱心網友回復：
嘗試：
import requests
from bs4 import BeautifulSoup
urls = [
"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3860262",
"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2521007",
"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3146924",
"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2488552",
"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3330134",
]
headers = {
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:100.0) Gecko/20100101 Firefox/100.0"
}
for url in urls:
soup = BeautifulSoup(
requests.get(url, headers=headers).content, "html.parser"
)
pdf_url = (
"https://papers.ssrn.com/sol3/"
soup.select_one("a[data-abstract-id]")["href"]
)
filename = url.split("=")[-1] ".pdf"
print(f"Downloading {pdf_url} as {filename}")
with open(filename, "wb") as f_out:
f_out.write(
requests.get(pdf_url, headers={**headers, "Referer": url}).content
)
印刷：
Downloading https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3860262_code1719241.pdf?abstractid=3860262&mirid=1 as 3860262.pdf
Downloading https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2521007_code576529.pdf?abstractid=2521007&mirid=1 as 2521007.pdf
Downloading https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID4066577_code104690.pdf?abstractid=3146924&mirid=1 as 3146924.pdf
Downloading https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2505208_code16198.pdf?abstractid=2488552&mirid=1 as 2488552.pdf
Downloading https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3506882_code16198.pdf?abstractid=3330134&mirid=1 as 3330134.pdf
并將 PDF 保存為：
andrej@PC:~$ ls -alF *pdf
-rw-r--r-- 1 root root 993466 máj 24 01:10 2488552.pdf
-rw-r--r-- 1 root root 3583616 máj 24 01:10 2521007.pdf
-rw-r--r-- 1 root root 1938284 máj 24 01:10 3146924.pdf
-rw-r--r-- 1 root root 685777 máj 24 01:10 3330134.pdf
-rw-r--r-- 1 root root 939157 máj 24 01:10 3860262.pdf
轉載請註明出處，本文鏈接：https://www.uj5u.com/net/480822.html
上一篇：cython加速3d串列操作