我制作了它正在讀取的代碼，UpdatedUrls.tmp但是這個檔案只包含一個 url。該代碼執行諸如廢舊網站尋找電子郵件之類的操作，但如果我在此檔案中放置 2 個或更多地址，則該代碼不起作用。為emails.txt空。
我需要放置一個回圈或更改某些內容以從 url 獲取兩封或多封電子郵件。
內容UpdatedUrls.tmp：
https://mobissom.com.br/contato/
我需要與：
https://mobissom.com.br/contato/
https://www.site2.com
https://www.site3.com
代碼在這里：
import re
import requests
from urllib.parse import urlsplit
from collections import deque
from bs4 import BeautifulSoup
import pandas as pd
with open("updatedUrls.tmp", "r") as smails:
original_url = smails.readlines()
original_url = ''.join(original_url)
# to save urls to be scraped
unscraped = deque([original_url])
# to save scraped urls
scraped = set()
# to save fetched emails
emails = set()
while len(unscraped):
# move unsraped_url to scraped_urls set
url = unscraped.popleft() # popleft(): Remove and return an element from the left side of the deque
scraped.add(url)
parts = urlsplit(url)
base_url = "{0.scheme}://{0.netloc}".format(parts)
if '/' in parts.path:
path = url[:url.rfind('/') 1]
else:
path = url
print("Crawling URL %s" % url)
try:
response = requests.get(url)
except (requests.exceptions.MissingSchema, requests.exceptions.ConnectionError):
continue
new_emails = set(re.findall(r"[a-z0-9\.\- _] @[a-z0-9\.\- _] \.com", response.text, re.I))
emails.update(new_emails)
soup = BeautifulSoup(response.text, 'lxml')
for anchor in soup.find_all("a"):
if "href" in anchor.attrs:
link = anchor.attrs["href"]
else:
link = ''
if link.startswith('/'):
link = base_url link
elif not link.startswith('http'):
link = path link
if not link.endswith(".gz"):
if not link in unscraped and not link in scraped:
unscraped.append(link)
df = pd.DataFrame(emails, columns=None)
df.to_csv('email.txt', index=False)
with open('email.txt', 'r') as fin:
data = fin.read().splitlines(True)
with open('email.txt', 'w') as fout:
fout.writelines(data[1:])
uj5u.com熱心網友回復：
您正在將檔案的所有內容original_url作為單個字串讀取，因為您在拆分它們后立即加入這些行。
改變
with open("updatedUrls.tmp", "r") as smails:
original_url = smails.readlines()
original_url = ''.join(original_url) # This joins all lines into one string
unscraped = deque([original_url]) # original_url is a string
# unscraped = deque(['url1\nurl2\nurl3'])
進入
with open("updatedUrls.tmp", "r") as smails:
# Gets rid of trailing newlines
original_url = smails.read().splitlines()
unscraped = deque(original_url) # original_url is a list of URLs
# unscraped = deque(['url1', 'url2', 'url3'])
轉載請註明出處，本文鏈接：https://www.uj5u.com/ruanti/438468.html