import requests
from bs4 import BeautifulSoup
url = 'https://www.basketball-reference.com/players/a/'
urlb = 'https://www.basketball-reference.com/players/b/'
urlc = 'https://www.basketball-reference.com/players/c/'
result = requests.get(url)
doc = BeautifulSoup(result.text, 'lxml')
college = doc.find_all(string="Kentucky")
result = requests.get(urlb)
doc = BeautifulSoup(result.text, 'lxml')
collegeb = doc.find_all(string='Kentucky')
result = requests.get(urlc)
doc = BeautifulSoup(result.text, 'lxml')
collegec = doc.find_all(string='Kentucky')
print(college)
print(collegeb)
print(collegec)
我需要為至少 30 所學校的每個字母做這個，我真的很想知道如何更有效地做到這一點
uj5u.com熱心網友回復：
使用回圈輸入和 a listor dictof 結果對幾乎相同的代碼進行去重：
import requests
from bs4 import BeautifulSoup
url_template = 'https://www.basketball-reference.com/players/{}/'
folders = ['a', 'b', 'c'] # The only varying thing in your original tripled code
colleges = [] # Store the results for each varied thing here in same order
for folder in folders: # Loop over varying component
result = requests.get(url_template.format(folder)) # Substitute it in template
doc = BeautifulSoup(result.text, 'lxml')
colleges.append(doc.find_all(string="Kentucky")) # Append result in same order
# Loop over results to print them
for college in colleges:
print(college)
如果您讓它適用于許多學校，對于字母表中的每個字母，您可能會使用 a dict(better, a defaultdict) 作為結果而不是 a list（因此您可以按學校對結果進行分組）和內部回圈決議出校資料：
import requests
from bs4 import BeautifulSoup
from collections import defaultdict
from string import ascii_lowercase
url_template = 'https://www.basketball-reference.com/players/{}/'
folders = ascii_lowercase # Will run for every lowercase alphabet letter
schoolnames = ("Kentucky", "Gonzaga", ...)
colleges = defaultdict(list) # Store a list of results for each school
for folder in folders: # Loop over varying component
result = requests.get(url_template.format(folder)) # Substitute it in template
doc = BeautifulSoup(result.text, 'lxml')
for schoolname in schoolnames:
colleges[schoolname].append(doc.find_all(schoolname=school))
# Loop over results to print them
for collegename, results in colleges.items():
print(collegename)
for result in results:
print(result)
uj5u.com熱心網友回復：
這是一個稍微簡單的代碼來做到這一點。我所做的只是拉入所有玩家表，然后.value_counts()在'Colleges'列上使用。這將使您獲得所有學校。然后，如果您只想查看一所學校，只需呼叫該索引值即可：
import pandas as pd
from string import ascii_lowercase
dfs_list = []
for letter in ascii_lowercase:
url = f'https://www.basketball-reference.com/players/{letter}/'
dfs_list.append(pd.read_html(url)[0])
print(url)
results = pd.concat(dfs_list, axis=0)
colleges_count = results['Colleges'].value_counts()
您甚至可以使用串列理解用更少的代碼行來轉換它：
import pandas as pd
from string import ascii_lowercase
results = pd.concat([pd.read_html(f'https://www.basketball-reference.com/players/{letter}/')[0] for letter in ascii_lowercase], axis=0)
colleges_count = results['Colleges'].value_counts()
輸出：
print(colleges_count)
Kentucky 112
UCLA 91
UNC 91
Duke 84
Kansas 72
Kansas, Houston 1
California Western Uiversity 1
Florida, Louisiana 1
NC State, Iona College 1
Seattle Pacific University, Washington 1
Name: Colleges, Length: 806, dtype: int64
或者只看某所學校：
print(colleges_count['Kentucky'])
112
uj5u.com熱心網友回復：
你可以只使用一個for回圈。
import requests
from bs4 import BeautifulSoup
colleges = []
for char in "abcdefghijklmnopqrstuvwxyz":
url = f"https://www.basketball-reference.com/players/{char}/"
result = requests.get(url)
doc = BeautifulSoup(result.text, 'lxml')
college = doc.find_all(string="Kentucky")
colleges.append(college)
print(*colleges, sep = "\n")
uj5u.com熱心網友回復：
您可以撰寫一個函式，其中包含您需要為每所學校“重做”的指令。然后，您為每所學校部署一個main()具有每個引數或特征/特征的函式。您的代碼似乎是一大塊行，您應該將它們分開在不同的指令中，并更多地依靠“整潔的編碼”來完成您的作業
轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/496239.html
標籤：python-3.x 网页抓取 代码清理