我是 Python 和編程的新手,我在網站決議專案中遇到了問題。
這是我設法撰寫的代碼:
import requests
from bs4 import BeautifulSoup
import pandas as pd
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', None)
pd.set_option('display.max_colwidth', -1)
import json
#necessary lists
url_list = [
"https://warframe.market/items/melee_riven_mod_(veiled)",
"https://warframe.market/items/zaw_riven_mod_(veiled)"
]
item_list = []
items_name = []
combined_data = []
iteration = 0
#looping for every url found in url_list
for url in url_list:
#requesting data
r = requests.get(url)
soup = BeautifulSoup(r.content, "html.parser")
#splitting the last part of the url which has the name of the item that I want to insert in the dataframe
name = url.split("/")[4]
items_name.append(name)
#Finding in the parsed HTML code where the JSON file starts ( it start from <script> n°2)
results = soup.find_all('script')[2].text.strip()
data = json.loads(results)
combined_data.append(data) #combining all the data into one list
#filtering only the users who sell the items and are either "ingame" or "online"
for payload in combined_data[iteration]["payload"]["orders"]:
if payload["order_type"] == "sell" and (payload["user"]["status"] == "online" or payload["user"]["status"] == "ingame"):
p = payload
item_list.append(p)
#adding the items names to the item list ???? PROBLEM ?????
item_list = [dict(item, **{'name':items_name[iteration]}) for item in item_list]
#trying to change the list from where the data gets taken from and the items name ????? PROBLEM ????
iteration = 1
#creating a dataframe with all the values
df = pd.DataFrame(item_list).sort_values(by=["platinum"])
我正在嘗試做但找不到解決方案的是將 url 所指專案的名稱添加到 item_list 中。
例如
| 指數 | 鉑 | 數量 | ... | 專案名稱(問題列) |
|---|---|---|---|---|
| 1 | 10 | 1 | ... | melee_riven_mod_(蒙面) |
| 2 | 11 | 1 | ... | melee_riven_mod_(蒙面) |
| 3 | 12 | 2 | ... | zaw_riven_mod_(蒙面) |
| 4 | ... | ... | ... | zaw_riven_mod_(蒙面) |
但是 items name 列對于所有行都具有相同的名稱,如下所示:
| 指數 | 鉑 | 數量 | ... | 專案名稱(問題列) |
|---|---|---|---|---|
| 1 | 10 | 1 | ... | melee_riven_mod_(蒙面) |
| 2 | 11 | 1 | ... | melee_riven_mod_(蒙面) |
| 3 | 12 | 2 | ... | melee_riven_mod_(蒙面) |
| 4 | ... | ... | ... | melee_riven_mod_(蒙面) |
所以我想問一下我在for回圈中做錯了什么?它迭代 2 次,這是 url 中的數量,url_list但它不會更改專案的名稱。我沒有看到什么?
uj5u.com熱心網友回復:
改變
if payload["order_type"] == "sell" and (payload["user"]["status"] == "online" or payload["user"]["status"] == "ingame"):
p = payload
item_list.append(p)
#adding the items names to the item list ???? PROBLEM ?????
item_list = [dict(item, **{'name':items_name[iteration]}) for item in item_list]
對此:
if payload["order_type"] == "sell" and (payload["user"]["status"] == "online" or payload["user"]["status"] == "ingame"):
payload['name'] = items_name[iteration]
item_list.append(payload)
請注意,這不是有一個獨立的變數iteration,并增加它,你就可以通過回圈url_list使用enumerate,既提供了專案和其在每次迭代指數:
for iteration, url in enumerate(url_list):
....
轉載請註明出處,本文鏈接:https://www.uj5u.com/qukuanlian/383039.html
標籤:Python json 熊猫 for循环 html解析
上一篇:如何聚合熊貓系列的預定義間隔?
下一篇:過濾列,指定區分大小寫
