我想抓取網站并將所需的資料放入 JSON 檔案中。我被反駁的問題是我得到一個文本并且只能列印它。但我只需要在 JSON 檔案中添加特定資料并在我的類中重用資料。我正在抓取的 WEB 和我的代碼：
import requests
from bs4 import BeautifulSoup
URL = 'https://lt.brcauto.eu/automobiliu-paieska/'
req = requests.get(URL)
soup = BeautifulSoup(req.text, 'lxml')
pages = soup.find_all('li', class_ = 'page-item')[-2] #biggest page -2 ">" we need only before the last
cars_printed_counter = 0
for number in range(1, int(pages.text)):
req = requests.get(URL '?page=' str(number))
soup = BeautifulSoup(req.text, 'lxml')
if cars_printed_counter == 20:
break
for single_car in soup.find_all('div', class_ = 'cars-wrapper'):
if cars_printed_counter == 20:
break
Car_Title = single_car.find('h2', class_ = 'cars__title')
Car_Specs = single_car.find('p', class_ = 'cars__subtitle')
print('\nCar number:', cars_printed_counter 1)
print(Car_Title.text)
print(Car_Specs.text)
cars_printed_counter = 1
我得到的資料如下所示：列印結果
Car number: 19
BMW 520 Gran Turismo M-Sport
2013 | 2.0 Diesel | Automation | 255229 km | 135 kW (184 AG) | Black
Car number: 20
BMW 750 i Automation
2005 | 5.0 Gasoline | Automation | 343906 km | 270 kW (367 AG) | Grey
問題是：我應該如何將資料放入 JSON 檔案中，它看起來像這樣：Desired json
[
{
"fuel": "diesel",
"title": "BMW 520 Gran Turismo M-Sport",
"year": 2013,
"run": 255229,
"type": "Black"
},
{
"fuel": "gasoline",
"title": "BMW 750 i Automation",
"year": 2005,
"run": 343906,
"type": "Grey"
},
uj5u.com熱心網友回復：
你可以做這樣的事情。查看此鏈接，了解如何在 python 中創建 dicts
import json
# this is going to store your dicts of cars
list_of_printed_cars = []
for single_car in soup.find_all('div', class_ = 'cars-wrapper'):
if cars_printed_counter == 20:
break
Car_Title = single_car.find('h2', class_ = 'cars__title')
Car_Specs = single_car.find('p', class_ = 'cars__subtitle')
# printed_car is a dictionary of the car's title and specs
printed_car = {
'title': Car_Title.text,
'specs': Car_Specs.text
}
# this appends to a list that stores each car's title and specs
list_of_printed_cars.append(printed_car)
# to use list_of_printed_cars, you need to convert it to a json add it to a file
with open('data.json', 'w') as f:
json.dump(list_of_printed_cars, f)
然后，您可以通過使用并將其保存到檔案list_of_printed_cars中來使用 as json的 dictjson.dumps
uj5u.com熱心網友回復：
開門見山：
import requests
from bs4 import BeautifulSoup
import json
URL = 'https://lt.brcauto.eu/automobiliu-paieska/'
req = requests.get(URL)
soup = BeautifulSoup(req.text, 'lxml')
pages = soup.find_all('li', class_='page-item')[-2] # biggest page -2 ">" we need only before the last
cars_printed_counter = 0
for number in range(1, int(pages.text)):
req = requests.get(URL '?page=' str(number))
soup = BeautifulSoup(req.text, 'lxml')
if cars_printed_counter == 20:
break
out = []
for single_car in soup.find_all('div', class_='cars-wrapper'):
if cars_printed_counter == 20:
break
Car_Title = single_car.find('h2', class_='cars__title')
Car_Specs = single_car.find('p', class_='cars__subtitle')
print('\nCar number:', cars_printed_counter 1)
print(Car_Title.text)
print(Car_Specs.text)
car = {}
car["title"] = Car_Title.text
subs = Car_Specs.text.split(' | ')
car["year"] = subs[0]
car["fuel"] = subs[1].split(" ")[1]
car["run"] = subs[3].split(" ")[0]
car["type"] = subs[5]
car["number"] = cars_printed_counter 1
out.append(car)
cars_printed_counter = 1
print(json.dumps(out))
with open("outfile.json", "w") as f:
f.write(json.dumps(out))
解釋：我們創建一個out變數來保存所有的汽車。當我們回圈它們時，我們創建了一個包含我們想要的值的字典。但是由于規范是一個字串，我們用“|”分割該字串以獲得單獨的組件。然后只需將每個組件映射到字典中的一個成員。然后我們獲取該字典并將其附加到out物件中。總而言之，我們有一個包含我們需要的所有資訊的字典串列。然后我們呼叫json.dumps()該串列以獲取 json 并將其保存到檔案中。
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/459897.html