我有n 個批次，每個批次包含 100 個 API 請求。第 1 批包含檔案 [1-100]，第 2 批包含檔案 [101-200] 等...
我想將這些中的每一個轉儲到一個 json 檔案中。這很好.. 但是，我想將 100k json 回應轉儲到 1 個檔案中，然后創建一個新檔案并將接下來的 100k 觀察值轉儲到另一個檔案中。
我需要配置一個基于批號創建檔案的功能，我嘗試了以下方法：
def open_file(self, batch):
if batch % 1000 == 0:
filename = f"data_{batch}.json"
else:
filename = ""
f = open(filename, "a")
return f
如果批處理 % 1000 == 0，那么我想更改名稱（作為批處理號 1000 -> 1000 批 100 個 json 請求 = 總共 100k）。但是，這顯然不起作用，因為當我評估批次 1001 時，舊檔案再次打開。如何為批次 1-999 創建一個檔案，為批次 1000-1999 創建另一個檔案，然后為 2000-2999 ......？
謝謝
編輯：附加資訊
def fetch_data(self, sequence, batch=1):
# fetch event list (event list size = 100)
event_list = self.events(sequence)
# open files to store data
f = self.open_file(batch)
# opening thread pool executor for multi-threading
with ThreadPoolExecutor(max_workers=4) as executor:
# self.thread_event_list multi-threads each event in event list by sending
# and API request.
for i, response in enumerate(executor.map(self.thread_event_list, event_list), 1):
json.dump(response.json(), f)
f.write("\n")
# continue from last sequence number (used in the recursive call)
last_sequence = response.json()["last_sequence"]
# Recursive call, use last sequence number of the
# event list, and continue with sequence 1 and batch 1
self.fetch_data(
sequence=last_sequence 1,
batch=batch 1
)
uj5u.com熱心網友回復：
import math
# Be sure that batch is never 0, otherwise this will create a file for batch #0 only.
def open_file(self, batch):
# If you're unsure, this handles it for you.
if batch < 1:
batch = 1
filename = f"data_{math.ceil(batch/1000)}.json"
f = open(filename, "a")
return f
uj5u.com熱心網友回復：
為什么不只使用范圍函式？`
if batch in range(1,1000):
filename = f"data_{batch_name1}.json"
elif batch in range(1000,2000):
filename = f"data_{batch_name2}.json"
elif batch in range(2000,3000):
filename = f"data_{batch_name3}.json"
`
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/435530.html
上一篇：如何在jqwalk中參考父物件？
下一篇：Python字典串列-訪問鍵