從包含 800 多個設備 ID 的 Excel 作業表中加載設備 ID 并在 http get 請求中傳遞這些設備 ID 的最快方法是什么。
我正在從 excel 表中獲取設備 ID，發出 http get 請求以獲取相關資料并將其轉儲到串列中，然后使用以下命令將其保存在 excel 檔案中：-
if __name__ == '__main__':
excel_file = openpyxl.load_workbook("D:\mypath\Book1.xlsx")
active_sheet = excel_file.get_sheet_by_name("Sheet4")
def iter_rows(active_sheet):
for row in active_sheet.iter_rows():
yield [cell.value for cell in row]
res = iter_rows(active_sheet)
keys = next(res)
final_data_to_dump = []
failed_data_dump = []
for new in res:
inventory_data = dict(zip(keys, new))
if None in inventory_data.values():
pass
else:
url_get_event = 'https://some_url&source={}'.format(inventory_data['DeviceID'])
header_events = {
'Authorization': 'Basic authkey_here'}
print(inventory_data['DeviceID'])
try:
r3 = requests.get(url_get_event, headers=header_events)
r3_json = json.loads(r3.content)
if r3_json['events']:
for object in r3_json['events']:
dict_excel_data = {
"DeviceID":object['source']['id'],
"Device Name":object['source']['name'],
"Start 1":object['Start1'],
"Start 2":object['Start2'],
"Watering Mode":object['WateringMode'],
"Duration":object['ActuationDetails']['Duration'],
"Type":object['type'],
"Creation Time":object['creationTime']
}
final_data_to_dump.append(dict_excel_data)
else:
no_dict_excel_data = {
"DeviceID":inventory_data["DeviceID"],
"Device Name":inventory_data["DeviceName"],
"Start 1":"",
"Start 2":"",
"Watering Mode":"",
"Duration":"",
"Type":"",
"Creation Time":""
}
final_data_to_dump.append(no_dict_excel_data)
except requests.ConnectionError:
failed_dict_excel_data = {
"DeviceID":inventory_data['DeviceID'],
"Device Name":inventory_data["DeviceName"],
"Status":"Connection Error"
}
failed_data_dump.append(failed_dict_excel_data)
df = pd.DataFrame.from_dict(final_data_to_dump)
df2 = pd.DataFrame.from_dict(failed_data_dump)
df.to_excel('D:\mypath\ReportReceived_10Apr.xlsx',sheet_name='Sheet1',index=False)
df2.to_excel('D:\mypath\Failed_ReportReceived_10Apr.xlsx',sheet_name='Sheet1',index=False)
但這可能需要 10-15 分鐘以上的時間，因為 Book1 表中有 800 多個設備，并且可能會增加。我怎樣才能使這個程序更快？
uj5u.com熱心網友回復：
您可以使用異步庫，但這里最簡單的解決方案是執行類似的操作
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor() as exc:
responses = exc.map(get, device_ids)
def get(device_id):
url_get_event = 'https://some_url&source={}'.format(device_id)
return requests.get(url_get_event)
如果您的代碼的其他部分很小，您可能希望將函式提交給執行程式并用于as_completed在主執行緒中處理它們，同時等待其他請求也運行。
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/460124.html
標籤：Python json python-3.x 熊猫 http