我正在使用氣流來安排和自動化安裝在 Ubuntu 服務器上的 Python 腳本。DAG 觸發 CURL 請求,該請求在實際運行腳本的同一臺機器上命中 Flask API。以下是流程的高級概述:
Airflow --> Curl 請求 --> Flask API --> Python 腳本
DAG 任務:
t2 = BashOperator (
task_id='extract_pcty_data',
bash_command=f"""curl -d '{dataset}' -H 'Content-Type: application/json' -X POST {base_url}{endpoint}""",
)
端點注冊:
api.add_resource(paylocity, "/api/v1/application/paylocity")
資源物件:
class paylocity(Resource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
def get(self):
return 200
def post(self):
try:
if request.json:
data = request.json
query = data['dataset']
pcty = PaylocityAPI()
pcty.auth()
pcty.get_employees()
pcty.get_paystatements()
pcty.load_dataset()
pcty.clean_up()
return 200
except Exception as e:
print(traceback.print_exc(e))
raise ValueError(e)
我遇到的問題是腳本將由于某種原因而失敗,該原因被 try/catch 塊捕獲,然后引發值錯誤 - 但它不會導致腳本失敗,因為回傳的 HTTP 請求回應是500 - Internal Server Error. 我正在尋找的是一種簡單而優雅的方式來解釋不是200 - OK“失敗”的 HTTP 回應并引發類似ValueError或AirflowException導致任務失敗的東西。任何指導或支持將不勝感激!
uj5u.com熱心網友回復:
對于那些從 Google 尋找這個或類似問題的簡單而優雅的答案的人。Curl 有一些標志,允許您指定您希望請求的失敗行為如何執行。對于我的具體情況:--fail是最合適的。還有--fail-with-body一個允許您獲取失敗回應的內容,而不僅僅是非零退出代碼。從他們的檔案中:
-f, --fail
(HTTP) Fail fast with no output at all on server errors. This is useful to enable scripts and users to better deal with failed attempts. In normal cases when an HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.
This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).
Example:
curl --fail https://example.com
轉載請註明出處,本文鏈接:https://www.uj5u.com/yidong/512982.html
上一篇:laravel錯誤:InvalidArgumentException:index.php中的Request::capture()
