更改默認Airflow配置選項（在Docker上運行）

為了學習，我一直在嘗試在一組 docker 容器上設定 Airflow 實體。我只知道 Docker 和 Airflow 的核心基礎知識。

我一直在關注如何使用 Docker 設定 Airflow 的指南。

遵循指南時，我可以啟動容器并且一切都按預期作業。

我的下一步是開始更改 docker-compose.yml 檔案中的不同配置選項。我首先嘗試設定不同的用戶名和密碼。

我在與 docker-compose.yml 檔案相同的目錄中創建了一個 .env 檔案。見下文。

AIRFLOW_UID=50000
USER='user'
PASSWORD='password'
DATABASE='airflow'
_AIRFLOW_WWW_USER_USERNAME='user'
_AIRFLOW_WWW_USER_PASSWORD='password'

為了使用這些環境變數，我還對 docker-compose.yml 中的連接字串進行了一些更改。原始行已被注釋掉。下面硒。

x-airflow-common:
  &airflow-common
  # In order to add custom dependencies or upgrade provider packages you can use your extended image.
  # Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
  # and uncomment the "build" line below, Then run `docker-compose build` to build the images.
  image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.4.2}
  # build: .
  environment:
    &airflow-common-env
    AIRFLOW__CORE__EXECUTOR: CeleryExecutor
    AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql psycopg2://${USER}:${PASSWORD}@postgres/${DATABASE}
    # For backward compatibility, with Airflow <2.3
    AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql psycopg2://${USER}:${PASSWORD}@postgres/${DATABASE}
    AIRFLOW__CELERY__RESULT_BACKEND: db postgresql://${USER}:${PASSWORD}@postgres/${DATABASE}
# -- Original settings --
#AIRFLOW__DATABASE__SQL_ALCHEMY_CONN:postgresql psycopg2://airflow:airflow@postgres/airflow
#AIRFLOW__CORE__SQL_ALCHEMY_CONN:postgresql psycopg2://airflow:airflow@postgres/airflow
#AIRFLOW__CELERY__RESULT_BACKEND: db postgresql://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
    AIRFLOW__CORE__FERNET_KEY: ''
    AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
    AIRFLOW__CORE__LOAD_EXAMPLES: 'true'
    AIRFLOW__API__AUTH_BACKENDS: 'airflow.api.auth.backend.basic_auth'
    _PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
  volumes:
    - ./dags:/opt/airflow/dags
    - ./logs:/opt/airflow/logs
    - ./plugins:/opt/airflow/plugins
  user: "${AIRFLOW_UID:-50000}:0"
  depends_on:
    &airflow-common-depends-on
    redis:
      condition: service_healthy
    postgres:
      condition: service_healthy

services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: ${USER}
      POSTGRES_PASSWORD: ${PASSWORD}
      POSTGRES_DB: ${DATABASE}
     # -- Original settings --
     #POSTGRES_USER: airflow
     #POSTGRES_PASSWORD: airflow
     #POSTGRES_DB: airflow
    volumes:
      - postgres-db-volume:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", '${USER}']
      interval: 5s
      retries: 5
    restart: always

首先我運行docker-compose down --volumes --rmi all是為了清理環境。然后我運行docker-compose up airflow-init。以下是終端中顯示的最后幾行。

data_collection_proj-airflow-init-1  | User "user" created with role "Admin"
data_collection_proj-airflow-init-1  | /home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py:367: FutureWarning: The auth_backends setting in [api] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
data_collection_proj-airflow-init-1  |   FutureWarning,
data_collection_proj-airflow-init-1  | 2.4.2
data_collection_proj-airflow-init-1 exited with code 0 

沒有顯示錯誤或警告，因此它似乎作業正常。

然后我運行docker-compose up并收到以下錯誤。

data_collection_proj-postgres-1           | 2022-11-12 17:47:08.939 UTC [165] FATAL:  database "user" does not exist

我的更改似乎破壞了設定，但我不明白為什么。

uj5u.com熱心網友回復：

你可以試試這個測驗語法，它對我來說很好：

services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: ${USER}
      POSTGRES_PASSWORD: ${PASSWORD}
      POSTGRES_DB: ${DATABASE}
    volumes:
      - postgres-db-volume:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "sh -c 'pg_isready -U ${USER} -d ${DATABASE}'"]
      interval: 5s
      retries: 5
    restart: always

標籤：码头工人码头工人撰写yaml空气流动

上一篇：復制子目錄和嵌套子目錄的所有檔案

下一篇：使用多個Proto構建Docker-未找到Proto檔案