You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/how-tos/airflow/generate-dags-from-yml.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,7 +34,7 @@ generate:
34
34
```
35
35
36
36
:::tip
37
-
If using an Extract and Load tool in your DAG you can dynamically generate your sources; however, additional configuration will be needed inside the config.yml file. See [Airbyte](./run-airbyte-sync-jobs.md#configure-transformdbt-covesconfigyml-file). For [Fivetran](./run-fivetran-sync-jobs.md#configure-transformdbt-covesconfigyml-file) contact us to complete the setup.
37
+
If using an Extract and Load tool in your DAG you can dynamically generate your sources; however, additional configuration will be needed inside the config.yml file. See [Airbyte](./run-airbyte-sync-jobs#generate-dags-from-yml-with-dbt-coves). For [Fivetran](./run-fivetran-sync-jobs#configure-your-transform-dbt-coves-config-yml-file) contact us to complete the setup.
Copy file name to clipboardExpand all lines: docs/how-tos/airflow/initial-setup.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ Each requires specific information to be provided during configuration. Our reco
39
39
-**Git branch name** The branch airflow will monitor for changes. If you have more than 1 environment (Development and Production), we suggest `airflow_development` for the development environment and `main` for the production environment. Note: You would need to create an `airflow_development` branch in your repo. If only have 1 environment, then the `Git branch name` should be `main`.
40
40
41
41
:::tip
42
-
We recommend combining your dbt transformations in the same project as your Airflow orchestration. However, you may wish to separate orchestration from transformation into different git projects. In Datacoves you can achieve this by having two projects. Each project will be associated with one git repo. Find out how to configure a [project](how-tos/datacoves/how_to_projects.md).
42
+
We recommend combining your dbt transformations in the same project as your Airflow orchestration. However, you may wish to separate orchestration from transformation into different git projects. In Datacoves you can achieve this by having two projects. Each project will be associated with one git repo. Find out how to configure a [project](../datacoves/how_to_projects/).
Copy file name to clipboardExpand all lines: docs/how-tos/airflow/run-fivetran-sync-jobs.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,7 +27,8 @@ Create a new connection using the following details:
27
27
:::tip
28
28
Once your Fivetran API key and secret have been generated, for security reasons, the secret cannot be viewed again through the Fivetran interface. If you lose or forget your API secret, you will need to generate a new API key and secret pair so be sure to store them somewhere secure for reference later. See <ahref="https://fivetran.com/docs/rest-api/getting-started"target="_blank"rel="noopener">Fivetran Documentation</a> on how to generate your Fivetran `API Key` and `API Secret`.
29
29
:::
30
-
### Configure your transform/.dbt-coves/config.yml file
30
+
31
+
### Configure your transform dbt-coves config yml file
31
32
32
33
By default, dbt-coves cannot query the necessary information for Fivetran connections. You will need to configure these in your yml DAG manually, or contact us to configure Datacoves with the necessary information.
33
34
Below are the configurations in for dbt-coves airflow-dags. You will need to configure these if using dbt-coves to generate DAGs from YML
@@ -120,12 +121,12 @@ You will need to define two operators: `fivetran_provider.operators.fivetran.Fiv
-**poke_interval**: The poke interval is the time in seconds that the sensor waits before rechecking if the connector is done loading data. Defaults to 60.
128
-
-**fivetran_conn_id**: This is the `connection_id` that was configured above in the Fivetran UI as seen [above](#id=fivetran-connection).
129
+
-**fivetran_conn_id**: This is the `connection_id` that was configured above in the Fivetran UI as seen [above](#fivetran-connection).
129
130
-**dependencies**: A list of tasks this task depends on.
Copy file name to clipboardExpand all lines: static/llms.txt
+7-6Lines changed: 7 additions & 6 deletions
Original file line number
Diff line number
Diff line change
@@ -1196,7 +1196,7 @@ Each requires specific information to be provided during configuration. Our reco
1196
1196
- **Git branch name** The branch airflow will monitor for changes. If you have more than 1 environment (Development and Production), we suggest `airflow_development` for the development environment and `main` for the production environment. Note: You would need to create an `airflow_development` branch in your repo. If only have 1 environment, then the `Git branch name` should be `main`.
1197
1197
1198
1198
:::tip
1199
-
We recommend combining your dbt transformations in the same project as your Airflow orchestration. However, you may wish to separate orchestration from transformation into different git projects. In Datacoves you can achieve this by having two projects. Each project will be associated with one git repo. Find out how to configure a [project](how-tos/datacoves/how_to_projects.md).
1199
+
We recommend combining your dbt transformations in the same project as your Airflow orchestration. However, you may wish to separate orchestration from transformation into different git projects. In Datacoves you can achieve this by having two projects. Each project will be associated with one git repo. Find out how to configure a [project](../datacoves/how_to_projects/).
1200
1200
:::
1201
1201
1202
1202
**S3 Sync**
@@ -1963,7 +1963,7 @@ generate:
1963
1963
```
1964
1964
1965
1965
:::tip
1966
-
If using an Extract and Load tool in your DAG you can dynamically generate your sources; however, additional configuration will be needed inside the config.yml file. See [Airbyte](./run-airbyte-sync-jobs.md#configure-transformdbt-covesconfigyml-file). For [Fivetran](./run-fivetran-sync-jobs.md#configure-transformdbt-covesconfigyml-file) contact us to complete the setup.
1966
+
If using an Extract and Load tool in your DAG you can dynamically generate your sources; however, additional configuration will be needed inside the config.yml file. See [Airbyte](./run-airbyte-sync-jobs#generate-dags-from-yml-with-dbt-coves). For [Fivetran](./run-fivetran-sync-jobs#configure-your-transform-dbt-coves-config-yml-file) contact us to complete the setup.
1967
1967
:::
1968
1968
## Create the yml file for your Airflow DAG
1969
1969
@@ -2828,7 +2828,8 @@ Create a new connection using the following details:
2828
2828
:::tip
2829
2829
Once your Fivetran API key and secret have been generated, for security reasons, the secret cannot be viewed again through the Fivetran interface. If you lose or forget your API secret, you will need to generate a new API key and secret pair so be sure to store them somewhere secure for reference later. See <a href="https://fivetran.com/docs/rest-api/getting-started" target="_blank" rel="noopener">Fivetran Documentation</a> on how to generate your Fivetran `API Key` and `API Secret`.
2830
2830
:::
2831
-
### Configure your transform/.dbt-coves/config.yml file
2831
+
2832
+
### Configure your transform dbt-coves config yml file
2832
2833
2833
2834
By default, dbt-coves cannot query the necessary information for Fivetran connections. You will need to configure these in your yml DAG manually, or contact us to configure Datacoves with the necessary information.
2834
2835
Below are the configurations in for dbt-coves airflow-dags. You will need to configure these if using dbt-coves to generate DAGs from YML
@@ -2921,12 +2922,12 @@ You will need to define two operators: `fivetran_provider.operators.fivetran.Fiv
- **poke_interval**: The poke interval is the time in seconds that the sensor waits before rechecking if the connector is done loading data. Defaults to 60.
2929
-
- **fivetran_conn_id**: This is the `connection_id` that was configured above in the Fivetran UI as seen [above](#id=fivetran-connection).
2930
+
- **fivetran_conn_id**: This is the `connection_id` that was configured above in the Fivetran UI as seen [above](#fivetran-connection).
2930
2931
- **dependencies**: A list of tasks this task depends on.
2931
2932
2932
2933
### YAML version
@@ -7624,7 +7625,7 @@ A Project is the highest grouping in Datacoves. It is what contains environments
7624
7625
The Datacoves landing page (Launch Pad) follows this hierarchy:
7625
7626
7626
7627
:::tip
7627
-
See our How To - [Projects](how-tos/datacoves/how_to_projects.md)
7628
+
See our How To - [Projects](/docs/how-tos/datacoves/how_to_projects/)
0 commit comments