Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIP-84: Migrating GET Dataset events for DAG runs api to fastAPI #43874

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

amoghrajesh
Copy link
Contributor

related #42370

Depends on #43783

Migrating the GET Dataset events for DAG runs to fastAPI

Responses:
Legacy
image

FastAPI
image

Swagger spec
image

image


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Nov 11, 2024
@amoghrajesh
Copy link
Contributor Author

Only the last 3 commits are relevant here

@pierrejeambrun pierrejeambrun added the AIP-84 Modern Rest API label Nov 12, 2024
@pierrejeambrun
Copy link
Member

pierrejeambrun commented Nov 12, 2024

The base branch has been merged, branch needs rebasing.

Also you can add the legacy api tag to allow the CI to build.

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice just a couple of comments.

I'll do a final in depth review when the branch is rebased, it will make the review easier to only have the 'definitive' diff and not also the one from the base branch.

airflow/api_fastapi/core_api/routes/public/dag_run.py Outdated Show resolved Hide resolved
Comment on lines 87 to 88
# piggyback on the fix for https://github.com/apache/airflow/issues/43845 for asset_uri
# meanwhile, unblock by adding uri below
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this comment, do you mind elaborate a little bit more please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My PR and this one: #43881 had a common issue here. #43881 was handling this portion and meanwhile these values were changed. I will accomodate the changes from #43881 onto this one so that the response is like legacy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made the change and it works as expected

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Response:

{
    "asset_events": [
        {
            "id": 2,
            "asset_id": 1,
            "asset_uri": "wasb://somepath/",
            "extra": {},
            "source_task_id": "producer2",
            "source_dag_id": "producer2_dag",
            "source_run_id": "manual__2024-11-12T12:40:53.566380+00:00",
            "source_map_index": -1,
            "created_dagruns": [
                {
                    "run_id": "asset_triggered__2024-11-12T12:40:54.754807+00:00",
                    "dag_id": "multi_consumer_dag",
                    "logical_date": "2024-11-12T12:40:54.754807Z",
                    "start_date": "2024-11-12T12:40:54.903281Z",
                    "end_date": "2024-11-12T12:40:55.936560Z",
                    "state": "success",
                    "data_interval_start": "2024-11-12T12:40:51.405849Z",
                    "data_interval_end": "2024-11-12T12:40:53.566380Z"
                },
                {
                    "run_id": "asset_triggered__2024-11-12T12:40:54.754809+00:00",
                    "dag_id": "consumer2_dag",
                    "logical_date": "2024-11-12T12:40:54.754809Z",
                    "start_date": "2024-11-12T12:40:54.903329Z",
                    "end_date": "2024-11-12T12:40:55.939573Z",
                    "state": "success",
                    "data_interval_start": "2024-11-12T12:40:53.566380Z",
                    "data_interval_end": "2024-11-12T12:40:53.566380Z"
                }
            ],
            "timestamp": "2024-11-12T12:40:54.753857Z"
        }
    ],
    "total_entries": 1
}

@amoghrajesh amoghrajesh added the legacy api Whether legacy API changes should be allowed in PR label Nov 13, 2024
@amoghrajesh amoghrajesh self-assigned this Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AIP-84 Modern Rest API area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. legacy api Whether legacy API changes should be allowed in PR
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

2 participants