psycopg2 vs psycopg2-binary

[brki]

Airflow's Postgres provider specifies psycopg2-binary.

According to the psycopg2 docs, this is not recommended for production use :

The psycopg2-binary package is meant for beginners to start playing with Python and PostgreSQL without the need to meet the build requirements.

If you are the maintainer of a published package depending on psycopg2 you shouldn’t use psycopg2-binary as a module dependency. For production use you are advised to use the source distribution.

Shouldn't Airflow use the non-binary package psycopg2?

I guess some people will point out that it requires additional effort to build, because the OS-level PostgreSQL dev libraries, and openssl libraries, need to be available.

If the goal is to have Airflow easily installable, perhaps there should be an alternative postgres provider, that has a dependency on the non-binary psycopg2 package?

[potiuk]

I think you are right - can you please make a PR and see if you can make the tests and images work? Shoudl be as easy as updating provider.yaml dependency. Creating a PR should run all tests and we should see if we can swap it out - it's really the matter if our image will nicely build with it,

[brki]

PR: #25710