PostgresHook: Add configurable UPSERT update fields

[SameerMesiah97]

Description

This change extends PostgreSQL UPSERT support by introducing configurable update fields for INSERT ... ON CONFLICT operations.

A new replace_target option is added to PostgresDialect.generate_replace_sql, allowing callers to specify which columns are updated when a conflict occurs. When omitted, the existing behavior of updating all non-conflict columns is preserved. When an empty list is provided, DO NOTHING semantics are generated.

The change also adds a convenience PostgresHook.upsert_rows method that wraps the existing insert_rows(replace=True) implementation and exposes PostgreSQL UPSERT concepts through conflict_fields and update_fields.

Rationale

While PostgreSQL UPSERT support already exists through insert_rows(replace=True), callers cannot currently control which columns are updated when conflicts occur. This change adds that flexibility while preserving existing behavior by default.

The upsert_rows convenience method provides a more intuitive PostgreSQL-specific API without requiring callers to use lower-level dialect arguments.

Tests

Added unit tests verifying that:

• UPSERT operations correctly support configurable update fields.
• UPSERT operations correctly support single and composite conflict fields.
• DO NOTHING behavior is generated when an empty update field list is provided.
• upsert_rows correctly delegates to the existing insert_rows(replace=True) implementation.

Notes

The entry for the parameter replace in the docstring for generate_replace_sql has been removed as the argument for it is not being used within the function.

Backwards Compatibility

This change is fully backwards compatible.

Existing insert_rows(replace=True) behavior is preserved. Callers that do not specify update fields continue to update all non-conflict columns exactly as before.

[justinpakzad]

Nice PR. Left a couple of comments. I know it's still in draft but figured I'd leave some feedback anyways.

[SameerMesiah97]

Nice PR. Left a couple of comments. I know it's still in draft but figured I'd leave some feedback anyways.

That’s perfectly fine. I will address the feedback once the dependency is merged.

[dabla]

Upsert rows is already supported in PostgresHook via insert_rows and insert_args={“replace”: True}

[SameerMesiah97]

Upsert rows is already supported in PostgresHook via insert_rows and insert_args={“replace”: True}

I will convert this to draft. This either needs to be reworked or closed.

Sorry for overlooking that.

[dabla]

Upsert rows is already supported in PostgresHook via insert_rows and insert_args={“replace”: True}

I will convert this to draft. This either needs to be reworked or closed.

Sorry for overlooking that.

You could still implement this method in the PostgresHook as a shortcut convenience method and delegate the call to the insert_rows method with replace=True. And then just test the delegation, I still think this is a good idea.

[SameerMesiah97]

@dabla

I have re-scoped the PR down to a convenience method i.e. upsert_rows that allows the update fields to be configured. This is not currently covered by insert_rows. Naturally, the underlying dialect for postgres had to be ajdusted slightly to accomodate this.

[Dev-iL]

Naturally, the underlying dialect for postgres had to be ajdusted slightly to accomodate this.

Please elaborate. This is important since we're in the process of switching the default driver for postgres (both sync and async) to psycopg v3.

[SameerMesiah97]

Naturally, the underlying dialect for postgres had to be ajdusted slightly to accomodate this.

Please elaborate. This is important since we're in the process of switching the default driver for postgres (both sync and async) to psycopg v3.

I have left a comment on the relevant part of the diff.

[dabla]

Naturally, the underlying dialect for postgres had to be ajdusted slightly to accomodate this.

Please elaborate. This is important since we're in the process of switching the default driver for postgres (both sync and async) to psycopg v3.

This is regarding to the Airflow postgres dialect, not the SQLAlchemy one to be clear, normally this should have zero effect on the used driver.

[dabla]

@SameerMesiah97 please resolve all comments that have been addressed, this makes review easier.

[dabla]

Congratulations @SameerMesiah97 , thank your for the great work!