Allow to set batch size for db clean CLI

[eladkal]

Body

The db clean CLI was added in #20838
There is issue of timeout if the running the CLI on a very large table(s), ideally we should allow running the command with some kind of batch size flag that will split the work and execute it batch after batch.

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[Riddhikshah21]

Hi @eladkal, can you assign this issue to me?

[shubham-pyc]

@eladkal I see that this is assigned to @Riddhikshah21. If collaboration is possible, I'd love to help!

[rawwar]

@eladkal I see that this is assigned to @Riddhikshah21. If collaboration is possible, I'd love to help!

@shubham-pyc : I suggest you pick another open unassigned issue. You should probably keep an eye on this Issue's PR and share feedback if you have any.

@Riddhikshah21 , Do you have any questions or need help with this issue?

[shubham-pyc]

I can work on this if no progress is being made. @rawwar

[ayushjain2809]

The Astronomer example provides a practical solution by finding the first candidate timestamp for cleanup, and using it to generate batches. But it is not doable with the public interface, it relies on internal APIs.
Would be nice to have a similar batch-size as CLI parameter.