feat: support datetime objects in literal instantiation

[jayceslesar]

Support datetime objects being passed into the literal function.

Allows situations like the following:

from datetime import datetime
...
row_filter=And(
            GreaterThanOrEqual("my_timestamp_column", datetime.now()),
        ),

Which previously failed

Closes #1456

[kevinjqliu]

Thanks for the PR! Lets add a test for the example in the description with row_filter

[jayceslesar]

Thanks for the PR! Lets add a test for the example in the description with row_filter

Added, let me know if we want anything more explicit there

[kevinjqliu]

nit: the table has datetime column, what if we add a row and then filter datetime before and after and assert the results.

maybe something like,

yesterday = ...
table.append({"datatime": yesterday)
assert len(table.scan(row_filter=GreaterThanOrEqual("datetime", datetime.now()))) == 0
assert len(table.scan(row_filter=LessThanOrEqual("datetime", datetime.now()))) == 1

[jayceslesar]

thanks, looking much better now, took me too long to figure out I can run make test-integration PYTEST_ARGS='-k test_scan_with_datetime' to only select that test hahaha

[kevinjqliu]

LGTM! Thanks again

[Fokko]

Looks good, thanks for adding this @jayceslesar 🙌 Thanks for the review @kevinjqliu 🚀

[Fokko]

The tzinfo is handled here:

iceberg-python/pyiceberg/utils/datetime.py

Lines 75 to 81 in 3981e62

	def datetime_to_micros(dt: datetime) -> int:
	"""Convert a datetime to microseconds from 1970-01-01T00:00:00.000000."""
	if dt.tzinfo:
	delta = dt - EPOCH_TIMESTAMPTZ
	else:
	delta = dt - EPOCH_TIMESTAMP
	return (delta.days * 86400 + delta.seconds) * 1_000_000 + delta.microseconds

[jayceslesar]

this introduces sort of a tricky typing problem...

tests/expressions/test_literals.py:910: error: Value of type variable "L" of "literal" cannot be "datetime"  [type-var]
tests/integration/test_reads.py:976: error: Value of type variable "L" of "GreaterThanOrEqual" cannot be "datetime"  [type-var]
tests/integration/test_reads.py:979: error: Value of type variable "L" of "LessThan" cannot be "datetime"  [type-var]

Will check it out tonight but can't just fix by adding datetime to the L type

[kevinjqliu]

Thanks for the contribution @jayceslesar! and thanks for the review @Fokko :)