MINOR: [Doc][CI] Fix doctests failing on pandas 3 string types examples

[rok]

See #48619 (comment)

Rationale for this change

Fix CI failures

What changes are included in this PR?

Tests are made more general to allow for Pandas 2 and Pandas 3 style string types

Are these changes tested?

By CI

Are there any user-facing changes?

No

[jorisvandenbossche]

FWIW, I think if we know the doctests will run on an instance with pandas>=3, I would just update those to use pandas 3 output, because that will be less confusing for people reading the docs ("what do those 3 dots mean here?").

Suggested change

	Categories (3, ...): ['foo', 'bar', 'baz']
	Categories (3, str): ['foo', 'bar', 'baz']

[rok]

Do we want to start testing on Pandas 3 exclusively? I believe we have both testing now. (One is not triggered for this PR because the scope is smaller)

[rok]

If we do agree to stop doctest on Pandas 2.x.y then we should also update CI appropriately.

[jorisvandenbossche]

For general tests (unit tests) we certainly have to test with various pandas versions (i.e. whichever range we claim to support), but personally I think for doctests it is perfectly fine to just have those passing in one specific environment (they are not meant as actual proper tests, but only to ensure the docs are correct and kept up to date)

[rok]

Thanks for the review @jorisvandenbossche ! Could you take another look before the merge?