Make `MemoryStream` public

[alamb]

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-11276

I found the need to take a copy of MemoryStream for use in another project.

It would be nice if we could expose this as a supported public API so that other projects building physical operators can re-use it.

[1nF0rmed]

Hey - is this something we are still looking at?

[houqp]

I believe so, it is still marked as pub(crate), but @alamb can confrim.

[alamb]

I think making MemoryStream pub and making sure it has sufficient test coverage sounds like a great idea to me 👍

[1nF0rmed]

@alamb if I understand correctly, the issue is looking at making this MemoryStream pub. But there is also another reference under Ballista for the same here, I assume that this is not related to the issue.

[alamb]

@alamb if I understand correctly, the issue is looking at making this MemoryStream pub. But there is also another reference under Ballista for the same here, I assume that this is not related to the issue.

That is a good find @1nF0rmed -- yes we are talking about making https://github.com/apache/arrow-datafusion/blob/295033492bbf335cdc02446cefb3751ab840f9b5/datafusion/src/physical_plan/memory.rs#L161-L171 public.

the fact that ballista has a copy/paste version of the same code is likely due to the fact that it is not pub lol -- so perhaps as part of the PR that makes MemoryStream pub we can remove the duplicate in Ballista