We've been running into issues with DoPut in Java. In particular:
-
Closing a FlightStream without draining it should not send a cancellation to the other side (or should send the cancellation, but also drain the queue). A server will have sent an explicit error message, or will simply just not want to read the entire stream. A client should explicitly cancel/gRPC will cancel for you anyways when you end the call. Also, the gRPC call may already have ended and cancelling the call may result in a runtime exception.
-
Cancelling a FlightStream explicitly should not immediately mark the stream as completed - it should wait for gRPC to acknowledge the cancellation as there may be undelivered messages.
-
Make sure there is no race between the gRPC observer in the FlightStream and the consumer. (Ideally the only way for a FlightStream to end is for the observer to end the stream; that does open us up to the possibility of a FlightStream being stuck forever for servers that do not respect cancellation.)
-
The server should close/clean up things properly in DoPut (it should act like DoExchange and tie closing of the stream to the onCompleted/onError callbacks). Otherwise trying to use it with ARROW-9586 becomes impossible (you need to close the FlightStream before ending the call, or you'll close the per-call allocator before you close the FlightStream)
I think this also ties into flakiness in unit tests.
Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm
PRs and other links:
Note: This issue was originally created as ARROW-9587. Please see the migration documentation for further details.
We've been running into issues with DoPut in Java. In particular:
Closing a FlightStream without draining it should not send a cancellation to the other side (or should send the cancellation, but also drain the queue). A server will have sent an explicit error message, or will simply just not want to read the entire stream. A client should explicitly cancel/gRPC will cancel for you anyways when you end the call. Also, the gRPC call may already have ended and cancelling the call may result in a runtime exception.
Cancelling a FlightStream explicitly should not immediately mark the stream as completed - it should wait for gRPC to acknowledge the cancellation as there may be undelivered messages.
Make sure there is no race between the gRPC observer in the FlightStream and the consumer. (Ideally the only way for a FlightStream to end is for the observer to end the stream; that does open us up to the possibility of a FlightStream being stuck forever for servers that do not respect cancellation.)
The server should close/clean up things properly in DoPut (it should act like DoExchange and tie closing of the stream to the onCompleted/onError callbacks). Otherwise trying to use it with ARROW-9586 becomes impossible (you need to close the FlightStream before ending the call, or you'll close the per-call allocator before you close the FlightStream)
I think this also ties into flakiness in unit tests.
Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm
PRs and other links:
Note: This issue was originally created as ARROW-9587. Please see the migration documentation for further details.