Skip to content

Fix concurrency, exception-safety, and correctness bugs in JNI layer …#326

Merged
bernardladenthin merged 1 commit into
bernardladenthin:mainfrom
vaiju1981:fix/bugs-and-issues
Jul 11, 2026
Merged

Fix concurrency, exception-safety, and correctness bugs in JNI layer …#326
bernardladenthin merged 1 commit into
bernardladenthin:mainfrom
vaiju1981:fix/bugs-and-issues

Conversation

@vaiju1981

Copy link
Copy Markdown

…and Java API

JNI / native fixes:

  • Guard rerank / embedding / streaming to_json() calls against uncaught C++ exceptions crossing the JNI boundary, which would otherwise abort the JVM.
  • Hold a jllama_context user reference for the attached NativeServer worker so closing the model cannot free the context out from under the running HTTP frontend (use-after-free).
  • Fix close()-during-inference hang: add a closing flag that unblocks readers parked in next()/wait_for_all(), and decrement the user count under the context mutex so the teardown wait cannot free the mutex/condvar before the releaser notifies (latent use-after-free in the ref-count itself).
  • Fix NativeServer jstring_to_utf8 calling getBytes with a Charset argument while resolving the String overload (deterministic break on every server start); cache the JNI IDs thread-safely.
  • Initialize parse_string_array slots to nullptr so an OOM path cannot free uninitialized memory.
  • Add a pending-exception check in parse_jstring.

Java fixes:

  • Widen token counts to long in the protocol translators and ChatResponseParser.extractUsageField to avoid truncation above ~2.1B tokens.
  • Honor a pre-cancelled CancellationToken in complete() (stop resetting it at entry); fix completeAsJson javadoc (the params object is not mutated).

Tests and docs:

  • Extract the OuteTTS codec filter into a testable helper; add rerank bounds and codec-filter unit tests, and a model-gated close()-during-inference test.
  • Remove the dead BUILD_SHARED_LIBS ON line in CMakeLists.txt.
  • Update plan and TODO docs to reflect the real teardown design.

Summary

Test plan

  • [ X] Affected unit / integration tests pass locally
  • CI is green on this branch
  • [ X] Docs / CHANGELOG updated where applicable

Related issues / PRs

Checklist

  • [ X] I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • [ X] My commits follow Conventional Commits
  • [ X] No security-sensitive changes (if there are, I have notified the maintainer privately per SECURITY.md)

…and Java API

JNI / native fixes:
- Guard rerank / embedding / streaming to_json() calls against uncaught C++
  exceptions crossing the JNI boundary, which would otherwise abort the JVM.
- Hold a jllama_context user reference for the attached NativeServer worker so
  closing the model cannot free the context out from under the running HTTP
  frontend (use-after-free).
- Fix close()-during-inference hang: add a closing flag that unblocks readers
  parked in next()/wait_for_all(), and decrement the user count under the
  context mutex so the teardown wait cannot free the mutex/condvar before the
  releaser notifies (latent use-after-free in the ref-count itself).
- Fix NativeServer jstring_to_utf8 calling getBytes with a Charset argument
  while resolving the String overload (deterministic break on every server
  start); cache the JNI IDs thread-safely.
- Initialize parse_string_array slots to nullptr so an OOM path cannot free
  uninitialized memory.
- Add a pending-exception check in parse_jstring.

Java fixes:
- Widen token counts to long in the protocol translators and
  ChatResponseParser.extractUsageField to avoid truncation above ~2.1B tokens.
- Honor a pre-cancelled CancellationToken in complete() (stop resetting it at
  entry); fix completeAsJson javadoc (the params object is not mutated).

Tests and docs:
- Extract the OuteTTS codec filter into a testable helper; add rerank bounds
  and codec-filter unit tests, and a model-gated close()-during-inference test.
- Remove the dead BUILD_SHARED_LIBS ON line in CMakeLists.txt.
- Update plan and TODO docs to reflect the real teardown design.
@vaiju1981 vaiju1981 force-pushed the fix/bugs-and-issues branch from 68989f6 to 88fdd6d Compare July 10, 2026 21:21
@vaiju1981 vaiju1981 deployed to startgate July 10, 2026 21:21 — with GitHub Actions Active
@bernardladenthin bernardladenthin merged commit 6f09d88 into bernardladenthin:main Jul 11, 2026
66 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants