Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 17 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.

Current llama.cpp pinned version: **b9957**
Current llama.cpp pinned version: **b9959**

## Upgrading CUDA Version

Expand Down Expand Up @@ -421,7 +421,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
ships no UI):
```bash
# needs node/npm + network; embed.cpp is plain C++17 (no npm)
git clone --depth 1 --branch b9957 https://github.com/ggml-org/llama.cpp /tmp/lc
git clone --depth 1 --branch b9959 https://github.com/ggml-org/llama.cpp /tmp/lc
( cd /tmp/lc/tools/ui && npm ci && npm run build \
&& ( cd dist && find . -type f -not -path './_gzip/*' \
| while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
Expand Down Expand Up @@ -461,7 +461,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
- `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
as the repo secret **`DEPOT_TOKEN`**.

Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b9957`), the
Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b9959`), the
~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
*changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
Expand Down Expand Up @@ -622,18 +622,27 @@ else step through the largest intermediate tag still under the threshold), the
`.github/scripts/llama-next-version.sh` helper that computes the next reviewable step, and the
edit/verify/commit loop below. Use it for any non-trivial bump; the steps here are the mechanical core.

To change the llama.cpp version, update the following **three** files (and re-verify `patches/`):
To change the llama.cpp version, update the following **four** files (and re-verify `patches/`):

1. **llama/CMakeLists.txt** — the `GIT_TAG` line for llama.cpp: `GIT_TAG b8831`
1. **llama/CMakeLists.txt** — the `GIT_TAG` line for llama.cpp: `GIT_TAG b8831` (and the
cosmetic `-DLLAMA_TAG=b8831` a few lines below, passed to the TTS generator — keep them equal)
2. **README.md** — the badge and link line with the version number
3. **CLAUDE.md** — the "Current llama.cpp pinned version" line
4. **llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java** — the
`LLAMA_CPP_VERSION` constant (the compile-time pin exposed to Java/Kotlin consumers, e.g. the
Android version badge). It is the *pure-Java mirror* of `GIT_TAG` and must stay equal to it —
the native `LlamaModel.getLlamaCppBuildInfo()` getter reports the actually-linked build
(`b<number>-<commit>`), and `NativeLibraryLoadSmokeTest.nativeBuildInfoMatchesPinnedVersionConstant`
**fails the build** if this constant and the linked binary drift apart.

Example: To upgrade from b8808 to b8831:
```bash
# Edit llama/CMakeLists.txt: change GIT_TAG b8808 to b8831
# Edit llama/CMakeLists.txt: change GIT_TAG b8808 to b8831 (and the -DLLAMA_TAG line)
# Edit README.md: change b8808 to b8831 (in both badge and link)
# Edit CLAUDE.md: change b8808 to b8831
git add llama/CMakeLists.txt README.md CLAUDE.md
# Edit LlamaCppVersion.java: change LLAMA_CPP_VERSION "b8808" to "b8831"
git add llama/CMakeLists.txt README.md CLAUDE.md \
llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java
git commit -m "Upgrade llama.cpp from b8808 to b8831"
git push -u origin <your-branch>
```
Expand Down Expand Up @@ -1228,7 +1237,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"

#### Upstream source location (in CMake build tree)

llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b9957`.
llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b9959`.

**GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
**Build:**
![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)
![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)
[![llama.cpp b9957](https://img.shields.io/badge/llama.cpp-%23b9957-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b9957)
[![llama.cpp b9959](https://img.shields.io/badge/llama.cpp-%23b9959-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b9959)
[![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)
![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)
[![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)
Expand Down
2 changes: 2 additions & 0 deletions docs/history/llama-cpp-breaking-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -471,3 +471,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
| b9941–b9947 | upstream verification (compare diff) | All **eight** patches (`0001`–`0008`) apply unchanged against b9947: the `common/arg.cpp` change is a localized `.set_examples()` metadata edit that does not overlap patch `0001`'s parse-function region, no `tools/server/*` file is touched (`0002`–`0008` unaffected), and no OuteTTS generator anchor (`tools/tts/tts.cpp` unchanged) — confirmed via the upstream `b9941...b9947` compare diff. Full configure (fail-loud patch apply + TTS extraction) + build + `ctest` to be confirmed by the CI pipeline. |
| b9947–b9957 | `tools/server/server-tools.{cpp,h}` + `common/arg.cpp` + `tools/cli/cli-context.cpp` + `ggml/src/ggml-opencl/**` + `tools/ui/**` + `tests/test-batch-alloc.cpp` | **Additive / help-text-only, no public-API break** (106 files, ~43k insertions — the bulk is the auto-followed `tools/ui` WebUI rework ("ui: export full message tree", tooltip-flicker fix) plus new OpenCL cluster-parallel Adreno flash-attention kernels and a `test-batch-alloc` unit test, none project-reviewed/-shipped). The only CPU-side non-WebUI change of note is upstream's built-in server-tools revamp ("server: improve tools, remove apply_diff"): `server-tools.{cpp,h}` drop the `apply_diff` tool and rework the others, and `common/arg.cpp`'s sole edit is the matching one-line `--tools` help-text (removing `apply_diff` from the advertised list) — a different region than patch `0001`'s `common_params_parse` block. `server-tools.{cpp,h}` are **not** patch targets (patches touch `server.cpp` / `server-context.cpp` / `server-models.cpp`, all byte-identical across the range). All **eight** priority headers byte-identical; no project source changes required. |
| b9947–b9957 | upstream verification (sandbox) | All **eight** patches (`0001`–`0008`) re-verified against b9957: applied in filename order onto a clean b9957 checkout, all clean (the range touches no patch-target file except `common/arg.cpp`'s `--tools` help-text edit, which is outside patch `0001`'s parse-function region). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b9957`). Full local verification: fresh configure (fail-loud patch apply + TTS extraction) + full build (`jllama` + `jllama_test`) + `ctest` **479/479 passing**; per-platform confirmation by the CI pipeline. |
| b9957–b9959 | `ggml/src/ggml-cpu/arch/arm/quants.c` + `scripts/sync-ggml.last` | **No public-API break** (2 files — the only source change wraps two ARM NEON `vqtbl1q_u8()` table-lookup calls in the CPU quant kernel with the `ggml_vqtbl1q_u8()` helper; `scripts/sync-ggml.last` is a metadata bump). Entirely inside upstream-compiled ggml-cpu TUs (ARM-only), not a patch target, not an OuteTTS generator anchor (`tools/tts/tts.cpp` unchanged), and no header in the priority-8 list is touched. All **eight** patches (`0001`–`0008`) apply unchanged; no project source changes required — confirmed via the upstream `b9957...b9959` compare diff. |
| b9957–b9959 | upstream verification (compare diff) | All **eight** patches (`0001`–`0008`) apply unchanged against b9959: the single-range diff touches only `ggml/src/ggml-cpu/arch/arm/quants.c` (a NEON intrinsic-wrapper edit) and `scripts/sync-ggml.last` — no patch-target file (`common/arg.*`, `tools/server/*`) and no OuteTTS generator anchor (`tools/tts/tts.cpp` unchanged). All eight priority headers byte-identical across the range. Full configure (fail-loud patch apply + TTS extraction) + build + `ctest` to be confirmed by the CI pipeline. |
10 changes: 8 additions & 2 deletions docs/upgrade/llama-cpp-version-bump.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,16 @@ Once you have the `b<cur> -> b<next>` step, apply it exactly as
[`CLAUDE.md § Upgrading/Downgrading`](../../CLAUDE.md#upgradingdowngrading-llamacpp-version) describes.
Concretely:

1. **Edit the pin — three files:**
1. **Edit the pin — four files:**
- `llama/CMakeLists.txt` — the `GIT_TAG b<cur>` line **and** the `-DLLAMA_TAG=b<cur>` used by the
WebUI/TTS extraction (both must move together).
- `README.md` — the llama.cpp badge and link (version appears twice).
- `CLAUDE.md` — the "Current llama.cpp pinned version" line (and any build-example `b<nnnn>`).
- `llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java` — the `LLAMA_CPP_VERSION`
constant (the pure-Java pin consumers read for a version badge/log line). It mirrors `GIT_TAG`;
if you forget it, `NativeLibraryLoadSmokeTest.nativeBuildInfoMatchesPinnedVersionConstant` fails
the build (it cross-checks the constant against `LlamaModel.getLlamaCppBuildInfo()`, which reads
llama.cpp's own linked-in `build-info`).
2. **Re-verify `patches/`** — a clean configure re-runs the fail-loud `PATCH_COMMAND`, so every patch
`0001`–`0006` must still apply. Use a **fresh** build dir (a stale one re-applies over an
already-patched tree and reports a false "does not apply"):
Expand All @@ -127,7 +132,8 @@ Concretely:
`b<cur> -> b<next>` range (what broke / what was new; "no source change" is a valid row).
4. **Commit + push** on the working branch (do not open a new PR if one already tracks the branch):
```bash
git add llama/CMakeLists.txt README.md CLAUDE.md docs/history/llama-cpp-breaking-changes.md
git add llama/CMakeLists.txt README.md CLAUDE.md docs/history/llama-cpp-breaking-changes.md \
llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java
git commit -m "Upgrade llama.cpp from b<cur> to b<next>"
git push -u origin <your-branch>
```
Expand Down
4 changes: 2 additions & 2 deletions llama/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
llama.cpp
GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
GIT_TAG b9957
GIT_TAG b9959
PATCH_COMMAND ${CMAKE_COMMAND}
-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
-DLLAMA_SRC=<SOURCE_DIR>
Expand All @@ -197,7 +197,7 @@ execute_process(
COMMAND ${CMAKE_COMMAND}
-DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp
-DOUT_CPP=${JLLAMA_TTS_GEN_CPP}
-DLLAMA_TAG=b9957
-DLLAMA_TAG=b9959
-P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake
RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT
)
Expand Down
7 changes: 7 additions & 0 deletions llama/src/main/cpp/jllama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1239,6 +1239,13 @@ JNIEXPORT jbyteArray JNICALL Java_net_ladenthin_llama_LlamaModel_jsonSchemaToGra
}
}

// Returns llama.cpp's own build identifier ("b<number>-<commit>") from the linked-in build-info
// symbols. Pure-ASCII, so NewStringUTF is safe here (the modified-UTF-8 caveat only applies to
// model-generated payload text).
JNIEXPORT jstring JNICALL Java_net_ladenthin_llama_LlamaModel_nativeLlamaCppBuildInfo(JNIEnv *env, jclass clazz) {
return env->NewStringUTF(llama_build_info());
}

JNIEXPORT jstring JNICALL Java_net_ladenthin_llama_LlamaModel_handleCompletions(JNIEnv *env, jobject obj,
jstring jparams) {
REQUIRE_SERVER_CONTEXT(nullptr);
Expand Down
7 changes: 7 additions & 0 deletions llama/src/main/cpp/jllama.h
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,13 @@ JNIEXPORT void JNICALL Java_net_ladenthin_llama_LlamaModel_releaseTask(JNIEnv *,
*/
JNIEXPORT jbyteArray JNICALL Java_net_ladenthin_llama_LlamaModel_jsonSchemaToGrammarBytes(JNIEnv *, jclass, jstring);

/*
* Class: net_ladenthin_llama_LlamaModel
* Method: nativeLlamaCppBuildInfo
* Signature: ()Ljava/lang/String;
*/
JNIEXPORT jstring JNICALL Java_net_ladenthin_llama_LlamaModel_nativeLlamaCppBuildInfo(JNIEnv *, jclass);

/*
* Class: net_ladenthin_llama_LlamaModel
* Method: handleRerank
Expand Down
17 changes: 17 additions & 0 deletions llama/src/main/java/net/ladenthin/llama/LlamaModel.java
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,21 @@ public String decode(int... tokens) {
*/
public static native void setLogger(LogFormat format, BiConsumer<LogLevel, String> callback);

/**
* Returns the upstream llama.cpp build identifier actually linked into the loaded native
* library, composed as {@code "b<build-number>-<commit>"} (e.g. {@code "b9959-0badc06ab"}).
*
* <p>Unlike the compile-time constant {@link net.ladenthin.llama.value.LlamaCppVersion#LLAMA_CPP_VERSION}
* (a pure-Java string that can be read without the native library), this reads llama.cpp's own
* {@code build-info} through JNI, so it reports the genuine build number and resolved commit of
* the binary and cannot silently drift from it. It requires {@code libjllama} to be loaded.</p>
*
* @return the linked llama.cpp build info string, e.g. {@code "b9959-0badc06ab"}
*/
public static String getLlamaCppBuildInfo() {
return nativeLlamaCppBuildInfo();
}

@Override
public synchronized void close() {
// synchronized so two concurrent close() calls cannot both observe a live native
Expand Down Expand Up @@ -486,6 +501,8 @@ protected final void finalize() {

private static native byte[] jsonSchemaToGrammarBytes(String schema);

private static native String nativeLlamaCppBuildInfo();

/**
* Converts a JSON schema to a grammar string usable by {@link net.ladenthin.llama.parameters.ModelParameters#setGrammar(String)}.
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// SPDX-FileCopyrightText: 2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>
// SPDX-FileCopyrightText: 2023-2025 Konstantin Herud
//
// SPDX-License-Identifier: MIT

package net.ladenthin.llama.value;

/**
* The pinned upstream <a href="https://github.com/ggml-org/llama.cpp">llama.cpp</a> build tag this
* library was compiled against, exposed as a compile-time constant so callers can render a badge or
* emit a startup log line without loading the native library.
*
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b9959"}) that mirrors the
* {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is
* absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a
* lightweight version badge in Android or other UIs.</p>
*
* <p>For the <em>authoritative</em> value that is baked into the native binary — the build number
* plus the resolved upstream commit, e.g. {@code "b9959-0badc06ab"} — call
* {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own
* {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires
* the native library to be loaded).</p>
*/
public final class LlamaCppVersion {

/**
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b9959"}.
*
* <p>Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the
* "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the
* compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the
* value actually linked into the native binary.</p>
*/
public static final String LLAMA_CPP_VERSION = "b9959";

// Constants holder — not instantiable.
private LlamaCppVersion() {}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,13 @@
package net.ladenthin.llama.loader;

import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assumptions.assumeTrue;

import net.ladenthin.llama.ClaudeGenerated;
import net.ladenthin.llama.LlamaModel;
import net.ladenthin.llama.value.LlamaCppVersion;
import org.junit.jupiter.api.Test;

/**
Expand Down Expand Up @@ -59,4 +63,24 @@ void loadingNativeLibraryRunsJniOnLoadWithoutError() {
"LlamaModel.<clinit> must load the native library and JNI_OnLoad must resolve "
+ "every FindClass'd Java class");
}

/**
* The native build-info getter must resolve (proving the {@code nativeLlamaCppBuildInfo} JNI
* symbol has C linkage and is reachable) and its value must agree with the compile-time
* {@link LlamaCppVersion#LLAMA_CPP_VERSION} pin — catching the exact drift the "Upgrading
* llama.cpp Version" checklist warns about (a {@code GIT_TAG} bump that forgets the constant).
* {@code llama_build_info()} returns {@code "b<number>-<commit>"}, so it must start with the
* pinned tag followed by {@code '-'}.
*/
@Test
void nativeBuildInfoMatchesPinnedVersionConstant() {
assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping native build-info check");
String buildInfo = LlamaModel.getLlamaCppBuildInfo();
assertNotNull(buildInfo, "getLlamaCppBuildInfo() must return the linked llama.cpp build identifier");
assertTrue(
buildInfo.startsWith(LlamaCppVersion.LLAMA_CPP_VERSION + "-"),
"Linked build-info \"" + buildInfo + "\" must start with the pinned tag \""
+ LlamaCppVersion.LLAMA_CPP_VERSION + "-\"; if this fails, GIT_TAG in "
+ "llama/CMakeLists.txt and LlamaCppVersion.LLAMA_CPP_VERSION have drifted apart");
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// SPDX-FileCopyrightText: 2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>
// SPDX-FileCopyrightText: 2023-2025 Konstantin Herud
//
// SPDX-License-Identifier: MIT

package net.ladenthin.llama.value;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.matchesRegex;
import static org.hamcrest.Matchers.notNullValue;

import java.lang.reflect.Constructor;
import java.lang.reflect.Modifier;
import org.junit.jupiter.api.Test;

/**
* Tests for the {@link LlamaCppVersion#LLAMA_CPP_VERSION} compile-time constant.
*
* <p>These are pure-Java assertions with no native dependency: they pin the shape of the pinned
* llama.cpp tag ({@code b} followed by digits, e.g. {@code "b9959"}) rather than a literal value, so
* a routine version bump does not break the test while a malformed edit (empty string, wrong prefix)
* still does. The authoritative runtime value ({@code LlamaModel#getLlamaCppBuildInfo()}) is
* exercised separately by the native-load smoke test.
*/
public class LlamaCppVersionTest {

Check warning on line 25 in llama/src/test/java/net/ladenthin/llama/value/LlamaCppVersionTest.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove redundant visibility modifiers from this test class and its methods.

See more on https://sonarcloud.io/project/issues?id=bernardladenthin_java-llama.cpp&issues=AZ9QFiS9OI8JwMZvpW0u&open=AZ9QFiS9OI8JwMZvpW0u&pullRequest=327

@Test
public void testVersionConstantIsPresentAndWellFormed() {
assertThat(LlamaCppVersion.LLAMA_CPP_VERSION, notNullValue());
// The pinned tag is always "b" followed by the upstream build number.
assertThat(LlamaCppVersion.LLAMA_CPP_VERSION, matchesRegex("^b\\d+$"));
}

@Test
public void testClassIsNotInstantiable() throws Exception {
Constructor<LlamaCppVersion> constructor = LlamaCppVersion.class.getDeclaredConstructor();
assertThat(Modifier.isPrivate(constructor.getModifiers()), org.hamcrest.Matchers.is(true));
}
}
Loading