Docs: modernize README — position as JVM-native LLM inference & serving engine

[mikepapadim]

Summary

Modernizes the README to position GPULlama3.java clearly as a JVM-native LLM inference & serving engine — the vLLM-style path for the Java ecosystem — while preserving all existing content. Trims 517 → 329 lines and makes it far more scannable.

Docs-only change: README.md.

What changed

Positioning

• New hero: "Think vLLM — but pure Java, and it runs on any GPU." Frames the project as an inference and serving engine, not just a model runner.
• Aligns tone with the TornadoVM site branding ("Java on GPUs. And more.", "no JNI glue, no second toolchain"): code-first, benchmark-driven, scannable.

New sections

• Why GPULlama3.java — pure Java, run-on-any-GPU, LangChain4j/Quarkus, serving, many models.
• Quickstart (60 seconds) — SDKMAN install + a JBang one-liner, up top.
• Serving: OpenAI-compatible (preview) — surfaces the serving direction, each item linked to its in-flight PR and clearly marked preview:
  • OpenAI-compatible server (--server, /v1/chat/completions, streaming) — OpenAI-compatible server (llama-tornado --server): /v1/chat/completions + /v1/completions, streaming, zero-dep #135
  • On-device greedy sampling (~500× less D2H) — On-device greedy sampling (-Dllama.deviceSample): GPU argmax, keep logits device-side (~500x less D2H) #134
  • Static batched decode (up to 41× aggregate throughput) — Static batched decode (LLaMA + Qwen3): B independent sequences per step, up to 41x aggregate throughput #129
  • Tensor-core (MMA) batch prefill — Add TornadoVM CUDA backend support and tensor-core (MMA) accelerated batch prefill (FP16 & Q8_0, Llama & Qwen3) #127
  • --bench llama-bench-style benchmarking — Add llama-bench-style benchmark: llama-tornado --bench (pp/tg/pg matrix, avg±stddev t/s, md/csv/json) #133

Preserved (verbatim where it matters)

• Performance table, all model collections, Maven coordinates (0.5.0 / 0.5.0-jdk25), the LangChain4j example, llama-tornado / llamaTornado / JBang / Docker usage, GPU-memory guidance, funding, MIT, and both demo GIFs.
• The full --help output and debug flags are kept behind a collapsible block, folded into an "Embed in your own tools" section alongside --show-command.

Restructured for scannability
Why → Quickstart → Performance → Serving → LangChain4j/Quarkus + Maven → Install → CLI (llama-tornado / llamaTornado / JBang / Docker) → Models → GPU memory → Embed & options → Roadmap → Acknowledgments → License. Emoji section accents throughout, consistent with the TornadoVM docs style.

Notes

• The serving features are unmerged PRs, so they are presented as preview with links rather than claimed as shipped — happy to promote them once merged.
• No code changes; badges and asset paths (docs/ll.gif, docs/inter-output.gif, roadmap/optimizations docs) are unchanged and verified to exist.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.