codeloopback
Guide contents

Build an Inference Engine in Java

A transformer inference engine in plain Java: GGUF loading with the FFM API, a tokenizer from GGUF metadata, the forward pass, quantized kernels with the Vector API, and a chat engine.

5 lessons

  1. 49.Loading GGUF with the FFM API
  2. 50.A Real Tokenizer from GGUF Metadata
  3. 51.The Forward Pass in Java
  4. 52.Quantized Kernels, the Vector API, and Threads
  5. 53.Capstone: A Chat Engine in Pure Java