Home
Data Structures and Algorithms
About

Guide contents

Local LLMs and Coding Agents in Java

How LLMs Work

1.What Is a Large Language Model?
2.Talking to a Model: Prompts, Context, and Limits
3.Agents and LLM Engineering: The Vocabulary
4.Introduction and First Run
5.What a Language Model Computes
6.Tokens, Vocabularies, and Special Tokens
7.Building a BPE Tokenizer
8.The Math Behind Transformers, in Java
9.Self-Attention Step by Step
10.The Transformer Block and the Full Model
11.Generation, Prefill, Decode, and the KV Cache
12.Sampling: Greedy, Temperature, Top-k, Top-p, and Min-p
13.Penalties, Logit Bias, and Sampler Chains
14.How Models Are Trained, and the Variants You Download
15.Chat Templates, Roles, and Special Tokens

Hardware, Numbers, and Quantization

16.CPUs, GPUs, and Memory for Inference
17.FLOPs, Bandwidth, and the Roofline Model
18.Number Formats: FP32, FP16, BF16, FP8, INT8, INT4, and FP4
19.Memory Math: Weights, KV Cache, and What Fits
20.Quantization Fundamentals
21.Building Q8_0 and Q4_0 Like ggml
22.k-quants, i-quants, and the Importance Matrix
23.GPTQ, AWQ, EXL3, NF4, and FP8 Serving
24.Making, Quantizing, and Testing Your Own GGUF

Model Files and Hugging Face

25.The Hugging Face Hub
26.The Hub HTTP API and Cache Layout in Java
27.The safetensors Format
28.The GGUF Format

The loop CLI and Model Management

29.picocli Basics
30.Subcommands, Configuration, and Packaging
31.Downloading Large Files Reliably in Java
32.Workshop: loop pull, list, show, rm, and fit

Inference Engines

33.llama.cpp Architecture
34.Running llama.cpp: Core Flags
35.Tuning llama.cpp
36.llama-server: Endpoints, Slots, and Prompt Cache
37.The Chat Completions Protocol and Streaming in Java
38.vLLM Architecture and Installation
39.Serving with vLLM
40.The Engine Landscape and How Ollama Works
41.Workshop: loop serve, run, and chat

Inference Engineering

42.Attention Optimizations and Long Context
43.Batching and Scheduling
44.Paged KV Cache and Prefix Caching
45.Speculative Decoding
46.Constrained Decoding and Structured Output
47.Multi-GPU Inference and Serving at Scale
48.Benchmarking Inference

Build an Inference Engine in Java

49.Loading GGUF with the FFM API
50.A Real Tokenizer from GGUF Metadata
51.The Forward Pass in Java
52.Quantized Kernels, the Vector API, and Threads
53.Capstone: A Chat Engine in Pure Java

Tool Calling and Coding Agents

54.Function Calling from First Principles
55.The Agent Loop and a Tool Framework
56.File Tools
57.The Shell Tool
58.Web Search and Fetch Tools
59.Permissions, Sandboxing, and Prompt Injection
60.Context Engineering for Coding Agents
61.Edit Formats
62.Making Small Local Models Reliable
63.Workshop: loop agent
64.The Model Context Protocol
65.Evaluating Models and Agents

An HTTP Gateway with Spring Boot

66.Spring Boot Essentials for an LLM Gateway
67.An OpenAI-Compatible API in Front of llama-server
68.Streaming Responses Through the Gateway
69.Loading, Unloading, and Scheduling Models
70.Securing and Observing the Gateway
71.Workshop: loop gateway

Reading Real Codebases

72.Reading Coding Harness Codebases
73.Reading Inference Engine Codebases

Local LLMs and Coding Agents in Java

How LLMs Work

1.What Is a Large Language Model?
2.Talking to a Model: Prompts, Context, and Limits
3.Agents and LLM Engineering: The Vocabulary
4.Introduction and First Run
5.What a Language Model Computes
6.Tokens, Vocabularies, and Special Tokens
7.Building a BPE Tokenizer
8.The Math Behind Transformers, in Java
9.Self-Attention Step by Step
10.The Transformer Block and the Full Model
11.Generation, Prefill, Decode, and the KV Cache
12.Sampling: Greedy, Temperature, Top-k, Top-p, and Min-p
13.Penalties, Logit Bias, and Sampler Chains
14.How Models Are Trained, and the Variants You Download
15.Chat Templates, Roles, and Special Tokens

Hardware, Numbers, and Quantization

16.CPUs, GPUs, and Memory for Inference
17.FLOPs, Bandwidth, and the Roofline Model
18.Number Formats: FP32, FP16, BF16, FP8, INT8, INT4, and FP4
19.Memory Math: Weights, KV Cache, and What Fits
20.Quantization Fundamentals
21.Building Q8_0 and Q4_0 Like ggml
22.k-quants, i-quants, and the Importance Matrix
23.GPTQ, AWQ, EXL3, NF4, and FP8 Serving
24.Making, Quantizing, and Testing Your Own GGUF

Model Files and Hugging Face

25.The Hugging Face Hub
26.The Hub HTTP API and Cache Layout in Java
27.The safetensors Format
28.The GGUF Format

The loop CLI and Model Management

29.picocli Basics
30.Subcommands, Configuration, and Packaging
31.Downloading Large Files Reliably in Java
32.Workshop: loop pull, list, show, rm, and fit

Inference Engines

33.llama.cpp Architecture
34.Running llama.cpp: Core Flags
35.Tuning llama.cpp
36.llama-server: Endpoints, Slots, and Prompt Cache
37.The Chat Completions Protocol and Streaming in Java
38.vLLM Architecture and Installation
39.Serving with vLLM
40.The Engine Landscape and How Ollama Works
41.Workshop: loop serve, run, and chat

Inference Engineering

42.Attention Optimizations and Long Context
43.Batching and Scheduling
44.Paged KV Cache and Prefix Caching
45.Speculative Decoding
46.Constrained Decoding and Structured Output
47.Multi-GPU Inference and Serving at Scale
48.Benchmarking Inference

Build an Inference Engine in Java

49.Loading GGUF with the FFM API
50.A Real Tokenizer from GGUF Metadata
51.The Forward Pass in Java
52.Quantized Kernels, the Vector API, and Threads
53.Capstone: A Chat Engine in Pure Java

Tool Calling and Coding Agents

54.Function Calling from First Principles
55.The Agent Loop and a Tool Framework
56.File Tools
57.The Shell Tool
58.Web Search and Fetch Tools
59.Permissions, Sandboxing, and Prompt Injection
60.Context Engineering for Coding Agents
61.Edit Formats
62.Making Small Local Models Reliable
63.Workshop: loop agent
64.The Model Context Protocol
65.Evaluating Models and Agents

An HTTP Gateway with Spring Boot

66.Spring Boot Essentials for an LLM Gateway
67.An OpenAI-Compatible API in Front of llama-server
68.Streaming Responses Through the Gateway
69.Loading, Unloading, and Scheduling Models
70.Securing and Observing the Gateway
71.Workshop: loop gateway

Reading Real Codebases

72.Reading Coding Harness Codebases
73.Reading Inference Engine Codebases

Home/Local LLMs and Coding Agents in Java/Reading Real Codebases

Reading Real Codebases

Guided reading of open-source coding harnesses (Pi, Aider, mini-swe-agent, OpenHands, Crush) and inference engines (llama.cpp, vLLM, Ollama, SGLang).

2 lessons

72.Reading Coding Harness Codebases
73.Reading Inference Engine Codebases

© 2026 codeloopback