llama-cpp

About this Skill

Operate, configure, benchmark, and troubleshoot llama.cpp across CPU, Metal, CUDA, HIP/ROCm, Vulkan, SYCL, and hybrid or multi-GPU systems. Use when installing or building llama.cpp, selecting or inspecting GGUF models, running llama-cli, serving an OpenAI-compatible API with llama-server, tuning memory and performance, or diagnosing backend, context, template, and API failures. Do not use for model training or fine-tuning, general inference-framework selection, llama-cpp-python or other bindings, LlamaIndex, Ollama, or LM Studio operation.

Use cases

• Build llama.cpp for CPU or accelerator backends
• Inspect and run GGUF models
• Serve an OpenAI-compatible local API
• Benchmark inference and serving performance
• Diagnose backend, memory, context, template, and API failures

Features

• Backend and device verification
• GGUF model and memory planning
• llama-cli and llama-server workflows
• Controlled performance tuning and benchmarking
• Structured troubleshooting with evidence and safety boundaries

Requirements

• A supported llama.cpp binary or build environment
• A compatible GGUF model file for model execution
• Matching accelerator drivers and SDKs when using an accelerator backend

Supported Agents

Skill files

Related Skills