Adaptive Token Compressor#
Adaptive Token Compressor is a pluggable Python library purpose-built for LLM agent systems. Through a single unified compressor interface it applies tailored compression to each part of an agent pipeline — system prompt (harness), tool schemas, and conversation context — to significantly reduce token usage and improve inference efficiency on edge hardware.
Use Cases#
Reducing LLM inference cost in agentic pipelines with large system prompts or tool catalogs.
Compressing tool schemas at runtime so only contextually relevant tools are included in the prompt.
Compressing long conversation history and tool outputs in multi-turn agent sessions.
Tracking per-compressor and cross-compressor token savings, latency, and compression ratio telemetry.
Drop-in integration into existing agent frameworks via a factory-based compressor API.
Key Capabilities#
Unified compressor API with three specialized types:
harness(conversation messages and system prompt),tool(tool schema selection), andcontext(experimental history and tool-output compression).HarnessCompressor — section-aware message compression using hybrid rule-based and model-based compression; preserves instruction-critical content while reducing token cost.
ToolCompressor — LLM-guided or reranker-based tool selection that scores and filters tool schemas from conversation context, with configurable injection placements to trade off token savings against prefix-cache hit rate.
ContextCompressor (Experimental) — turn-aware optimization for long, tool-heavy conversations. It organizes history around tool-interaction turns, keeping tool calls and their results connected while applying selective compression across older turns. Recent-turn retention, recovery-guided historical stubs, and optional edit-aware safeguards help maintain continuity as the conversation evolves. Eligible content can also be compressed through Claw Compactor or Compress Server.
Compress Server runtimes — supports PyTorch and OpenVINO™ runtimes for the compression service, including Intel GPU (XPU) acceleration.
CompressionManager — orchestrates multiple compressors in a single pipeline and provides cross-compressor metric aggregation (tokens saved, compression ratio, average latency per request).
Multi-key metrics aggregation — independently track metrics by provider, user, or other application-defined dimensions alongside global metrics, with per-key snapshots through
CompressionManager.Factory construction — create and inspect compressors by type name at runtime using
create_compressor(),available_compressor_types(), andconfig_schema().
Get Started#
If you want to reduce token usage and improve inference efficiency, start here with the basics:
Installation — set up the library and required dependencies.
Quick Start — run your first compression workflow in just a few steps.
Once it is up and running, explore the full Get Started guide for configuration options, compressor principles and workflow, compressor metrics, single- and multi-compressor metrics collection, testing, and additional resources.
Next Steps#
Deploy Compress Server — set up the compress server required by HarnessCompressor and ContextCompressor.
Deploy LLM for Tool Prediction — set up the predictor service required by ToolCompressor.
Release Notes — version history and changelog.