Handheld (Soldier System) Blueprint#
The Handheld Blueprint is a full-stack AI inference and observability software collection consisting of both single- and multi-modal components that are optimized for Intel® edge hardware in handheld deployment scenarios.
This composite application combines a conversational agent exposed via Chat UI that is backed by a LLM inference server, a speech-to-text service and Visual Pipeline and Platform Evaluation Tool. All components of the composite application share the visual pipeline solution’s Docker network.
The diagram below shows how these components are layered on top of the Edge Node Infrastructure software stack.
The Handheld Multi-Modal application is deployed on top of the Edge Node Infrastructure software - an edge computing platform, which enables hardware acceleration capabilities. See Infrastructure Setup for build and provisioning steps.
Deployment and Usage#
Follow these steps to deploy the Handheld (Soldier System) Blueprint:
Infrastructure Setup — Build the OS image, flash it to a bootable USB, and validate the provisioned platform.
Install OEP SDKs — Verify hardware accelerators and install the OEP Vision AI SDK on the provisioned target.
Install Handheld Multi-Modal Application — Download and deploy the composite application stack.
Access Application User Interface — Connect to the application endpoints and explore each component.
Benchmarks — Use the bundled ViPPET tool to benchmark AI pipelines across available hardware.
Components#
The Handheld application combines a conversational agent (Chat UI) exposed as Open WebUI component backed by LLM model served through the OpenVINO Model Server platform, a speech-to-text transcription functionality realized by the Whisper model, and observability dashboard exposed via Grafana dashboard for a live view of platform utilization and application metrics.
Visual Pipeline and Platform Evaluation Tool#
The Visual Pipeline and Platform Evaluation Tool simplifies hardware selection for AI workloads by enabling configuration of workload parameters, performance benchmarking, and analysis of key metrics such as throughput, CPU usage, and GPU usage. With its intuitive interface, the tool provides actionable insights that support optimized hardware selection and performance tuning.
For more information, see ViPPET documentation.
Speech To Text (Whisper Model)#
This component is responsible for speech to text functionality and uses Whisper model. Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.
For more information, see Whisper documentation.
Web UI#
Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various runners, such as Ollama and OpenAI-compatible APIs, with a built-in inference engine for RAG, making it a powerful AI deployment solution.
For more information, see Web UI documentation.
Observability#
The application includes Grafana Open Source (OSS), a data visualization and analytics tool. A Grafana Dashboard is
supplied that aggregates and presents metrics from the components of the application
and from the underlying platform. Metrics are streamed over websocket to Grafana
for a live, ephemeral on-device view. Additionally, a Prometheus endpoint is exposed at
localhost:9273/metrics address, from which data can be scraped for
long-term persistence.
AI Playground#
AI Playground is an optional open source generative AI application suite for offline AI chat, code assistance, document search, image analysis, and image and video generation, running on the same Edge Node Infrastructure software platform. See the AI Playground — FedAero Setup Guide for installation and setup steps.
AI Agent SKILLs#
The Handheld Blueprint supports AI agent skills for GitHub Copilot and Claude Code, covering both platform-level automation (image build, provisioning, power and thermal tuning) and application-level pipeline generation with Intel DL Streamer. See AI Agent SKILLs for the available skills and how to use them.