# Get Started The Chat Question & Answer (Chat Q&A) sample application is a Retrieval Augmented Generation (RAG) pipeline designed to be used in a single user non concurrent deployment primarily targeting IntelĀ® Core™ deployments. The application is intended to help developers create personal intelligent chatbots that can answer questions based on their private data. This guide will help you set up, run, and modify the Chat Q&A sample application on Intel Edge AI systems. By following this guide, you will learn how to: - **Set up the sample application**: Use Docker Compose to quickly deploy the application in your environment. - **Run the application**: Execute the application to see real-time question answering based on your data. - **Modify application parameters**: Customize settings like inference models and deployment configurations to adapt the application to your specific requirements. ## Prerequisites - Verify that your system meets the [minimum requirements](./get-started/system-requirements.md). - Install Docker: [Installation Guide](https://docs.docker.com/get-docker/). - Install Docker Compose: [Installation Guide](https://docs.docker.com/compose/install/). - HF Token is mandatory if the model used is a private or gated model. Else, it is optional. ## Running the application using Docker Compose 1. **Clone the Repository**: Clone the repository. ```bash # Clone the latest on mainline git clone https://github.com/open-edge-platform/edge-ai-libraries.git edge-ai-libraries # Alternatively, Clone a specific release branch git clone https://github.com/open-edge-platform/edge-ai-libraries.git edge-ai-libraries -b release-2026.3.0 ``` > [!NOTE] > Adjust the repo link appropriately in case of forked repo. 2. **Navigate to the Directory**: Go to the directory where the Docker Compose file is located: ```bash cd edge-ai-libraries/sample-applications/chat-question-and-answer-core ``` 3. **Configure Image Pulling Registry and Tag Environment Variables**: To utilize the release images for the Chat Q&A sample application from the registry, set the following environment variables: ```bash export REGISTRY="intel/" export UI_TAG=core_2026.3.0-rc1 # For OpenVINO framework, set the following: export BACKEND_TAG=core_2026.3.0-rc1 # For those who prefer Ollama framework, set the following: export BACKEND_TAG=core_ollama_2026.3.0-rc1 ``` Skip this step if you prefer to build the sample application from source. For detailed instructions, refer to **[How to Build from Source](./build-from-source.md)** guide for details. 4. **Set Up Environment Variables**: Choose one of the following options based on your hardware setups: - **OpenVINO toolkit framework** - Support CPU, GPU and NPU devices (Default): ```bash # HF Token is mandatory for gated or private model. Else, it can be skipped. export HUGGINGFACEHUB_API_TOKEN= source scripts/setup_env.sh ``` - **Ollama framework** - CPU-only supported ```bash source scripts/setup_env.sh -b ollama ``` 5. **Model Configuration Setup**: - You can define model settings through a YAML configuration file. To assist you in getting started, sample templates for the respective framework are available in the [Sample directory](https://github.com/open-edge-platform/edge-ai-libraries/tree/release-2026.3.0/sample-applications/chat-question-and-answer-core/model_config/sample). After creating and customizing your YAML file, set the `MODEL_CONFIG_PATH` environment variable to point to its location. You can specify the path to your YAML configuration file using either an absolute path or a relative path. If you are not familiar with Docker volume mount logic, Intel recommends using a full path to avoid any potential issues. ```bash # Recommended: Using a full path to your YAML file export MODEL_CONFIG_PATH= ``` If `MODEL_CONFIG_PATH` is not set, the application will automatically fall back to its built-in default configuration, which uses the same [sample template YAML](https://github.com/open-edge-platform/edge-ai-libraries/tree/release-2026.3.0/sample-applications/chat-question-and-answer-core/model_config/sample) file for the respective framework. - A collection of ready-to-use YAML configuration files featuring validated combinations of LLMs, Embedding models, and Rerankers is available in the [model_config folder](https://github.com/open-edge-platform/edge-ai-libraries/tree/release-2026.3.0/sample-applications/chat-question-and-answer-core/model_config). Feel free to explore these examples and use them as a starting point. - If you wish to assign the inference workload to other dedicated device such as CPU/GPU/NPU device independently, you can assign each model's inference workload to a specific device(e.g., CPU, GPU or NPU) independently by configuring the `device_settings` section in the YAML as below: ```bash # EMBEDDING_DEVICE: Specifies the device for embedding model inference. # RERANKER_DEVICE: Specifies the device for reranker model inference. # LLM_DEVICE: Specifies the device for LLM model inference. # Make sure that you are using the correct backend framework image if you wish to use GPU or NPU inferencing. device_settings: EMBEDDING_DEVICE: "GPU" RERANKER_DEVICE: "GPU" LLM_DEVICE: "GPU" ``` > [!NOTE] > > - **GPU inferencing only supported for OpenVINO toolkit framework not Ollama framework.** > - If the system has an integrated GPU, its id is always 0 (GPU.0). The GPU is an alias for GPU.0. If a system has multiple GPUs (for example, an integrated and a discrete IntelĀ® GPU) It is done by specifying GPU.0, GPU.1 > > ```bash > device_settings: > EMBEDDING_DEVICE: "" > RERANKER_DEVICE: "" > LLM_DEVICE: "" > ``` 6. **Start the Application**: Start the application using Docker Compose tool: ```bash docker compose -f docker/compose.yaml up ``` 7. **Verify the Application**: The following log will be printed on the console, confirming that the application is ready for use. ```text # chatqna-core | INFO: Application startup complete. # chatqna-core | INFO: Uvicorn running on http://0.0.0.0:8888 ``` 8. **Access the Application**: Open a browser and go to `http://:8102` to access the application dashboard. The application dashboard allows the user to: - Create and manage context by adding documents (pdf, docx, etc. Note: Web links are not supported for the Core version of the sample application. Note: There are restrictions on the max size of the document allowed.) - Start Q&A session with the created context. ## Advanced Setup Options For alternative ways to set up the sample application, see: - [How to Build from Source](./build-from-source.md) ## Supporting Resources - [Docker Compose Documentation](https://docs.docker.com/compose/) - [API Reference](./api-reference.md) :::{toctree} :hidden: get-started/system-requirements :::