ChatRTX is a demo application developed by NVIDIA that allows users to create a personalized chatbot powered by a large language model (LLM) connected to their own content.
Local processing: ChatRTX runs locally on a user's Windows PC or workstation with an NVIDIA RTX GPU, ensuring fast and secure results without sending data to the cloud.
Custom content integration: Users can connect the LLM to their own documents, notes, photos, and other data.
Supported file formats: ChatRTX can process various file types, including text, PDF, DOC/DOCX, XML, PNG, JPG, and BMP.
Retrieval-augmented generation (RAG): The application leverages RAG technology to provide contextually relevant answers based on the user's personal data.
GPU acceleration: ChatRTX utilizes TensorRT-LLM and RTX acceleration for improved performance.
Multiple AI models: The application supports various models, including LLaMa 2 13B, Mistral 7B, ChatGLM3 6B, Whisper Medium (for voice input), and CLIP (for image processing).
Voice input: ChatRTX includes audio-to-text translation using the Whisper model, allowing users to interact with the chatbot using voice commands.
Image retrieval: The application can find and display images matching voice or text input.
ChatRTX is designed to help users quickly access and analyze their personal data, offering a secure and efficient way to interact with AI-powered language models without compromising privacy.