NVIDIA brings custom AI to your local PC

If you want an AI to organize your private notes but hate the idea of uploading sensitive files to cloud servers, your graphics card just found a new job.

Nvidia just launched Chat with RTX, a free tech demo that lets Windows users run a personalized AI chatbot locally on their own hardware. All you need is an RTX 30- or 40-series GPU with at least 8GB of VRAM.

Local documents and YouTube transcripts

The app uses retrieval-augmented generation (RAG) paired with Nvidia TensorRT-LLM software to connect open-source models like Mistral and Llama 2 to your own files. Point it at a folder filled with .txt, .pdf, .doc, .docx, or .xml files, and it indexes them into a local library within seconds.

You can also paste in YouTube video or playlist links. Chat with RTX pulls the transcripts, letting you search through hours of podcasts or tutorials for specific quotes or generate fast video summaries.

According to early hands-on testing by The Verge, parsing messy legal files from the FTC v. Microsoft case delivered near-instant answers without the lag typical of cloud services. It also handled complex PDF summaries far better than Microsoft Copilot inside Word.

A very rough developer demo

While promising, Chat with RTX is unmistakably an early developer playground rather than polished consumer software.

For starters, it takes up nearly 40GB of storage and takes roughly 30 minutes to install, launching a local web server, a Python instance eating roughly 3GB of RAM, and an open command prompt spitting error codes. Nvidia even noted a bug where changing the default installation directory breaks the setup entirely.

Testers also ran into limitations quickly. Trying to index 25,000 documents crashed the app. The YouTube tool occasionally downloaded transcripts for completely unrelated videos. Crucially, the chatbot lacks conversational memory, meaning you cannot ask follow-up questions based on previous answers. It also dumps JSON index files straight into the target folders.

Why it matters

Nvidia is showing how local AI can solve the two biggest complaints about cloud models: privacy and subscription fatigue. For journalists, lawyers, or anyone handling sensitive data offline, running a capable model without sending data to third parties is a massive win over paying $20 a month for ChatGPT Plus or Copilot Pro.

We think it is not quite ready for casual users yet. Between the massive 40GB download, lack of conversational context, and buggy indexing, this is strictly a proof of concept for enthusiasts with beefy rigs.

Strategically, this is Nvidia defending its moat by giving developers an off-the-shelf reference architecture. By proving that TensorRT-LLM can deliver snappy, local RAG on desktop hardware, Nvidia is nudging the industry toward hybrid AI—and reminding PC owners why that expensive GeForce card in their tower is worth keeping.