Hugging Face brings multimodal AI agents to JavaScript
Hugging Face just launched Agents.js, a new library that brings agentic tool use to JavaScript developers across browsers and Node environments. It ships with multimodal tools like text-to-image and captioning out of the box, while supporting custom models and APIs. Why it matters: Web developers can now build tool-using agents in native JS, though Hugging Face warns that evaluating model-generated code carries security risks.
Agents.js is a new library from Hugging Face that lets developers give language models tool access directly inside JavaScript, running either in the browser or on Node.js.
While most agent frameworks live in the Python ecosystem, Agents.js translates plain-text prompts directly into executable JavaScript code that chains multimodal APIs together.
What you can do with it
Getting started takes a single npm install of @huggingface/agents. Once you create an HfAgent instance with your Hugging Face access token, you can start passing natural language prompts.
For example, if you ask it to draw a rubber duck with a top hat and caption the result, the agent writes an async JavaScript function that calls text-to-image, feeds the resulting image into image-to-text, and returns the output as structured updates.
It handles multimodal inputs out of the box. You can pass raw HTML FileList objects directly into the agent, letting users upload an image and have the model caption it and read the caption aloud via text-to-speech.
Custom setups and code execution warnings
By default, the library runs on Hugging Face's hosted OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 model. You can swap that out for any custom model, including OpenAI's API, by passing an async function that handles prompts and returns text completions.
Adding custom tools is equally straightforward. You provide a name, description, usage examples, and an async call function, then pass your tool array right into the HfAgent constructor.
The catch is how it executes tasks. Agents.js evaluates arbitrary model-generated JavaScript directly in your browser or Node runtime. Hugging Face explicitly warns against running this in untrusted environments and suggests using generateCode to inspect the script before running evaluateCode.
A hosted demo powered by Open Assistant 30B is available to test the tool-use workflow without writing code.
Why it matters
For frontend and full-stack developers, Agents.js lowers the barrier to building agentic workflows without touching Python. Its native support for browser primitives like FileList and Blobs makes rich media manipulation feel surprisingly natural.
Evaluating un-sandboxed model code remains a serious security hurdle for production apps. We think it is a great playground for prototypes and internal tooling right now, but deploying it to end users will require much tighter sandboxing.
Sources (1)
- huggingface.co(opens in a new tab)Originalhuggingface.co