Getting Started

Run GGUF models from Hugging Face on iOS and Android, on the device. Pick one that fits in the phone’s memory: Model Management lists some that do. The provider is built on llama.rn, a third-party React Native binding for llama.cpp.

Model	Create with	Use with
Language	llama.languageModel(path)	generateText, streamText
Embedding	llama.embeddingModel(path)	embed, embedMany
Reranking	llama.rerankingModel(path)	rerank
Speech	llama.speechModel(path, ...)	generateSpeech

Installation

npm

npm install @react-native-ai/llama llama.rn ai

yarn

yarn add @react-native-ai/llama llama.rn ai

pnpm

pnpm add @react-native-ai/llama llama.rn ai

bun

bun add @react-native-ai/llama llama.rn ai

The provider works with AI SDK v7 and needs its polyfills. It requires React Native 0.76 or newer and llama.rn 0.13.0-rc.8 or newer.

Expo

Add the llama.rn config plugin to your app config:

app.json

{
  "expo": {
    "plugins": ["llama.rn"]
  }
}

See llama.rn’s Expo setup for its options, such as entitlements and OpenCL on Android.

Usage

Download a model, load it, and generate:

import { downloadModel, llama } from '@react-native-ai/llama'
import { streamText } from 'ai'

const modelPath = await downloadModel(
  'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf'
)

const model = llama.languageModel(modelPath)
await model.prepare()

const { textStream } = streamText({
  model,
  prompt: 'Explain quantum computing in simple terms',
})

for await (const delta of textStream) {
  console.log(delta)
}

await model.unload()

• downloadModel fetches the file from Hugging Face once and returns its local path. Model IDs are owner/repo/filename.gguf.
• prepare() loads the model into memory. Without it, the first request loads the model and logs a warning.
• unload() frees the memory when you are done.

The rest of these docs start from a prepared model. See Model Management for downloading, recommended models, and all model options.