Practical Machine Learning in the Browser with JavaScript

Machine learning is no longer something only a server can do. Browsers now ship WebAssembly with SIMD and threads, and increasingly WebGPU, so a web page can classify images, transcribe speech, embed text for semantic search or run a small language model entirely on the user’s device. Nothing leaves the device, and you pay no per-request inference bill. This guide covers how to do it in practice with TensorFlow.js, ONNX Runtime Web and Transformers.js, when it is the right choice, and how to handle the biggest constraint: model size.

Why run machine learning in the browser?#

Client-side inference isn’t always the right architecture, but when it fits, the benefits are concrete:

• Privacy by design. The user’s photos, voice recordings or documents are processed locally. For sensitive inputs such as ID documents, health notes or private messages, not having to transmit the data at all is the strongest privacy guarantee you can give.
• Low latency. Once the model is loaded, there is no network round trip, so real-time uses like webcam pose tracking, live captioning or as-you-type suggestions become feasible.
• Cost that doesn’t scale with usage. The user’s hardware does the compute. Your costs are hosting static model files, which a CDN caches cheaply.
• Offline capability. With the model cached, features keep working without a connection, which suits progressive web apps and field tools.

The trade-offs are just as real: a one-off model download, wide variation in device capability, and memory limits, especially on mobile. Most of this guide is about managing those.

Use cases that work well today#

It helps to think in terms of tasks, not models. These categories are well served by small, browser-friendly models and mature tooling:

Vision#

• Image classification and tagging, for example suggesting categories when a user uploads a product photo, or rejecting blurry or wrong-orientation document scans before upload.
• Object and pose detection for fitness apps, gesture controls, virtual try-on and accessibility tools, often combined with a canvas overlay like the one in our Canvas animation tutorial.
• Background removal and segmentation for profile pictures and product images, without uploading the original photo.

Text#

• Embeddings and semantic search over help centres, documentation or product catalogues.
• Classification: sentiment, topic routing for contact forms, detecting language before choosing a translation path.
• Small generative models for autocomplete, rewriting or summarising short passages, where a quantised model of a few hundred million parameters is often enough.

Audio#

• Speech recognition with compact Whisper-family models for dictation and captions, keeping recordings on the device.
• Keyword spotting and audio classification for voice commands or sound-event detection.

What these have in common is bounded input, tolerance for occasional imperfect output, and a clear user benefit from privacy or immediacy. If a feature needs encyclopaedic knowledge or long, high-stakes reasoning, a large server-hosted model is usually the better tool.

The runtime layer: WebAssembly, WebGL, WebGPU and WebNN#

Every browser ML library sits on top of one or more backends, the browser technologies that actually run the maths. Understanding them explains most of the performance differences you’ll see.

Backend	Runs on	Strengths	Limitations
WebAssembly (Wasm)	CPU	Available practically everywhere; broadest operator coverage; SIMD and multithreading speed it up considerably	Slower than GPU for large models; threads require cross-origin isolation
WebGL	GPU	Wide support; mature in TensorFlow.js	A graphics API repurposed for compute; overheads and precision quirks
WebGPU	GPU	Real compute shaders; much better suited to modern ML workloads	Not yet available in every browser/OS combination; secure contexts only
WebNN	CPU, GPU or NPU	Lets the browser hand work to OS-level ML acceleration, including neural processing units	Emerging API with limited availability; treat as progressive enhancement

MDN’s WebGPU documentation currently marks the API as having limited availability and notes that it only works in secure contexts. The practical pattern is to detect WebGPU, use it when present, and fall back to WebAssembly otherwise:

async function pickDevice() {
  if ('gpu' in navigator) {
    try {
      const adapter = await navigator.gpu.requestAdapter();
      if (adapter) return 'webgpu';
    } catch (err) {
      // Adapter request can fail on blocklisted drivers; fall through
    }
  }
  return 'wasm';
}

Choosing a library#

Three libraries cover most real-world projects. They overlap, but each has a clear sweet spot.

TensorFlow.js#

TensorFlow.js is the most established option. It runs pre-trained models, converts TensorFlow/Keras models, and is unusual in supporting training in JavaScript, including transfer learning on the user’s own data in the browser. It has CPU, WebGL, Wasm and WebGPU backends and a large collection of ready-made models for pose detection, face landmarks, object detection and more.

ONNX Runtime Web#

ONNX is an open model format that most frameworks can export to, PyTorch in particular. ONNX Runtime Web runs those models in the browser with Wasm, WebGL, WebGPU and WebNN execution providers. The Wasm provider supports all ONNX operators, while the GPU providers support a subset. That makes it the natural choice when your data science team trains in Python and you need a reliable path to production on the web.

Transformers.js#

Transformers.js (published on npm as @huggingface/transformers) mirrors the Python transformers pipeline API and runs ONNX-converted models from the Hugging Face Hub, using ONNX Runtime underneath. In a few lines you get tasks like sentiment analysis, embeddings, zero-shot classification, translation, summarisation, speech recognition and image segmentation. It is the fastest way to prototype.

	TensorFlow.js	ONNX Runtime Web	Transformers.js
Best for	TF/Keras models, in-browser training	Custom models exported from PyTorch and other frameworks	Ready-made NLP, vision and audio tasks
Abstraction level	Tensors and layers	Inference sessions	High-level task pipelines
Model source	TF Hub/Kaggle models, converted models	Any ONNX file	Hugging Face Hub (ONNX weights)
GPU path	WebGL, WebGPU	WebGPU, WebGL, WebNN	WebGPU (via ONNX Runtime)
Training	Yes	Inference-focused	Inference only

Example: semantic search with Transformers.js in a Web Worker#

A useful, privacy-friendly feature for many sites is semantic search over your own content, matching meaning rather than exact keywords. The pattern: precompute embeddings for your articles at build time, ship them as JSON, and embed only the user’s query in the browser.

Run the model in a Web Worker so that loading and inference never block the main thread, which would hurt responsiveness and your INP score:

// embed-worker.js (load with: new Worker('embed-worker.js', { type: 'module' }))
import { pipeline } from '@huggingface/transformers';

let extractorPromise = null;

function getExtractor(device) {
  if (!extractorPromise) {
    extractorPromise = pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', {
      device,            // 'webgpu' or 'wasm'
      dtype: 'q8',       // quantised weights: smaller download
      progress_callback: (p) => self.postMessage({ type: 'progress', p }),
    });
  }
  return extractorPromise;
}

self.onmessage = async ({ data }) => {
  const extractor = await getExtractor(data.device);
  const output = await extractor(data.text, { pooling: 'mean', normalize: true });
  self.postMessage({ type: 'result', vector: Array.from(output.data) });
};

On the main thread, the query vector is compared with your precomputed article vectors. Because the vectors are normalised, cosine similarity reduces to a dot product:

function dot(a, b) {
  let sum = 0;
  for (let i = 0; i < a.length; i++) sum += a[i] * b[i];
  return sum;
}

function rank(queryVector, articles, topK = 5) {
  return articles
    .map((article) => ({ ...article, score: dot(queryVector, article.vector) }))
    .sort((x, y) => y.score - x.score)
    .slice(0, topK);
}

The search query never touches your server. For a few hundred articles, a brute-force comparison like this takes milliseconds, and you don’t need a vector database.

Example: a custom model with ONNX Runtime Web#

When the model is your own, for example a product-image classifier fine-tuned in PyTorch and exported with torch.onnx.export, ONNX Runtime Web gives you direct control over the session. List execution providers in order of preference and the runtime falls back if one isn’t available:

import * as ort from 'onnxruntime-web/webgpu';

const session = await ort.InferenceSession.create('/models/classifier.onnx', {
  executionProviders: ['webgpu', 'wasm'],
  graphOptimizationLevel: 'all',
});

// Preprocessed image: Float32Array of shape [1, 3, 224, 224]
async function classify(pixels) {
  const input = new ort.Tensor('float32', pixels, [1, 3, 224, 224]);
  const results = await session.run({ [session.inputNames[0]]: input });
  const logits = results[session.outputNames[0]].data;
  return softmax(Array.from(logits));
}

function softmax(values) {
  const max = Math.max(...values);
  const exps = values.map((v) => Math.exp(v - max));
  const total = exps.reduce((a, b) => a + b, 0);
  return exps.map((e) => e / total);
}

Preprocessing has to match training exactly: resize method, channel order, normalisation mean and standard deviation. In our experience, mismatched preprocessing causes more “the model works in Python but not in the browser” bugs than anything else. Validate the outputs against the Python version on a handful of test images before shipping.

Example: TensorFlow.js and memory discipline#

TensorFlow.js exposes tensors directly, which gives you flexibility and one responsibility: GPU-backed tensors are not garbage-collected like ordinary JavaScript objects. Wrap synchronous work in tf.tidy() and dispose of anything you keep:

import * as tf from '@tensorflow/tfjs';

await tf.setBackend('webgl');   // or 'wasm' / 'webgpu' with the matching backend package
await tf.ready();

const model = await tf.loadGraphModel('/models/mobilenet/model.json');

function predict(imageElement) {
  return tf.tidy(() => {
    const input = tf.browser.fromPixels(imageElement)
      .resizeBilinear([224, 224])
      .toFloat()
      .div(127.5).sub(1)       // scale to [-1, 1] as the model expects
      .expandDims(0);
    return model.predict(input).dataSync();
  });
}

// Watch for leaks during development
console.log(tf.memory().numTensors);

If numTensors keeps climbing as the user interacts, you have a leak. On a phone, a leak like that ends in a crashed tab within minutes.

Model size: the trade-off that shapes everything#

A server can load a model once. A browser has to download it, at least on the first visit. A model’s download size is roughly its parameter count multiplied by the bytes stored per parameter, which is why quantisation, storing weights at lower precision, matters so much on the web.

Precision	Bytes per parameter	Approx. size of a 100M-parameter model	Typical trade-off
fp32	4	~400 MB	Reference accuracy; usually too large for casual web use
fp16	2	~200 MB	Near-identical accuracy for most models; good fit for WebGPU
int8 (q8)	1	~100 MB	Small accuracy loss for many tasks; good default for Wasm
4-bit (q4)	0.5	~50 MB	Larger quality impact; often what makes small LLMs feasible

These figures are simple arithmetic and leave out file overheads, but they show the scale of the decision. Transformers.js exposes this directly through its dtype option. Its documentation lists fp32 as the default on WebGPU and q8 on Wasm, with fp16 and q4 available depending on the model.

Strategies for managing size#

• Choose task-specific small models. A distilled sentence-embedding model or a MobileNet-class vision model is tens of megabytes or less, and often good enough. Start small and move up only if quality requires it.
• Load on intent, not on page load. Download the model when the user opens the feature, not when they land on the page. Your Core Web Vitals will thank you.
• Cache aggressively. Serve model files with long-lived, immutable cache headers and versioned filenames. Transformers.js also caches downloaded files using the browser’s Cache API by default, so repeat visits skip the download.
• Show honest progress. A progress bar with megabytes remaining is far better than an unexplained spinner.
• Respect constrained users. Check navigator.connection?.saveData where available, and offer a server-side or non-ML fallback for low-memory devices.

Privacy, security and knowing when not to#

Keeping the privacy promise#

“Runs on your device” is a strong claim, so make sure it is true. Audit the feature’s network traffic in DevTools. Analytics events, error-tracking payloads and session-replay tools can quietly capture the very inputs you promised to keep local. If you describe the feature as private in your UI or privacy policy, back it up by checking what actually leaves the device.

Your model is public#

Anything you ship to the browser can be downloaded and inspected. Don’t deploy a model whose weights are a trade secret, and never rely on client-side inference for security decisions such as fraud checks or content moderation that must not be bypassed. A user can always modify the result before it reaches your server.

When a server is the better choice#

• The model is too large to download comfortably (large generative models, high-accuracy vision models).
• You need consistent results and latency on every device, including low-end phones.
• The model or its outputs must stay confidential, or results must be trusted server-side.
• Inference needs data that lives on the server anyway.

Hybrid setups work well too: a small on-device model handles the common, latency-sensitive cases and escalates hard ones to a server, with the user’s consent where data is sensitive.

Bringing ML features to your product#

On-device machine learning has moved from demos to dependable product features: private document scanning, smart search, accessibility aids like live captions, and interactive visual tools. The libraries are mature enough that most of the work is ordinary engineering: choosing a model that fits, keeping the main thread free and treating megabytes with respect. If you are weighing whether you need a framework around all this, our take on modern JavaScript without the bloat applies here too. For presenting model output visually, see our comparison of SVG, Canvas and WebGL.

Have an idea for an ML-powered feature, or a model that needs to run on the web? Our Custom Web Application Development service covers prototyping, model integration and production hardening. Tell us what you’re building and we’ll help you decide whether it belongs in the browser, on a server or in both.

Frequently asked questions

Can a browser really run machine learning models fast enough to be useful?

Yes, for the right models. Image classification, object detection, text embeddings, sentiment analysis, speech recognition and small language models all run well in modern browsers, especially with WebGPU. Very large generative models are still better served from a server or a dedicated API.

Is browser-based ML private?

Inference happens on the user's device, so input data such as photos, audio or text does not need to leave the browser. That is a real privacy benefit, but it only holds if your page doesn't also send that data to analytics or logging services. The model file itself is public, since anyone can download it.

Which library should I start with: TensorFlow.js, ONNX Runtime Web or Transformers.js?

For ready-made NLP, vision or audio tasks using models from the Hugging Face Hub, start with Transformers.js. If you have a model trained in PyTorch or another framework, export it to ONNX and use ONNX Runtime Web. Choose TensorFlow.js if your model comes from the TensorFlow ecosystem or you want to train or fine-tune in JavaScript.

Does WebGPU work in every browser?

Not yet universally. MDN still marks WebGPU as having limited availability, and it requires a secure (HTTPS) context. Always feature-detect with navigator.gpu and fall back to the WebAssembly backend, which runs on effectively every modern browser.