Convert Figma logo to code with AI

tensorflow logotfjs-models

Pretrained models for TensorFlow.js

14,047
4,337
14,047
205

Top Related Projects

27,075

Cross-platform, customizable ML solutions for live and streaming media.

17,765

Open standard for machine learning interoperability

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

82,049

Tensors and Dynamic neural networks in Python with strong GPU acceleration

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Turi Create simplifies the development of custom machine learning models.

Quick Overview

TensorFlow.js Models is a collection of pre-trained machine learning models that can be used directly in the browser or Node.js environments. These models cover a wide range of applications, including image classification, object detection, pose estimation, and natural language processing, all optimized for performance in JavaScript environments.

Pros

  • Easy integration with web applications, enabling ML capabilities directly in the browser
  • Wide variety of pre-trained models for different tasks
  • Optimized for performance in JavaScript environments
  • Regular updates and community support

Cons

  • May have lower performance compared to native implementations
  • Limited customization options for some models
  • Dependency on TensorFlow.js core library
  • Larger file sizes for some models, which may impact load times

Code Examples

  1. Image Classification using MobileNet:
import * as mobilenet from '@tensorflow-models/mobilenet';

const img = document.getElementById('img');
const model = await mobilenet.load();
const predictions = await model.classify(img);
console.log(predictions);
  1. Pose Estimation using PoseNet:
import * as posenet from '@tensorflow-models/posenet';

const video = document.getElementById('video');
const net = await posenet.load();
const pose = await net.estimateSinglePose(video);
console.log(pose);
  1. Object Detection using COCO-SSD:
import * as cocoSsd from '@tensorflow-models/coco-ssd';

const img = document.getElementById('img');
const model = await cocoSsd.load();
const predictions = await model.detect(img);
console.log(predictions);

Getting Started

To use TensorFlow.js Models in your project:

  1. Install the desired model package:

    npm install @tensorflow/tfjs @tensorflow-models/mobilenet
    
  2. Import and use the model in your code:

    import * as tf from '@tensorflow/tfjs';
    import * as mobilenet from '@tensorflow-models/mobilenet';
    
    async function classifyImage(img) {
      const model = await mobilenet.load();
      const predictions = await model.classify(img);
      console.log(predictions);
    }
    
    // Use the function with an image element
    const img = document.getElementById('myImage');
    classifyImage(img);
    

Remember to include the TensorFlow.js core library as well as the specific model package you want to use in your project.

Competitor Comparisons

27,075

Cross-platform, customizable ML solutions for live and streaming media.

Pros of MediaPipe

  • More comprehensive suite of pre-built solutions for various ML tasks
  • Better cross-platform support, including mobile and embedded devices
  • Optimized for real-time performance on edge devices

Cons of MediaPipe

  • Steeper learning curve due to more complex architecture
  • Less JavaScript-specific focus compared to tfjs-models
  • Requires more setup and configuration for web-based projects

Code Comparison

MediaPipe (C++):

#include "mediapipe/framework/calculator_framework.h"
#include "mediapipe/framework/formats/landmark.pb.h"

class HandLandmarkCalculator : public CalculatorBase {
 public:
  static ::mediapipe::Status GetContract(CalculatorContract* cc);
  ::mediapipe::Status Process(CalculatorContext* cc) override;
};

tfjs-models (JavaScript):

import * as handpose from '@tensorflow-models/handpose';

const video = document.getElementById('video');
const model = await handpose.load();
const predictions = await model.estimateHands(video);

MediaPipe offers a more comprehensive and optimized solution for various ML tasks, especially on edge devices, but comes with a steeper learning curve. tfjs-models provides a simpler, JavaScript-focused approach that's easier to integrate into web projects but may have limitations in cross-platform support and edge device optimization.

17,765

Open standard for machine learning interoperability

Pros of ONNX

  • Broader ecosystem support and interoperability across different frameworks
  • More flexible and framework-agnostic approach to model representation
  • Stronger focus on model portability and standardization

Cons of ONNX

  • Less integrated with specific JavaScript/web environments
  • May require additional steps for deployment in browser-based applications
  • Potentially steeper learning curve for developers primarily focused on TensorFlow.js

Code Comparison

ONNX example (Python):

import onnx
model = onnx.load("model.onnx")
onnx.checker.check_model(model)

TensorFlow.js Models example (JavaScript):

import * as mobilenet from '@tensorflow-models/mobilenet';

const model = await mobilenet.load();
const predictions = await model.classify(img);

Summary

ONNX offers greater flexibility and interoperability across different machine learning frameworks, making it ideal for projects involving multiple tools or platforms. However, tfjs-models provides a more streamlined experience for web-based applications, especially those already using TensorFlow.js. The choice between the two depends on the specific requirements of your project, such as deployment environment, existing tech stack, and desired level of framework independence.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Pros of onnxruntime

  • Supports a wider range of machine learning frameworks (not limited to TensorFlow)
  • Offers better performance optimization across different hardware platforms
  • Provides more extensive language support, including C++, C#, and Java

Cons of onnxruntime

  • Steeper learning curve for beginners compared to tfjs-models
  • Less focus on pre-trained models and more on runtime optimization
  • May require more setup and configuration for specific use cases

Code Comparison

tfjs-models:

import * as mobilenet from '@tensorflow-models/mobilenet';

const img = document.getElementById('img');
const model = await mobilenet.load();
const predictions = await model.classify(img);
console.log(predictions);

onnxruntime:

import * as ort from 'onnxruntime-web';

const session = await ort.InferenceSession.create('mobilenet.onnx');
const input = new ort.Tensor('float32', imageData, [1, 3, 224, 224]);
const outputs = await session.run({ input });
console.log(outputs.classification);

Both repositories offer tools for machine learning model deployment, but tfjs-models focuses on pre-trained models for TensorFlow.js, while onnxruntime provides a more versatile runtime for various frameworks. tfjs-models is easier to use for web-based projects, while onnxruntime offers broader platform support and potentially better performance.

82,049

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Pros of PyTorch

  • More flexible and dynamic computational graph
  • Easier debugging and more intuitive coding experience
  • Stronger community support and faster adoption in research

Cons of PyTorch

  • Slower deployment in production environments
  • Less comprehensive ecosystem for mobile and web deployment
  • Steeper learning curve for beginners

Code Comparison

PyTorch:

import torch

x = torch.tensor([1, 2, 3])
y = torch.tensor([4, 5, 6])
z = torch.add(x, y)

TensorFlow.js Models:

import * as tf from '@tensorflow/tfjs';

const x = tf.tensor([1, 2, 3]);
const y = tf.tensor([4, 5, 6]);
const z = x.add(y);

Summary

PyTorch offers more flexibility and ease of use for research and prototyping, while TensorFlow.js Models excels in web and mobile deployment. PyTorch has a steeper learning curve but provides better debugging capabilities. TensorFlow.js Models has a more comprehensive ecosystem for JavaScript environments but may be less flexible for complex model architectures. The choice between the two depends on the specific use case, deployment requirements, and developer preferences.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Pros of transformers

  • Broader range of pre-trained models and tasks
  • More extensive documentation and community support
  • Easier integration with popular NLP libraries

Cons of transformers

  • Larger library size and potentially higher resource requirements
  • Steeper learning curve for beginners
  • Less optimized for browser-based deployment

Code Comparison

tfjs-models:

import * as mobilenet from '@tensorflow-models/mobilenet';

const model = await mobilenet.load();
const predictions = await model.classify(img);

transformers:

from transformers import pipeline

classifier = pipeline("image-classification")
predictions = classifier(image)

Both repositories offer pre-trained models for various tasks, but tfjs-models focuses on browser-based deployment using TensorFlow.js, while transformers provides a wider range of models and tasks primarily for Python-based environments. tfjs-models is more lightweight and optimized for web applications, whereas transformers offers more flexibility and advanced NLP capabilities.

Turi Create simplifies the development of custom machine learning models.

Pros of Turi Create

  • Offers a high-level, user-friendly API for various machine learning tasks
  • Provides built-in visualization tools for data exploration and model evaluation
  • Supports deployment to iOS and macOS devices with Core ML integration

Cons of Turi Create

  • Limited to Python programming language
  • Primarily focused on Apple ecosystem, potentially limiting cross-platform compatibility
  • Smaller community and ecosystem compared to TensorFlow.js models

Code Comparison

Turi Create example:

import turicreate as tc

# Load data
data = tc.SFrame('path/to/data.csv')

# Create and train model
model = tc.image_classifier.create(data, target='label', model='resnet-50')

# Make predictions
predictions = model.predict(new_data)

TensorFlow.js models example:

import * as mobilenet from '@tensorflow-models/mobilenet';

// Load pre-trained model
const model = await mobilenet.load();

// Make predictions
const predictions = await model.classify(imageElement);

Both repositories offer pre-trained models and easy-to-use APIs for various machine learning tasks. TensorFlow.js models focus on browser and Node.js environments, while Turi Create targets desktop and mobile Apple platforms. TensorFlow.js models provide a wider range of pre-trained models and greater flexibility for web-based applications, whereas Turi Create excels in simplifying the machine learning workflow for Apple ecosystem developers.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Pre-trained TensorFlow.js models

This repository hosts a set of pre-trained models that have been ported to TensorFlow.js.

The models are hosted on NPM and unpkg so they can be used in any project out of the box. They can be used directly or used in a transfer learning setting with TensorFlow.js.

To find out about APIs for models, look at the README in each of the respective directories. In general, we try to hide tensors so the API can be used by non-machine learning experts.

For those interested in contributing a model, please file a GitHub issue on tfjs to gauge interest. We are trying to add models that complement the existing set of models and can be used as building blocks in other apps.

Models

Type Model Demo Details Install
Images
MobileNet
live Classify images with labels from the ImageNet database. npm i @tensorflow-models/mobilenet
source
Hand
live Real-time hand pose detection in the browser using TensorFlow.js. npm i @tensorflow-models/hand-pose-detection
source
Pose
live An API for real-time human pose detection in the browser. npm i @tensorflow-models/pose-detection
source
Coco SSD
Object detection model that aims to localize and identify multiple objects in a single image. Based on the TensorFlow object detection API. npm i @tensorflow-models/coco-ssd
source
DeepLab v3
Semantic segmentation npm i @tensorflow-models/deeplab
source
Face Landmark Detection
live Real-time 3D facial landmarks detection to infer the approximate surface geometry of a human face npm i @tensorflow-models/face-landmarks-detection
source
Audio
Speech Commands
live Classify 1 second audio snippets from the speech commands dataset. npm i @tensorflow-models/speech-commands
source
Text
Universal Sentence Encoder
Encode text into a 512-dimensional embedding to be used as inputs to natural language processing tasks such as sentiment classification and textual similarity. npm i @tensorflow-models/universal-sentence-encoder
source
Text Toxicity
live Score the perceived impact a comment might have on a conversation, from "Very toxic" to "Very healthy". npm i @tensorflow-models/toxicity
source
Depth Estimation
Portrait Depth
live Estimate per-pixel depth (the distance to the camera center) for a single portrait image, which can be further used for creative applications such as 3D photo and relighting. npm i @tensorflow-models/depth-estimation
source
General Utilities
KNN Classifier
This package provides a utility for creating a classifier using the K-Nearest Neighbors algorithm. Can be used for transfer learning. npm i @tensorflow-models/knn-classifier
source

Development

You can run the unit tests for any of the models by running the following inside a directory:

yarn test

New models should have a test NPM script (see this package.json and run_tests.ts helper for reference).

To run all of the tests, you can run the following command from the root of this repo:

yarn presubmit