Convert Figma logo to code with AI

deeppavlov logoDeepPavlov

An open source library for deep learning end-to-end dialog systems and chatbots.

6,649
1,141
6,649
45

Top Related Projects

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Bot Framework provides the most comprehensive experience for building conversation applications.

12,494

The open-source hub to build & deploy GPT/LLM Agents ⚡️

29,635

💫 Industrial-strength Natural Language Processing (NLP) in Python

10,485

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.

Quick Overview

DeepPavlov is an open-source conversational AI framework that provides a set of pre-trained models and tools for building and deploying conversational AI applications. It is designed to simplify the process of developing and deploying conversational AI systems, making it easier for developers to create chatbots, virtual assistants, and other conversational interfaces.

Pros

  • Extensive Pre-trained Models: DeepPavlov comes with a wide range of pre-trained models for various conversational AI tasks, such as intent recognition, named entity recognition, and dialogue management, which can be easily fine-tuned for specific use cases.
  • Modular and Flexible Architecture: The framework's modular design allows developers to easily integrate and customize different components of the conversational AI system, making it highly flexible and adaptable.
  • Active Community and Documentation: DeepPavlov has an active community of contributors and users, and the project is well-documented, making it easier for developers to get started and find support.
  • Supports Multiple Languages: The framework supports multiple languages, including English, Russian, and Chinese, making it suitable for a wide range of use cases.

Cons

  • Steep Learning Curve: While the framework is well-documented, the overall complexity of the system and the number of available components can make it challenging for beginners to get started.
  • Limited Deployment Options: DeepPavlov primarily focuses on the development and training of conversational AI models, but it may not provide comprehensive deployment and hosting solutions, which could be a limitation for some users.
  • Performance Concerns: Depending on the complexity of the models and the specific use case, the performance of the conversational AI system built with DeepPavlov may vary, and some users may encounter performance issues.
  • Dependency on External Libraries: DeepPavlov relies on several external libraries and frameworks, such as TensorFlow and PyTorch, which can introduce additional dependencies and potential compatibility issues.

Code Examples

Here are a few code examples demonstrating the usage of DeepPavlov:

  1. Intent Recognition:
from deeppavlov import build_model, configs

intent_model = build_model(configs.classifiers.intents_dstc2, download=True)
intent = intent_model(['I would like to book a flight'])
print(intent)  # Output: ['inform']

This code demonstrates how to use the pre-trained intent recognition model in DeepPavlov to classify the intent of a user's input.

  1. Named Entity Recognition:
from deeppavlov import build_model, configs

ner_model = build_model(configs.ner.ner_ontonotes_bert_mult, download=True)
entities = ner_model(['My name is John and I live in New York'])
print(entities)  # Output: [('John', 'PER'), ('New York', 'LOC')]

This code shows how to use the pre-trained named entity recognition model in DeepPavlov to extract entities from a given text.

  1. Dialogue Management:
from deeppavlov import build_model, configs

dialogue_model = build_model(configs.dialog.dialog_system_seq2seq_atten, download=True)
response = dialogue_model(['Hi, how can I help you?', 'I would like to book a flight'])
print(response)  # Output: ['Sure, let me help you with that. What are the details of your flight?']

This code demonstrates the usage of the pre-trained dialogue management model in DeepPavlov to generate a response based on the user's input.

Getting Started

To get started with DeepPavlov, follow these steps:

  1. Install the required dependencies:
pip install deeppavlov
  1. Download the pre-trained models:
from deeppavlov import download_model
download_model('intents_dstc2')
download_model('ner_ontonotes_bert_mult')
download_model('dialog_system_seq2seq_atten')
  1. Build and use the models:
from deeppavlov import build_model, configs

intent_model = build_model(configs.classifiers.intents_dstc2)
ner_model = build_model(

Competitor Comparisons

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Pros of Transformers

  • Larger community and more frequent updates
  • Broader range of pre-trained models and tasks
  • Seamless integration with PyTorch and TensorFlow

Cons of Transformers

  • Steeper learning curve for beginners
  • Less focus on end-to-end dialogue systems

Code Comparison

DeepPavlov:

from deeppavlov import build_model, configs

model = build_model(configs.squad.squad_bert, download=True)
result = model(["Text", "Question"])

Transformers:

from transformers import pipeline

qa_pipeline = pipeline("question-answering")
result = qa_pipeline(question="Question", context="Text")

Both repositories offer powerful NLP capabilities, but Transformers provides a wider range of models and tasks, while DeepPavlov focuses more on dialogue systems and offers a simpler API for certain tasks. Transformers has a larger community and more frequent updates, but may be more challenging for beginners. DeepPavlov provides an easier entry point for specific dialogue-related tasks but has a narrower scope overall.

Bot Framework provides the most comprehensive experience for building conversation applications.

Pros of Bot Framework SDK

  • Comprehensive ecosystem with tools for building, testing, and deploying bots
  • Strong integration with Azure services and other Microsoft technologies
  • Extensive documentation and community support

Cons of Bot Framework SDK

  • Steeper learning curve, especially for developers new to Microsoft technologies
  • More complex setup and configuration compared to DeepPavlov
  • Primarily focused on chatbots, while DeepPavlov offers a broader range of NLP tasks

Code Comparison

Bot Framework SDK (C#):

[LuisIntent("Greeting")]
public async Task Greeting(IDialogContext context, LuisResult result)
{
    await context.PostAsync("Hello! How can I help you today?");
    context.Wait(MessageReceived);
}

DeepPavlov (Python):

from deeppavlov import build_model, configs

model = build_model(configs.ner.ner_ontonotes_bert_mult, download=True)
result = model(["Hello, how are you?"])
print(result)

The Bot Framework SDK example shows a simple intent handler for a greeting, while the DeepPavlov example demonstrates how to build and use a pre-trained NER model. DeepPavlov's approach is more straightforward for specific NLP tasks, while Bot Framework SDK provides a more comprehensive structure for building conversational agents.

12,494

The open-source hub to build & deploy GPT/LLM Agents ⚡️

Pros of Botpress

  • User-friendly visual interface for bot design and management
  • Extensive built-in NLU capabilities and integrations
  • Active community and regular updates

Cons of Botpress

  • Steeper learning curve for developers
  • Less flexibility for custom NLP models compared to DeepPavlov

Code Comparison

Botpress (JavaScript):

bp.hear(/hello/i, async (bp, event) => {
  await bp.messaging.sendText(event.channel.id, 'Hello, human!')
})

DeepPavlov (Python):

from deeppavlov import build_model, configs

model = build_model(configs.ner.ner_ontonotes_bert_mult, download=True)
result = model(["Hello, how are you?"])

Botpress focuses on a more visual approach to bot creation, with a user-friendly interface and built-in NLU capabilities. It's suitable for both developers and non-technical users. However, it may have a steeper learning curve for developers accustomed to more code-centric approaches.

DeepPavlov, on the other hand, is more flexible for custom NLP model development and integration. It's primarily designed for researchers and developers working on advanced NLP tasks. While it offers powerful tools for building complex conversational AI systems, it may require more technical expertise to implement effectively.

29,635

💫 Industrial-strength Natural Language Processing (NLP) in Python

Pros of spaCy

  • More mature and widely adopted in industry
  • Faster processing speed for large-scale text analysis
  • Extensive documentation and community support

Cons of spaCy

  • Less flexibility for customizing deep learning models
  • Fewer out-of-the-box task-specific models compared to DeepPavlov
  • Limited support for languages other than English

Code Comparison

spaCy:

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp("Apple is looking at buying U.K. startup for $1 billion")
for ent in doc.ents:
    print(ent.text, ent.label_)

DeepPavlov:

from deeppavlov import build_model, configs

ner_model = build_model(configs.ner.ner_ontonotes_bert_mult, download=True)
results = ner_model(["Apple is looking at buying U.K. startup for $1 billion"])
for entity in results[0]:
    print(entity)

Both libraries offer NLP capabilities, but spaCy is more focused on efficiency and industrial applications, while DeepPavlov provides a wider range of pre-built models for various NLP tasks. spaCy's syntax is generally more concise, while DeepPavlov offers more flexibility in model selection and configuration.

10,485

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.

Pros of ParlAI

  • Extensive library of pre-built tasks and datasets for dialogue research
  • Strong integration with Facebook's AI research ecosystem
  • Active community and frequent updates

Cons of ParlAI

  • Steeper learning curve for beginners
  • More focused on research than production deployment
  • Heavier resource requirements for some models

Code Comparison

ParlAI:

from parlai.core.agents import Agent
from parlai.core.worlds import DialogPartnerWorld

class MyAgent(Agent):
    def act(self):
        observation = self.observation
        return {'text': 'Hello, ' + observation['text']}

DeepPavlov:

from deeppavlov import build_model, configs

model = build_model(configs.squad.squad_bert, download=True)
result = model(["Text", "Question"])
print(result)

ParlAI offers a more flexible, agent-based approach for dialogue systems, while DeepPavlov provides a simpler API for quick model deployment. ParlAI's code structure is geared towards research and experimentation, whereas DeepPavlov focuses on ease of use for production scenarios.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

DeepPavlov 1.0

License Apache 2.0 Python 3.6, 3.7, 3.8, 3.9, 3.10, 3.11 Downloads Static Badge Static Badge

DeepPavlov 1.0 is an open-source NLP framework built on PyTorch and transformers. DeepPavlov 1.0 is created for modular and configuration-driven development of state-of-the-art NLP models and supports a wide range of NLP model applications. DeepPavlov 1.0 is designed for practitioners with limited knowledge of NLP/ML.

Quick Links

nameDescription
⭐️ DemoCheck out our NLP models in the online demo
📚 DocumentationHow to use DeepPavlov 1.0 and its features
🚀 Model ListFind the NLP model you need in the list of available models
🪐 Contribution GuidePlease read the contribution guidelines before making a contribution
🎛 IssuesIf you have an issue with DeepPavlov, please let us know
⏩ ForumPlease let us know if you have a problem with DeepPavlov
📦 BlogsRead about our current development
🦙 Extended colab tutorialsCheck out the code tutorials for our models
🌌 Docker HubCheck out the Docker images for rapid deployment
👩‍🏫 FeedbackPlease leave us your feedback to make DeepPavlov better

Installation

  1. DeepPavlov supports Linux, Windows 10+ (through WSL/WSL2), MacOS (Big Sur+) platforms, Python 3.6, 3.7, 3.8, 3.9 and 3.10. Depending on the model used, you may need from 4 to 16 GB RAM.

  2. Create and activate a virtual environment:

    • Linux
    python -m venv env
    source ./env/bin/activate
    
  3. Install the package inside the environment:

    pip install deeppavlov
    

QuickStart

There is a bunch of great pre-trained NLP models in DeepPavlov. Each model is determined by its config file.

List of models is available on the doc page in the deeppavlov.configs (Python):

from deeppavlov import configs

When you're decided on the model (+ config file), there are two ways to train, evaluate and infer it:

GPU requirements

By default, DeepPavlov installs models requirements from PyPI. PyTorch from PyPI could not support your device CUDA capability. To run supported DeepPavlov models on GPU you should have CUDA compatible with used GPU and PyTorch version required by DeepPavlov models. See docs for details. GPU with Pascal or newer architecture and 4+ GB VRAM is recommended.

Command line interface (CLI)

To get predictions from a model interactively through CLI, run

python -m deeppavlov interact <config_path> [-d] [-i]
  • -d downloads required data - pretrained model files and embeddings (optional).
  • -i installs model requirements (optional).

You can train it in the same simple way:

python -m deeppavlov train <config_path> [-d] [-i]

Dataset will be downloaded regardless of whether there was -d flag or not.

To train on your own data you need to modify dataset reader path in the train config doc. The data format is specified in the corresponding model doc page.

There are even more actions you can perform with configs:

python -m deeppavlov <action> <config_path> [-d] [-i]
  • <action> can be
    • install to install model requirements (same as -i),
    • download to download model's data (same as -d),
    • train to train the model on the data specified in the config file,
    • evaluate to calculate metrics on the same dataset,
    • interact to interact via CLI,
    • riseapi to run a REST API server (see doc),
    • predict to get prediction for samples from stdin or from <file_path> if -f <file_path> is specified.
  • <config_path> specifies path (or name) of model's config file
  • -d downloads required data
  • -i installs model requirements

Python

To get predictions from a model interactively through Python, run

from deeppavlov import build_model

model = build_model(<config_path>, install=True, download=True)

# get predictions for 'input_text1', 'input_text2'
model(['input_text1', 'input_text2'])

where

  • install=True installs model requirements (optional),
  • download=True downloads required data from web - pretrained model files and embeddings (optional),
  • <config_path> is model name (e.g. 'ner_ontonotes_bert_mult'), path to the chosen model's config file (e.g. "deeppavlov/configs/ner/ner_ontonotes_bert_mult.json"), or deeppavlov.configs attribute (e.g. deeppavlov.configs.ner.ner_ontonotes_bert_mult without quotation marks).

You can train it in the same simple way:

from deeppavlov import train_model 

model = train_model(<config_path>, install=True, download=True)

To train on your own data you need to modify dataset reader path in the train config doc. The data format is specified in the corresponding model doc page.

You can also calculate metrics on the dataset specified in your config file:

from deeppavlov import evaluate_model 

model = evaluate_model(<config_path>, install=True, download=True)

DeepPavlov also allows to build a model from components for inference using Python.

License

DeepPavlov is Apache 2.0 - licensed.