Convert Figma logo to code with AI

aurelio-labs logosemantic-router

Superfast AI decision making and intelligent processing of multi-modal data.

1,994
213
1,994
82

Top Related Projects

93,526

🦜🔗 Build context-aware reasoning applications

Integrate cutting-edge LLM technology quickly and easily into your apps

Examples and guides for using the OpenAI API

16,603

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

Quick Overview

Semantic Router is an open-source library that provides intelligent routing capabilities for large language models (LLMs). It aims to enhance the efficiency and accuracy of LLM-based applications by dynamically routing queries to the most appropriate model or tool based on the semantic content of the input.

Pros

  • Improves efficiency by directing queries to the most suitable model or tool
  • Enhances accuracy of LLM responses by leveraging specialized models for specific tasks
  • Reduces computational costs by avoiding unnecessary processing of queries by inappropriate models
  • Easily integrable with existing LLM-based applications

Cons

  • May introduce additional latency due to the routing process
  • Requires careful configuration and maintenance of routing rules
  • Potential for misrouting if the semantic analysis is not accurate
  • Limited documentation and examples available for advanced use cases

Code Examples

  1. Basic usage of Semantic Router:
from semantic_router import Router
from semantic_router.encoders import OpenAIEncoder
from semantic_router.routes import Route

router = Router(encoder=OpenAIEncoder())

routes = [
    Route(name="greeting", utterances=["hello", "hi", "hey"]),
    Route(name="farewell", utterances=["goodbye", "bye", "see you later"])
]

router.add_routes(routes)

result = router.route("Hello, how are you?")
print(result.name)  # Output: greeting
  1. Using custom routes with functions:
from semantic_router import Router
from semantic_router.encoders import OpenAIEncoder

def handle_greeting(query):
    return f"Hello! Nice to meet you. You said: {query}"

def handle_farewell(query):
    return f"Goodbye! Have a great day. You said: {query}"

router = Router(encoder=OpenAIEncoder())

router.add_route("greeting", ["hello", "hi", "hey"], handle_greeting)
router.add_route("farewell", ["goodbye", "bye", "see you later"], handle_farewell)

result = router.route("Hi there!")
print(result.function("Hi there!"))  # Output: Hello! Nice to meet you. You said: Hi there!
  1. Using Semantic Router with async functions:
import asyncio
from semantic_router import Router
from semantic_router.encoders import OpenAIEncoder

async def async_handle_greeting(query):
    await asyncio.sleep(1)  # Simulating async operation
    return f"Hello! Nice to meet you. You said: {query}"

router = Router(encoder=OpenAIEncoder())
router.add_route("greeting", ["hello", "hi", "hey"], async_handle_greeting)

async def main():
    result = await router.aroute("Hello, how are you?")
    response = await result.function("Hello, how are you?")
    print(response)

asyncio.run(main())

Getting Started

To get started with Semantic Router, follow these steps:

  1. Install the library:

    pip install semantic-router
    
  2. Import and initialize the Router:

    from semantic_router import Router
    from semantic_router.encoders import OpenAIEncoder
    
    router = Router(encoder=OpenAIEncoder())
    
  3. Add routes and start routing queries:

    router.add_route("greeting", ["hello", "hi", "hey"])
    router.add_route("farewell", ["goodbye", "bye", "see you later"])
    
    result = router.route("Hello, how are you?")
    print(result.name)  # Output: greeting
    

Competitor Comparisons

93,526

🦜🔗 Build context-aware reasoning applications

Pros of LangChain

  • Comprehensive framework with a wide range of tools and integrations
  • Extensive documentation and active community support
  • Flexible architecture allowing for customization and extension

Cons of LangChain

  • Steeper learning curve due to its extensive feature set
  • Potential overhead for simpler projects that don't require all functionalities
  • May require more setup and configuration compared to lightweight alternatives

Code Comparison

Semantic Router:

from semantic_router import Route, RouteLayer

routes = [
    Route(name="greeting", utterances=["hello", "hi", "hey"]),
    Route(name="farewell", utterances=["goodbye", "bye", "see you"])
]
layer = RouteLayer(routes)
result = layer("Hello there!")

LangChain:

from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI

prompt = PromptTemplate.from_template("Say {text}")
llm = OpenAI()
chain = prompt | llm
result = chain.invoke({"text": "Hello there!"})

Both libraries offer ways to process and route text, but LangChain provides a more comprehensive set of tools for building complex language model applications, while Semantic Router focuses specifically on semantic routing functionality.

Integrate cutting-edge LLM technology quickly and easily into your apps

Pros of Semantic Kernel

  • More comprehensive framework with broader functionality
  • Extensive documentation and community support
  • Integration with various AI services and models

Cons of Semantic Kernel

  • Steeper learning curve due to its complexity
  • Potentially overkill for simpler projects
  • Tighter coupling with Microsoft ecosystem

Code Comparison

Semantic Router:

from semantic_router import Route

route = Route(
    name="weather",
    utterances=["What's the weather like?", "Is it going to rain today?"]
)

Semantic Kernel:

using Microsoft.SemanticKernel;

var kernel = Kernel.Builder.Build();
var weatherSkill = kernel.ImportSkill(new WeatherSkill());
var result = await kernel.RunAsync("What's the weather like?", weatherSkill["GetWeather"]);

The Semantic Router focuses on routing intents, while Semantic Kernel provides a more comprehensive framework for building AI-powered applications. Semantic Router offers a simpler approach for specific use cases, whereas Semantic Kernel provides more flexibility and integration options at the cost of increased complexity.

Examples and guides for using the OpenAI API

Pros of openai-cookbook

  • Comprehensive collection of examples and best practices for using OpenAI's APIs
  • Covers a wide range of use cases and techniques, from basic to advanced
  • Regularly updated with new features and improvements from OpenAI

Cons of openai-cookbook

  • Focuses solely on OpenAI's offerings, limiting its applicability to other AI platforms
  • May be overwhelming for beginners due to its extensive content
  • Lacks a specific focus on routing or intent classification

Code Comparison

semantic-router:

from semantic_router import Route

route = Route(
    name="greeting",
    utterances=["hi", "hello", "hey"],
    function=lambda x: "Hello there!"
)

openai-cookbook:

import openai

response = openai.Completion.create(
  engine="text-davinci-002",
  prompt="Translate the following English text to French: 'Hello, how are you?'",
  max_tokens=60
)

The semantic-router example demonstrates a simple route definition for intent classification, while the openai-cookbook example shows how to use OpenAI's API for text completion tasks. semantic-router is more focused on routing and intent classification, whereas openai-cookbook provides a broader range of examples for various AI tasks using OpenAI's services.

16,603

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

Pros of Haystack

  • More comprehensive framework for building end-to-end NLP applications
  • Larger community and ecosystem with extensive documentation
  • Supports multiple languages and integrates with various NLP models

Cons of Haystack

  • Steeper learning curve due to its extensive features
  • Potentially overkill for simpler routing tasks
  • Heavier resource requirements for deployment

Code Comparison

Semantic-router:

from semantic_router import Route, RouteLayer

routes = [
    Route(name="greeting", utterances=["hi", "hello"]),
    Route(name="farewell", utterances=["bye", "goodbye"])
]
layer = RouteLayer(routes)
result = layer("hello there")

Haystack:

from haystack import Pipeline
from haystack.nodes import TextConverter, Preprocessor, FARMReader

pipeline = Pipeline()
pipeline.add_node(component=TextConverter(), name="TextConverter", inputs=["File"])
pipeline.add_node(component=Preprocessor(), name="Preprocessor", inputs=["TextConverter"])
pipeline.add_node(component=FARMReader(model_name_or_path="deepset/roberta-base-squad2"), name="Reader", inputs=["Preprocessor"])

Summary

Semantic-router is more focused on lightweight semantic routing, while Haystack offers a comprehensive NLP pipeline framework. Semantic-router is easier to set up for simple routing tasks, whereas Haystack provides more advanced features but with increased complexity.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Semantic Router

PyPI - Python Version GitHub Contributors GitHub Last Commit GitHub Issues GitHub Pull Requests Github License

Semantic Router is a superfast decision-making layer for your LLMs and agents. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions — routing our requests using semantic meaning.

Read the Docs


Quickstart

To get started with semantic-router we install it like so:

pip install -qU semantic-router

❗️ If wanting to use a fully local version of semantic router you can use HuggingFaceEncoder and LlamaCppLLM (pip install -qU "semantic-router[local]", see here). To use the HybridRouteLayer you must pip install -qU "semantic-router[hybrid]".

We begin by defining a set of Route objects. These are the decision paths that the semantic router can decide to use, let's try two simple routes for now — one for talk on politics and another for chitchat:

from semantic_router import Route

# we could use this as a guide for our chatbot to avoid political conversations
politics = Route(
    name="politics",
    utterances=[
        "isn't politics the best thing ever",
        "why don't you tell me about your political opinions",
        "don't you just love the president",
        "they're going to destroy this country!",
        "they will save the country!",
    ],
)

# this could be used as an indicator to our chatbot to switch to a more
# conversational prompt
chitchat = Route(
    name="chitchat",
    utterances=[
        "how's the weather today?",
        "how are things going?",
        "lovely weather today",
        "the weather is horrendous",
        "let's go to the chippy",
    ],
)

# we place both of our decisions together into single list
routes = [politics, chitchat]

We have our routes ready, now we initialize an embedding / encoder model. We currently support a CohereEncoder and OpenAIEncoder — more encoders will be added soon. To initialize them we do:

import os
from semantic_router.encoders import CohereEncoder, OpenAIEncoder

# for Cohere
os.environ["COHERE_API_KEY"] = "<YOUR_API_KEY>"
encoder = CohereEncoder()

# or for OpenAI
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()

With our routes and encoder defined we now create a RouteLayer. The route layer handles our semantic decision making.

from semantic_router.layer import RouteLayer

rl = RouteLayer(encoder=encoder, routes=routes)

We can now use our route layer to make super fast decisions based on user queries. Let's try with two queries that should trigger our route decisions:

rl("don't you love politics?").name
[Out]: 'politics'

Correct decision, let's try another:

rl("how's the weather today?").name
[Out]: 'chitchat'

We get both decisions correct! Now lets try sending an unrelated query:

rl("I'm interested in learning about llama 2").name
[Out]:

In this case, no decision could be made as we had no matches — so our route layer returned None!

Integrations

The encoders of semantic router include easy-to-use integrations with Cohere, OpenAI, Hugging Face, FastEmbed, and more — we even support multi-modality!.

Our utterance vector space also integrates with Pinecone and Qdrant!


📚 Resources

Docs

NotebookDescription
IntroductionIntroduction to Semantic Router and static routes
Dynamic RoutesDynamic routes for parameter generation and functionc calls
Save/Load LayersHow to save and load RouteLayer from file
LangChain IntegrationHow to integrate Semantic Router with LangChain Agents
Local ExecutionFully local Semantic Router with dynamic routes — local models such as Mistral 7B outperform GPT-3.5 in most tests
Route OptimizationHow to train route layer thresholds to optimize performance
Multi-Modal RoutesUsing multi-modal routes to identify Shrek vs. not-Shrek pictures

Online Course

Semantic Router Course

Community