Convert Figma logo to code with AI

xlang-ai logoOpenAgents

[COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild

3,892
425
3,892
12
View on GitHub

Top Related Projects

langchain-ai's avatar

langchain

93,526

🦜🔗 Build context-aware reasoning applications

microsoft's avatar

semantic-kernel

21,280

Integrate cutting-edge LLM technology quickly and easily into your apps

geekan's avatar

MetaGPT

44,314

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

Significant-Gravitas's avatar

AutoGPT

167,533

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

reworkd's avatar

AgentGPT

31,565

🤖 Assemble, configure, and deploy autonomous AI Agents in your browser.

Quick Overview

OpenAgents is an open-source project that provides a platform for building and deploying AI agents. It aims to create a comprehensive ecosystem for developing, testing, and deploying various types of AI agents, including conversational agents, task-oriented agents, and autonomous agents.

Pros

  • Offers a flexible and extensible framework for creating diverse AI agents
  • Provides pre-built components and tools to accelerate agent development
  • Supports integration with popular AI and machine learning libraries
  • Encourages community collaboration and contributions

Cons

  • May have a steep learning curve for beginners in AI agent development
  • Documentation might be incomplete or outdated in some areas
  • Limited support for certain specialized agent types or advanced features
  • Potential performance issues with complex agent configurations

Code Examples

# Creating a simple conversational agent
from openagents import ConversationalAgent

agent = ConversationalAgent(name="ChatBot")
response = agent.respond("Hello, how are you?")
print(response)

# Defining a custom task for a task-oriented agent
from openagents import TaskOrientedAgent, Task

class WeatherTask(Task):
    def execute(self, location):
        # Implement weather fetching logic here
        return f"The weather in {location} is sunny."

agent = TaskOrientedAgent(name="WeatherBot")
agent.add_task(WeatherTask())
result = agent.execute_task("WeatherTask", "New York")
print(result)

# Setting up an autonomous agent with a goal
from openagents import AutonomousAgent

agent = AutonomousAgent(name="ExplorerBot")
agent.set_goal("Explore and map a virtual environment")
agent.start()

Getting Started

To get started with OpenAgents, follow these steps:

  1. Install the library:

    pip install openagents

  2. Import the necessary components:

    from openagents import ConversationalAgent, TaskOrientedAgent, AutonomousAgent

  3. Create and configure an agent:

    agent = ConversationalAgent(name="MyFirstAgent")

  4. Use the agent in your application:

    response = agent.respond("Hello, agent!")
print(response)

For more detailed information and advanced usage, refer to the official documentation.

Competitor Comparisons

langchain-ai logo

langchain

93,526

🦜🔗 Build context-aware reasoning applications

Pros of LangChain

  • More mature and widely adopted project with extensive documentation
  • Broader ecosystem with integrations for various LLMs and tools
  • Flexible architecture allowing for custom components and workflows

Cons of LangChain

  • Steeper learning curve due to its extensive features and abstractions
  • Can be overkill for simpler projects, potentially adding unnecessary complexity

Code Comparison

LangChain:

from langchain import OpenAI, LLMChain, PromptTemplate

llm = OpenAI(temperature=0.9)
prompt = PromptTemplate(input_variables=["product"], template="What is a good name for a company that makes {product}?")
chain = LLMChain(llm=llm, prompt=prompt)

print(chain.run("colorful socks"))

OpenAgents:

from openagents import Agent

agent = Agent()
response = agent.run("What is a good name for a company that makes colorful socks?")
print(response)

OpenAgents appears to have a simpler API for basic tasks, while LangChain offers more granular control over the components involved in the language model interaction. LangChain's approach allows for more customization but requires more setup code.

microsoft logo

semantic-kernel

21,280

Integrate cutting-edge LLM technology quickly and easily into your apps

Pros of Semantic Kernel

  • More mature project with extensive documentation and examples
  • Stronger integration with Azure AI services and Microsoft ecosystem
  • Supports multiple programming languages (C#, Python, Java)

Cons of Semantic Kernel

  • Less focus on autonomous agents compared to OpenAgents
  • May have a steeper learning curve for non-Microsoft developers
  • More complex architecture, potentially overkill for simpler projects

Code Comparison

Semantic Kernel (C#):

var kernel = Kernel.Builder.Build();
var function = kernel.CreateSemanticFunction("Generate a joke about {{$input}}");
var result = await kernel.RunAsync("cats", function);

OpenAgents (Python):

from openagents import Agent

agent = Agent()
result = agent.run("Generate a joke about cats")

Summary

Semantic Kernel offers a more comprehensive framework for building AI-powered applications, particularly within the Microsoft ecosystem. It provides robust tools for integrating various AI services and supports multiple programming languages. However, it may be more complex for simple projects and has less focus on autonomous agents.

OpenAgents, on the other hand, appears to be more specialized in creating autonomous agents with a simpler API. It might be more suitable for projects specifically focused on agent-based AI interactions but may lack the broader capabilities and ecosystem integration of Semantic Kernel.

geekan logo

MetaGPT

44,314

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

Pros of MetaGPT

  • More focused on software development and project management
  • Provides a structured approach to generating entire software projects
  • Includes role-based agents for different aspects of development

Cons of MetaGPT

  • Less versatile in terms of general task completion
  • May require more setup and configuration for specific use cases
  • Limited integration with external tools and services

Code Comparison

MetaGPT:

from metagpt.roles import ProjectManager, Architect, Engineer
from metagpt.team import Team

team = Team()
team.hire([ProjectManager(), Architect(), Engineer()])
team.run("Create a web application for task management")

OpenAgents:

from openagents import Agent, Tool

agent = Agent()
agent.add_tool(Tool("web_search"))
agent.add_tool(Tool("code_interpreter"))
response = agent.run("Create a task management application")

Summary

MetaGPT is more specialized for software development workflows, offering a structured approach with role-based agents. OpenAgents provides a more flexible framework for general task completion and integrates various tools. The choice between them depends on the specific use case and desired level of automation in the development process.

Significant-Gravitas logo

AutoGPT

167,533

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

Pros of AutoGPT

  • More established project with a larger community and contributor base
  • Offers a wider range of features and capabilities out of the box
  • Better documentation and setup guides for beginners

Cons of AutoGPT

  • Can be more complex to set up and configure
  • May require more computational resources to run effectively
  • Less focused on specific use cases, which can lead to a steeper learning curve

Code Comparison

AutoGPT:

def execute_command(command_name: str, arguments: Dict[str, Any]) -> str:
    """Execute a command and return the result"""
    command = COMMAND_CATEGORIES.get(command_name)
    if not command:
        return f"Unknown command '{command_name}'. Please refer to the list of available commands."
    return command(**arguments)

OpenAgents:

def run_agent(agent_type: str, task: str) -> str:
    """Run a specific agent with a given task"""
    agent = AGENT_TYPES.get(agent_type)
    if not agent:
        return f"Unknown agent type '{agent_type}'. Please choose from available agent types."
    return agent.execute(task)

The code snippets show that AutoGPT focuses on executing commands with arguments, while OpenAgents emphasizes running specific agent types for given tasks. This reflects the different approaches of the two projects, with AutoGPT offering a more general-purpose framework and OpenAgents providing specialized agents for specific use cases.

reworkd logo

AgentGPT

31,565

🤖 Assemble, configure, and deploy autonomous AI Agents in your browser.

Pros of AgentGPT

  • User-friendly web interface for easy interaction and task management
  • Supports multiple AI models, including GPT-3.5 and GPT-4
  • Active community with frequent updates and improvements

Cons of AgentGPT

  • Limited customization options for advanced users
  • Fewer built-in tools and integrations compared to OpenAgents
  • Less focus on multi-modal capabilities (e.g., image processing)

Code Comparison

OpenAgents:

from openagents import Agent

agent = Agent(model="gpt-3.5-turbo")
response = agent.run("Summarize this text: ...")
print(response)

AgentGPT:

import { useAgent } from "@/hooks/useAgent";

const { agent, isRunning, handleStart } = useAgent();
handleStart("Summarize this text: ...");

The code snippets show that OpenAgents uses a Python-based approach with a straightforward API, while AgentGPT employs a JavaScript/React-based implementation with hooks for managing agent state and actions. OpenAgents appears to have a more traditional programming interface, whereas AgentGPT is designed for web-based applications.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

OpenAgents: An Open Platform for Language Agents in the Wild

OpenAgents Paper Online Demos XLangNLPLab User Manual License: apache-2-0 GitHub Stars Open Issues Twitter Follow Join Slack Discord

English • 中文 • 日本語 • 한국어

Current language agent frameworks aim to facilitate the construction of proof-of-concept language agents while neglecting the non-expert user access to agents and paying little attention to application-level designs. We built OpenAgents, an open platform for using and hosting language agents in the wild of everyday life.

We have now implemented three agents in OpenAgents, and we host them on demo for free use!

  1. Data Agent for data analysis with Python/SQL and data tools;
  2. Plugins Agent with 200+ daily tools;
  3. Web Agent for autonomous web browsing.

OpenAgents can analyze data, call plugins, control your browser as ChatGPT Plus, but with OPEN Code for

  1. Easy deployment
  2. Full stack
  3. Chat Web UI
  4. Agent methods
  5. …

OpenAgents enables general users to interact with agent functionalities through a web UI optimized for swift responses and common failures, while offering developers and researchers a seamless deployment experience on local setups, providing a foundation for crafting innovative language agents and facilitating real-world evaluations. We elucidate both the challenges and promising opportunities, aspiring to set a foundation for future research and development of real-world language agents.

We welcome contributions from everyone. Before you start, please take a moment to read our CONTRIBUTING.md guidelines for issues and PRs. This will help ensure that your contribution process is smooth and consistent with the project’s standards.

🔫 Trouble Shooting

Join our Discord for help if you encounter any issues with our online demo or local deployment. Alternatively, create an issue if you have trouble with features or code.

🔥 News

  • [2023, Oct 26] We've reached 3,000 users! 🚀 Heartfelt thanks to all our users and contributors! 🙏 Please bear with us as we navigate through unexpectedly high traffic on our servers. We appreciate your patience and ready to assist as soon as possible!
  • [2023, Oct 18] Try out our Lemur, the SOTA and open-sourced foundation models for language agents, matching ChatGPT on 15 agent tasks!
  • [2023, Oct 17] Check out the OpenAgents paper here!
  • [2023, Oct 13] We've released OpenAgents platform code for all three agents, server backend and frontend! Feel free to setup your localhost one, and play with OpenAgents!
  • [2023, Aug 17] Our platform has officially reached 500 users! 🚀
  • [2023, Aug 8] We've released OpenAgents demos, including Data, Plugins, and Web agents! Check tutorials and use cases!

🥑 OpenAgents

We built three real-world agents with chat-based web UI as demonstration(check OpenAgents demos). Here is a brief overview of our OpenAgents platform. You can find more details about concepts & designs in our documentation.

Data Agent

Data Agent is a comprehensive toolkit designed for efficient data operations. It provides capabilities to:

  • 🔍 Search: Quickly locate the data you need.
  • 🛠️ Handle: Streamline data acquisition and processing.
  • 🔄 Manipulate: Modify data to suit specific requirements.
  • 📊 Visualize: Represent data in a clear and insightful manner.

With its proficiency in writing and executing code, Data Agent simplifies a wide range of data-centric tasks. Discover its potential through various use cases.

Click to see more use case screenshots

Plugins Agent

Plugins Agent seamlessly integrates with over 200 third-party plugins, each handpicked to enrich various facets of your daily life. With these plugins at its disposal, the agent empowers you to tackle a wide range of tasks and activities more efficiently.

🔌 Sample Plugins Include:

  • 🛍️ Shopping: Klarna Shopping
  • ☁️ Weather: XWeather
  • 🔬 Scientific Exploration: Wolfram Alpha

Combined Plugin Usage

Harness the power of synergy! Plugins Agent supports the concurrent use of multiple plugins. Planning a trip? Seamlessly integrate functionalities from Klook, Currency converter, and WeatherViz.

Auto Plugin Selection

Simplify your choices with our Auto Plugin Selection feature. Let the agent intuitively search and suggest the best plugins tailored to your needs.

Dive into more use cases to see Plugins Agent in action.

Click to see more use case screenshots

Web Agent

Web Agent harnesses the power of a Chrome extension to navigate and explore websites automatically. This agent streamlines the web browsing experience, making it easier to find relevant information, access desired resources, and so on.

Examples of What Web Agent Can Do:

  • 📍 Google Maps Navigation: Planning a journey? Simply relay your starting point and destination to Web Agent. It will navigate Google Maps for you and present the best routes.
  • 🐦 Twitter Postings: Engage in a conversation with Web Agent and wish to share something on Twitter? Mention the content, and Web Agent will handle your tweet effortlessly.
  • 📝 Google Form Assistance: Need to sign up for an event or activity? Share the Google Form link and the required details. Web Agent will populate the form for you.

Witness the full potential of Web Agent in these use cases.

Click to see more use case screenshots

💻 Localhost Deployment

We've released the OpenAgents platform code. Feel free to deploy on your own localhost!

Here is a brief system design of OpenAgents:

From Source Code

Please check the following folders and README files to set up & localhost:

  1. Backend: the flask backend to host our three agents.
  2. Frontend: the frontend UI and WeBot Chrome extension.

p.s.: We have renamed some arguments in code for better readability. If you have pulled the code before 10/26/2023, just a reminder that if you want to you pull the latest code, previous local chat history will be lost because of different key names.

Docker

Please follow the following steps to use the docker-compose to deploy the OpenAgents platform.

Note: the docker is under development, so there may be functions not working properly as expected and slower response. Please feel free to open an issue if you have any questions. If you want a more robust version, currently we recommend you to deploy from source code.

  1. If you want to use kaggle's dataset, you must modify the information in the Dockerfile to your correct information.
ENV KAGGLE_USER="" \
    KAGGLE_KEY=""
  1. If you are not running locally, you need to modify the accessible IP to the backend service in frontend/Dockerfile
ENV NEXT_PUBLIC_BACKEND_ENDPOINT http://x.x.x.x:8000
  1. Run the docker compose build command in the project root directory.
  2. If you use openai unofficial services, such as FastChat, you need to modify OPENAI_API_BASE in docker-compose.yml;otherwise you only to put your OPENAI_API_KEY in docker-compose.yml
  3. After completing the above steps, you can run docker compose up -d to start all services.

Notice:

  1. If you want to use GPU, you need install Nvidia Container Toolkit,and uncomment the the docker-compose.yml Lines 56-62.
  2. Use Auto Plugin will download the weight file from huggingface. In some areas, connection timeout may occur. Please solve the network problem by yourself.

📜 Tutorial on Extending OpenAgents

Code Structure

Before we dive into how to extend OpenAgents, let's first take a glance at the code structure for better understanding. The code structure of OpenAgents is shown below:

├── backend  # backend code
│   ├── README.md  # backend README for setup
│   ├── api  # RESTful APIs, to be called by the frontend
│   ├── app.py  # main flask app
│   ├── display_streaming.py  # rendering the streaming response
│   ├── kernel_publisher.py  # queue for code execution
│   ├── main.py  # main entry for the backend
│   ├── memory.py  # memory(storage) for the backend
│   ├── schemas.py  # constant definitions
│   ├── setup_script.sh  # one-click setup script for the backend
│   ├── static  # static files, e.g., cache and figs
│   └── utils  # utilities
├── frontend  # frontend code
│   ├── README.md  # frontend README for setup
│   ├── components  # React components
│   ├── hooks  # custom React hooks
│   ├── icons  # icon assets
│   ├── next-env.d.ts  # TypeScript declarations for Next.js environment variables
│   ├── next-i18next.config.js  # configuration settings for internationalization
│   ├── next.config.js  # configuration settings for Next.js
│   ├── package-lock.json  # generated by npm that describes the exact dependency tree
│   ├── package.json  # manifest file that describes the dependencies
│   ├── pages  # Next.js pages
│   ├── postcss.config.js  # configuration settings for PostCSS
│   ├── prettier.config.js  # configuration settings for Prettier
│   ├── public  # static assets
│   ├── styles  # global styles
│   ├── tailwind.config.js  # configuration settings for Tailwind CSS
│   ├── tsconfig.json  # configuration settings for TypeScript
│   ├── types  # type declarations
│   ├── utils  # utilities or helper functions
│   ├── vitest.config.ts  # configuration settings for ViTest
│   └── webot_extension.zip  # Chrome extension for Web Agent
└── real_agents  # language agents
    ├── adapters  # shared components for the three agents to adapt to the backend
    ├── data_agent  # data agent implementation
    ├── plugins_agent  # plugins agent implementation
    └── web_agent  # web agent implementation

As shown, backend/ and frontend/ are self-contained and directly deployable (see here). It does not mean they cannot be modified. Instead, you can just follow the conventional client-server architecture to extend the backend and frontend as you wish. For real_agents/, we design it to be "one agent, one folder", so that it is easy to extend a new agent. It is worth noting that we name it "real agents" because not only the conceptual language agent part is included, but also the gaps between the language agent and the backend are filled here. For example, adapters/ contains the shared adapter components like stream parsing, data model, memory, callbacks, etc. We refer interested readers to our paper for concepts and implementation designs. And we thank LangChain as we base on their code to build real agents.

Extend A New Agent

If you want to build a new agent beyond the three agents we provide, you can follow the steps below:

  • Refer to the real_agents/ folder to see how previous agents are implemented, and create a new folder for your agent.
  • Implement the agent logic in the new folder. Use the components under adapters/ folder when needed.
  • Add a chat_<new_agent>.py file under backend/api/ folder to define the chat API for the new agent, which will be called by the frontend.
  • Register new constants in backend/schemas.py if needed.
  • Add a new OpenAgentID in frontend/types/agent.ts and the corresponding API in frontend/utils/app/api.ts and frontend/utils/app/const.ts.
  • Implement the agent UI in frontend/components/Chat/Chat.tsx and frontend/components/Chat/ChatMessage.tsx when needed.
  • Run localhost script and test your new agent.

Note, if new data types, i.e., beyond text, image, table, and json, you may need to implement its parsing logic in backend/display_streaming.py and add new data models.

Extend A New LLM

Extending a new LLM as the agent backbone is simpler if the LLM is already hosted and can be called via API. Just register your new model in backend/api/language_model.py. Just refer to lemur-chat as a template.

If the LLM is not hosted yet, we have a tutorial on how to deploy a new LLM and expose it as an API here (LLM hosting to todo).

Extend A New Tool

If you want to extend a new tool in Plugins Agent, you can follow the steps below:

  • Refer to the already built plugins in real_agents/plugins_agent/plugins/, and create a new folder for your tool.
  • Implement the tool logic in the new folder. Note that ai-plugin.json and openapi.yaml are essential for the tool to be recognized(which can be generated by LLM following the others rather than manually written). And the paths/ are for the actual tool API call.
  • Register the new tool name in real_agents/plugins_agent/plugins/plugin_names.py.

👏 Contributing

Thanks to open-sourced communities’ efforts, such as LangChain, ChatBot UI, Taxy.ai browser extension and others. We are able to build our interface prototype much more conveniently and efficiently.

We welcome contributions and suggestions, together we move further to make it better! Following the steps will be well-received:

  • Step1: Post an issue if you want to add any additional features, enhancements, or encounter any problems during your experience. We would appreciate it if you follow the issue template. The issues will be discussed and assigned there.
  • Step2: Whenever an issue is assigned, you can contribute by creating a Pull Request by following the PR template. You can also claim for any open issues. Together we can make OpenAgents better!
  • Step3: PR will be merged or iterated after review and discussion. Thanks for your contribution!

Before you start, we highly recommend taking a moment to check here before contribution.

📖 Documentation

Please check here for full documentation, which will be updated to stay on pace with the demo changes and the code release.

🧙‍Participants

Tech Lead

Co-Lead Contributors

Key Contributors

Valuable Contributors

Acknowledgments (beyond code)

Heartfelt appreciation to Ziyi Huang, Roxy Rong, Haotian Li, Xingbo Wang, Jansen Wong, and Chen Henry Wu for their valuable contributions to the OpenAgents. Their expertise and insights were instrumental in bringing this project to fruition!

Open Source Contributors

Thanks to all the contributors!

Citation

If you find our work helpful, please cite us:

@misc{OpenAgents,
      title={OpenAgents: An Open Platform for Language Agents in the Wild}, 
      author={Tianbao Xie and Fan Zhou and Zhoujun Cheng and Peng Shi and Luoxuan Weng and Yitao Liu and Toh Jing Hua and Junning Zhao and Qian Liu and Che Liu and Leo Z. Liu and Yiheng Xu and Hongjin Su and Dongchan Shin and Caiming Xiong and Tao Yu},
      year={2023},
      eprint={2310.10634},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Acknowledgments

We would like to thank Google Research, Amazon AWS, and Salesforce Research for their research gift funds to this open-source effort!

⭐️ Star History

Star History Chart

A ⭐️ to OpenAgents is to make it shine brighter and benefit more people.