Top Related Projects
Qlib is an AI-oriented quantitative investment platform that aims to realize the potential, empower research, and create value using AI technologies in quantitative investment, from exploring ideas to implementing productions. Qlib supports diverse machine learning modeling paradigms. including supervised learning, market dynamics modeling, and RL.
An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.
Python Backtesting library for trading strategies
Python library for backtesting trading strategies & analyzing financial markets (formerly pythalesians)
MlFinLab helps portfolio managers and traders who want to leverage the power of machine learning by providing reproducible, interpretable, and easy to use tools.
Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
Quick Overview
FinRL is an open-source library for financial reinforcement learning. It provides a unified framework for various financial tasks, including portfolio management, algorithmic trading, and risk management. The library aims to bridge the gap between reinforcement learning and quantitative finance.
Pros
- Comprehensive framework for applying RL to finance
- Supports multiple market environments and RL algorithms
- Includes pre-built datasets and data processing tools
- Active community and regular updates
Cons
- Steep learning curve for those new to RL or finance
- Limited documentation for some advanced features
- Performance can be slow for large-scale simulations
- Requires careful hyperparameter tuning for optimal results
Code Examples
- Creating an environment:
from finrl.apps import stocks_trading
from finrl.config import TRAINED_MODEL_DIR, RESULTS_DIR
env = stocks_trading.StockTradingEnv(df=processed_df,
stock_dim=len(stock_list),
hmax=100,
initial_amount=1000000,
transaction_cost_pct=0.001)
- Training an agent:
from finrl.agents.stablebaselines3 import DRLAgent
agent = DRLAgent(env=env)
model = agent.get_model("ppo")
trained_model = agent.train_model(model=model,
tb_log_name='ppo',
total_timesteps=50000)
- Backtesting:
from finrl.plot import backtest_stats, backtest_plot
df_account_value, df_actions = DRLAgent.DRL_prediction(
model=trained_model,
environment=env)
perf_stats_all = backtest_stats(account_value=df_account_value)
backtest_plot(df_account_value,
baseline_ticker='SPY',
baseline_start=df_account_value.index[0],
baseline_end=df_account_value.index[-1])
Getting Started
- Install FinRL:
pip install finrl
- Import necessary modules:
from finrl.apps import stocks_trading
from finrl.agents.stablebaselines3 import DRLAgent
from finrl.config import TRAINED_MODEL_DIR, RESULTS_DIR
import pandas as pd
- Prepare data and create environment:
# Load and preprocess your data
df = pd.read_csv('your_data.csv')
processed_df = stocks_trading.preprocess_data(df)
# Create environment
env = stocks_trading.StockTradingEnv(df=processed_df,
stock_dim=len(stock_list),
hmax=100,
initial_amount=1000000,
transaction_cost_pct=0.001)
- Train and evaluate a model:
agent = DRLAgent(env=env)
model = agent.get_model("ppo")
trained_model = agent.train_model(model=model,
tb_log_name='ppo',
total_timesteps=50000)
# Backtest the trained model
df_account_value, df_actions = DRLAgent.DRL_prediction(
model=trained_model,
environment=env)
Competitor Comparisons
Qlib is an AI-oriented quantitative investment platform that aims to realize the potential, empower research, and create value using AI technologies in quantitative investment, from exploring ideas to implementing productions. Qlib supports diverse machine learning modeling paradigms. including supervised learning, market dynamics modeling, and RL.
Pros of qlib
- More comprehensive and feature-rich, offering a wider range of tools and models
- Better documentation and extensive examples for various use cases
- Stronger industry backing and support from Microsoft
Cons of qlib
- Steeper learning curve due to its complexity and extensive features
- Less focus on reinforcement learning compared to FinRL
- May be overkill for simpler financial modeling tasks
Code Comparison
FinRL example:
from finrl.apps import config
from finrl.finrl_meta.preprocessor.yahoodownloader import YahooDownloader
from finrl.finrl_meta.preprocessor.preprocessors import FeatureEngineer
df = YahooDownloader(start_date="2009-01-01",
end_date="2021-01-01",
ticker_list=config.DOW_30_TICKER).fetch_data()
qlib example:
import qlib
from qlib.data import D
from qlib.config import REG_CN
qlib.init(provider_uri='~/.qlib/qlib_data/cn_data', region=REG_CN)
instruments = ['SH000300']
fields = ['$close', '$volume', '$factor', '$paused']
df = D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31')
Both libraries provide data fetching and preprocessing capabilities, but qlib offers more flexibility in data sources and feature engineering.
An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.
Pros of TensorTrade
- More flexible and customizable trading environment
- Supports a wider range of financial instruments and markets
- Better documentation and examples for getting started
Cons of TensorTrade
- Less focus on deep reinforcement learning algorithms
- Smaller community and fewer contributions
- Limited built-in data sources and preprocessing tools
Code Comparison
FinRL example:
from finrl.agents.stablebaselines3.models import DRLAgent
from finrl.finrl_meta.preprocessor.yahoodownloader import YahooDownloader
df = YahooDownloader(start_date="2009-01-01", end_date="2021-01-01", ticker_list=["AAPL"]).fetch_data()
agent = DRLAgent(env=env)
model = agent.get_model("ppo")
TensorTrade example:
from tensortrade.env import default
from tensortrade.feed.core import Stream, DataFeed
from tensortrade.oms.instruments import USD, BTC
env = default.create(
instrument=USD/BTC,
features=DataFeed([Stream.source(exchange.streams["USD-BTC"], dtype="float")]),
action_scheme="managed-risk",
reward_scheme="risk-adjusted"
)
Both libraries provide tools for reinforcement learning in financial markets, but TensorTrade offers more flexibility in environment creation and supports a broader range of financial instruments. FinRL, on the other hand, focuses more on deep reinforcement learning algorithms and provides better integration with popular data sources.
Python Backtesting library for trading strategies
Pros of backtrader
- More mature and established project with a larger community
- Comprehensive documentation and extensive examples
- Flexible and customizable framework for backtesting trading strategies
Cons of backtrader
- Primarily focused on backtesting, with limited support for live trading
- Steeper learning curve for beginners due to its extensive features
- Less emphasis on machine learning and AI-driven strategies
Code Comparison
backtrader:
class MyStrategy(bt.Strategy):
def __init__(self):
self.sma = bt.indicators.SimpleMovingAverage(self.data.close, period=20)
def next(self):
if self.data.close[0] > self.sma[0]:
self.buy()
elif self.data.close[0] < self.sma[0]:
self.sell()
FinRL:
class DRLAgent(object):
def __init__(self, env, price_array, tech_array, turbulence_array):
self.env = env
self.price_array = price_array
self.tech_array = tech_array
self.turbulence_array = turbulence_array
def train(self):
# Training logic here
Python library for backtesting trading strategies & analyzing financial markets (formerly pythalesians)
Pros of finmarketpy
- More comprehensive financial market analysis tools, including backtesting and market data handling
- Better documentation and examples for various financial market scenarios
- Wider range of financial instruments and asset classes supported
Cons of finmarketpy
- Less focus on reinforcement learning for financial applications
- Not as actively maintained or updated as FinRL
- Steeper learning curve for users new to financial market analysis
Code Comparison
FinRL example (portfolio optimization):
env = StockPortfolioEnv(df=df,
stock_dim=stock_dimension,
hmax=100,
initial_amount=1000000,
transaction_cost_pct=0.001)
model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=50000)
finmarketpy example (backtesting):
backtest = Backtest(market_data,
trading_model,
parameters,
date_start=start_date,
date_end=end_date)
backtest.run_backtest()
backtest.plot_results()
MlFinLab helps portfolio managers and traders who want to leverage the power of machine learning by providing reproducible, interpretable, and easy to use tools.
Pros of mlfinlab
- More comprehensive library for financial machine learning tasks
- Implements advanced techniques from academic research papers
- Larger community and more frequent updates
Cons of mlfinlab
- Steeper learning curve due to more complex implementations
- Less focus on reinforcement learning for trading strategies
- Requires more domain knowledge to use effectively
Code Comparison
mlfinlab:
from mlfinlab.data_structures import imbalance_data_structures as ds
tick_data = ds.get_tick_data(file_path, num_prev_ticks=10)
dollar_bars = ds.get_dollar_bars(tick_data, threshold=70000000)
FinRL:
from finrl.meta.preprocessor.yahoodownloader import YahooDownloader
from finrl.meta.env_stock_trading.env_stocktrading import StockTradingEnv
df = YahooDownloader(start_date, end_date, ticker_list).fetch_data()
env = StockTradingEnv(df, initial_amount=100000, hmax=1000, transaction_cost_pct=0.001)
Both libraries offer valuable tools for financial machine learning, but mlfinlab focuses more on data structures and advanced techniques, while FinRL emphasizes reinforcement learning for trading strategies. The choice between them depends on the specific requirements of your project and your level of expertise in financial machine learning.
Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
Pros of Lean
- More comprehensive and production-ready algorithmic trading platform
- Larger community and extensive documentation
- Supports multiple asset classes and brokers
Cons of Lean
- Steeper learning curve for beginners
- Less focus on reinforcement learning techniques
- Requires more setup and configuration
Code Comparison
FinRL example (reinforcement learning approach):
env = StockTradingEnv(df=df, ...)
model = PPO("MlpPolicy", env, ...)
model.learn(total_timesteps=50000)
Lean example (traditional algorithmic approach):
public class MyAlgorithm : QCAlgorithm
{
public override void Initialize()
{
SetStartDate(2020, 1, 1);
SetCash(100000);
AddEquity("SPY", Resolution.Daily);
}
}
FinRL focuses on applying reinforcement learning to financial trading, making it easier to implement RL-based strategies. Lean, on the other hand, provides a more traditional algorithmic trading framework with a wider range of features and asset classes. FinRL is more suitable for researchers and those interested in RL applications in finance, while Lean is better suited for production-ready algorithmic trading systems.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
FinRL: Financial Reinforcement Learning
FinGPT: Open-source for open-finance! Revolutionize FinTech.
Check out our latest competition: ACM ICAIF 2023 FinRL Contest
Financial reinforcement learning (FinRL) (Document website) is the first open-source framework for financial reinforcement learning. FinRL has evolved into an ecosystem
Dev Roadmap | Stage | Users | Project | Desription |
---|---|---|---|---|
0.0 (Preparation) | entrance | practitioners | FinRL-Meta | gym-style market environments |
1.0 (Proof-of-Concept) | full-stack | developers | this repo | automatic pipeline |
2.0 (Professional) | profession | experts | ElegantRL | algorithms |
3.0 (Production) | service | hedge funds | Podracer | cloud-native deployment |
Outline
- Overview
- File Structure
- Supported Data Sources
- Installation
- Status Update
- Tutorials
- Publications
- News
- Citing FinRL
- Join and Contribute
- LICENSE
Overview
FinRL has three layers: market environments, agents, and applications. For a trading task (on the top), an agent (in the middle) interacts with a market environment (at the bottom), making sequential decisions.
A quick start: Stock_NeurIPS2018.ipynb. Videos FinRL at AI4Finance Youtube Channel.
File Structure
The main folder finrl has three subfolders applications, agents, meta. We employ a train-test-trade pipeline with three files: train.py, test.py, and trade.py.
FinRL
âââ finrl (main folder)
â âââ applications
â âââ Stock_NeurIPS2018
â âââ imitation_learning
â âââ cryptocurrency_trading
â âââ high_frequency_trading
â âââ portfolio_allocation
â âââ stock_trading
â âââ agents
â âââ elegantrl
â âââ rllib
â âââ stablebaseline3
â âââ meta
â âââ data_processors
â âââ env_cryptocurrency_trading
â âââ env_portfolio_allocation
â âââ env_stock_trading
â âââ preprocessor
â âââ data_processor.py
â âââ meta_config_tickers.py
â âââ meta_config.py
â âââ config.py
â âââ config_tickers.py
â âââ main.py
â âââ plot.py
â âââ train.py
â âââ test.py
â âââ trade.py
â
âââ examples
âââ unit_tests (unit tests to verify codes on env & data)
â âââ environments
â âââ test_env_cashpenalty.py
â âââ downloaders
â âââ test_yahoodownload.py
â âââ test_alpaca_downloader.py
âââ setup.py
âââ requirements.txt
âââ README.md
Supported Data Sources
Data Source | Type | Range and Frequency | Request Limits | Raw Data | Preprocessed Data |
---|---|---|---|---|---|
Akshare | CN Securities | 2015-now, 1day | Account-specific | OHLCV | Prices&Indicators |
Alpaca | US Stocks, ETFs | 2015-now, 1min | Account-specific | OHLCV | Prices&Indicators |
Baostock | CN Securities | 1990-12-19-now, 5min | Account-specific | OHLCV | Prices&Indicators |
Binance | Cryptocurrency | API-specific, 1s, 1min | API-specific | Tick-level daily aggegrated trades, OHLCV | Prices&Indicators |
CCXT | Cryptocurrency | API-specific, 1min | API-specific | OHLCV | Prices&Indicators |
EODhistoricaldata | US Securities | Frequency-specific, 1min | API-specific | OHLCV | Prices&Indicators |
IEXCloud | NMS US securities | 1970-now, 1 day | 100 per second per IP | OHLCV | Prices&Indicators |
JoinQuant | CN Securities | 2005-now, 1min | 3 requests each time | OHLCV | Prices&Indicators |
QuantConnect | US Securities | 1998-now, 1s | NA | OHLCV | Prices&Indicators |
RiceQuant | CN Securities | 2005-now, 1ms | Account-specific | OHLCV | Prices&Indicators |
Sinopac | Taiwan securities | 2023-04-13~now, 1min | Account-specific | OHLCV | Prices&Indicators |
Tushare | CN Securities, A share | -now, 1 min | Account-specific | OHLCV | Prices&Indicators |
WRDS | US Securities | 2003-now, 1ms | 5 requests each time | Intraday Trades | Prices&Indicators |
YahooFinance | US Securities | Frequency-specific, 1min | 2,000/hour | OHLCV | Prices&Indicators |
OHLCV: open, high, low, and close prices; volume. adjusted_close: adjusted close price
Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'. Users also can add new features.
Installation
- Install description for all operating systems (MAC OS, Ubuntu, Windows 10)
- FinRL for Quantitative Finance: Install and Setup Tutorial for Beginners
Status Update
Version History [click to expand]
- 2022-06-25 0.3.5: Formal release of FinRL, neo_finrl is chenged to FinRL-Meta with related files in directory: meta.
- 2021-08-25 0.3.1: pytorch version with a three-layer architecture, apps (financial tasks), drl_agents (drl algorithms), neo_finrl (gym env)
- 2020-12-14 Upgraded to Pytorch with stable-baselines3; Remove tensorflow 1.0 at this moment, under development to support tensorflow 2.0
- 2020-11-27 0.1: Beta version with tensorflow 1.5
Tutorials
- [Towardsdatascience] Deep Reinforcement Learning for Automated Stock Trading
A complete list at blogs
Publications
Title | Conference/Journal | Link | Citations | Year |
---|---|---|---|---|
Dynamic Datasets and Market Environments for Financial Reinforcement Learning | Machine Learning - Springer Nature | paper code | 7 | 2024 |
FinRL-Meta: FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning | NeurIPS 2022 | paper code | 37 | 2022 |
FinRL: Deep reinforcement learning framework to automate trading in quantitative finance | ACM International Conference on AI in Finance (ICAIF) | paper | 49 | 2021 |
FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance | NeurIPS 2020 Deep RL Workshop | paper | 87 | 2020 |
Deep reinforcement learning for automated stock trading: An ensemble strategy | ACM International Conference on AI in Finance (ICAIF) | paper code | 154 | 2020 |
Practical deep reinforcement learning approach for stock trading | NeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services | paper code | 164 | 2018 |
News
- [央广ç½] 2021 IDEA大ä¼äºç¦ç°å满è½å¹ï¼ç¾¤è±èè论éAI å¤é¡¹ç®åå¸äº®ç¹çº·å
- [央广ç½] 2021 IDEA大ä¼å¼å¯AIææ³çå®´ æ²åæ´çäºé¿åå¸å 大å沿产å
- [IDEAæ°é»] 2021 IDEA大ä¼åå¸äº§åFinRL-Metaââåºäºæ°æ®é©±å¨ç强åå¦ä¹ éèé£é©æ¨¡æç³»ç»
- [ç¥ä¹] FinRL-Metaåºäºæ°æ®é©±å¨ç强åå¦ä¹ éèå å®å®
- [éåæèµä¸æºå¨å¦ä¹ ] åºäºæ·±åº¦å¼ºåå¦ä¹ çè¡ç¥¨äº¤æçç¥æ¡æ¶ï¼ä»£ç +ææ¡£)
- [è¿ç¹OR帷å¹] é¢è¯»è®¡åNO.10 | åºäºæ·±åº¦å¢å¼ºå¦ä¹ çéå交ææºå¨äººï¼ä»AlphaGoå°FinRLçæ¼åè¿ç¨
- [深度强åå®éªå®¤] ãéç£ æ¨èãå¥å¤§å¼æºâFinRLâ: ä¸ä¸ªç¨äºéåéèèªå¨äº¤æç深度强åå¦ä¹ åº
- [åä¸æ°ç¥] éèç§æ讲座å顾|AI4Finance: ä»AlphaGoå°FinRL
- [Kaggle] Jane Street Market Prediction
- [ç©æ± äºMatpool] å¨ç©æ± äºä¸å¦ä½è¿è¡FinRLè¡ç¥¨äº¤æçç¥æ¡æ¶
- [è´¢æºæ ç] éèå¦ä¼å¸¸å¡çäºéå¦å½¬: 深度强åå¦ä¹ å¨éèèµäº§ç®¡çä¸çåºç¨
- [Neurohive] FinRL: глÑбокое обÑÑение Ñ Ð¿Ð¾Ð´ÐºÑеплением Ð´Ð»Ñ ÑÑейдинга
- [ICHI.PRO] ìì ê¸ìµììí FinRL: ë¨ì¼ 주ì ê±°ë를ìí íí 리ì¼
- [ç¥ä¹] åºäºæ·±åº¦å¼ºåå¦ä¹ çéè交æçç¥ï¼FinRL+Stable baselines3ï¼ä»¥éç¼æ¯30è¡ç¥¨ä¸ºä¾ï¼
- [ç¥ä¹] å¨ææ°æ®é©±å¨çéè强åå¦ä¹
- [ç¥ä¹] FinRLçW&Bå+è¶ åæ°æç´¢å模åä¼å(åºäºStable Baselines 3ï¼
- [ç¥ä¹] FinRL-Meta: æªæ¥éè强åå¦ä¹ çå å®å®
Citing FinRL
@article{dynamic_datasets,
author = {Liu, Xiao-Yang and Xia, Ziyi and Yang, Hongyang and Gao, Jiechao and Zha, Daochen and Zhu, Ming and Wang, Christina Dan and Wang, Zhaoran and Guo, Jian},
title = {Dynamic Datasets and Market Environments for Financial Reinforcement Learning},
journal = {Machine Learning - Springer Nature},
year = {2024}
}
@article{liu2022finrl_meta,
title={FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning},
author={Liu, Xiao-Yang and Xia, Ziyi and Rui, Jingyang and Gao, Jiechao and Yang, Hongyang and Zhu, Ming and Wang, Christina Dan and Wang, Zhaoran and Guo, Jian},
journal={NeurIPS},
year={2022}
}
@article{liu2021finrl,
author = {Liu, Xiao-Yang and Yang, Hongyang and Gao, Jiechao and Wang, Christina Dan},
title = {{FinRL}: Deep reinforcement learning framework to automate trading in quantitative finance},
journal = {ACM International Conference on AI in Finance (ICAIF)},
year = {2021}
}
@article{finrl2020,
author = {Liu, Xiao-Yang and Yang, Hongyang and Chen, Qian and Zhang, Runjia and Yang, Liuqing and Xiao, Bowen and Wang, Christina Dan},
title = {{FinRL}: A deep reinforcement learning library for automated stock trading in quantitative finance},
journal = {Deep RL Workshop, NeurIPS 2020},
year = {2020}
}
@article{liu2018practical,
title={Practical deep reinforcement learning approach for stock trading},
author={Liu, Xiao-Yang and Xiong, Zhuoran and Zhong, Shan and Yang, Hongyang and Walid, Anwar},
journal={NeurIPS Workshop on Deep Reinforcement Learning},
year={2018}
}
We published FinRL papers that are listed at Google Scholar. Previous papers are given in the list.
Join and Contribute
Welcome to AI4Finance community!
Please check Contributing Guidances.
Contributors
Thank you!
LICENSE
MIT License
Disclaimer: We are sharing codes for academic purpose under the MIT education license. Nothing herein is financial advice, and NOT a recommendation to trade real money. Please use common sense and always first consult a professional before trading or investing.
Top Related Projects
Qlib is an AI-oriented quantitative investment platform that aims to realize the potential, empower research, and create value using AI technologies in quantitative investment, from exploring ideas to implementing productions. Qlib supports diverse machine learning modeling paradigms. including supervised learning, market dynamics modeling, and RL.
An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.
Python Backtesting library for trading strategies
Python library for backtesting trading strategies & analyzing financial markets (formerly pythalesians)
MlFinLab helps portfolio managers and traders who want to leverage the power of machine learning by providing reproducible, interpretable, and easy to use tools.
Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot