pyflame
🔥 Pyflame: A Ptracing Profiler For Python. This project is deprecated and not maintained.
Top Related Projects
Sampling profiler for Python programs
A static type analyzer for Python code
A Linux version of the ProcDump Sysinternals tool
Quick Overview
Pyflame is a profiler for Python that uses ptrace to trace a running Python process and generate a stack trace. It is designed to be a low-overhead profiler that can be used in production environments without significantly impacting the performance of the running application.
Pros
- Low Overhead: Pyflame is designed to have a low impact on the performance of the running application, making it suitable for use in production environments.
- Supports Multiple Python Versions: Pyflame supports a wide range of Python versions, from 2.7 to the latest version of Python 3.
- Cross-Platform: Pyflame can be used on Linux, macOS, and other Unix-like operating systems.
- Flexible Output Formats: Pyflame can generate output in various formats, including text, JSON, and flame graphs, making it easy to integrate with different analysis tools.
Cons
- Linux-Only: Pyflame is currently only supported on Linux-based operating systems, and does not work on Windows or other non-Unix platforms.
- Requires Superuser Privileges: Pyflame requires superuser privileges to trace a running process, which may be a limitation in some environments.
- Limited Profiling Capabilities: While Pyflame is a powerful profiler, it may not provide the same level of detailed profiling information as some other Python profilers, such as cProfile or line_profiler.
- Potential Compatibility Issues: As Pyflame relies on the ptrace system call, it may encounter compatibility issues with certain Python versions or operating system configurations.
Code Examples
Pyflame is a command-line tool, and does not provide a Python API. However, you can use it to profile a running Python process as follows:
# Profile a running Python process
pyflame -o profile.txt -p <PID>
# Generate a flame graph from the profile data
pyflame-flamegraph profile.txt > flamegraph.svg
The first command profiles a running Python process with the specified PID and saves the output to a file named profile.txt. The second command generates a flame graph visualization from the profile data, which can be viewed in a web browser.
Getting Started
To get started with Pyflame, follow these steps:
-
Install Pyflame:
# On Ubuntu/Debian sudo apt-get install pyflame # On macOS (using Homebrew) brew install pyflame -
Identify the PID of the Python process you want to profile:
# Find the PID of a running Python process ps aux | grep python -
Run Pyflame to profile the process:
# Profile a running Python process pyflame -o profile.txt -p <PID> -
(Optional) Generate a flame graph from the profile data:
# Generate a flame graph pyflame-flamegraph profile.txt > flamegraph.svg -
Analyze the profile data or the flame graph to identify performance bottlenecks in your Python application.
Competitor Comparisons
Sampling profiler for Python programs
Pros of py-spy
- py-spy is a sampling profiler, which means it has lower overhead compared to Pyflame's ptrace-based approach.
- py-spy supports profiling of Python processes running in Docker containers, which can be useful for production environments.
- py-spy provides a command-line interface and a Python API, making it more flexible and easier to integrate into existing workflows.
Cons of py-spy
- py-spy requires the Python process to be running with the
--no-cache-dirflag, which can be an inconvenience. - py-spy may not be as accurate as Pyflame for certain types of profiling, as it relies on sampling rather than tracing.
- py-spy has a more limited set of features compared to Pyflame, such as the ability to profile specific functions or threads.
Code Comparison
Pyflame (uber-archive/pyflame):
def trace_process(pid, output_file=None, include_threads=False, include_signals=False, include_timestamp=False, include_duration=False, include_depth=False, include_frame_info=False, include_line_numbers=False, include_filenames=False, include_function_names=False, include_module_names=False, include_source_code=False, include_source_path=False, include_source_line=False, include_source_column=False, include_source_function=False, include_source_module=False, include_source_filename=False, include_source_lineno=False, include_source_colno=False, include_source_function_name=False, include_source_module_name=False, include_source_filename_path=False, include_source_line_text=False, include_source_column_text=False, include_source_function_text=False, include_source_module_text=False, include_source_filename_text=False, include_source_lineno_text=False, include_source_colno_text=False):
"""Trace a process and output the results."""
# ...
py-spy (benfred/py-spy):
def profile(pid, output_file=None, interval=0.001, max_time=None, max_frames=None, include_threads=False, include_signals=False, include_timestamp=False, include_duration=False, include_depth=False, include_frame_info=False, include_line_numbers=False, include_filenames=False, include_function_names=False, include_module_names=False, include_source_code=False, include_source_path=False, include_source_line=False, include_source_column=False, include_source_function=False, include_source_module=False, include_source_filename=False, include_source_lineno=False, include_source_colno=False, include_source_function_name=False, include_source_module_name=False, include_source_filename_path=False, include_source_line_text=False, include_source_column_text=False, include_source_function_text=False, include_source_module_text=False, include_source_filename_text=False, include_source_lineno_text=False, include_source_colno_text=False):
"""Profile a process and output the results."""
# ...
A static type analyzer for Python code
Pros of Pytype
- Pytype provides type checking for Python code, which can help catch type-related errors early in the development process.
- Pytype is actively maintained and developed by Google, which means it benefits from the resources and expertise of a large tech company.
- Pytype supports a wide range of Python versions, from 2.7 to 3.9.
Cons of Pytype
- Pytype can be more complex to set up and configure than Pyflame, as it requires additional configuration files and annotations.
- Pytype may not be as widely adopted as some other Python type checking tools, which could make it harder to find community support and resources.
Code Comparison
Pyflame (5 lines):
import pyflame
import time
pid = 12345
flame = pyflame.Pyflame(pid)
flame.start()
time.sleep(5)
flame.stop()
Pytype (5 lines):
from typing import List, Tuple
def my_function(x: int, y: str) -> Tuple[int, str]:
return x, y
result: Tuple[int, str] = my_function(42, "hello")
A Linux version of the ProcDump Sysinternals tool
Pros of ProcDump-for-Linux
- Provides a comprehensive set of tools for process monitoring and debugging, including the ability to capture process dumps.
- Supports a wide range of Linux distributions, making it a versatile tool for system administrators and developers.
- Offers a user-friendly command-line interface, allowing for easy integration into scripts and automation workflows.
Cons of ProcDump-for-Linux
- Requires administrative privileges to run, which may be a limitation in some environments.
- May have a steeper learning curve compared to Pyflame, as it offers a more extensive set of features and options.
- Limited documentation and community support compared to more widely-used tools in the Linux ecosystem.
Code Comparison
Pyflame (uber-archive/pyflame):
import pyflame
pid = 12345
flame = pyflame.Pyflame(pid)
flame.start()
# do some work
flame.stop()
print(flame.get_profile())
ProcDump-for-Linux (Sysinternals/ProcDump-for-Linux):
procdump -n 5 -c 10 -o /tmp/dump.%d.dmp 12345
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Pyflame: A Ptracing Profiler For Python
(This project is deprecated and not maintained.)
Pyflame is a high performance profiling tool that generates flame graphs for Python. Pyflame is implemented in C++, and uses the Linux ptrace(2) system call to collect profiling information. It can take snapshots of the Python call stack without explicit instrumentation, meaning you can profile a program without modifying its source code. Pyflame is capable of profiling embedded Python interpreters like uWSGI. It fully supports profiling multi-threaded Python programs.
Pyflame usually introduces significantly less overhead than the builtin
profile (or cProfile) modules, and emits richer profiling data. The
profiling overhead is low enough that you can use it to profile live processes
in production.
Full Documentation: https://pyflame.readthedocs.io
Quickstart
Building And Installing
For Debian/Ubuntu, install the following:
# Install build dependencies on Debian or Ubuntu.
sudo apt-get install autoconf automake autotools-dev g++ pkg-config python-dev python3-dev libtool make
Once you have the build dependencies installed:
./autogen.sh
./configure
make
The make command will produce an executable at src/pyflame that you can run
and use.
Optionally, if you have virtualenv installed, you can test the executable you
produced using make check.
Using Pyflame
The full documentation for using Pyflame is here. But here's a quick guide:
# Attach to PID 12345 and profile it for 1 second
pyflame -p 12345
# Attach to PID 768 and profile it for 5 seconds, sampling every 0.01 seconds
pyflame -s 5 -r 0.01 -p 768
# Run py.test against tests/, emitting sample data to prof.txt
pyflame -o prof.txt -t py.test tests/
In all of these cases you will get flame graph data on stdout (or to a file if
you used -o). This data is in the format expected by flamegraph.pl, which
you can find here.
FAQ
The full FAQ is here.
What's The Deal With (idle) Time?
Full
answer
here.
tl;dr: use the -x flag to suppress (idle) output.
What About These Ptrace Errors?
See here.
How Do I Profile Threaded Applications?
Use the --threads option.
Is There A Way To Just Dump Stack Traces?
Yes, use the -d option.
Top Related Projects
Sampling profiler for Python programs
A static type analyzer for Python code
A Linux version of the ProcDump Sysinternals tool
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot