Convert Figma logo to code with AI

gaogaotiantian logoviztracer

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

4,838
362
4,838
25

Top Related Projects

🚴 Call stack profiler for Python. Shows you why your code is slow!

12,439

Sampling profiler for Python programs

11,555

Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals

Line-by-line profiling for Python

Monitor Memory usage of Python code

Quick Overview

VizTracer is a low-overhead profiling tool for Python that can trace and visualize your Python program's execution. It generates an interactive HTML report that allows you to explore the program's performance and behavior, including function calls, time spent in each function, and system events.

Pros

  • Low overhead, suitable for production environments
  • Generates detailed, interactive visualizations of program execution
  • Supports multi-threading and multi-processing
  • Integrates with existing Python debugging and profiling tools

Cons

  • May slightly impact performance when tracing is enabled
  • Large trace files can be generated for long-running programs
  • Learning curve for interpreting complex visualizations
  • Limited support for certain Python implementations (e.g., PyPy)

Code Examples

  1. Basic usage:
from viztracer import VizTracer

tracer = VizTracer()
tracer.start()

# Your code here
def fibonacci(n):
    if n <= 1:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

fibonacci(10)

tracer.stop()
tracer.save()  # Saves the report as 'result.html'
  1. Tracing only specific functions:
from viztracer import VizTracer

def trace_function(func):
    def wrapper(*args, **kwargs):
        tracer = VizTracer(output_file=f"{func.__name__}.html")
        tracer.start()
        result = func(*args, **kwargs)
        tracer.stop()
        tracer.save()
        return result
    return wrapper

@trace_function
def expensive_operation():
    # Your code here
    pass

expensive_operation()
  1. Using VizTracer as a context manager:
from viztracer import VizTracer

with VizTracer(output_file="context_trace.html"):
    # Your code here
    for i in range(10000):
        _ = i ** 2

Getting Started

  1. Install VizTracer:

    pip install viztracer
    
  2. Add tracing to your Python script:

    from viztracer import VizTracer
    
    tracer = VizTracer()
    tracer.start()
    
    # Your code here
    
    tracer.stop()
    tracer.save()  # Saves the report as 'result.html'
    
  3. Run your script as usual.

  4. Open the generated HTML file (e.g., 'result.html') in a web browser to view the visualization.

Competitor Comparisons

🚴 Call stack profiler for Python. Shows you why your code is slow!

Pros of Pyinstrument

  • Simpler setup and usage, requiring minimal code changes
  • Provides a more intuitive flame graph visualization
  • Lower overhead, suitable for production environments

Cons of Pyinstrument

  • Less detailed profiling information compared to VizTracer
  • Limited support for asynchronous code profiling
  • Fewer customization options for output and data collection

Code Comparison

Pyinstrument:

from pyinstrument import Profiler

profiler = Profiler()
profiler.start()
# Your code here
profiler.stop()
profiler.print()

VizTracer:

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Your code here
tracer.stop()
tracer.save()

Both tools offer straightforward integration, but Pyinstrument requires slightly less setup. VizTracer provides more advanced features and customization options, while Pyinstrument focuses on simplicity and ease of use. The choice between the two depends on the specific profiling needs and the complexity of the project being analyzed.

12,439

Sampling profiler for Python programs

Pros of py-spy

  • Low-overhead sampling profiler, minimal impact on program performance
  • Can profile running Python processes without code modifications
  • Supports profiling multi-threaded Python programs

Cons of py-spy

  • Limited visualization options compared to VizTracer
  • Doesn't provide as detailed function-level tracing
  • May require root access for some features on certain systems

Code Comparison

VizTracer:

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Your code here
tracer.stop()
tracer.save("result.json")

py-spy:

# No code changes required
# Run from command line:
# py-spy record -o profile.svg -- python your_script.py

VizTracer requires code modification to instrument the program, while py-spy can profile existing processes without changes. VizTracer offers more detailed tracing and visualization options, but py-spy provides a simpler, lower-overhead sampling approach. VizTracer is better suited for in-depth analysis of specific code sections, while py-spy excels at getting quick performance overviews of entire programs or live processes.

11,555

Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals

Pros of Scalene

  • Provides detailed memory profiling, including per-line memory usage
  • Offers CPU and GPU profiling capabilities
  • Generates interactive HTML reports for easy analysis

Cons of Scalene

  • May have higher overhead compared to VizTracer for certain workloads
  • Requires Python 3.6+ and only works on Unix-like systems (Linux, macOS)

Code Comparison

VizTracer:

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Your code here
tracer.stop()
tracer.save("result.json")

Scalene:

# No code changes required
# Run your script with:
# python -m scalene your_script.py

Summary

Scalene focuses on comprehensive profiling, including memory usage and GPU profiling, making it suitable for complex applications where resource usage is critical. It generates detailed reports but may have higher overhead.

VizTracer emphasizes low-overhead tracing and visualization, making it ideal for performance analysis of large-scale applications. It requires minimal code changes but may not provide as detailed memory profiling as Scalene.

Choose Scalene for in-depth resource profiling, especially for memory-intensive applications. Opt for VizTracer when you need a lightweight solution for tracing and visualizing program execution.

Line-by-line profiling for Python

Pros of line_profiler

  • Provides detailed line-by-line profiling information
  • Lightweight and focused on a specific task
  • Easy to integrate with existing Python projects

Cons of line_profiler

  • Limited to line-level profiling, lacking broader performance insights
  • Requires manual decoration of functions to profile
  • Less visual representation of profiling data

Code Comparison

line_profiler:

@profile
def my_function():
    # Function code here
    pass

# Run the profiler
lprofiler = LineProfiler()
lprofiler.add_function(my_function)
lprofiler.run('my_function()')

VizTracer:

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Code to profile
tracer.stop()
tracer.save()

VizTracer offers a more comprehensive approach to profiling, capturing function calls, arguments, and return values. It provides a visual timeline of program execution, making it easier to identify performance bottlenecks. However, line_profiler excels at providing detailed line-by-line timing information for specific functions, which can be crucial for optimizing critical code sections.

While line_profiler requires explicit function decoration, VizTracer can profile entire programs with minimal code changes. This makes VizTracer more suitable for large-scale profiling, while line_profiler is ideal for targeted optimization of specific functions.

Monitor Memory usage of Python code

Pros of memory_profiler

  • Focused specifically on memory usage profiling
  • Provides line-by-line memory consumption analysis
  • Can be used as a command-line tool or as a Python module

Cons of memory_profiler

  • Limited to memory profiling, doesn't provide comprehensive performance analysis
  • May have higher overhead compared to VizTracer for large-scale applications
  • Lacks advanced visualization features for complex program flows

Code Comparison

memory_profiler:

from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

VizTracer:

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Your code here
tracer.stop()
tracer.save()

memory_profiler is more straightforward for memory-specific profiling, while VizTracer offers a broader range of profiling capabilities with its tracing approach. VizTracer provides more comprehensive performance insights but may require more setup for memory-specific analysis.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

VizTracer

build flake8 readthedocs coverage pypi Visual Studio Marketplace Version support-version license commit sponsor

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

The front-end UI is powered by Perfetto. Use "AWSD" to zoom/navigate. More help can be found in "Support - Controls".

example_img

Highlights

  • Detailed function entry/exit information on timeline with source code
  • Super easy to use, no source code change for most features, no package dependency
  • Supports threading, multiprocessing, subprocess and async
  • Powerful front-end, able to render GB-level trace smoothly
  • Works on Linux/MacOS/Windows

Install

The preferred way to install VizTracer is via pip

pip install viztracer

Basic Usage

Command Line

Assume you have a python script to run:

python3 my_script.py arg1 arg2

You can simply use VizTracer by

viztracer my_script.py arg1 arg2
A result.json file will be generated, which you can open with vizviewer

vizviewer will host an HTTP server on http://localhost:9001. You can also open your browser and use that address.

If you do not want vizviewer to open the webbrowser automatically, you can use

vizviewer --server_only result.json

If you just need to bring up the trace report once, and do not want the persistent server, use

vizviewer --once result.json
vizviewer result.json
# You can display all the files in a directory and open them in browser too
vizviewer ./
# For very large trace files, try external trace processor
vizviewer --use_external_processor result.json

A VS Code Extension is available to make your life even easier.

Add --open to open the reports right after tracing
viztracer --open my_script.py arg1 arg2
viztracer -o result.html --open my_script.py arg1 arg2
modules and console scripts(like flask) are supported as well
viztracer -m your_module
viztracer flask run

Inline

You can also manually start/stop VizTracer in your script as well.

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Something happens here
tracer.stop()
tracer.save() # also takes output_file as an optional argument

Or, you can do it with with statement

with VizTracer(output_file="optional.json") as tracer:
    # Something happens here

Jupyter

If you are using Jupyter, you can use viztracer cell magics.

# You need to load the extension first
%load_ext viztracer
%%viztracer
# Your code after

A VizTracer Report button will appear after the cell and you can click it to view the results

Advanced Usage

Trace Filter

VizTracer can filter out the data you don't want to reduce overhead and keep info of a longer time period before you dump the log.

Extra Logs without Code Change

VizTracer can log extra information without changing your source code

Add Custom Event

VizTracer supports inserting custom events while the program is running. This works like a print debug, but you can know when this print happens while looking at trace data.

Misc

Multi Thread Support

VizTracer supports python native threading module without the need to do any modification to your code. Just start VizTracer before you create threads and it will just work.

For other multi-thread scenarios, you can use enable_thread_tracing() to notice VizTracer about the thread to trace it.

example_img

Refer to multi thread docs for details

Multi Process Support

VizTracer supports subprocess, multiprocessing, os.fork(), concurrent.futures, and loky out of the box.

For more general multi-process cases, VizTracer can support with some extra steps.

example_img

Refer to multi process docs for details

Async Support

VizTracer supports asyncio natively, but could enhance the report by using --log_async.

example_img

Refer to async docs for details

Flamegraph

VizTracer can show flamegraph of traced data.

vizviewer --flamegraph result.json

example_img

Remote attach

VizTracer supports remote attach to an arbitrary Python process to trace it, as long as viztracer is importable

Refer to remote attach docs

JSON alternative

VizTracer needs to dump the internal data to json format. It is recommended for the users to install orjson, which is much faster than the builtin json library. VizTracer will try to import orjson and fall back to the builtin json library if orjson does not exist.

Performance

VizTracer will introduce 2x to 3x overhead in the worst case. The overhead is much better if there are less function calls or if filters are applied correctly.

An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VM
fib:
0.000678067(1.00)[origin]
0.019880272(29.32)[py] 0.011103901(16.38)[parse] 0.021165599(31.21)[json]
0.001344933(1.98)[c] 0.008181911(12.07)[parse] 0.015789866(23.29)[json]
0.001472846(2.17)[cProfile]

hanoi     (6148, 4100):
0.000550255(1.00)[origin]
0.016343521(29.70)[py] 0.007299123(13.26)[parse] 0.016779364(30.49)[json]
0.001062505(1.93)[c] 0.006416136(11.66)[parse] 0.011463236(20.83)[json]
0.001144914(2.08)[cProfile]

qsort     (8289, 5377):
0.002817679(1.00)[origin]
0.052747431(18.72)[py] 0.011339725(4.02)[parse] 0.023644345(8.39)[json]
0.004767673(1.69)[c] 0.008735166(3.10)[parse] 0.017173703(6.09)[json]
0.007248019(2.57)[cProfile]

slow_fib  (1135, 758):
0.028759652(1.00)[origin]
0.033994071(1.18)[py] 0.001630461(0.06)[parse] 0.003386635(0.12)[json]
0.029481623(1.03)[c] 0.001152415(0.04)[parse] 0.002191417(0.08)[json]
0.028289305(0.98)[cProfile]

Documentation

For full documentation, please see https://viztracer.readthedocs.io/en/stable

Bugs/Requests

Please send bug reports and feature requests through github issue tracker. VizTracer is currently under development now and it's open to any constructive suggestions.

License

Copyright 2020-2024 Tian Gao.

Distributed under the terms of the Apache 2.0 license.