Never use print for debugging again

PySnooper - Never use print for debugging again

PySnooper is a poor man's debugger. If you've used Bash, it's like set -x for Python, except it's fancier.

Your story: You're trying to figure out why your Python code isn't doing what you think it should be doing. You'd love to use a full-fledged debugger with breakpoints and watches, but you can't be bothered to set one up right now.

You want to know which lines are running and which aren't, and what the values of the local variables are.

Most people would use print lines, in strategic locations, some of them showing the values of variables.

PySnooper lets you do the same, except instead of carefully crafting the right print lines, you just add one decorator line to the function you're interested in. You'll get a play-by-play log of your function, including which lines ran and when, and exactly when local variables were changed.

What makes PySnooper stand out from all other code intelligence tools? You can use it in your shitty, sprawling enterprise codebase without having to do any setup. Just slap the decorator on, as shown below, and redirect the output to a dedicated log file by specifying its path as the first argument.

Example

We're writing a function that converts a number to binary, by returning a list of bits. Let's snoop on it by adding the @pysnooper.snoop() decorator:

import pysnooper

@pysnooper.snoop()
def number_to_bits(number):
    if number:
        bits = []
        while number:
            number, remainder = divmod(number, 2)
            bits.insert(0, remainder)
        return bits
    else:
        return [0]

number_to_bits(6)

The output to stderr is:

Source path:... /my_code/foo.py
Starting var:.. number = 6
15:29:11.327032 call         4 def number_to_bits(number):
15:29:11.327032 line         5     if number:
15:29:11.327032 line         6         bits = []
New var:....... bits = []
15:29:11.327032 line         7         while number:
15:29:11.327032 line         8             number, remainder = divmod(number, 2)
New var:....... remainder = 0
Modified var:.. number = 3
15:29:11.327032 line         9             bits.insert(0, remainder)
Modified var:.. bits = [0]
15:29:11.327032 line         7         while number:
15:29:11.327032 line         8             number, remainder = divmod(number, 2)
Modified var:.. number = 1
Modified var:.. remainder = 1
15:29:11.327032 line         9             bits.insert(0, remainder)
Modified var:.. bits = [1, 0]
15:29:11.327032 line         7         while number:
15:29:11.327032 line         8             number, remainder = divmod(number, 2)
Modified var:.. number = 0
15:29:11.327032 line         9             bits.insert(0, remainder)
Modified var:.. bits = [1, 1, 0]
15:29:11.327032 line         7         while number:
15:29:11.327032 line        10         return bits
15:29:11.327032 return      10         return bits
Return value:.. [1, 1, 0]
Elapsed time: 00:00:00.000001

Or if you don't want to trace an entire function, you can wrap the relevant part in a with block:

import pysnooper
import random

def foo():
    lst = []
    for i in range(10):
        lst.append(random.randrange(1, 1000))

    with pysnooper.snoop():
        lower = min(lst)
        upper = max(lst)
        mid = (lower + upper) / 2
        print(lower, mid, upper)

foo()

which outputs something like:

New var:....... i = 9
New var:....... lst = [681, 267, 74, 832, 284, 678, ...]
09:37:35.881721 line        10         lower = min(lst)
New var:....... lower = 74
09:37:35.882137 line        11         upper = max(lst)
New var:....... upper = 832
09:37:35.882304 line        12         mid = (lower + upper) / 2
74 453.0 832
New var:....... mid = 453.0
09:37:35.882486 line        13         print(lower, mid, upper)
Elapsed time: 00:00:00.000344

Features

If stderr is not easily accessible for you, you can redirect the output to a file:

@pysnooper.snoop('/my/log/file.log')

You can also pass a stream or a callable instead, and they'll be used.

See values of some expressions that aren't local variables:

@pysnooper.snoop(watch=('foo.bar', 'self.x["whatever"]'))

Show snoop lines for functions that your function calls:

@pysnooper.snoop(depth=2)

See Advanced Usage for more options. <------

Installation with Pip

The best way to install PySnooper is with Pip:

$ pip install pysnooper

Other installation options

Conda with conda-forge channel:

$ conda install -c conda-forge pysnooper

Arch Linux:

$ yay -S python-pysnooper

License

Copyright (c) 2019 Ram Rachum and collaborators, released under the MIT license.

I provide Development services in Python and Django and I give Python workshops to teach people Python and related topics.

Media Coverage

Hacker News thread and /r/Python Reddit thread (22 April 2019)

Owner
Ram Rachum
Fellow of the @psf · Organizer at PyWeb-IL · Working on Google Cloud · My views are my own
Ram Rachum
Comments
  • Add support for thread identifiers

    Add support for thread identifiers

    Like @zzzeek as suggested these changes display thread infos (identifier and name) on output to help user to track execution on apps who use threading.

    Solve #79

  • Add classes to automatically track attributes, keys, etc. of variables

    Add classes to automatically track attributes, keys, etc. of variables

    Closes #59

    Sample usage:

    from types import SimpleNamespace
    
    import pysnooper
    from pysnooper.variables import Attrs, Indices, Keys, Enumerate
    
    
    @pysnooper.snoop(variables=(
            Keys('d', exclude='c'),
            Attrs('point'),
            Indices('lst')[-3:],
            Enumerate('s'),
    ))
    def foo():
        d = {'a': 1, 'b': 2, 'c': 'ignore'}
        point = SimpleNamespace(x=3, y=4)
        lst = list(range(1000))
        s = {7, 8, 9}
    
    
    foo()
    

    Output:

    19:01:11.689207 call        13 def foo():
    19:01:11.779673 line        14     d = {'a': 1, 'b': 2, 'c': 'ignore'}
    New var:....... (d)['a'] = 1
    New var:....... (d)['b'] = 2
    New var:....... d = {'a': 1, 'b': 2, 'c': 'ignore'}
    19:01:11.872593 line        15     point = SimpleNamespace(x=3, y=4)
    New var:....... (point).x = 3
    New var:....... (point).y = 4
    New var:....... point = namespace(x=3, y=4)
    19:01:12.000584 line        16     lst = list(range(1000))
    New var:....... (lst)[997] = 997
    New var:....... (lst)[998] = 998
    New var:....... (lst)[999] = 999
    New var:....... lst = [0, 1, 2, 3, 4, 5, ...]
    19:01:12.128228 line        17     s = {7, 8, 9}
    New var:....... (s)<0> = 8
    New var:....... (s)<1> = 9
    New var:....... (s)<2> = 7
    New var:....... s = {7, 8, 9}
    19:01:12.260525 return      17     s = {7, 8, 9}
    Return value:.. None
    
  • Runtime error (IronPython)

    Runtime error (IronPython)

    I get the following runtime error with the demo code provided in the readme:

    Runtime error (MissingMemberException): 'NoneType' object has no attribute 'get'
    Traceback:
      line 72, in decorate, "pysnooper.py"
      line 59, in get_source_from_frame, "pysnooper\tracer.py"
      line 209, in trace, "pysnooper\tracer.py"
      line 2, in number_to_bits, "<decorator-gen-10>"
      line 20, in script
    

    (line 20 calls the "number_to_bits" function)

  • Add elapsed time and elapsed_time format option.

    Add elapsed time and elapsed_time format option.

    Thank you for your awesome debug tool!

    This PR added a printing total elapsed time and elapsed_time format option. The test code is as follows.

    import argparse
    import time
    
    import pysnooper
    
    
    def test1(elapsed_time=False):
        with pysnooper.snoop(elapsed_time=elapsed_time):
            time.sleep(1.0)
    
    
    def test2(elapsed_time=False):
        a = 1
        b = 2
        c = 3
        with pysnooper.snoop(elapsed_time=elapsed_time):
            tmp = a + b
            tmp2 = a - b
    
    
    test1(elapsed_time=False)
    print()
    test1(elapsed_time=True)
    print()
    test2(elapsed_time=False)
    print()
    test2(elapsed_time=True)
    

    The result is as follows.

    Source path:... <ipython-input-14-f06356abdfce>
    New var:....... elapsed_time = False
    22:14:11.952148 line         9         time.sleep(1.0)
    Total elapsed time: 00:00:01.002691
    
    Source path:... <ipython-input-14-f06356abdfce>
    New var:....... elapsed_time = True
    00:00:00.000007 line         9         time.sleep(1.0)
    Total elapsed time: 00:00:01.003509
    
    Source path:... <ipython-input-14-f06356abdfce>
    New var:....... elapsed_time = False
    New var:....... a = 1
    New var:....... b = 2
    New var:....... c = 3
    22:14:13.959043 line        17         tmp = a + b
    New var:....... tmp = 3
    22:14:13.959235 line        18         tmp2 = a - b
    Total elapsed time: 00:00:00.000334
    
    Source path:... <ipython-input-14-f06356abdfce>
    New var:....... elapsed_time = True
    New var:....... a = 1
    New var:....... b = 2
    New var:....... c = 3
    00:00:00.000011 line        17         tmp = a + b
    New var:....... tmp = 3
    00:00:00.000295 line        18         tmp2 = a - b
    Total elapsed time: 00:00:00.000409
    
  • pysnooper/tracer.py

    pysnooper/tracer.py", line 75, in get_source_from_frame raise NotImplementedError

    When I ran a simple script:

    import pysnooper
    import numpy as np
    
    @pysnooper.snoop()
    def multi_matmul(times):
        x = np.random.rand(2,2)
        w = np.random.rand(2,2)
        
        for i in range(times):
            x = np.matmul(x,w)
        return x
    
    multi_matmul(3)
    

    The error happened:

    Starting var:.. times = 3
    Traceback (most recent call last):
      File "<tmp 1>", line 14, in <module>
        multi_matmul(3)
      File "</usr/local/lib/python3.6/dist-packages/decorator.py:decorator-gen-121>", line 2, in multi_matmul
      File "/usr/local/lib/python3.6/dist-packages/pysnooper/pysnooper.py", line 72, in decorate
        return function(*args, **kwargs)
      File "<tmp 1>", line 4, in multi_matmul
        @pysnooper.snoop()
      File "/usr/local/lib/python3.6/dist-packages/pysnooper/tracer.py", line 182, in trace
        source_line = get_source_from_frame(frame)[frame.f_lineno - 1]
      File "/usr/local/lib/python3.6/dist-packages/pysnooper/tracer.py", line 75, in get_source_from_frame
        raise NotImplementedError
    NotImplementedError
    
  • Allow using PySnooper as a command line program

    Allow using PySnooper as a command line program

    A POC for running pysnooper as an external command (e.g. pysnooper /path/to/script.py)

    The script will create sitecustomize.py script and will add it first in PYTHONPATH.

  • Remember state of the local vars beforehand

    Remember state of the local vars beforehand

    Now the local variables state is examined when the first callback fires. This can be too late, when the variables were created earlier, before the call. For example this code:

    import pysnooper
    import sys
    
    class C:
        pass
    
    def f():
        pass
    
    a = 1
    
    with pysnooper.snoop(depth=2):
        2 + 2
    

    produces this output

    New var:....... __name__ = '__main__'
    New var:....... __doc__ = None
    New var:....... __package__ = None
    New var:....... __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7f015d3adfd0>
    New var:....... __spec__ = None
    New var:....... __annotations__ = {}
    New var:....... __builtins__ = <module 'builtins' (built-in)>
    New var:....... __file__ = 'q.py'
    New var:....... __cached__ = None
    New var:....... pysnooper = <module 'pysnooper' from '/home/bay/tmp/249_pysnooper/PySnooper/pysnooper/__init__.py'>
    New var:....... sys = <module 'sys' (built-in)>
    New var:....... C = <class '__main__.C'>
    New var:....... f = <function f at 0x7f015d3e61e0>
    New var:....... a = 1
    00:51:25.580167 line        14     2 + 2
    

    Here we can see functions, classes, modules, and special variables, declared earlier.

    The idea of this one-line patch is to remember the local variables state before the first call, in the __enter__ function:

            self.frame_to_local_reprs[calling_frame] = get_local_reprs(calling_frame)
    

    This adds the symmetry with the __exit__ function, because it contains

            self.frame_to_local_reprs.pop(calling_frame, None)
    

    and also solves the problem:

    00:51:43.395931 line        14     2 + 2
    
  • Python 3.10 compatibility

    Python 3.10 compatibility

    Fixes: #218

    This is the minimum I had to implement to make the tests compatible with Python 3.10. I also did some improvements in configs for CI and tests.

    I'm not sure what is your approach to flake8 and pylint because there are already many errors and it's hard to find the ones possible caused by me here. Also, it's kinda verbose to use the names for keyword arguments instead of **kwargs for the optional ones but I've followed the current codebase.

    All unit tests are passing locally so we'll see what CI thinks about it.

  • Feature/normalize output

    Feature/normalize output

    normalizing output - removing machine specific data:

    absolute paths are replaced with the basename of the path.
    time stamps are removed.
    default (CPython) reprs of the form <.... at 0x[0-9a-fA-F] > will not contain the 'at 0x...' ending.
    

    tests:

    normalization does not affect the original test results so every test now runs with normalize=True
    and normalize=False to preserve compatability.
    assert_output can now verify the output of a normalized trace.
    

    readme:

    • updated accordingly
  • Allow snooping on all methods of a class (issue #150)

    Allow snooping on all methods of a class (issue #150)

    This implements https://github.com/cool-RR/PySnooper/issues/150 by allowing the same decorator to be applied to a decorator or to a class.

    Based on input from @alexmojaki I have added tests to verify that snoop at least does not actively break properties or other methods with decorators. The output is probably suboptimal/confusing in those cases, but I think that should be OK given the warning in README.md.

  • Major refactor separating tracing, writing, and formatting

    Major refactor separating tracing, writing, and formatting

    This separates the concerns of tracing (gathering data), formatting the data, and writing the formatted data. The code is much more organised and easier to read, modify, and test, and the library is easier to externally customise and extend.

    The next stage is to remove the use of regexes from tests. Instead, test entries will be compared with the unformatted data directly.

    After that, I'd like to make my own separate library which uses pysnooper at its core while adding additional functionality through dependencies. For example, I'd like to have output colouring including syntax highlighting using pygments and colorama.

    In the long term, this will also enable a radically different infrastructure such as writing to a database and then viewing the logs in a web UI.

    And then, I'd like to understand how to git.

  • Snoop recursively, but only in my code.

    Snoop recursively, but only in my code.

    Hi,

    I would like to trace the http request handling of Django in my development environment.

    But I am sure that the bug is in my code, not in the code of Django.

    I would like to see every line of my code, excluding Django, standard library and other libraries.

    I had a look at the arguments which snoop() accepts, but I think it is not possible up to now.

    Would you accept a PR which implements this?

    Do you have a hint how to implement this?

  • Restore previous f_trace after snoop is done

    Restore previous f_trace after snoop is done

    See https://github.com/alexmojaki/snoop/pull/34

    Basically you need to do this:

    https://github.com/alexmojaki/snoop/blob/437956c661184ab458a3f60fcf97c81c31ea2db1/snoop/tracer.py#L213-L217

    In particular you need calling_frame.f_trace = previous_trace in addition to the existing sys.settrace(previous_trace).

    This is obviously good for restoring the other tracer properly, which I confirmed with the PyCharm debugger. But I stumbled across this because pp stopped working after with snoop: exited. It turns out that if you don't reset f_trace then the frame line number becomes wrong. Here's a demo:

    import inspect
    import pysnooper
    
    with pysnooper.snoop():
        pass
    
    print(inspect.currentframe().f_trace)
    print(inspect.currentframe().f_lineno)  # wrong lineno
    1/0  # wrong lineno in traceback
    

    In particular note the weird traceback:

    Traceback (most recent call last):
      File "...", line 5, in <module>
        pass
    ZeroDivisionError: division by zero
    

    Here's a pure python version of the demo:

    import inspect
    import sys
    
    frame = inspect.currentframe()
    
    frame.f_trace = lambda *_: print('tracing', frame.f_lineno) or frame.f_trace
    sys.settrace(frame.f_trace)
    
    # frame.f_trace = None  # uncomment to fix
    sys.settrace(None)
    
    print('after trace', frame.f_lineno)  # wrong lineno
    1/0  # wrong lineno in traceback
    

    This is a bug which is apparently fixed in 3.10: https://bugs.python.org/issue42823

  • `relative_time` fails with generator

    `relative_time` fails with generator

    I found a bug with relative_time when used with a generator. The times shown seem to be reset at some point. I don't know whether this bug could also happen without generators.

    @iory Can you take a look at this bug?

    @alexmojaki I suspect that this has something to do with the start_times dict you suggested.

    Sample:

    import pysnooper
    import time
    
    def g():
        time.sleep(0.1)
        yield 8
        time.sleep(0.1)
    
    @pysnooper.snoop(relative_time=True)
    def f():
        time.sleep(0.1)
        yield from g()
        time.sleep(0.1)
    
    
    tuple(f())
    

    Output:

    Source path:... C:\Users\Administrator\Desktop\fuck.py
    00:00:00.000000 call        10 def f():
    00:00:00.000000 line        11     time.sleep(0.1)
    00:00:00.100000 line        12     yield from g()
    00:00:00.200000 return      12     yield from g()
    Return value:.. 8
    Elapsed time: 00:00:00.200000
    00:00:00.000000 call        12     yield from g()
    00:00:00.100000 exception   12     yield from g()
    StopIteration
    00:00:00.100000 line        13     time.sleep(0.1)
    00:00:00.207002 return      13     time.sleep(0.1)
    Return value:.. None
    Elapsed time: 00:00:00.207002
    
  • Minor bug in nesting

    Minor bug in nesting

    There is a minor bug at the nesting case. Following is the example.

    import time
    import pysnooper
    
    snoop = pysnooper.snoop()
    
    def test():
        with snoop:
            a = 10
            with snoop:
                time.sleep(1.0)
            a = 20
            time.sleep(1.0)
    
    
    test()
    

    The output is the following.

    Source path:... <ipython-input-2-9ee7fdb55b80>
    22:57:41.232785 line        11         a = 10
    New var:....... a = 10
    22:57:41.232850 line        12         with snoop:
    22:57:41.232936 line        13             time.sleep(1.0)
    

    The debug outputs of the a = 20 and time.sleep(1.0) are vanished. I don't think we'll use it like this, but I'll report it.

  • Feature request: Implement tool to explore long PySnooper outputs

    Feature request: Implement tool to explore long PySnooper outputs

    It happened that I was using PySnooper with depth=10000 to explore a piece of running code. Of course, the output was huge with many levels of indentations. I wanted to explore it, but it was difficult to track the execution of a single function.

    It would be cool if there was a tool that accepted PySnooper outputs and allow some useful use cases of exploring them to find what you want.

printstack is a Python package that adds stack trace links to the builtin print function, so that editors such as PyCharm can link you to the source of the print call.
printstack is a Python package that adds stack trace links to the builtin print function, so that editors such as PyCharm can link you to the source of the print call.

printstack is a Python package that adds stack trace links to the builtin print function, so that editors such as PyCharm can link to the source of the print call.

Sep 10, 2021
Debugging manhole for python applications.

Overview docs tests package Manhole is in-process service that will accept unix domain socket connections and present the stacktraces for all threads

Jun 29, 2022
A toolbar overlay for debugging Flask applications

Flask Debug-toolbar This is a port of the excellent django-debug-toolbar for Flask applications. Installation Installing is simple with pip: $ pip ins

Jun 25, 2022
A powerful set of Python debugging tools, based on PySnooper
A powerful set of Python debugging tools, based on PySnooper

snoop snoop is a powerful set of Python debugging tools. It's primarily meant to be a more featureful and refined version of PySnooper. It also includ

Jun 26, 2022
Cyberbrain: Python debugging, redefined.
 Cyberbrain: Python debugging, redefined.

Cyberbrain1(电子脑) aims to free programmers from debugging.

Jun 28, 2022
VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.
VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

Jul 3, 2022
A package containing a lot of useful utilities for Python developing and debugging.
A package containing a lot of useful utilities for Python developing and debugging.

Vpack A package containing a lot of useful utilities for Python developing and debugging. Features Sigview: press Ctrl+C to print the current stack in

Jun 20, 2022
A web-based visualization and debugging platform for NuPIC

Cerebro 2 A web-based visualization and debugging platform for NuPIC. Usage Set up cerebro2.server to export your model state. Then, run: cd static py

Oct 13, 2021
Hypothesis debugging with vscode

Hypothesis debugging with vscode

Feb 9, 2022
Python's missing debug print command and other development tools.
Python's missing debug print command and other development tools.

python devtools Python's missing debug print command and other development tools. For more information, see documentation. Install Just pip install de

Jul 4, 2022
🍦 Never use print() to debug again.

IceCream -- Never use print() to debug again Do you ever use print() or log() to debug your code? Of course you do. IceCream, or ic for short, makes p

Jun 30, 2022
You'll never want to use cd again.
You'll never want to use cd again.

Jmp Description Have you ever used the cd command? You'll never touch that outdated thing again when you try jmp. Navigate your filesystem with unprec

May 24, 2022
A python script that will automate the boring task of login to the captive portal again and again

A python script that will automate the boring task of login to the captive portal again and again

Feb 9, 2022
Never miss a deadline again

Hack the Opportunities Never miss a deadline again! Link to the excel sheet Contribution This list is not complete and I alone cannot make it whole. T

Jun 29, 2022
Never get booted from a game for inactivity ever again

Anti AFK Bot Never get booted from a game for inactivity ever again! Built With Python Installation Clone the repo git clone https://github.com/lippie

Dec 5, 2021
Never get kicked for inactivity ever again!

FFXIV AFK Bot Tired of getting kicked from games due to inactivity? This Bot will make random movements in random intervals to prevent you from gettin

Jan 12, 2022
Qt-creator-boost-debugging-helper - Qt Creator Debugging Helper for Boost Library
Qt-creator-boost-debugging-helper - Qt Creator Debugging Helper for Boost Library

Go to Tools > Options > Debugger > Locals & Expressions. Paste the script path t

Apr 22, 2022
printstack is a Python package that adds stack trace links to the builtin print function, so that editors such as PyCharm can link you to the source of the print call.
printstack is a Python package that adds stack trace links to the builtin print function, so that editors such as PyCharm can link you to the source of the print call.

printstack is a Python package that adds stack trace links to the builtin print function, so that editors such as PyCharm can link to the source of the print call.

Sep 10, 2021
This is discord nitro code generator and checker made with python. This will generate nitro codes and checks if the code is valid or not. If code is valid then it will print the code leaving 2 lines and if not then it will print '*'.

Discord Nitro Generator And Checker ⚙️ Rᴜɴ Oɴ Rᴇᴘʟɪᴛ ??️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs If you are taking code from this repository without a fork, then atleast

May 20, 2022
Generate your own QR Code and scan it to see the results! Never use it for malicious purposes.

QR-Code-Generator-Python Choose the name of your generated QR .png file. If it happens to open the .py file (the application), there are specific comm

Dec 23, 2021