pipreqs - Generate pip requirements.txt file based on imports of any project. Looking for maintainers to move this project forward.

pipreqs - Generate requirements.txt file for any project based on imports

Installation

pip install pipreqs

Usage

Usage:
    pipreqs [options] <path>

Options:
    --use-local           Use ONLY local package info instead of querying PyPI
    --pypi-server <url>   Use custom PyPi server
    --proxy <url>         Use Proxy, parameter will be passed to requests library. You can also just set the
                          environments parameter in your terminal:
                          $ export HTTP_PROXY="http://10.10.1.10:3128"
                          $ export HTTPS_PROXY="https://10.10.1.10:1080"
    --debug               Print debug information
    --ignore <dirs>...    Ignore extra directories
    --encoding <charset>  Use encoding parameter for file open
    --savepath <file>     Save the list of requirements in the given file
    --print               Output the list of requirements in the standard output
    --force               Overwrite existing requirements.txt
    --diff <file>         Compare modules in requirements.txt to project imports.
    --clean <file>        Clean up requirements.txt by removing modules that are not imported in project.
    --no-pin              Omit version of output packages.

Example

$ pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt

Contents of requirements.txt

wheel==0.23.0
Yarg==0.1.9
docopt==0.6.2

Why not pip freeze?

  • pip freeze only saves the packages that are installed with pip install in your environment.
  • pip freeze saves all packages in the environment including those that you don't use in your current project. (if you don't have virtualenv)
  • and sometimes you just need to create requirements.txt for a new project without installing modules.
Owner
Vadim Kravcenko
CTO @ Mindnow AG
Vadim Kravcenko
Comments
  • Error while creating requirements.txt

    Error while creating requirements.txt

    Hello!

    C:\Users\ipetrash\Desktop\PyScripts\hideme.ru-proxy-list>pipreqs C:\Users\ipetrash\Desktop\PyScripts\hideme.ru-proxy-list
    Traceback (most recent call last):
      File "C:\Python34\lib\runpy.py", line 170, in _run_module_as_main
        "__main__", mod_spec)
      File "C:\Python34\lib\runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "C:\Python34\Scripts\pipreqs.exe\__main__.py", line 9, in <module>
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 178, in main
        init(args)
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 157, in init
        local = get_import_local(candidates)
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 112, in get_import_local
        local = get_locally_installed_packages()
      File "C:\Python34\lib\site-packages\pipreqs\pipreqs.py", line 105, in get_locally_installed_packages
        'version': package[1].replace(".dist", ""),
    IndexError: list index out of range
    

    Used for: https://github.com/gil9red/hideme.ru-proxy-list

    Sincerely yours, gil9red

  • New Option for Dynamic Versioning, e.g. wheel==0.23.*, wheel==0.* or wheel

    New Option for Dynamic Versioning, e.g. wheel==0.23.*, wheel==0.* or wheel

    Added new option "dynamic", for dynamic versioning.

    Example: wheel==0.23.0

    There will be 3 possible args for this option, micro, minor or all. The result in requirements.txt will be as below.

    • wheel==0.23.* (micro)
    • wheel==0.* (minor)
    • wheel (all)

    This will enable more targeted installations, esp if one is only looking at bug fixes, security patches or minor/micro version updates, or simply just install most recent updates.

    --no-pin option is merged within this new option as the "all" arg.

    @jcch94

  • UnicodeDecodeError: 'utf-8' codec can't decode byte

    UnicodeDecodeError: 'utf-8' codec can't decode byte

    I just installed pipreqs for the first time. This is what it does:

    $ pipreqs $(pwd)
    Traceback (most recent call last):
      File "/Users/tillmann/dev/textSum/.direnv/python-3.8.3/bin/pipreqs", line 10, in <module>
        sys.exit(main())
      File "/Users/tillmann/dev/textSum/.direnv/python-3.8.3/lib/python3.8/site-packages/pipreqs/pipreqs.py", line 470, in main
        init(args)
      File "/Users/tillmann/dev/textSum/.direnv/python-3.8.3/lib/python3.8/site-packages/pipreqs/pipreqs.py", line 406, in init
        candidates = get_all_imports(input_path,
      File "/Users/tillmann/dev/textSum/.direnv/python-3.8.3/lib/python3.8/site-packages/pipreqs/pipreqs.py", line 122, in get_all_imports
        contents = f.read()
      File "/usr/local/bin/../Cellar/[email protected]/3.8.3_2/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/codecs.py", line 322, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 81: invalid start byte
    

    Am I doing something wrong?

    $ neofetch           
                        'c.          [email protected] 
                     ,xNMM.          -------------------- 
                   .OMMMMo           OS: macOS Catalina 10.15.5 19F101 x86_64 
                   OMMM0,            Host: MacBookPro15,1 
         .;loddo:' loolloddol;.      Kernel: 19.5.0 
       cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 17 days, 14 hours, 5 mins 
     .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 115 (brew) 
     XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.7.1 
    ;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: [email protected], [email protected], [email protected] 
    :MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua 
    .MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor 
     kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Light) 
     .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: Apple_Terminal 
      .XMMMMMMMMMMMMMMMMMMMMMMMMK.   Terminal Font: HackNerdFontComplete-Regular 
        kMMMMMMMMMMMMMMMMMMMMMMd     CPU: Intel i7-8850H (12) @ 2.60GHz 
         ;KMMMMMMMWXXWMMMMMMMk.      GPU: Intel UHD Graphics 630, Radeon Pro 560X 
           .cooc,.    .,coo:.        Memory: 10080MiB / 16384MiB 
    $ python -V                      
    Python 3.8.3
    
  • Pipreqs missing some requirements

    Pipreqs missing some requirements

    I develop this project: https://github.com/tuwien-musicir/rp_extract

    As you can see from the imports it uses a lot of libraries (numpy, scipy, matplotlib, argparse, unicsv, cPickle, sklearn, ...) - many of which are missed by pipreqs:

    pipreqs --debug --force . DEBUG: Found packages: set(['argparse', 'cPickle', 'uuid', 'tempfile', 'numpy', 'pylab', 'matplotlib', 'wave', 'sys', 'subprocess', 'os', 'sklearn']) DEBUG: Found imports: matplotlib, numpy, matplotlib, scikit_learn DEBUG: Getting packages information from Local/PyPI INFO: Starting new HTTPS connection (1): pypi.python.org DEBUG: Setting read timeout to None DEBUG: "GET /pypi/numpy/json HTTP/1.1" 200 65657 INFO: Starting new HTTPS connection (1): pypi.python.org DEBUG: Setting read timeout to None DEBUG: "GET /pypi/scikit_learn/json HTTP/1.1" 301 0 DEBUG: Setting read timeout to None DEBUG: "GET /pypi/scikit-learn/json HTTP/1.1" 200 102410 DEBUG: Writing 4 requirements: matplotlib, matplotlib, numpy, scikit_learn to ./requirements.txt INFO: Successfully saved requirements file in ./requirements.txt [email protected]:~/rp_extract$ cat requirements.txt matplotlib == 1.3.1.egg matplotlib == 1.3.1.egg numpy == 1.10.1 scikit_learn == 0.17

    Why does it miss many of the library requirements? (some of them are even mentioned in the debug output, some are not, e.g. scipy, unicsv) (and matplotlib on the other hand is put twice into the requirements.txt)

  • Error executing newly installed pipreqs

    Error executing newly installed pipreqs

    I installed pipreqs using pip install pipreqs within my local virtualenv and then executed it, got this error:

    ~/code % pipreqs web
    Looking for imports
    Traceback (most recent call last):
      File "/home/satiani/code/web/venv/bin/pipreqs", line 8, in <module>
        load_entry_point('pipreqs==0.1.8', 'console_scripts', 'pipreqs')()
      File "/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 109, in main
        init(args)
      File "/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 92, in init
        imports = get_all_imports(args['<path>'])
      File "/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 62, in get_all_imports
        with open(os.path.join(os.path.dirname(__file__), "stdlib"), "r") as f:
    IOError: [Errno 2] No such file or directory: '/home/satiani/code/web/venv/local/lib/python2.7/site-packages/pipreqs/stdlib'
    zsh: exit 1     pipreqs web
    
  • Difference between package name and import name makes problem

    Difference between package name and import name makes problem

    For example, to import beautifulsoup4 package, it should be import bs4. So pipreqs cannot find the beautifulsoup4 package from pypi server. I don't have an idea to resolve this problem... :(

  • whitelines at the beginning of the file

    whitelines at the beginning of the file

    pipreqs fails on python files which have blank lines at the beginning of the file.

    System:

    • Python 3.6.8
    • 16.04.1-Ubuntu
    • pipreqs 0.4.10

    Stacktrace:

    ERROR: Failed on file: /home/test/file.py
    Traceback (most recent call last):
      File "/usr/local/bin/pipreqs", line 10, in <module>
        sys.exit(main())
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 470, in main
        init(args)
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 409, in init
        follow_links=follow_links)
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 138, in get_all_imports
        raise exc
      File "/usr/local/lib/python3.6/dist-packages/pipreqs/pipreqs.py", line 124, in get_all_imports
        tree = ast.parse(contents)
      File "/usr/lib/python3.6/ast.py", line 35, in parse
        return compile(source, filename, mode, PyCF_ONLY_AST)
      File "<unknown>", line 1
        
        ^
    SyntaxError: invalid character in identifier
    
  • Is there a way to output >= instead of == in requirements.txt?

    Is there a way to output >= instead of == in requirements.txt?

    Is there a way to output >= instead of == in requirements.txt?

    pygsheets>=1.1.3
    retrying>=1.3.3
    

    Instead of

    pygsheets==1.1.3
    retrying==1.3.3
    

    Thanks!

  • -bash: pipreqs: command not found

    -bash: pipreqs: command not found

    When I try to run the pipreqs from cli, I got ther is no command found. I miss something? The installation process and the usage is jus this:

    $ pip install pipreqs
    $ pipreqs [path/to/project]
    
  • Exception when 'import' is used in comments

    Exception when 'import' is used in comments

    A freshly installed pipreqs (in a venv) crashes on pylint source (in a venv in my case)

    Offending line is: https://bitbucket.org/logilab/pylint/src/cc5990f6a12a467d09567bfc1b210ba999f6cb13/pylint/test/functional/unused_import.py?at=default#cl-3

    ~/src/c3 $ pipreqs .
    Traceback (most recent call last):
      File "/home/koert/src/c3/.ve2/bin/pipreqs", line 11, in <module>
        sys.exit(main())
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 167, in main
        init(args)
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 139, in init
        candidates = get_all_imports(args['<path>'])
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 48, in get_all_imports
        res = map(get_name_without_alias, item.split(","))
      File "/home/koert/src/c3/.ve2/local/lib/python2.7/site-packages/pipreqs/pipreqs.py", line 130, in get_name_without_alias
        name = REGEXP[0].match(name.strip()).groups(0)[0]
    AttributeError: 'NoneType' object has no attribute 'groups'
    ~/src/c3 $ pip freeze
    docopt==0.6.2
    pipreqs==0.2.4
    requests==2.7.0
    yarg==0.1.9
    
  • Incorret package included for Hydra

    Incorret package included for Hydra

    I'm using Facebook hydra-core package (see Hydra.cc). But this package is not correctly handled because of another package with the same name (hydra - the python bloom filter).

    Example

    # content of my.py, the only file in my dir to show the behavior with pipreqs
    import hydra
    
    if __name__ == "__main__":
        print(hydra.__version__)
    

    Case 1:

    Only hydra-core is locally installed

    # Name Version Build Channel hydra-core 1.0.6 pypi_0 pypi

    Then output of pipreqs --print is:

    hydra_core==1.0.6
    hydra==2.5
    INFO: Successfully output requirements
    

    And output of pipreqs --print --use-local is:

    hydra_core==1.0.6
    INFO: Successfully output requirements
    

    Case 2:

    When both hydra-core and hydra are installed (this easily happens to anyone using fb hydra-core package because of accidentally using pip install hydra instead of pip install hydra-core), then for both commands (with and without --use-local flag) the output is following:

    Hydra==2.5
    INFO: Successfully output requirements
    

    Question

    I was thinking about updating mapping with a line of hydra:hydra-core (similarly to https://github.com/bndr/pipreqs/pull/234). Is that the right way how to deal with this issue? I was just a bit wondering about this since the behavior described above was quite strange to me (I was also confused with e.g. why hydra starts sometimes with capital letter and sometimes not).

    Environment

    Python 3.8.6 pipreqs 0.4.10

  • Question: is there any way to add additional libraries to the requirements file, which aren't imported anywhere?

    Question: is there any way to add additional libraries to the requirements file, which aren't imported anywhere?

    I want the pipreqs library itself to the requirements file, but without importing it anywhere. Is there any way to do this? Also, I'd rather not add it manually.

  • Pipreqs generates bad requirement besides pymongo

    Pipreqs generates bad requirement besides pymongo

    I'm generating the requirements for a project that uses the pymongo library and it seems that pipreqs also generates the following requirement, which is invalid: ~ymongo==4.1.1

  • The program does not recognize packages with names containing capital letters.

    The program does not recognize packages with names containing capital letters.

    For example cx_Oracle. I run the program with --use-local parameter. I was reading the code and suppose the issue are in ‘get_import_local()’ function. It uses item.lower() method.

  • 'pipreqs' is not recognized as an internal or external command, operable program or batch file.

    'pipreqs' is not recognized as an internal or external command, operable program or batch file.

    I've tried everything, ranging from adding it through the path by editing system variables, to reinstalling pip and python but nothing seems to work. Could anyone please tell me how to fix this error?

  • fix when calling from outside directory

    fix when calling from outside directory

    Hello, tanks for the great package. The current directory is always included in the search path. This can be a problem when calling pipreqs from another directory (e.g.: pipreqs my_dir), as the current directory and the desired directory (my_dir) will be included in the search path. This PR ensures that only the desired directory is scanned.

Package manager based on libdnf and libsolv. Replaces YUM.

Dandified YUM Dandified YUM (DNF) is the next upcoming major version of YUM. It does package management using RPM, libsolv and hawkey libraries. For m

Jul 2, 2022
Easy to use, fast, git sourced based, C/C++ package manager.

Yet Another C/C++ Package Manager Easy to use, fast, git sourced based, C/C++ package manager. Features No need to install a program, just include the

May 14, 2022
Example for how to package a Python library based on Cython.

Cython sample module This project is an example of a module that can be built using Cython. It is an upgrade from a similar model developed by Arin Kh

Jan 17, 2022
:package: :fire: Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.
:package: :fire: Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.

THE PROJECT IS ARCHIVED Forks: https://github.com/orsinium/forks DepHell -- project management for Python. Why it is better than all other tools: Form

Jun 28, 2022
If you have stars in your Pipfile and you don't want them, this project is for you!

unstar-pipfile If you have stars in your Pipfile, this project is for you! unstar-pipfile is a tool to scan Pipfile.lock and replace any stars in Pipf

Mar 30, 2022
Workon - A simple project manager for conda, windows 10 and vscode

WORK ON A simple project manager for conda, windows 10 and vscode Installation p

Jan 16, 2022
to-requirements.txt allows to automatically add and delete modules to requirements.txt installing them using pip.
to-requirements.txt allows to automatically add and delete modules to requirements.txt installing them using pip.

to-requirements.txt | Automatically update requirements.txt to-requirements.txt allows to automatically add and delete modules to requirements.txt ins

Jan 8, 2022
Listreqs is a simple requirements.txt generator. It's an alternative to pipreqs
Listreqs is a simple requirements.txt generator. It's an alternative to pipreqs

⚡ Listreqs Listreqs is a simple requirements.txt generator. It's an alternative to pipreqs. Where in Pipreqs, it helps you to Generate requirements.tx

Oct 15, 2021
Create pinned requirements.txt inside a Docker image using pip-tools
Create pinned requirements.txt inside a Docker image using pip-tools

Pin your Python dependencies! pin-requirements.py is a script that lets you pin your Python dependencies inside a Docker container. Pinning your depen

Apr 22, 2022
Tool for pinpointing circular imports in Python. Find cyclic imports in any project
Tool for pinpointing circular imports in Python. Find cyclic imports in any project

Pycycle: Find and fix circular imports in python projects Pycycle is an experimental project that aims to help python developers fix their circular de

Jun 29, 2022
theHasher Tool created for generate strong and unbreakable passwords by using Hash Functions.Generate Hashes and store them in txt files.Use the txt files as lists to execute Brute Force Attacks!
theHasher Tool created for generate strong and unbreakable passwords by using Hash Functions.Generate Hashes and store them in txt files.Use the txt files as lists to execute Brute Force Attacks!

$theHasher theHasher is a Tool for generating hashes using some of the most Famous Hashes Functions ever created. You can save your hashes to correspo

Feb 2, 2022
🖍️This is a feature-complete clone of the awesome Chalk (JavaScript) library.
🖍️This is a feature-complete clone of the awesome Chalk (JavaScript) library.

Terminal string styling done right This is a feature-complete clone of the awesome Chalk (JavaScript) library. All credits go to Sindre Sorhus. Highli

Jun 12, 2022
An assistant to guess your pip dependencies from your code, without using a requirements file.

Pip Sala Bim is an assistant to guess your pip dependencies from your code, without using a requirements file. Pip Sala Bim will tell you which packag

Mar 24, 2022
With the help of json txt you can use your txt file as a json file in a very simple way

json txt With the help of json txt you can use your txt file as a json file in a very simple way Dependencies re filemod pip install filemod Installat

Jan 5, 2022
Annotate your Python requirements.txt file with summaries of each package.

Summarize Requirements ?? ?? Annotate your Python requirements.txt file with a short summary of each package. This tool: takes a Python requirements.t

Apr 22, 2022
Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Dec 17, 2021
VSCode extension to sort and refactor python imports using reorder-python-imports.
VSCode extension to sort and refactor python imports using reorder-python-imports.

reorder-python-imports VSCode extension to sort and refactor python imports using reorder-python-imports. Unlike other import organizers, reorder-pyth

Mar 3, 2022
New AidForBlind - Various Libraries used like OpenCV and other mentioned in Requirements.txt

AidForBlind Recommended PyCharm IDE Various Libraries used like OpenCV and other

Jan 13, 2022
Python requirements.txt Guesser
 Python requirements.txt Guesser

Python-Requirements-Guesser ⚠️ This is alpha quality software. Work in progress Attempt to guess requirements.txt modules versions based on Git histor

May 24, 2022
AutoSub is a CLI application to generate subtitle files (.srt, .vtt, and .txt transcript) for any video file using Mozilla DeepSpeech.

AutoSub About Motivation Installation Docker How-to example How it works TO-DO Contributing References About AutoSub is a CLI application to generate

Jul 4, 2022