Python library and shell utilities to monitor filesystem events.

Watchdog

https://travis-ci.org/gorakhargosh/watchdog.svg?branch=master

Python API and shell utilities to monitor file system events.

Works on 3.6+.

If you want to use Python 2.6, you should stick with watchdog < 0.10.0.

If you want to use Python 2.7, 3.4 or 3.5, you should stick with watchdog < 1.0.0.

Example API Usage

A simple program that uses watchdog to monitor directories specified as command-line arguments and logs events generated:

import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO,
                        format='%(asctime)s - %(message)s',
                        datefmt='%Y-%m-%d %H:%M:%S')
    path = sys.argv[1] if len(sys.argv) > 1 else '.'
    event_handler = LoggingEventHandler()
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    finally:
        observer.stop()
        observer.join()

Shell Utilities

Watchdog comes with an optional utility script called watchmedo. Please type watchmedo --help at the shell prompt to know more about this tool.

Here is how you can log the current directory recursively for events related only to *.py and *.txt files while ignoring all directory events:

watchmedo log \
    --patterns="*.py;*.txt" \
    --ignore-directories \
    --recursive \
    .

You can use the shell-command subcommand to execute shell commands in response to events:

watchmedo shell-command \
    --patterns="*.py;*.txt" \
    --recursive \
    --command='echo "${watch_src_path}"' \
    .

Please see the help information for these commands by typing:

watchmedo [command] --help

About watchmedo Tricks

watchmedo can read tricks.yaml files and execute tricks within them in response to file system events. Tricks are actually event handlers that subclass watchdog.tricks.Trick and are written by plugin authors. Trick classes are augmented with a few additional features that regular event handlers don't need.

An example tricks.yaml file:

tricks:
- watchdog.tricks.LoggerTrick:
    patterns: ["*.py", "*.js"]
- watchmedo_webtricks.GoogleClosureTrick:
    patterns: ['*.js']
    hash_names: true
    mappings_format: json                  # json|yaml|python
    mappings_module: app/javascript_mappings
    suffix: .min.js
    compilation_level: advanced            # simple|advanced
    source_directory: app/static/js/
    destination_directory: app/public/js/
    files:
      index-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/index-page.js
      about-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/about-page/**/*.js

The directory containing the tricks.yaml file will be monitored. Each trick class is initialized with its corresponding keys in the tricks.yaml file as arguments and events are fed to an instance of this class as they arrive.

Installation

Install from PyPI using pip:

$ python -m pip install -U watchdog

# or to install the watchmedo utility:
$ python -m pip install -U watchdog[watchmedo]

Install from source:

$ python -m pip install -e .

# or to install the watchmedo utility:
$ python -m pip install -e ".[watchmedo]"

Installation Caveats

The watchmedo script depends on PyYAML which links with LibYAML, which brings a performance boost to the PyYAML parser. However, installing LibYAML is optional but recommended. On Mac OS X, you can use homebrew to install LibYAML:

$ brew install libyaml

On Linux, use your favorite package manager to install LibYAML. Here's how you do it on Ubuntu:

$ sudo apt install libyaml-dev

On Windows, please install PyYAML using the binaries they provide.

Documentation

You can browse the latest release documentation online.

Contribute

Fork the repository on GitHub and send a pull request, or file an issue ticket at the issue tracker. For general help and questions use the official mailing list or ask on stackoverflow with tag python-watchdog.

Create and activate your virtual environment, then:

python -m pip install pytest pytest-cov
python -m pip install -e ".[watchmedo]"
python -m pytest tests

If you are making a substantial change, add an entry to the "Unreleased" section of the changelog.

Supported Platforms

  • Linux 2.6 (inotify)
  • Mac OS X (FSEvents, kqueue)
  • FreeBSD/BSD (kqueue)
  • Windows (ReadDirectoryChangesW with I/O completion ports; ReadDirectoryChangesW worker threads)
  • OS-independent (polling the disk for directory snapshots and comparing them periodically; slow and not recommended)

Note that when using watchdog with kqueue, you need the number of file descriptors allowed to be opened by programs running on your system to be increased to more than the number of files that you will be monitoring. The easiest way to do that is to edit your ~/.profile file and add a line similar to:

ulimit -n 1024

This is an inherent problem with kqueue because it uses file descriptors to monitor files. That plus the enormous amount of bookkeeping that watchdog needs to do in order to monitor file descriptors just makes this a painful way to monitor files and directories. In essence, kqueue is not a very scalable way to monitor a deeply nested directory of files and directories with a large number of files.

About using watchdog with editors like Vim

Vim does not modify files unless directed to do so. It creates backup files and then swaps them in to replace the files you are editing on the disk. This means that if you use Vim to edit your files, the on-modified events for those files will not be triggered by watchdog. You may need to configure Vim appropriately to disable this feature.

About using watchdog with CIFS

When you want to watch changes in CIFS, you need to explicitly tell watchdog to use PollingObserver, that is, instead of letting watchdog decide an appropriate observer like in the example above, do:

from watchdog.observers.polling import PollingObserver as Observer

Dependencies

  1. Python 3.6 or above.
  2. XCode (only on Mac OS X)
  3. PyYAML (only for watchmedo script)
  4. argh (only for watchmedo script)

Licensing

Watchdog is licensed under the terms of the Apache License, version 2.0.

Copyright 2011 Yesudeep Mangalapilly.

Copyright 2012 Google, Inc & contributors.

Project source code is available at Github. Please report bugs and file enhancement requests at the issue tracker.

Why Watchdog?

Too many people tried to do the same thing and none did what I needed Python to do:

Comments
  • [0.9.0] Build fails on test

    [0.9.0] Build fails on test

    I see 2 problems:

    • First the test fails
    • The module is installed at the wrong location. Because it contains C/C++ code it mist be placed under /usr/lib64 but the installer will place it under /usr/lib/ witch is for python code only.

    here the build log:

    Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.TCew4J
    + umask 022
    + cd /builddir/build/BUILD
    + cd /builddir/build/BUILD
    + rm -rf watchdog-0.9.0
    + /usr/bin/gzip -dc /builddir/build/SOURCES/watchdog-0.9.0.tar.gz
    + /usr/bin/tar -xf -
    + STATUS=0
    + '[' 0 -ne 0 ']'
    + cd watchdog-0.9.0
    + /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
    + rm -rf watchdog.egg-info
    + exit 0
    Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.HU8XdO
    + umask 022
    + cd /builddir/build/BUILD
    + cd watchdog-0.9.0
    + CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic'
    + /usr/bin/python3.4 setup.py build '--executable=/usr/bin/python3.4 -s'
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/watchdog
    copying src/watchdog/watchmedo.py -> build/lib/watchdog
    copying src/watchdog/version.py -> build/lib/watchdog
    copying src/watchdog/events.py -> build/lib/watchdog
    copying src/watchdog/__init__.py -> build/lib/watchdog
    creating build/lib/watchdog/utils
    copying src/watchdog/utils/win32stat.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/unicode_paths.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/platform.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/importlib2.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/event_backport.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/echo.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/dirsnapshot.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/delayed_queue.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/decorators.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/compat.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/bricks.py -> build/lib/watchdog/utils
    copying src/watchdog/utils/__init__.py -> build/lib/watchdog/utils
    creating build/lib/watchdog/tricks
    copying src/watchdog/tricks/__init__.py -> build/lib/watchdog/tricks
    creating build/lib/watchdog/observers
    copying src/watchdog/observers/winapi.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/read_directory_changes.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/polling.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/kqueue.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/inotify_c.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/inotify_buffer.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/inotify.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/fsevents2.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/fsevents.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/api.py -> build/lib/watchdog/observers
    copying src/watchdog/observers/__init__.py -> build/lib/watchdog/observers
    running egg_info
    writing dependency_links to src/watchdog.egg-info/dependency_links.txt
    writing entry points to src/watchdog.egg-info/entry_points.txt
    writing top-level names to src/watchdog.egg-info/top_level.txt
    writing requirements to src/watchdog.egg-info/requires.txt
    writing src/watchdog.egg-info/PKG-INFO
    reading manifest file 'src/watchdog.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.h' under directory 'src'
    writing manifest file 'src/watchdog.egg-info/SOURCES.txt'
    + sleep 1
    + rm -rf html/.doctrees html/.buildinfo
    + exit 0
    Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.8QiunX
    + umask 022
    + cd /builddir/build/BUILD
    + '[' /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64 '!=' / ']'
    + rm -rf /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64
    ++ dirname /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64
    + mkdir -p /builddir/build/BUILDROOT
    + mkdir /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64
    + cd watchdog-0.9.0
    + CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic'
    + /usr/bin/python3.4 setup.py install -O1 --skip-build --root /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64
    running install
    running install_lib
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/__init__.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/api.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/fsevents.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/fsevents2.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/inotify.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/inotify_buffer.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/inotify_c.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/kqueue.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/polling.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/read_directory_changes.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    copying build/lib/watchdog/observers/winapi.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/tricks
    copying build/lib/watchdog/tricks/__init__.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/tricks
    creating /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/__init__.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/bricks.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/compat.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/decorators.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/delayed_queue.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/dirsnapshot.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/echo.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/event_backport.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/importlib2.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/platform.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/unicode_paths.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/utils/win32stat.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils
    copying build/lib/watchdog/__init__.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog
    copying build/lib/watchdog/events.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog
    copying build/lib/watchdog/version.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog
    copying build/lib/watchdog/watchmedo.py -> /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/__init__.py to __init__.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/api.py to api.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/fsevents.py to fsevents.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/fsevents2.py to fsevents2.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/inotify.py to inotify.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/inotify_buffer.py to inotify_buffer.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/inotify_c.py to inotify_c.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/kqueue.py to kqueue.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/polling.py to polling.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/read_directory_changes.py to read_directory_changes.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/observers/winapi.py to winapi.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/tricks/__init__.py to __init__.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/__init__.py to __init__.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/bricks.py to bricks.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/compat.py to compat.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/decorators.py to decorators.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/delayed_queue.py to delayed_queue.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/dirsnapshot.py to dirsnapshot.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/echo.py to echo.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/event_backport.py to event_backport.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/importlib2.py to importlib2.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/platform.py to platform.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/unicode_paths.py to unicode_paths.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/utils/win32stat.py to win32stat.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/__init__.py to __init__.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/events.py to events.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/version.py to version.cpython-34.pyc
    byte-compiling /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog/watchmedo.py to watchmedo.cpython-34.pyc
    writing byte-compilation script '/tmp/tmpyiqk3hge.py'
    /usr/bin/python3.4 -O /tmp/tmpyiqk3hge.py
    removing /tmp/tmpyiqk3hge.py
    running install_egg_info
    running egg_info
    writing dependency_links to src/watchdog.egg-info/dependency_links.txt
    writing entry points to src/watchdog.egg-info/entry_points.txt
    writing requirements to src/watchdog.egg-info/requires.txt
    writing src/watchdog.egg-info/PKG-INFO
    writing top-level names to src/watchdog.egg-info/top_level.txt
    reading manifest file 'src/watchdog.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.h' under directory 'src'
    writing manifest file 'src/watchdog.egg-info/SOURCES.txt'
    Copying src/watchdog.egg-info to /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/lib/python3.4/site-packages/watchdog-0.9.0-py3.4.egg-info
    running install_scripts
    Installing watchmedo script to /builddir/build/BUILDROOT/python-watchdog-0.9.0-1.el7.x86_64/usr/bin
    + /usr/lib/rpm/find-debuginfo.sh --strict-build-id -m --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 /builddir/build/BUILD/watchdog-0.9.0
    /usr/lib/rpm/sepdebugcrcfix: Updated 0 CRC32s, 0 CRC32s did match.
    find: 'debug': No such file or directory
    + /usr/lib/rpm/check-buildroot
    + /usr/lib/rpm/redhat/brp-compress
    + /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
    + /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
    + /usr/lib/rpm/redhat/brp-python-hardlink
    + /usr/lib/rpm/redhat/brp-java-repack-jars
    Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.ou8A87
    + umask 022
    + cd /builddir/build/BUILD
    + cd watchdog-0.9.0
    + /usr/bin/python3.4 setup.py test
    running test
    running egg_info
    writing entry points to src/watchdog.egg-info/entry_points.txt
    writing requirements to src/watchdog.egg-info/requires.txt
    writing top-level names to src/watchdog.egg-info/top_level.txt
    writing src/watchdog.egg-info/PKG-INFO
    writing dependency_links to src/watchdog.egg-info/dependency_links.txt
    reading manifest file 'src/watchdog.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.h' under directory 'src'
    writing manifest file 'src/watchdog.egg-info/SOURCES.txt'
    running build_ext
    Traceback (most recent call last):
      File "setup.py", line 167, in <module>
        zip_safe=False
      File "/usr/lib64/python3.4/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib64/python3.4/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.4/site-packages/setuptools/command/test.py", line 159, in run
        self.with_project_on_sys_path(self.run_tests)
      File "/usr/lib/python3.4/site-packages/setuptools/command/test.py", line 140, in with_project_on_sys_path
        func()
      File "setup.py", line 77, in run_tests
        errno = pytest.main(self.test_args)
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 38, in main
        config = _prepareconfig(args, plugins)
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 117, in _prepareconfig
        pluginmanager=pluginmanager, args=args)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
        return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
        return self._inner_hookexec(hook, methods, kwargs)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
        _MultiCall(methods, kwargs, hook.spec_opts).execute()
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
        return _wrapped_call(hook_impl.function(*args), self.execute)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
        wrap_controller.send(call_outcome)
      File "/usr/lib/python3.4/site-packages/_pytest/helpconfig.py", line 28, in pytest_cmdline_parse
        config = outcome.get_result()
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
        raise ex[1].with_traceback(ex[2])
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
        self.result = func()
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
        res = hook_impl.function(*args)
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 852, in pytest_cmdline_parse
        self.parse(args)
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 957, in parse
        self._preparse(args)
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 918, in _preparse
        self.pluginmanager.load_setuptools_entrypoints("pytest11")
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 504, in load_setuptools_entrypoints
        self.register(plugin, name=ep.name)
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 207, in register
        ret = super(PytestPluginManager, self).register(plugin, name)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 370, in register
        hook._maybe_apply_history(hookimpl)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 747, in _maybe_apply_history
        res = self._hookexec(self, [method], kwargs)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
        return self._inner_hookexec(hook, methods, kwargs)
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
        _MultiCall(methods, kwargs, hook.spec_opts).execute()
      File "/usr/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
        res = hook_impl.function(*args)
      File "/usr/lib/python3.4/site-packages/pytest_timeout.py", line 65, in pytest_addoption
        parser.addini('timeout_func_only', FUNC_ONLY_DESC, type='bool')
      File "/usr/lib/python3.4/site-packages/_pytest/config.py", line 510, in addini
        assert type in (None, "pathlist", "args", "linelist")
    AssertionError
    
  • Support coalesced `fsevents`

    Support coalesced `fsevents`

    Starting this as a draft PR to get some early feedback.

    The general idea is to expose an is_coalesced property on NativeEvent instances. This allows the NativeEvent -> FileSystemEvent conversion to perform additional checks on NativeEvent.path if necessary. With these additional checks the NativeEvent -> FileSystemEvent mechanism then has a chance to execute "a chain of properly sorted if-clauses instead of if-else clauses".

    Do note that in case of coalesced events it is impossible to know how many times NativeEvent.path was created, removed or renamed. It is only possible to know that such an event occurred at least once within an fseventsd update interval.

    @SamSchott - please let me know if you think the list of "coalesced event" bit masks is sufficient or not. As far as I can imagine right now then the tricky bits with coalesced events are around creation, removal, and rename of an item in an observed path.

  • 'pip install watchdog' fails on macOS 10.15.2 Catalina

    'pip install watchdog' fails on macOS 10.15.2 Catalina

    pip install watchdog fails after I upgraded to macOS 10.15.2 with this error:

        Complete output (202 lines):
        running install
        running build
        running build_py
        creating build
        creating build/lib.macosx-10.14-x86_64-3.5
        creating build/lib.macosx-10.14-x86_64-3.5/watchdog
        copying src/watchdog/watchmedo.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog
        copying src/watchdog/version.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog
        copying src/watchdog/events.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog
        copying src/watchdog/__init__.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog
        creating build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/unicode_paths.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/compat.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/win32stat.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/__init__.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/dirsnapshot.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/delayed_queue.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/platform.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/bricks.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        copying src/watchdog/utils/echo.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/utils
        creating build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/fsevents.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/inotify.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/__init__.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/api.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/inotify_buffer.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/winapi.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/read_directory_changes.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/kqueue.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/inotify_c.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/polling.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        copying src/watchdog/observers/fsevents2.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/observers
        creating build/lib.macosx-10.14-x86_64-3.5/watchdog/tricks
        copying src/watchdog/tricks/__init__.py -> build/lib.macosx-10.14-x86_64-3.5/watchdog/tricks
        running egg_info
        writing top-level names to src/watchdog.egg-info/top_level.txt
        writing requirements to src/watchdog.egg-info/requires.txt
        writing src/watchdog.egg-info/PKG-INFO
        writing entry points to src/watchdog.egg-info/entry_points.txt
        writing dependency_links to src/watchdog.egg-info/dependency_links.txt
        reading manifest file 'src/watchdog.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        warning: no files found matching '*.h' under directory 'src'
        writing manifest file 'src/watchdog.egg-info/SOURCES.txt'
        running build_ext
        building '_watchdog_fsevents' extension
        creating build/temp.macosx-10.14-x86_64-3.5
        creating build/temp.macosx-10.14-x86_64-3.5/src
        clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include -DWATCHDOG_VERSION_STRING="0.10.0" -DWATCHDOG_VERSION_MAJOR=0 -DWATCHDOG_VERSION_MINOR=10 -DWATCHDOG_VERSION_BUILD=0 -I/Users/ericvanegmond/.pyenv/versions/3.5.3/include/python3.5m -c src/watchdog_fsevents.c -o build/temp.macosx-10.14-x86_64-3.5/src/watchdog_fsevents.o -std=c99 -pedantic -Wall -Werror -Wextra -fPIC -Wno-error=unused-command-line-argument
        In file included from src/watchdog_fsevents.c:22:
        In file included from /Users/ericvanegmond/.pyenv/versions/3.5.3/include/python3.5m/Python.h:25:
        In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/stdio.h:64:
        /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/_stdio.h:137:9: error: type nullability specifier '_Nullable' is a Clang extension [-Werror,-Wnullability-extension]
                int     (* _Nullable _close)(void *);
                           ^
        /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/_stdio.h:93:16: error: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Werror,-Wnullability-completeness]
                unsigned char   *_base;
                                ^
        /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/_stdio.h:93:16: note: insert '_Nullable' if the pointer may be null
                unsigned char   *_base;
                                ^
                                  _Nullable
    .
    .
    .
        /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/stdio.h:391:44: note: insert '_Nullable' if the pointer may be null
                         fpos_t (* _Nullable)(void *, fpos_t, int),
                                                   ^
                                                    _Nullable
        /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/stdio.h:391:44: note: insert '_Nonnull' if the pointer should never be null
                         fpos_t (* _Nullable)(void *, fpos_t, int),
                                                   ^
                                                    _Nonnull
        /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/stdio.h:392:25: error: type nullability specifier '_Nullable' is a Clang extension [-Werror,-Wnullability-extension]
                         int (* _Nullable)(void *));
                                ^
        fatal error: too many errors emitted, stopping now [-ferror-limit=]
        20 errors generated.
        error: command 'clang' failed with exit status 1
        ----------------------------------------
    

    Any suggestions on how to resolve this? So far I've tried removing and reinstalling both xcode and command line tools which hasn't helped. Not sure what else to try here.

  • Release cycle

    Release cycle

    @danilobellini what do you think to publish releases more often? For instance there are several bug fixes in the current unreleased version. Perhaps could you publish a 0.9.1 with those changes?

  • Modifed event triggered twice

    Modifed event triggered twice

    if I copy and paste a file on OSX lion it triggered created event once and modified event twice should it be just trigger once modified event?

    INFO:root:Created file: /Users/ouyangjichao/BoxSync/rename/testwatchdog.py INFO:root:Modified file: /Users/ouyangjichao/BoxSync/rename/testwatchdog.py INFO:root:Modified file: /Users/ouyangjichao/BoxSync/rename/testwatchdog.py

  • second install of watchdog on non-macos incorrectly requires darwin dependencies

    second install of watchdog on non-macos incorrectly requires darwin dependencies

    If I run pip install watchdog on a clean python environment it works fine and doesn't install the pyobjc dependencies, although pip does output what look like warnings or errors:

    watchdog 0.10.0 requires pyobjc-framework-Cocoa>=4.2.2, which is not installed.
    watchdog 0.10.0 requires pyobjc-framework-FSEvents>=4.2.2, which is not installed.
    

    If I run pip install watchdog again pip does see it as already installed but then tries to install the pyobjc dependencies and fails. Here's the full command output:

    (_venv) [email protected]:~/py3$ python --version
    Python 3.6.9
    (_venv) [email protected]:~/py3$ pip install watchdog
    Collecting watchdog
      Using cached https://files.pythonhosted.org/packages/01/01/cc9b2fd111e19687dd42155f066718bccb414c8a728eb7df0b229742caf7/watchdog-0.10.0.tar.gz
      Ignoring pyobjc-framework-Cocoa: markers 'sys_platform == "darwin"' don't match your environment
      Ignoring pyobjc-framework-FSEvents: markers 'sys_platform == "darwin"' don't match your environment
    Collecting pathtools>=0.1.1 (from watchdog)
      Using cached https://files.pythonhosted.org/packages/e7/7f/470d6fcdf23f9f3518f6b0b76be9df16dcc8630ad409947f8be2eb0ed13a/pathtools-0.1.2.tar.gz
    Building wheels for collected packages: watchdog, pathtools
      Running setup.py bdist_wheel for watchdog ... done
      Stored in directory: /home/vagrant/.cache/pip/wheels/d4/74/90/4e0d4d5050776ce9a03e8a2c873e523b0149bf6412546dfc3f
      Running setup.py bdist_wheel for pathtools ... done
      Stored in directory: /home/vagrant/.cache/pip/wheels/0b/04/79/c3b0c3a0266a3cb4376da31e5bfe8bba0c489246968a68e843
    Successfully built watchdog pathtools
    watchdog 0.10.0 requires pyobjc-framework-Cocoa>=4.2.2, which is not installed.
    watchdog 0.10.0 requires pyobjc-framework-FSEvents>=4.2.2, which is not installed.
    Installing collected packages: pathtools, watchdog
    Successfully installed pathtools-0.1.2 watchdog-0.10.0
    You are using pip version 18.0, however version 20.0.2 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    (_venv) [email protected]:~/py3$ pip install watchdog
    Requirement already satisfied: watchdog in ./_venv/lib/python3.6/site-packages (0.10.0)
    Requirement already satisfied: pathtools>=0.1.1 in ./_venv/lib/python3.6/site-packages (from watchdog) (0.1.2)
    Collecting pyobjc-framework-Cocoa>=4.2.2 (from watchdog)
      Downloading https://files.pythonhosted.org/packages/7a/99/bfac679ee346f344bca72ee315249e56f4f34bc974345f66d7810380ce7a/pyobjc-framework-Cocoa-6.1.tar.gz (3.9MB)
        100% |████████████████████████████████| 3.9MB 4.1MB/s
        Complete output from command python setup.py egg_info:
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pip-install-vnyl8c9t/pyobjc-framework-Cocoa/setup.py", line 25, in <module>
            extra_link_args=["-framework", "CoreFoundation"],
          File "/tmp/pip-install-vnyl8c9t/pyobjc-framework-Cocoa/pyobjc_setup.py", line 408, in Extension
            os_level = get_os_level()
          File "/tmp/pip-install-vnyl8c9t/pyobjc-framework-Cocoa/pyobjc_setup.py", line 218, in get_os_level
            pl = plistlib.readPlist("/System/Library/CoreServices/SystemVersion.plist")
          File "/usr/lib/python3.6/plistlib.py", line 162, in readPlist
            with _maybe_open(pathOrFile, 'rb') as fp:
          File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
            return next(self.gen)
          File "/usr/lib/python3.6/plistlib.py", line 120, in _maybe_open
            with open(pathOrFile, mode) as fp:
        FileNotFoundError: [Errno 2] No such file or directory: '/System/Library/CoreServices/SystemVersion.plist'
    
        ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-vnyl8c9t/pyobjc-framework-Cocoa/
    You are using pip version 18.0, however version 20.0.2 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    

    I have tested this with python 2.7 and 3.6 and the behavior is the same.

    Additionally, if you try installing while the wheel version is cached, you also get the same error, even if watchdog is not installed:

    (_venv) [email protected]:~/py3$ pip install watchdog
    Processing /home/vagrant/.cache/pip/wheels/d4/74/90/4e0d4d5050776ce9a03e8a2c873e523b0149bf6412546dfc3f/watchdog-0.10.0-cp36-none-any.whl
    Collecting pyobjc-framework-FSEvents>=4.2.2
      Downloading pyobjc-framework-FSEvents-6.1.tar.gz (24 kB)
        ERROR: Command errored out with exit status 1:
         command: /home/vagrant/py3/_venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/setup.py'"'"'; __file__='"'"'/tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/pip-egg-info
             cwd: /tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/
        Complete output (15 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/setup.py", line 20, in <module>
            ext_modules=[Extension("FSEvents._callbacks", ["Modules/_callbacks.m"])],
          File "/tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/pyobjc_setup.py", line 408, in Extension
            os_level = get_os_level()
          File "/tmp/pip-install-2slm3k7e/pyobjc-framework-FSEvents/pyobjc_setup.py", line 218, in get_os_level
            pl = plistlib.readPlist("/System/Library/CoreServices/SystemVersion.plist")
          File "/usr/lib/python3.6/plistlib.py", line 162, in readPlist
            with _maybe_open(pathOrFile, 'rb') as fp:
          File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
            return next(self.gen)
          File "/usr/lib/python3.6/plistlib.py", line 120, in _maybe_open
            with open(pathOrFile, mode) as fp:
        FileNotFoundError: [Errno 2] No such file or directory: '/System/Library/CoreServices/SystemVersion.plist'
        ----------------------------------------
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    

    Upgrading to the latest pip (20.0.2) exhibits the same error.

  • watchmedo shell-command: drop events during execution of command

    watchmedo shell-command: drop events during execution of command

    (original title): watchmedo shell-command --wait seems to fire more often than necessary

    Version: watchdog 0.6.0

    When I execute the watchmedo script with "--wait --command=..." seems to be triggered to often. The command seems to be executed for each detected event according to the generated command-output. This seems to unnecessary and inefficient. Normally, I would expect the following behavior

    # -- PSEUDO-CODE:
    loop forever:
        Wait until event occurs.
        Remove all pending events (from the scheduler queue).
        Execute the command (once for all removed/pending events)
    

    REASON: When the command is executed it should rebuild whatever and be responsible for all accumulated events.

    The efficient behaviour could also be enabled by a new, additional command-line option.

    NOTE: If you do not use the "--wait" option you can still use the nice debug/diag feature:

    watchmedo shell-command ... --command='echo "${watch_src_path}"'
    

    to detect which filesystem events occurred.

  • Fixes missing notification for deletion of watched dir on macOS

    Fixes missing notification for deletion of watched dir on macOS

    This PR fixes an issue which would prevent us from being notified when the watched directory is deleted. This is achieved by passing the kFSEventStreamCreateFlagWatchRoot to the watch.

  • [WIP] FSEventsEmitter order events by mtime

    [WIP] FSEventsEmitter order events by mtime

    This is an updated version, similar to the somewhat outdated PR #311. This essentially orders emitted events by mtime and prevents confusion when FileDeletedEvents and FileCreatedEvents appear out of order. This would fix for instance #538.

    This issue only appears when using DirectorySnapshot to poll for events, as FSEventsEmitter does, and therefore does not seem to affect most observers.

    Let me know if this is the right place to fix this, or if a modification of DirectorySnapshot is more appropriate.

  • Installation of pyobjc-Core makes watchdog unusable on non MacOS hardware

    Installation of pyobjc-Core makes watchdog unusable on non MacOS hardware

    Collecting pyobjc-core==6.1
    
      Downloading https://artifactory.us-east-1.bamgrid.net/api/pypi/dataengineering-pypi/packages/packages/3a/80/0cdb2130b86a984e4765beb93135aa9c2120fd64be1a27e1c7cb31c719bc/pyobjc-core-6.1.tar.gz (791 kB)
    
        ERROR: Command errored out with exit status 1:
    
         command: /databricks/python3/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-09k75p2e/pyobjc-core/setup.py'"'"'; __file__='"'"'/tmp/pip-install-09k75p2e/pyobjc-core/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --no-user-cfg egg_info --egg-base /tmp/pip-install-09k75p2e/pyobjc-core/pip-egg-info
    
             cwd: /tmp/pip-install-09k75p2e/pyobjc-core/
    
        Complete output (2 lines):
    
        running egg_info
    
        error: PyObjC requires macOS to build
    
        ----------------------------------------
    
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    
    script returned exit code 1
    
    
  • SEGFAULT on macOS with the latest version of watchdog

    SEGFAULT on macOS with the latest version of watchdog

    Unfortunately, this will be tough to describe, since I don't have a mac and only my CI is complaining.

    I'm using watchdog as a submodule and today tried to update it. One specific test consistently makes pytest segfault.

    This is the log from a SEGFAULTING run: https://dev.azure.com/borisstaletic/3ce92110-caa5-4c49-b8c3-44a433da676b/_apis/build/builds/1535/logs/64 The way I'm using watchdog is by talking to, in this case, jdt.ls. When I open this file it sends... these patterns:

    [
      {
        "globPattern": "**/*.java"
      },
      {
        "globPattern": "**/.project"
      },
      {
        "globPattern": "**/.classpath"
      },
      {
        "globPattern": "**/.settings/*.prefs"
      },
      {
        "globPattern": "**/src/**"
      },
      {
        "globPattern": "**/*.gradle"
      },
      {
        "globPattern": "**/gradle.properties"
      },
      {
        "globPattern": "**/pom.xml"
      },
      {
        "globPattern": "/home/bstaletic/work/ycmd/ycmd/tests/java/testdata/simple_eclipse_project",
        "kind": 4
      }
    ]
    

    Which we then edit, like this: https://github.com/ycm-core/ycmd/blob/master/ycmd/completers/language_server/language_server_completer.py#L590-L591

    And finally use that with this handler: https://github.com/ycm-core/ycmd/blob/master/ycmd/completers/language_server/language_server_completer.py#L3219

    Any further information, as well as running the CI, I can gladly provide.

  • LoggingEventHandler is not triggered for files copied using SCP

    LoggingEventHandler is not triggered for files copied using SCP

    Hello, I have the simple code below monitoring an NFS mounted directory (/homes/myname/subdir). The host is a CentOS7.9 system.

    If I manually create a file by using touch (e.g: touch /homes/myname/subdir/a.txt) the event registers with LoggingEventHandler and I can see the output.

    But when I copy a large file (~500MB) to the same directory I don't see any response from LoggingEventHandler. I found the same behavior with FileSystemEventHandler, PatternMatchingEventHandler. They respond when I log into the CentOS system and touch a file to create it. But when a scp copies the file (large or small) to the same directory it is not caught by watchdog.

    from watchdog.observers import Observer
    from watchdog.events import LoggingEventHandler
    import argparse
    import logging
    import time
        
    if __name__ == "__main__":
        parser = argparse.ArgumentParser(description="Push new executables from CL folders to Nexus")
        parser.add_argument("--dir",help="Directory containing CL folders",required=True)
        args = parser.parse_args()
    
        logging.basicConfig(level=logging.INFO,
                            format='%(asctime)s - %(message)s',
                            datefmt='%Y-%m-%d %H:%M:%S')
    
        event_handler = LoggingEventHandler()
        observer = Observer()
        observer.schedule(event_handler, path=args.dir, recursive=True)
        logging.info("Monitoring started")
        observer.start()
    
        try:
            while observer.is_alive():
                time.sleep(1)
        except KeyboardInterrupt:
            logging.error("Process Killed by Ctrl+C")
            observer.stop()
            observer.join()
        finally:
            observer.stop()
            observer.join()
    
    

    Here is the output when I manually create the file by touching it on the CentOS system.

    myneni/[email protected]:50:15> cd /homes/homer3b/myneni/prog/py/push2nexus ; /usr/bin/env /homes/homp2n/bin/python /homes/homer3b/myneni/.vscode-server/extensions/ms-python.python-2022.18.2/py/../debugpy/launcher 45501 -- /homes/homer3b/myneni/prog/py/push2nexus/push2nexus.py --dir /homes/homer3a/osqa/ca_os_binaries 
    2022-11-15 22:51:11 - Monitoring started
    2022-11-15 22:51:19 - Created file: /homes/homer3a/osqa/ca_os_binaries/209900/osbla.tmp
    2022-11-15 22:51:19 - Modified directory: /homes/homer3a/osqa/ca_os_binaries/209900
    2022-11-15 22:51:19 - Modified file: /homes/homer3a/osqa/ca_os_binaries/209900/osbla.tmp
    2022-11-15 22:51:19 - Modified directory: /homes/homer3a/osqa/ca_os_binaries/209900
    2022-11-15 22:51:25 - Deleted file: /homes/homer3a/osqa/ca_os_binaries/209900/osbla.tmp
    2022-11-15 22:51:25 - Modified directory: /homes/homer3a/osqa/ca_os_binaries/209900
    

    I don't see this kind of behavior when the file is created by scp (same code as above).

  • Custom tricks cannot inspect their target directories

    Custom tricks cannot inspect their target directories

    It would be very helpful if pathname was passed to the TrickClass constructor by watchmedo: https://github.com/gorakhargosh/watchdog/blob/21e9f4c6466b884f87ab39d704101fb4b8b9f3c6/src/watchdog/watchmedo.py#L206

    Without this, a Trick subclass cannot do directory inspection prior to the .dispatch(...) loop kicking off, eg, to set its internal state on restart or similar.

  • watchmedo auto-restart behavior change with 2.1.9

    watchmedo auto-restart behavior change with 2.1.9

    We were using watchmedo auto-restart to call mypy and some flake8 tooling upon file changes. With the changes introduced by https://github.com/gorakhargosh/watchdog/pull/896, these processes re-run continually every time they exit whereas before they were re-ran upon a watched file/directory change.

    The name auto-restart makes me believe that this is actually desirable behavior and perhaps we were misusing that sub-command, however there doesn't appear to be an alternative to get the behavior we once had. watchmedo shell-command only runs the command on change with no option to run it once and also upon watched file change.

    Is there a way to add a flag to auto-restart which only restarts on an exit code, or maybe a way to use shell-command to call the command initially and then repeat on a file change?

  • watchmedo argument to set shell to something other than /bin/sh

    watchmedo argument to set shell to something other than /bin/sh

    Currently the executable argument is not passed. A literal is shown in the example below but I believe this should default to /bin/sh and allow an override through an argument. Of course sh can start bash too but this seems wasteful.

    subprocess.Popen(comman, shell=True, executable='/bin/bash')

  • Multiple observes watching same path

    Multiple observes watching same path

    According to this Issue, it should be fixed. However, I'm getting an error while trying to listen to the same folder from two different observers:

    observer = Observer()
    input_dir = "/tmp"
    observer.schedule(PatternMatchingEventHandler(), input_dir, recursive=True)
    observer.start()
    observer2 = Observer()
    observer2.schedule(PatternMatchingEventHandler(), input_dir, recursive=True)
    observer2.start()
    

    Triggers:

    ERROR:Unhandled exception in FSEventsEmitter
    Traceback (most recent call last):
      File "/opt/homebrew/lib/python3.10/site-packages/watchdog/observers/fsevents.py", line 315, in run
        _fsevents.add_watch(self, self.watch, self.events_callback, self.pathnames)
    RuntimeError: Cannot add watch <ObservedWatch: path=/tmp, is_recursive=True> - it is already scheduled
    

    python 3.10 macOS 12.6 M1 (arm64) watchdog-2.1.9

Python's Filesystem abstraction layer

PyFilesystem2 Python's Filesystem abstraction layer. Documentation Wiki API Documentation GitHub Repository Blog Introduction Think of PyFilesystem's

Dec 1, 2022
FUSE filesystem Python scripts for Nintendo console files
FUSE filesystem Python scripts for Nintendo console files

ninfs (formerly fuse-3ds) is a FUSE program to extract data from Nintendo game consoles. It works by presenting a virtual filesystem with the contents of your games, NAND, or SD card contents, and you can browse and copy out just the files that you need.

Nov 21, 2022
Quick and dirty FAT12 filesystem to ZIP file converter

Quick and Dirty FAT12 Filesystem Converter This is a really crappy Python script I wrote to convert a semi-compatible FAT12 filesystem from my HP150's

Feb 12, 2022
RMfuse provides access to your reMarkable Cloud files in the form of a FUSE filesystem

RMfuse provides access to your reMarkable Cloud files in the form of a FUSE filesystem. These files are exposed either in their original format, or as PDF files that contain your annotations. This lets you manage files in the reMarkable Cloud using the same tools you use on your local system.

Nov 24, 2022
Uproot is a library for reading and writing ROOT files in pure Python and NumPy.
Uproot is a library for reading and writing ROOT files in pure Python and NumPy.

Uproot is a library for reading and writing ROOT files in pure Python and NumPy. Unlike the standard C++ ROOT implementation, Uproot is only an I/O li

Nov 30, 2022
Powerful Python library for atomic file writes.

Powerful Python library for atomic file writes.

Oct 19, 2022
ValveVMF - A python library to parse Valve's VMF files

ValveVMF ValveVMF is a Python library for parsing .vmf files for the Source Engi

Jan 2, 2022
A Python library that provides basic functions to read / write Aseprite format files

A Python library that provides basic functions to read / write Aseprite format files

Jan 13, 2022
Kartothek - a Python library to manage large amounts of tabular data in a blob store

Kartothek - a Python library to manage (create, read, update, delete) large amounts of tabular data in a blob store

Nov 22, 2022
Simple, convenient and cross-platform file date changing library. 📝📅

Simple, convenient and cross-platform file date changing library.

Nov 24, 2022
shred - A cross-platform library for securely deleting files beyond recovery.
shred - A cross-platform library for securely deleting files beyond recovery.

shred Help the project financially: Donate: https://smartlegion.github.io/donate/ Yandex Money: https://yoomoney.ru/to/4100115206129186 PayPal: https:

Sep 4, 2021
An easy-to-use library for emulating code in minidump files.

dumpulator Note: This is a work-in-progress prototype, please treat it as such. An easy-to-use library for emulating code in minidump files. Example T

Nov 28, 2022
A simple library for temporary storage of small files

TemporaryStorage An simple library for temporary storage of small files. Navigation Install Usage In Python console As a standalone application List o

Apr 17, 2022
Python Fstab Generator is a small Python script to write and generate /etc/fstab files based on yaml file on Unix-like systems.

PyFstab Generator PyFstab Generator is a small Python script to write and generate /etc/fstab files based on yaml file on Unix-like systems. NOTE : Th

Nov 9, 2021
This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it
This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it

This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it. In the current state, it outputs a PrettyTable to txt file as well as the raw data from that table into a csv.

Nov 9, 2021
Better directory iterator and faster os.walk(), now in the Python 3.5 stdlib

scandir, a better directory iterator and faster os.walk() scandir() is a directory iteration function like os.listdir(), except that instead of return

Nov 27, 2022
Vericopy - This Python script provides various usage modes for secure local file copying and hashing.

Vericopy This Python script provides various usage modes for secure local file copying and hashing. Hash data is captured and logged for paths before

Nov 5, 2022
pydicom - Read, modify and write DICOM files with python code

pydicom is a pure Python package for working with DICOM files. It lets you read, modify and write DICOM data in an easy "pythonic" way.

Dec 5, 2022
fast change directory with python and ruby

fcdir fast change directory with python and ruby run run python script , chose drirectoy and change your directory need you need python and ruby deskt

Jun 20, 2022