A formatter for Python files

YAPF

PyPI version Build status Coverage status

Introduction

Most of the current formatters for Python --- e.g., autopep8, and pep8ify --- are made to remove lint errors from code. This has some obvious limitations. For instance, code that conforms to the PEP 8 guidelines may not be reformatted. But it doesn't mean that the code looks good.

YAPF takes a different approach. It's based off of 'clang-format', developed by Daniel Jasper. In essence, the algorithm takes the code and reformats it to the best formatting that conforms to the style guide, even if the original code didn't violate the style guide. The idea is also similar to the 'gofmt' tool for the Go programming language: end all holy wars about formatting - if the whole codebase of a project is simply piped through YAPF whenever modifications are made, the style remains consistent throughout the project and there's no point arguing about style in every code review.

The ultimate goal is that the code YAPF produces is as good as the code that a programmer would write if they were following the style guide. It takes away some of the drudgery of maintaining your code.

Installation

To install YAPF from PyPI:

$ pip install yapf

(optional) If you are using Python 2.7 and want to enable multiprocessing:

$ pip install futures

YAPF is still considered in "alpha" stage, and the released version may change often; therefore, the best way to keep up-to-date with the latest development is to clone this repository.

Note that if you intend to use YAPF as a command-line tool rather than as a library, installation is not necessary. YAPF supports being run as a directory by the Python interpreter. If you cloned/unzipped YAPF into DIR, it's possible to run:

$ PYTHONPATH=DIR python DIR/yapf [options] ...

Python versions

YAPF supports Python 2.7 and 3.6.4+. (Note that some Python 3 features may fail to parse with Python versions before 3.6.4.)

YAPF requires the code it formats to be valid Python for the version YAPF itself runs under. Therefore, if you format Python 3 code with YAPF, run YAPF itself under Python 3 (and similarly for Python 2).

Usage

Options:

usage: yapf [-h] [-v] [-d | -i] [-r | -l START-END] [-e PATTERN]
            [--style STYLE] [--style-help] [--no-local-style] [-p]
            [-vv]
            [files [files ...]]

Formatter for Python code.

positional arguments:
  files

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show version number and exit
  -d, --diff            print the diff for the fixed source
  -i, --in-place        make changes to files in place
  -r, --recursive       run recursively over directories
  -l START-END, --lines START-END
                        range of lines to reformat, one-based
  -e PATTERN, --exclude PATTERN
                        patterns for files to exclude from formatting
  --style STYLE         specify formatting style: either a style name (for
                        example "pep8" or "google"), or the name of a file
                        with style settings. The default is pep8 unless a
                        .style.yapf or setup.cfg file located in the same
                        directory as the source or one of its parent
                        directories (for stdin, the current directory is
                        used).
  --style-help          show style settings and exit; this output can be saved
                        to .style.yapf to make your settings permanent
  --no-local-style      don't search for local style definition
  -p, --parallel        Run yapf in parallel when formatting multiple files.
                        Requires concurrent.futures in Python 2.X
  -vv, --verbose        Print out file names while processing

Return Codes

Normally YAPF returns zero on successful program termination and non-zero otherwise.

If --diff is supplied, YAPF returns zero when no changes were necessary, non-zero otherwise (including program error). You can use this in a CI workflow to test that code has been YAPF-formatted.

Excluding files from formatting (.yapfignore)

In addition to exclude patterns provided on commandline, YAPF looks for additional patterns specified in a file named .yapfignore located in the working directory from which YAPF is invoked.

.yapfignore's syntax is similar to UNIX's filename pattern matching:

*       matches everything
?       matches any single character
[seq]   matches any character in seq
[!seq]  matches any character not in seq

Note that no entry should begin with ./.

Formatting style

The formatting style used by YAPF is configurable and there are many "knobs" that can be used to tune how YAPF does formatting. See the style.py module for the full list.

To control the style, run YAPF with the --style argument. It accepts one of the predefined styles (e.g., pep8 or google), a path to a configuration file that specifies the desired style, or a dictionary of key/value pairs.

The config file is a simple listing of (case-insensitive) key = value pairs with a [yapf] heading. For example:

[yapf]
based_on_style = pep8
spaces_before_comment = 4
split_before_logical_operator = true

The based_on_style setting determines which of the predefined styles this custom style is based on (think of it like subclassing). Four styles are predefined: pep8 (default), google, yapf, and facebook (see _STYLE_NAME_TO_FACTORY in style.py).

It's also possible to do the same on the command line with a dictionary. For example:

--style='{based_on_style: pep8, indent_width: 2}'

This will take the pep8 base style and modify it to have two space indentations.

YAPF will search for the formatting style in the following manner:

  1. Specified on the command line
  2. In the [style] section of a .style.yapf file in either the current directory or one of its parent directories.
  3. In the [yapf] section of a setup.cfg file in either the current directory or one of its parent directories.
  4. In the [style] section of a ~/.config/yapf/style file in your home directory.

If none of those files are found, the default style is used (PEP8).

Example

An example of the type of formatting that YAPF can do, it will take this ugly code:

x = {  'a':37,'b':42,

'c':927}

y = 'hello ''world'
z = 'hello '+'world'
a = 'hello {}'.format('world')
class foo  (     object  ):
  def f    (self   ):
    return       37*-+2
  def g(self, x,y=42):
      return y
def f  (   a ) :
  return      37+-+a[42-x :  y**3]

and reformat it into:

x = {'a': 37, 'b': 42, 'c': 927}

y = 'hello ' 'world'
z = 'hello ' + 'world'
a = 'hello {}'.format('world')


class foo(object):
    def f(self):
        return 37 * -+2

    def g(self, x, y=42):
        return y


def f(a):
    return 37 + -+a[42 - x:y**3]

Example as a module

The two main APIs for calling yapf are FormatCode and FormatFile, these share several arguments which are described below:

>>> from yapf.yapflib.yapf_api import FormatCode  # reformat a string of code

>>> FormatCode("f ( a = 1, b = 2 )")
'f(a=1, b=2)\n'

A style_config argument: Either a style name or a path to a file that contains formatting style settings. If None is specified, use the default style as set in style.DEFAULT_STYLE_FACTORY.

>>> FormatCode("def g():\n  return True", style_config='pep8')
'def g():\n    return True\n'

A lines argument: A list of tuples of lines (ints), [start, end], that we want to format. The lines are 1-based indexed. It can be used by third-party code (e.g., IDEs) when reformatting a snippet of code rather than a whole file.

>>> FormatCode("def g( ):\n    a=1\n    b = 2\n    return a==b", lines=[(1, 1), (2, 3)])
'def g():\n    a = 1\n    b = 2\n    return a==b\n'

A print_diff (bool): Instead of returning the reformatted source, return a diff that turns the formatted source into reformatter source.

>>> print(FormatCode("a==b", filename="foo.py", print_diff=True))
--- foo.py (original)
+++ foo.py (reformatted)
@@ -1 +1 @@
-a==b
+a == b

Note: the filename argument for FormatCode is what is inserted into the diff, the default is <unknown>.

FormatFile returns reformatted code from the passed file along with its encoding:

>>> from yapf.yapflib.yapf_api import FormatFile  # reformat a file

>>> print(open("foo.py").read())  # contents of file
a==b

>>> FormatFile("foo.py")
('a == b\n', 'utf-8')

The in_place argument saves the reformatted code back to the file:

>>> FormatFile("foo.py", in_place=True)
(None, 'utf-8')

>>> print(open("foo.py").read())  # contents of file (now fixed)
a == b

Formatting diffs

Options:

usage: yapf-diff [-h] [-i] [-p NUM] [--regex PATTERN] [--iregex PATTERN][-v]
                 [--style STYLE] [--binary BINARY]

This script reads input from a unified diff and reformats all the changed
lines. This is useful to reformat all the lines touched by a specific patch.
Example usage for git/svn users:

  git diff -U0 --no-color --relative HEAD^ | yapf-diff -i
  svn diff --diff-cmd=diff -x-U0 | yapf-diff -p0 -i

It should be noted that the filename contained in the diff is used
unmodified to determine the source file to update. Users calling this script
directly should be careful to ensure that the path in the diff is correct
relative to the current working directory.

optional arguments:
  -h, --help            show this help message and exit
  -i, --in-place        apply edits to files instead of displaying a diff
  -p NUM, --prefix NUM  strip the smallest prefix containing P slashes
  --regex PATTERN       custom pattern selecting file paths to reformat
                        (case sensitive, overrides -iregex)
  --iregex PATTERN      custom pattern selecting file paths to reformat
                        (case insensitive, overridden by -regex)
  -v, --verbose         be more verbose, ineffective without -i
  --style STYLE         specify formatting style: either a style name (for
                        example "pep8" or "google"), or the name of a file
                        with style settings. The default is pep8 unless a
                        .style.yapf or setup.cfg file located in one of the
                        parent directories of the source file (or current
                        directory for stdin)
  --binary BINARY       location of binary to use for yapf

Knobs

ALIGN_CLOSING_BRACKET_WITH_VISUAL_INDENT
Align closing bracket with visual indentation.
ALLOW_MULTILINE_LAMBDAS
Allow lambdas to be formatted on more than one line.
ALLOW_MULTILINE_DICTIONARY_KEYS

Allow dictionary keys to exist on multiple lines. For example:

x = {
    ('this is the first element of a tuple',
     'this is the second element of a tuple'):
         value,
}
ALLOW_SPLIT_BEFORE_DEFAULT_OR_NAMED_ASSIGNS
Allow splitting before a default / named assignment in an argument list.
ALLOW_SPLIT_BEFORE_DICT_VALUE
Allow splits before the dictionary value.
ARITHMETIC_PRECEDENCE_INDICATION

Let spacing indicate operator precedence. For example:

a = 1 * 2 + 3 / 4
b = 1 / 2 - 3 * 4
c = (1 + 2) * (3 - 4)
d = (1 - 2) / (3 + 4)
e = 1 * 2 - 3
f = 1 + 2 + 3 + 4

will be formatted as follows to indicate precedence:

a = 1*2 + 3/4
b = 1/2 - 3*4
c = (1+2) * (3-4)
d = (1-2) / (3+4)
e = 1*2 - 3
f = 1 + 2 + 3 + 4
BLANK_LINE_BEFORE_NESTED_CLASS_OR_DEF

Insert a blank line before a def or class immediately nested within another def or class. For example:

class Foo:
                   # <------ this blank line
    def method():
        pass
BLANK_LINE_BEFORE_MODULE_DOCSTRING
Insert a blank line before a module docstring.
BLANK_LINE_BEFORE_CLASS_DOCSTRING
Insert a blank line before a class-level docstring.
BLANK_LINES_AROUND_TOP_LEVEL_DEFINITION

Sets the number of desired blank lines surrounding top-level function and class definitions. For example:

class Foo:
    pass
                   # <------ having two blank lines here
                   # <------ is the default setting
class Bar:
    pass
BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES
Sets the number of desired blank lines between top-level imports and variable definitions. Useful for compatibility with tools like isort.
COALESCE_BRACKETS

Do not split consecutive brackets. Only relevant when DEDENT_CLOSING_BRACKETS or INDENT_CLOSING_BRACKETS is set. For example:

call_func_that_takes_a_dict(
    {
        'key1': 'value1',
        'key2': 'value2',
    }
)

would reformat to:

call_func_that_takes_a_dict({
    'key1': 'value1',
    'key2': 'value2',
})
COLUMN_LIMIT
The column limit (or max line-length)
CONTINUATION_ALIGN_STYLE

The style for continuation alignment. Possible values are:

  • SPACE: Use spaces for continuation alignment. This is default behavior.
  • FIXED: Use fixed number (CONTINUATION_INDENT_WIDTH) of columns (ie: CONTINUATION_INDENT_WIDTH/INDENT_WIDTH tabs or CONTINUATION_INDENT_WIDTH spaces) for continuation alignment.
  • VALIGN-RIGHT: Vertically align continuation lines to multiple of INDENT_WIDTH columns. Slightly right (one tab or a few spaces) if cannot vertically align continuation lines with indent characters.
CONTINUATION_INDENT_WIDTH
Indent width used for line continuations.
DEDENT_CLOSING_BRACKETS

Put closing brackets on a separate line, dedented, if the bracketed expression can't fit in a single line. Applies to all kinds of brackets, including function definitions and calls. For example:

config = {
    'key1': 'value1',
    'key2': 'value2',
}  # <--- this bracket is dedented and on a separate line

time_series = self.remote_client.query_entity_counters(
    entity='dev3246.region1',
    key='dns.query_latency_tcp',
    transform=Transformation.AVERAGE(window=timedelta(seconds=60)),
    start_ts=now()-timedelta(days=3),
    end_ts=now(),
)  # <--- this bracket is dedented and on a separate line
DISABLE_ENDING_COMMA_HEURISTIC
Disable the heuristic which places each list element on a separate line if the list is comma-terminated.
EACH_DICT_ENTRY_ON_SEPARATE_LINE
Place each dictionary entry onto its own line.
FORCE_MULTILINE_DICT
Respect EACH_DICT_ENTRY_ON_SEPARATE_LINE even if the line is shorter than COLUMN_LIMIT.
I18N_COMMENT
The regex for an internationalization comment. The presence of this comment stops reformatting of that line, because the comments are required to be next to the string they translate.
I18N_FUNCTION_CALL
The internationalization function call names. The presence of this function stops reformatting on that line, because the string it has cannot be moved away from the i18n comment.
INDENT_DICTIONARY_VALUE

Indent the dictionary value if it cannot fit on the same line as the dictionary key. For example:

config = {
    'key1':
        'value1',
    'key2': value1 +
            value2,
}
INDENT_WIDTH
The number of columns to use for indentation.
INDENT_BLANK_LINES
Set to True to prefer indented blank lines rather than empty
INDENT_CLOSING_BRACKETS

Put closing brackets on a separate line, indented, if the bracketed expression can't fit in a single line. Applies to all kinds of brackets, including function definitions and calls. For example:

config = {
    'key1': 'value1',
    'key2': 'value2',
    }  # <--- this bracket is indented and on a separate line

time_series = self.remote_client.query_entity_counters(
    entity='dev3246.region1',
    key='dns.query_latency_tcp',
    transform=Transformation.AVERAGE(window=timedelta(seconds=60)),
    start_ts=now()-timedelta(days=3),
    end_ts=now(),
    )  # <--- this bracket is indented and on a separate line
JOIN_MULTIPLE_LINES
Join short lines into one line. E.g., single line if statements.
NO_SPACES_AROUND_SELECTED_BINARY_OPERATORS

Do not include spaces around selected binary operators. For example:

1 + 2 * 3 - 4 / 5

will be formatted as follows when configured with *, /:

1 + 2*3 - 4/5
SPACES_AROUND_POWER_OPERATOR
Set to True to prefer using spaces around **.
SPACES_AROUND_DEFAULT_OR_NAMED_ASSIGN
Set to True to prefer spaces around the assignment operator for default or keyword arguments.
SPACES_AROUND_DICT_DELIMITERS

Adds a space after the opening '{' and before the ending '}' dict delimiters.

{1: 2}

will be formatted as:

{ 1: 2 }
SPACES_AROUND_LIST_DELIMITERS

Adds a space after the opening '[' and before the ending ']' list delimiters.

[1, 2]

will be formatted as:

[ 1, 2 ]
SPACES_AROUND_SUBSCRIPT_COLON

Use spaces around the subscript / slice operator. For example:

my_list[1 : 10 : 2]
SPACES_AROUND_TUPLE_DELIMITERS

Adds a space after the opening '(' and before the ending ')' tuple delimiters.

(1, 2, 3)

will be formatted as:

( 1, 2, 3 )
SPACES_BEFORE_COMMENT

The number of spaces required before a trailing comment. This can be a single value (representing the number of spaces before each trailing comment) or list of of values (representing alignment column values; trailing comments within a block will be aligned to the first column value that is greater than the maximum line length within the block). For example:

With spaces_before_comment=5:

1 + 1 # Adding values

will be formatted as:

1 + 1     # Adding values <-- 5 spaces between the end of the statement and comment

With spaces_before_comment=15, 20:

1 + 1 # Adding values
two + two # More adding

longer_statement # This is a longer statement
short # This is a shorter statement

a_very_long_statement_that_extends_beyond_the_final_column # Comment
short # This is a shorter statement

will be formatted as:

1 + 1          # Adding values <-- end of line comments in block aligned to col 15
two + two      # More adding

longer_statement    # This is a longer statement <-- end of line comments in block aligned to col 20
short               # This is a shorter statement

a_very_long_statement_that_extends_beyond_the_final_column  # Comment <-- the end of line comments are aligned based on the line length
short                                                       # This is a shorter statement
SPACE_BETWEEN_ENDING_COMMA_AND_CLOSING_BRACKET
Insert a space between the ending comma and closing bracket of a list, etc.
SPACE_INSIDE_BRACKETS

Use spaces inside brackets, braces, and parentheses. For example:

method_call( 1 )
my_dict[ 3 ][ 1 ][ get_index( *args, **kwargs ) ]
my_set = { 1, 2, 3 }
SPLIT_ARGUMENTS_WHEN_COMMA_TERMINATED
Split before arguments if the argument list is terminated by a comma.
SPLIT_ALL_COMMA_SEPARATED_VALUES
If a comma separated list (dict, list, tuple, or function def) is on a line that is too long, split such that all elements are on a single line.
SPLIT_ALL_TOP_LEVEL_COMMA_SEPARATED_VALUES

Variation on SPLIT_ALL_COMMA_SEPARATED_VALUES in which, if a subexpression with a comma fits in its starting line, then the subexpression is not split. This avoids splits like the one for b in this code:

abcdef(
    aReallyLongThing: int,
    b: [Int,
        Int])

With the new knob this is split as:

abcdef(
    aReallyLongThing: int,
    b: [Int, Int])
SPLIT_BEFORE_BITWISE_OPERATOR
Set to True to prefer splitting before &, | or ^ rather than after.
SPLIT_BEFORE_ARITHMETIC_OPERATOR
Set to True to prefer splitting before +, -, *, /, //, or @ rather than after.
SPLIT_BEFORE_CLOSING_BRACKET
Split before the closing bracket if a list or dict literal doesn't fit on a single line.
SPLIT_BEFORE_DICT_SET_GENERATOR

Split before a dictionary or set generator (comp_for). For example, note the split before the for:

foo = {
    variable: 'Hello world, have a nice day!'
    for variable in bar if variable != 42
}
SPLIT_BEFORE_DOT

Split before the . if we need to split a longer expression:

foo = ('This is a really long string: {}, {}, {}, {}'.format(a, b, c, d))

would reformat to something like:

foo = ('This is a really long string: {}, {}, {}, {}'
       .format(a, b, c, d))
SPLIT_BEFORE_EXPRESSION_AFTER_OPENING_PAREN
Split after the opening paren which surrounds an expression if it doesn't fit on a single line.
SPLIT_BEFORE_FIRST_ARGUMENT
If an argument / parameter list is going to be split, then split before the first argument.
SPLIT_BEFORE_LOGICAL_OPERATOR
Set to True to prefer splitting before and or or rather than after.
SPLIT_BEFORE_NAMED_ASSIGNS
Split named assignments onto individual lines.
SPLIT_COMPLEX_COMPREHENSION

For list comprehensions and generator expressions with multiple clauses (e.g multiple for calls, if filter expressions) and which need to be reflowed, split each clause onto its own line. For example:

result = [
    a_var + b_var for a_var in xrange(1000) for b_var in xrange(1000)
    if a_var % b_var]

would reformat to something like:

result = [
    a_var + b_var
    for a_var in xrange(1000)
    for b_var in xrange(1000)
    if a_var % b_var]
SPLIT_PENALTY_AFTER_OPENING_BRACKET
The penalty for splitting right after the opening bracket.
SPLIT_PENALTY_AFTER_UNARY_OPERATOR
The penalty for splitting the line after a unary operator.
SPLIT_PENALTY_ARITHMETIC_OPERATOR
The penalty of splitting the line around the +, -, *, /, //, %, and @ operators.
SPLIT_PENALTY_BEFORE_IF_EXPR
The penalty for splitting right before an if expression.
SPLIT_PENALTY_BITWISE_OPERATOR
The penalty of splitting the line around the &, |, and ^ operators.
SPLIT_PENALTY_COMPREHENSION
The penalty for splitting a list comprehension or generator expression.
SPLIT_PENALTY_EXCESS_CHARACTER
The penalty for characters over the column limit.
SPLIT_PENALTY_FOR_ADDED_LINE_SPLIT
The penalty incurred by adding a line split to the unwrapped line. The more line splits added the higher the penalty.
SPLIT_PENALTY_IMPORT_NAMES

The penalty of splitting a list of import as names. For example:

from a_very_long_or_indented_module_name_yada_yad import (long_argument_1,
                                                          long_argument_2,
                                                          long_argument_3)

would reformat to something like:

from a_very_long_or_indented_module_name_yada_yad import (
    long_argument_1, long_argument_2, long_argument_3)
SPLIT_PENALTY_LOGICAL_OPERATOR
The penalty of splitting the line around the and and or operators.
USE_TABS
Use the Tab character for indentation.

(Potentially) Frequently Asked Questions

Why does YAPF destroy my awesome formatting?

YAPF tries very hard to get the formatting correct. But for some code, it won't be as good as hand-formatting. In particular, large data literals may become horribly disfigured under YAPF.

The reasons for this are manyfold. In short, YAPF is simply a tool to help with development. It will format things to coincide with the style guide, but that may not equate with readability.

What can be done to alleviate this situation is to indicate regions YAPF should ignore when reformatting something:

# yapf: disable
FOO = {
    # ... some very large, complex data literal.
}

BAR = [
    # ... another large data literal.
]
# yapf: enable

You can also disable formatting for a single literal like this:

BAZ = {
    (1, 2, 3, 4),
    (5, 6, 7, 8),
    (9, 10, 11, 12),
}  # yapf: disable

To preserve the nice dedented closing brackets, use the dedent_closing_brackets in your style. Note that in this case all brackets, including function definitions and calls, are going to use that style. This provides consistency across the formatted codebase.

Why Not Improve Existing Tools?

We wanted to use clang-format's reformatting algorithm. It's very powerful and designed to come up with the best formatting possible. Existing tools were created with different goals in mind, and would require extensive modifications to convert to using clang-format's algorithm.

Can I Use YAPF In My Program?

Please do! YAPF was designed to be used as a library as well as a command line tool. This means that a tool or IDE plugin is free to use YAPF.

I still get non Pep8 compliant code! Why?

YAPF tries very hard to be fully PEP 8 compliant. However, it is paramount to not risk altering the semantics of your code. Thus, YAPF tries to be as safe as possible and does not change the token stream (e.g., by adding parentheses). All these cases however, can be easily fixed manually. For instance,

from my_package import my_function_1, my_function_2, my_function_3, my_function_4, my_function_5

FOO = my_variable_1 + my_variable_2 + my_variable_3 + my_variable_4 + my_variable_5 + my_variable_6 + my_variable_7 + my_variable_8

won't be split, but you can easily get it right by just adding parentheses:

from my_package import (my_function_1, my_function_2, my_function_3,
                        my_function_4, my_function_5)

FOO = (my_variable_1 + my_variable_2 + my_variable_3 + my_variable_4 +
       my_variable_5 + my_variable_6 + my_variable_7 + my_variable_8)

Gory Details

Algorithm Design

The main data structure in YAPF is the UnwrappedLine object. It holds a list of FormatTokens, that we would want to place on a single line if there were no column limit. An exception being a comment in the middle of an expression statement will force the line to be formatted on more than one line. The formatter works on one UnwrappedLine object at a time.

An UnwrappedLine typically won't affect the formatting of lines before or after it. There is a part of the algorithm that may join two or more UnwrappedLines into one line. For instance, an if-then statement with a short body can be placed on a single line:

if a == 42: continue

YAPF's formatting algorithm creates a weighted tree that acts as the solution space for the algorithm. Each node in the tree represents the result of a formatting decision --- i.e., whether to split or not to split before a token. Each formatting decision has a cost associated with it. Therefore, the cost is realized on the edge between two nodes. (In reality, the weighted tree doesn't have separate edge objects, so the cost resides on the nodes themselves.)

For example, take the following Python code snippet. For the sake of this example, assume that line (1) violates the column limit restriction and needs to be reformatted.

def xxxxxxxxxxx(aaaaaaaaaaaa, bbbbbbbbb, cccccccc, dddddddd, eeeeee):  # 1
    pass                                                               # 2

For line (1), the algorithm will build a tree where each node (a FormattingDecisionState object) is the state of the line at that token given the decision to split before the token or not. Note: the FormatDecisionState objects are copied by value so each node in the graph is unique and a change in one doesn't affect other nodes.

Heuristics are used to determine the costs of splitting or not splitting. Because a node holds the state of the tree up to a token's insertion, it can easily determine if a splitting decision will violate one of the style requirements. For instance, the heuristic is able to apply an extra penalty to the edge when not splitting between the previous token and the one being added.

There are some instances where we will never want to split the line, because doing so will always be detrimental (i.e., it will require a backslash-newline, which is very rarely desirable). For line (1), we will never want to split the first three tokens: def, xxxxxxxxxxx, and (. Nor will we want to split between the ) and the : at the end. These regions are said to be "unbreakable." This is reflected in the tree by there not being a "split" decision (left hand branch) within the unbreakable region.

Now that we have the tree, we determine what the "best" formatting is by finding the path through the tree with the lowest cost.

And that's it!


YAPF is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.
Owner
Google
Google ❀️ Open Source
Google
Comments
  • Support for 3.6 f-string syntax

    Support for 3.6 f-string syntax

    Apologies for the brief report, but here's a simple bug reproduction session in the shell:

    $ mktmpenv -p python3.6  # this command is from virtualenvwrapper
    (tmp-38ce2dcf0268df9) $ python --version
    Python 3.6.0b2
    (tmp-38ce2dcf0268df9) $ cat test.py 
    target = "world"
    print(f"Hello, {target}!")
    (tmp-38ce2dcf0268df9) $ python test.py 
    Hello, world!
    (tmp-38ce2dcf0268df9) $ pip install yapf==0.14.0
    (tmp-38ce2dcf0268df9) $ yapf -i test.py
    Traceback (most recent call last):
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 102, in ParseCodeToTree
        tree = parser_driver.parse_string(code, debug=False)
      File "/home/dpassaro/.pyenv/versions/3.6.0b2/lib/python3.6/lib2to3/pgen2/driver.py", line 106, in parse_string
        return self.parse_tokens(tokens, debug)
      File "/home/dpassaro/.pyenv/versions/3.6.0b2/lib/python3.6/lib2to3/pgen2/driver.py", line 71, in parse_tokens
        if p.addtoken(type, value, (prefix, start)):
      File "/home/dpassaro/.pyenv/versions/3.6.0b2/lib/python3.6/lib2to3/pgen2/parse.py", line 159, in addtoken
        raise ParseError("bad input", type, value, context)
    lib2to3.pgen2.parse.ParseError: bad input: type=3, value='"Hello, {target}!"', context=('', (2, 7))
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/bin/yapf", line 11, in <module>
        sys.exit(run_main())
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/__init__.py", line 296, in run_main
        sys.exit(main(sys.argv))
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/__init__.py", line 188, in main
        parallel=args.parallel)
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/__init__.py", line 236, in FormatFiles
        in_place, print_diff, verify)
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/__init__.py", line 259, in _FormatFile
        logger=logging.warning)
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 90, in FormatFile
        verify=verify)
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 126, in FormatCode
        tree = pytree_utils.ParseCodeToTree(unformatted_source)
      File "/home/dpassaro/.virtualenvs/tmp-38ce2dcf0268df9/lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 108, in ParseCodeToTree
        tree = parser_driver.parse_string(code, debug=False)
      File "/home/dpassaro/.pyenv/versions/3.6.0b2/lib/python3.6/lib2to3/pgen2/driver.py", line 106, in parse_string
        return self.parse_tokens(tokens, debug)
      File "/home/dpassaro/.pyenv/versions/3.6.0b2/lib/python3.6/lib2to3/pgen2/driver.py", line 71, in parse_tokens
        if p.addtoken(type, value, (prefix, start)):
      File "/home/dpassaro/.pyenv/versions/3.6.0b2/lib/python3.6/lib2to3/pgen2/parse.py", line 159, in addtoken
        raise ParseError("bad input", type, value, context)
    lib2to3.pgen2.parse.ParseError: bad input: type=3, value='"Hello, {target}!"', context=('', (2, 7))
    
  • fixed #228: make exit code customizable to indicate whether files were changed

    fixed #228: make exit code customizable to indicate whether files were changed

    As with others on #228, I was surprised to learn that yapf returns a non-success error code even when it completes successfully. This breaks long-standing convention (that, yes, is broken by other tools also).

    The attached patch makes the return code in the case of changed files customizable. By default, 0 is returned when files are formatted according to the effective style, regardless of whether changes occurred or not. Callers may request that another exit code be used to indicate successful execution AND that files were modified. For example, yapf -c 2 recovers the previous behavior to exit with code 2.

  • Move builtin styles into config files

    Move builtin styles into config files

    This change is quite a bit more intrusive than my previous few. Feel free to push back if you don't like it. It certainly makes sense to me this way (i didn't expect the style definitions in code when I first poked around the project). On thing I wasn't sure about was the file extension to use for styles. I went with .ini because this caused vim to choose the right syntax highlighting :)

    For my next change I was thinking of having yapf look for a project wide style file (.yapf) that would work like .clang-format does.. again not sure if that fits with your direction, but I thought it would be useful for yapf itself for example.

    Move the builtin styles into config files rather than hardcoding them. This allows them to be easily located and modified and also to act as an example to those who want to write their own styles.

    It also keeps us honest and ensures we don't encode anything into the builtin styles that cannot be parsed from the config file format.

    It also allows new builtin styles to be added by simply dropping files in the 'styles' directory.

  • Add knob to never ever break line after

    Add knob to never ever break line after "'dictkey':"

    Currently if one or more of dict's entries are longer than column_limit (or simply contain multiline string), yapf will break line after all dict's keys in dict.

    I've tried disabling indent_dictionary_value but this makes yapf just align value right under key, which is something I dislike more than breaking line and inlining value ;)

    Eventually, I've developed custom utility for my project that performs my way cleanup after yapf, but after 12h of programmering I can't help but feel that its hacky and fragile, which is why I'm opening ticket here.

  • Ignore files

    Ignore files

    It was great if one could add files to ignore to a knobs config. This will apply for recursive mode. Sometimes there are auto generated files where it makes no sense to improve the style.

  • yapf not support PEP-0492

    yapf not support PEP-0492

    PEP-0492 add async and await to keyword.

    ➜  ~  yapf --style pep8 ~/Desktop/3.5_asyncio_async.py
    import asyncio
    import time
    
    async
    
    def slow_operation(n):
        await asyncio.sleep(1)
        # print("Slow operation {} complete".format(n))
    
        async
    
    def main():
        start = time.time()
        await asyncio.wait([slow_operation(i) for i in range(10000)])
        end = time.time()
        print('Complete in {} second(s)'.format(end - start))
    
    
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    ➜  ~  yapf --style google ~/Desktop/3.5_asyncio_async.py
    import asyncio
    import time
    
    async
    
    def slow_operation(n):
        await asyncio.sleep(1)
        # print("Slow operation {} complete".format(n))
    
        async
    
    def main():
        start = time.time()
        await asyncio.wait([slow_operation(i) for i in range(10000)])
        end = time.time()
        print('Complete in {} second(s)'.format(end - start))
    
    
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    
  • Pathologically slow example

    Pathologically slow example

    The following takes 25 seconds to reformat on my machine. In comparison, many significantly larger files run in under a second:

    specialElements = frozenset((
        (namespaces['html'], 'address'),
        (namespaces['html'], 'applet'),
        (namespaces['html'], 'area'),
        (namespaces['html'], 'article'),
        (namespaces['html'], 'aside'),
        (namespaces['html'], 'base'),
        (namespaces['html'], 'basefont'),
        (namespaces['html'], 'bgsound'),
        (namespaces['html'], 'blockquote'),
        (namespaces['html'], 'body'),
        (namespaces['html'], 'br'),
        (namespaces['html'], 'button'),
        (namespaces['html'], 'caption'),
        (namespaces['html'], 'center'),
        (namespaces['html'], 'col'),
        (namespaces['html'], 'colgroup'),
        (namespaces['html'], 'command'),
        (namespaces['html'], 'dd'),
        (namespaces['html'], 'details'),
        (namespaces['html'], 'dir'),
        (namespaces['html'], 'div'),
        (namespaces['html'], 'dl'),
        (namespaces['html'], 'dt'),
        (namespaces['html'], 'embed'),
        (namespaces['html'], 'fieldset'),
        (namespaces['html'], 'figure'),
        (namespaces['html'], 'footer'),
        (namespaces['html'], 'form'),
        (namespaces['html'], 'frame'),
        (namespaces['html'], 'frameset'),
        (namespaces['html'], 'h1'),
        (namespaces['html'], 'h2'),
        (namespaces['html'], 'h3'),
        (namespaces['html'], 'h4'),
        (namespaces['html'], 'h5'),
        (namespaces['html'], 'h6'),
        (namespaces['html'], 'head'),
        (namespaces['html'], 'header'),
        (namespaces['html'], 'hr'),
        (namespaces['html'], 'html'),
        (namespaces['html'], 'iframe'),
        # Note that image is commented out in the spec as "this isn't an
        # element that can end up on the stack, so it doesn't matter,"
        (namespaces['html'], 'image'),
        (namespaces['html'], 'img'),
        (namespaces['html'], 'input'),
        (namespaces['html'], 'isindex'),
        (namespaces['html'], 'li'),
        (namespaces['html'], 'link'),
        (namespaces['html'], 'listing'),
        (namespaces['html'], 'marquee'),
        (namespaces['html'], 'menu'),
        (namespaces['html'], 'meta'),
        (namespaces['html'], 'nav'),
        (namespaces['html'], 'noembed'),
        (namespaces['html'], 'noframes'),
        (namespaces['html'], 'noscript'),
        (namespaces['html'], 'object'),
        (namespaces['html'], 'ol'),
        (namespaces['html'], 'p'),
        (namespaces['html'], 'param'),
        (namespaces['html'], 'plaintext'),
        (namespaces['html'], 'pre'),
        (namespaces['html'], 'script'),
        (namespaces['html'], 'section'),
        (namespaces['html'], 'select'),
        (namespaces['html'], 'style'),
        (namespaces['html'], 'table'),
        (namespaces['html'], 'tbody'),
        (namespaces['html'], 'td'),
        (namespaces['html'], 'textarea'),
        (namespaces['html'], 'tfoot'),
        (namespaces['html'], 'th'),
        (namespaces['html'], 'thead'),
        (namespaces['html'], 'title'),
        (namespaces['html'], 'tr'),
        (namespaces['html'], 'ul'),
        (namespaces['html'], 'wbr'),
        (namespaces['html'], 'xmp'),
        (namespaces['svg'], 'foreignObject')
    ))
    
  • Fix removing of extra lines on the edge of enabled lines boundaries.

    Fix removing of extra lines on the edge of enabled lines boundaries.

    This was broken by https://github.com/google/yapf/commit/dd9f72b9ec74267b84d5b0c0f70a12595f920166#diff-1020ef5a942231c7da662fa974a70a3bR145 which added a hacky fix that prevented removing newlines.

    I haven't fixed the test that was added in that commit because I haven't yet work out how to tell if a range is affected by a disable comment. The correct solution will be to handle that comment instead of disallowing removal of lines.

    Help welcome.

  • yapf (facebook style) loves to make a two-step reformat of over-long line

    yapf (facebook style) loves to make a two-step reformat of over-long line

    YAPF loves to make two steps if run twice in fixing this:

    # last character is column 87 with our indent
    self.assertEqual(result.reason_not_added, "current preflight is still running")
    

    becomes

    self.assertEqual(result.reason_not_added,
                     "current preflight is still running")
    

    next run it becomes

    self.assertEqual(
        result.reason_not_added, "current preflight is still running"
    )
    

    Now, I like the last option best, and it's what we want, but the fact we take consistently two steps to get there is a surprise. I'm not sure why it prefers the intermediate form once, but it happens often enough to feel "routine".

  • Added an option to split on all args even if not comma terminated.

    Added an option to split on all args even if not comma terminated.

    I noticed that there was an option to split args to individual lines, but only when comma terminated. I wanted an option to do this on all args, so I implemented something that seemed to work.

    What else is needed to get this change accepted? I reviewed the contributing guidelines and seem to be in compliance with all of them, but there were not many.

  • Add handling for unpacking with dict expressions (currently 3.6 only)

    Add handling for unpacking with dict expressions (currently 3.6 only)

    Only works on Python 3.6, since lib2to3 in Python 3.5 doesn't recognize the unpacking token.

    Prior to this patch, {**x} would raise an error like this in Python 3.5: lib2to3.pgen2.parse.ParseError: bad input: type=36, value='**', context=(' ', (1, 11)).

    This is a known issue.

    In Python 3.6, this is apparently fixed, so that we can indeed support PEP448 cases.

  • incorrect split with default settings

    incorrect split with default settings

            for j in range(i+1, samples):
                if gIds[i] == gIds[j] and gX[i,0] == gX[j,0] and gY[i,0] == gY[j,0]:
    

    to

            for j in range(i + 1, samples):
                if gIds[i] == gIds[j] and gX[i, 0] == gX[j,
                                                         0] and gY[i,
                                                                   0] == gY[j,
                                                                            0]:
    

    any suggestion for workaround?

  • Add formatting style search in a tox.ini file

    Add formatting style search in a tox.ini file

    Make yapf search search for the formatting style in the [yapf] section of a tox.ini file in either the current directory or one of its parent directories.

  • Leave single argument with trailing comma on own line

    Leave single argument with trailing comma on own line

    Right now, I don't believe there's a way to have yapf put a single argument to a function on its own line when there's a trailing comma. For example,

    bar = foo(
    	"only item",
    )
    

    gets formatted as

    bar = foo("only item",)
    

    whereas I prefer it to remain as-is, given the presence of a trailing comma.

    On the other hand, lines such as

    bar = foo(
    	"first item",
    	"second item",
    )
    

    are left alone, as desired.

    Have I missed a config setting for this behaviour, or is this feature not yet implemented?

  • Bad DEDENT_CLOSING_BRACKETS

    Bad DEDENT_CLOSING_BRACKETS

    I have code:

        return Response(
            content=json.dumps([contact.to_dict() for contact in contacts],
                               ensure_ascii=False),
            headers={
                "Access-Control-Allow-Origin":
                    "http://localhost:3000",
                "Content-Range":
                    f"{pagination[0]}-{pagination[1]}/{total_contacts_count}"
                    if pagination else ""
            }
        )
    

    But by idea it should be like this:

        return Response(
            content=json.dumps(
                [contact.to_dict() for contact in contacts],
                ensure_ascii=False
            ),
            headers={
                "Access-Control-Allow-Origin":
                    "http://localhost:3000",
                "Content-Range":
                    f"{pagination[0]}-{pagination[1]}/{total_contacts_count}"
                    if pagination else ""
            }
        )
    

    my .style.yapf:

    [style]
    based_on_style = pep8
    spaces_before_comment = 2
    indent_width = 4
    dedent_closing_brackets = true
    coalesce_brackets = true
    column_limit = 80
    split_complex_comprehension = true
    split_penalty_import_names = 0
    split_penalty_after_opening_bracket = 4
    align_closing_bracket_with_visual_indent = false
    blank_line_before_nested_class_or_def = false
    indent_dictionary_value = true
    
  • unable to format code with yapf

    unable to format code with yapf

    I'm unable to format code with yapf. My setup is using poetry and I use .style.yapf file to handle config for yapf. With VS Code format on save functionality, it is not formatting code. Using CLI, it keeps giving me yapf: toml package is needed for using pyproject.toml as a configuration file if I use .style.yapf and Invalid TOML file /home/wrench1815/Documents/git/vis-dash/vd-back/pyproject.toml: Unexpected character: 'p' at line 16 col 17 if I use pyproject.toml to handle the config. Also, I tried with yapf and yapf[pyproject]. With yapf[pyproject], the extra does not install TOML although, as in #964, it is supposed to install it.

    Yes, I'm using yapf from my env, I tried using with poetry run yapf and yapf both. But the errors persist.

    If only TOML is manually installed, yapf works unless it won't.

  • [usage question] Can I configure yapf to indent after trailing comma only when in a list, indent the items and put the list closing/opening brackets on there own lines (allman style)?

    [usage question] Can I configure yapf to indent after trailing comma only when in a list, indent the items and put the list closing/opening brackets on there own lines (allman style)?

    Would like it to enforce that function parameters stay on a single line (perhaps up to a line threshold), but that lists are displayed allmancase style with each element on its own line

    Desired output format:

    	page.add(
    	    flet.Container(expand=True, bgcolor="#252526", border_radius=5.5, margin=-7, padding=3, content=flet.Stack(controls=
    	    [
    	        flet.Column(controls=
    	        [
    	            windowDragArea,
    	            result,
    	            flet.Stack(ref=dope,animate_scale=80,controls=
    	            [
    	                flet.Image(src="obi-wan pfp.jpg", width=100),
    	                flet.Image(src="lisa/icons/blobs/blobSharp_Inverse.png", width=100, color="#252526",),
    	                flet.Container(width=100, height=100, on_hover=scale),
    	            ]),
    	            flexGrid,
    	        ])
    	    ])),
    	)
    

    Any thoughts appreciated πŸ’

Inspects Python source files and provides information about type and location of classes, methods etc

prospector About Prospector is a tool to analyse Python code and output information about errors, potential problems, convention violations and comple

Nov 26, 2022
pycallgraph is a Python module that creates call graphs for Python programs.
pycallgraph is a Python module that creates call graphs for Python programs.

Project Abandoned Many apologies. I've stopped maintaining this project due to personal time constraints. Blog post with more information. I'm happy t

Nov 29, 2022
Turn your Python and Javascript code into DOT flowcharts
Turn your Python and Javascript code into DOT flowcharts

Notes from 2017 This is an older project which I am no longer working on. It was built before ES6 existed and before Python 3 had much usage. While it

Nov 28, 2022
Find dead Python code

Vulture - Find dead code Vulture finds unused code in Python programs. This is useful for cleaning up and finding errors in large code bases. If you r

Nov 30, 2022
Code audit tool for python.

Pylama Code audit tool for Python and JavaScript. Pylama wraps these tools: pycodestyle (formerly pep8) Β© 2012-2013, Florent Xicluna; pydocstyle (form

Nov 30, 2022
The strictest and most opinionated python linter ever!
The strictest and most opinionated python linter ever!

wemake-python-styleguide Welcome to the strictest and most opinionated python linter ever. wemake-python-styleguide is actually a flake8 plugin with s

Dec 5, 2022
A Python utility / library to sort imports.
A Python utility / library to sort imports.

Read Latest Documentation - Browse GitHub Code Repository isort your imports, so you don't have to. isort is a Python utility / library to sort import

Dec 4, 2022
Performant type-checking for python.
Performant type-checking for python.

Pyre is a performant type checker for Python compliant with PEP 484. Pyre can analyze codebases with millions of lines of code incrementally – providi

Dec 3, 2022
Collection of library stubs for Python, with static types

typeshed About Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages as con

Dec 2, 2022
A system for Python that generates static type annotations by collecting runtime types

MonkeyType MonkeyType collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type

Dec 2, 2022
A static type analyzer for Python code

pytype - ? βœ” Pytype checks and infers types for your Python code - without requiring type annotations. Pytype can: Lint plain Python code, flagging c

Nov 30, 2022
Optional static typing for Python 3 and 2 (PEP 484)

Mypy: Optional Static Typing for Python Got a question? Join us on Gitter! We don't have a mailing list; but we are always happy to answer questions o

Dec 3, 2022
Static type checker for Python
Static type checker for Python

Static type checker for Python Speed Pyright is a fast type checker meant for large Python source bases. It can run in a β€œwatch” mode and performs fas

Dec 4, 2022
A static analysis tool for Python

pyanalyze Pyanalyze is a tool for programmatically detecting common mistakes in Python code, such as references to undefined variables and some catego

Dec 6, 2022
Unbearably fast O(1) runtime type-checking in pure Python.
Unbearably fast O(1) runtime type-checking in pure Python.

Look for the bare necessities, the simple bare necessities. Forget about your worries and your strife. β€” The Jungle Book.

Nov 23, 2022
Typing-toolbox for Python 3 _and_ 2.7 w.r.t. PEP 484.
Typing-toolbox for Python 3 _and_ 2.7 w.r.t. PEP 484.

Welcome to the pytypes project pytypes is a typing toolbox w.r.t. PEP 484 (PEP 526 on the road map, later also 544 if it gets accepted). Its main feat

Dec 4, 2022
Data parsing and validation using Python type hints

pydantic Data validation and settings management using Python type hinting. Fast and extensible, pydantic plays nicely with your linters/IDE/brain. De

Dec 5, 2022
Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

Nov 30, 2022
Typical: Fast, simple, & correct data-validation using Python 3 typing.
Typical: Fast, simple, & correct data-validation using Python 3 typing.

typical: Python's Typing Toolkit Introduction Typical is a library devoted to runtime analysis, inference, validation, and enforcement of Python types

Nov 25, 2022