Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.

Japronto!

irc: #japronto Gitter japronto/Lobby Build Status PyPI PyPI version

There is no new project development happening at the moment, but it's not abandoned either. Pull requests and new maintainers are welcome.

If you are a novice Python programmer, you don't like plumbing yourself or you don't have basic understanding of C, this project is not probably what you are looking for.

Japronto (from Portuguese "já pronto" /ˈʒa pɾõtu/ meaning "already done") is a screaming-fast, scalable, asynchronous Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser. It's targeted at speed enthusiasts, people who like plumbing and early adopters.

You can read more in the release announcement on medium

Performance

Here's a chart to help you imagine what kind of things you can do with Japronto:

Requests per second

As user @heppu points out Go’s stdlib HTTP server can be 12% faster than the graph shows when written more carefully. Also there is the awesome fasthttp server for Go that apparently is only 18% slower than Japronto in this particular benchmark. Awesome! For details see https://github.com/squeaky-pl/japronto/pull/12 and https://github.com/squeaky-pl/japronto/pull/14.

These results of a simple "Hello world" application were obtained on AWS c4.2xlarge instance. To be fair all the contestants (including Go) were running single worker process. Servers were load tested using wrk with 1 thread, 100 connections and 24 simultaneous (pipelined) requests per connection (cumulative parallelism of 2400 requests).

The source code for the benchmark can be found in benchmarks directory.

The server is written in hand tweaked C trying to take advantage of modern CPUs. It relies on picohttpparser for header & chunked-encoding parsing while uvloop provides asynchronous I/O. It also tries to save up on system calls by combining writes together when possible.

Early preview

This is an early preview with alpha quality implementation. APIs are provisional meaning that they will change between versions and more testing is needed. Don't use it for anything serious for now and definitely don't use it in production. Please try it though and report back feedback. If you are shopping for your next project's framework I would recommend Sanic.

At the moment the work is focused on CPython but I have PyPy on my radar, though I am not gonna look into it until PyPy reaches 3.5 compatibility somewhere later this year and most known JIT regressions are removed.

Hello world

Here is how a simple web application looks like in Japronto:

from japronto import Application


def hello(request):
    return request.Response(text='Hello world!')


app = Application()
app.router.add_route('/', hello)
app.run(debug=True)

Tutorial

  1. Getting started
  2. Asynchronous handlers
  3. Router
  4. Request object
  5. Response object
  6. Handling exceptions
  7. Extending request

Features

  • HTTP 1.x implementation with support for chunked uploads
  • Full support for HTTP pipelining
  • Keep-alive connections with configurable reaper
  • Support for synchronous and asynchronous views
  • Master-multiworker model based on forking
  • Support for code reloading on changes
  • Simple routing

License

This software is distributed under MIT License. This is a very permissive license that lets you use this software for any commercial and non-commercial work. Full text of the license is included in LICENSE.txt file.

The source distribution of this software includes a copy of picohttpparser which is distributed under MIT license as well.

Owner
Paweł Piotr Przeradowski
Python programmer 🐍 , Rust apprentice 🦀 DevOps 🛠 , PostgreSQL 🐘 , Open source, Windows Subsystem for Linux 🐧 user
Paweł Piotr Przeradowski
Comments
  • Installing fails on OSX

    Installing fails on OSX

    There is no requirements.txt and installation fails:

    $ python3 setup.py install
    Traceback (most recent call last):
      File "setup.py", line 10, in <module>
        import build
      File "/Users/r0fls/Documents/code/foss/japronto/build.py", line 11, in <module>
        import pytoml
    ImportError: No module named 'pytoml'
    

    Installing this way should install the dependencies.

  • CPU hits illegal instruction signal on older hardware

    CPU hits illegal instruction signal on older hardware

    I installed Japronto on OSX via setup.py, and attempted to run the hello world example. The example threw an error when I tried to connect to it via my browser or wrk. The error message was

    Worker exited with code -4!

    Note: I am using a 2010 Macbook pro with Sierra, and received this error straight from the fresh git clone. The installation ran fine but I received a few warnings about uvloop.

  • connection_made errors out with latest uvloop.

    connection_made errors out with latest uvloop.

    I tried a very basic tutorial at https://github.com/squeaky-pl/japronto/blob/master/tutorial/1_hello.md with pip 10.0.1, japronto 0.1.1, uvloop 0.9.1, and ujson 1.35 on Ubuntu 18.04 64-bit running Python 3.6.5. I have tried the latest Sanic with this uvloop and it doesn't cause any errors with it.

    When I start the server, it instantly shows this in the terminal:
    Accepting connections on http://0.0.0.0:8080

    but when I go to that in my browser, the browser is stuck in a loading animation and it immediately errors out in the terminal with this output:

    Exception in callback UVTransport._call_connection_made
    handle: <Handle UVTransport._call_connection_made>
    Traceback (most recent call last):
      File "uvloop/cbhandles.pyx", line 52, in uvloop.loop.Handle._run
      File "uvloop/handles/tcp.pyx", line 149, in uvloop.loop.TCPTransport._call_connection_made
      File "uvloop/handles/basetransport.pyx", line 156, in uvloop.loop.UVBaseTransport._call_connection_made
      File "uvloop/handles/basetransport.pyx", line 153, in uvloop.loop.UVBaseTransport._call_connection_made
    SystemError: NULL result without error in PyObject_Call
    Exception in callback UVTransport._call_connection_made
    handle: <Handle UVTransport._call_connection_made>
    Traceback (most recent call last):
      File "uvloop/cbhandles.pyx", line 52, in uvloop.loop.Handle._run
      File "uvloop/handles/tcp.pyx", line 149, in uvloop.loop.TCPTransport._call_connection_made
      File "uvloop/handles/basetransport.pyx", line 156, in uvloop.loop.UVBaseTransport._call_connection_made
      File "uvloop/handles/basetransport.pyx", line 153, in uvloop.loop.UVBaseTransport._call_connection_made
    SystemError: NULL result without error in PyObject_Call
    

    I know this project hasn't been maintained in a while, but a bunch of benchmarking sites like https://github.com/tbrand/which_is_the_fastest are starting to list it as the best Python server competing with Rust, Go, and Node.

    Will there be updates to this in the near future and if so, will there be more testing?

  • Benchmark is correct?

    Benchmark is correct?

    I am doing tests since its benchmark seems to me too high, I am doing the benchmark and they give me very different results to those that appear in the image:

    CPU: Intel (R) Core (TM) i7-6700 CPU @ 3.40 GHz

    japronto

    docker run williamyeh/wrk -c 200 -t 1 -d 4 http://172.17.0.2:8080/ Running 4s test @ http://172.17.0.2:8080/ 1 threads and 200 connections Thread Stats Avg Stdev Max +/- Stdev Latency 0.85ms 307.50us 3.17ms 70.66% Req/Sec 227.29k 3.84k 232.01k 75.00% 904138 requests in 4.01s, 79.33MB read Requests/sec: 225703.67 Transfer/sec: 19.80MB

    meinheld

    docker run williamyeh/wrk -c 200 -t 1 -d 4 http://172.17.0.2:8080/ Running 4s test @ http://172.17.0.2:8080/ 1 threads and 200 connections Thread Stats Avg Stdev Max +/- Stdev Latency 5.03ms 9.90ms 215.48ms 99.25% Req/Sec 46.38k 768.92 48.44k 75.00% 184537 requests in 4.01s, 31.15MB read Requests/sec: 46049.91 Transfer/sec: 7.77MB

  • When run the examples I have got issue

    When run the examples I have got issue "SystemError: NULL result without error in PyObject_Call"

    I have checked the post in https://medium.freecodecamp.org/million-requests-per-second-with-python-95c137af319 and git clone source code when run the example then got below issue

    I have Python to 3.6 and pip version 9.0.1

    command : python3.6 hello.py

    Accepting connections on http://0.0.0.0:8080 Exception in callback UVTransport._call_connection_made handle: Traceback (most recent call last): File "uvloop/cbhandles.pyx", line 52, in uvloop.loop.Handle._run File "uvloop/handles/tcp.pyx", line 149, in uvloop.loop.TCPTransport._call_connection_made File "uvloop/handles/basetransport.pyx", line 140, in uvloop.loop.UVBaseTransport._call_connection_made File "uvloop/handles/basetransport.pyx", line 137, in uvloop.loop.UVBaseTransport._call_connection_made SystemError: NULL result without error in PyObject_Call

    Please let me know where I change the configuration to work on it. I new to Python.

  • Using Python-Kafka with Japronto

    Using Python-Kafka with Japronto

    I really didn't understand what is causing the issue here. I have initialised producer like below. But when we call the request is failing. But the same thing if we execute in terminal or same structured code in Sanic is working fine.

    Any ideas or suggestions to fix this?

    # General Settings
    import json
    
    # Web Server specific imports
    from japronto import Application
    from json import JSONDecodeError
    from kafka import KafkaProducer
    from kafka.errors import KafkaError
    
    producer = KafkaProducer(bootstrap_servers=["35.154.159.4:9092", "35.154.159.4:9093", "35.154.159.4:9094"], retries=5,value_serializer=lambda m: json.dumps(m).encode('ascii'))
    
    def visuG(request):
    	print(producer.config)
    	print(producer)
    	jstore = request.json
    	print(jstore)	
    	jevents = json.loads(jstore['e'])
    	print(jevents)
    	raw_data = {'topic': 'vnk-desd', 'routes': 'clst/desd', 'messages': jevents}
    	f = producer.send("vnk-raw", raw_data)
    	print(f)
    	r = f.get(timeout=1)
    	return request.Response(code=200,json=r)
    
    
    app = Application()
    
    
    # The Router instance lets you register your handlers and execute
    # them depending on the url path and methods
    r = app.router
    r.add_route('/user-activity-poc', visuG, methods=["POST"])
    r.add_route('/spcb', visuG, methods=["POST"])
    
    # Finally start our server and handle requests until termination is
    # requested. Enabling debug lets you see request logs and stack traces.
    app.run(debug=True)
    

    Error

    Traceback (most recent call last):
      File "jap_server.py", line 22, in visuG
        r = f.get(timeout=1)
      File "/usr/local/lib/python3.6/site-packages/kafka/producer/future.py", line 60, in get
        "Timeout after waiting for %s secs." % timeout)
    kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Timeout after waiting for 1 secs.
    

    I don't think it is because of timeout as I tried it with 10 seconds. It didn't work. Any ideas it is failing ?

  • Templating example?

    Templating example?

    I apologize if this is too noobie of a question, but could someone write an example of how to use templating with Japronto? I'm used to Flask with built-in templating support and I don't know how to get started without it.

  • Need suggestions to work with motor (MongoDB async driver)

    Need suggestions to work with motor (MongoDB async driver)

    My app structure (simplified):

    app/
       templates/
       __init__.py
    manage.py
    

    app/__init__.py

    from japronto import Application
    from motor.motor_asyncio
    
    client = motor.motor_asyncio.AsyncIOMotorClient()
    db = client.mydb
    
    async def index(request):
        result = await db.find_one()
        print(result)
        return request.Response(text=str(result), mime_type='text/html')
    
    def create_app():
       app = Application()
       app.router.add_route('/', index)
    
    

    manage.py

    from app import create_app
    
    if __name__ == '__main__':
       app = create_app()
       app.run(debug=True)
    
    

    When I ran it, I got an exception:

    RuntimeError: Task <Task pending coro=<pkg() running at  .../app/__init__.py:64> cb=[Pipeline._task_done()]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /miniconda/envs/flecom/lib/python3.6/asyncio/futures.py:407]> attached to a different loop
    

    I suspect putting client = motor.motor_asyncio.AsyncIOMotorClient() inside index would work, but that would create new connections for every request. Reusing connection would be more preferable.

  • uvLoop/Protocol Error?

    uvLoop/Protocol Error?

    I was testing a server using Japronto in an Ubuntu-14.04 64bit Virtualbox VM when i came across this error:

    Warning: Host CPU doesn't support SSE 4.2, selecting slower implementation Accepting connections on http://0.0.0.0:80 protocol.pause_writing() failed protocol: <cprotocol.Protocol object at 0x240f7f0> transport: <TCPTransport closed=False reading=True 0x233ac38> Traceback (most recent call last): File "uvloop/handles/basetransport.pyx", line 92, in uvloop.loop.UVBaseTransport._maybe_pause_protocol (uvloop/loop.c:64797) AttributeError: 'cprotocol.Protocol' object has no attribute 'pause_writing' protocol.resume_writing() failed protocol: <cprotocol.Protocol object at 0x240f7f0> transport: <TCPTransport closed=False reading=True 0x233ac38> Traceback (most recent call last): File "uvloop/handles/basetransport.pyx", line 108, in uvloop.loop.UVBaseTransport._maybe_resume_protocol (uvloop/loop.c:65168) AttributeError: 'cprotocol.Protocol' object has no attribute 'resume_writing'

    here is the source code that caused the error. It happens every time i load the page "/register"

    from base64 import b64encode
    from datetime import datetime
    from japronto import Application, RouteNotFoundException
    
    #***Variables***#
    with open("images/img.jpg", "rb") as f: imgdata = b64encode(f.read()).decode("utf-8")
    basehtml = """
    <!DOCTYPE html>
    <html>
    <head>
        <title>Registration</title>
        <style>
            html {{
                background: url("data:image/png;base64,%s") no-repeat center center fixed;
                -webkit-background-size: cover;
                -moz-background-size: cover;
                -o-background-size: cover;
                background-size: cover;
            }}
            .center {{
                width: 310px;
                height: 300px;
                position: absolute;
                top:0;
                bottom: 0;
                left: 0;
                right: 0;
                margin: auto;
            }}
        </style>
    </head>
    <body>
    <form id="register" method="POST" autocomplete="off">
        <div class="center">
            <div id="formobject" style="width:310px">
                <h3 align="center" style="color:rgb(255, 255, 255)">Registration</h3>
                <input name="firstname" required="required" pattern="[A-Za-z0-9]{{1,20}}" type="text" style="width:300px" placeholder="first-name">
                <br>
                <input name="lastname" required="required" pattern="[A-Za-z0-9]{{1,20}}" type="text" style="width:300px" placeholder="last-name">
                <br>
                <input name="email" required="required" pattern="[^@][email protected][^@]+\.[a-zA-Z]{{2,6}}" type="text" style="width:300px" placeholder="[email protected]">
                <br><br>
                <button style="float:right">Submit</button>
            </div>
            <div id="noticeobject">{notice}</div>
        </div>
    </form>
    </body>
    </html>
    """%(imgdata)
    
    #***Functions***#
    def page(request):
        html = basehtml.format(notice="")
        return request.Response(text=html, headers=server_headers(), mime_type="text/html")
    
    def server_headers():
        return {
            "Server": "nginx",
            "Date": datetime.now().strftime("%a, %d %Y %H:%M:%S GMT"),
            "Last Modified": "Sun, 19 2017 09:00:01 GMT",
            "Accept Ranges": "bytes",
        }
    
    def handle_404(request, exception):
        """Handles any 404 errors within Server"""
        print("[404] FROM: %s, TO: %s"%(request.remote_addr,request.path))
        """Handles 404 Requests"""
        html="""<html>
    <head><title>404 Not Found</title></head>
    <body bgcolor="white">
    <center><h1>404 Not Found</h1></center>
    <hr><center>nginx</center>
    </body>
    </html>
    """
        return request.Response(text=html, headers=server_headers(), code=404, mime_type="text/html")
    
    #***Classes***#
    class NewApp(Application):
        """Used to modify the error handler to provide a better response on server and client"""
        def protocol_error_handler(self, error):
            """System Error Handler"""
            if error == "malformed_headers": error+=" (Possible Port Scan or Attack)"
            print("System Error: %s"%(error))
            error = error.encode('utf-8')
            response = """<html>
    <head><title>400 Bad Request</title></head>
    <body bgcolor="white">
    <center><h1>400 Bad Request</h1></center>
    <hr><center>nginx</center>
    </body>
    </html>
    """
            return response.encode('utf-8')
    
    #***Start***#
    app = NewApp()
    app.router.add_route('/register', page)
    app.add_error_handler(RouteNotFoundException, handle_404)
    app.run(port=80,debug=False)
    

    This error does not occur when i remove the base64 encoded image from the html, is uvloop having trouble writing all the data at once?

  • AttributeError: 'File' object has no attribute 'save'

    AttributeError: 'File' object has no attribute 'save'

    My server works perfectly with JSON and text requests, but when i try to receive files I have an error. Python Traceback ->

    File "template.py", line 85, in uploadFile file.save(f) AttributeError: 'File' object has no attribute 'save'

    The code I'm using : -> def uploadFile(request): file = request.files["data"] f = open("/Users/pewdepiebestthantseries/Documents/Japronto/uploader/" + request.files["data"].name, 'wb', 10000) file.save(f) print("All right") return request.Response(text="'Completed!'")

    The content of data is an array that contains an image.

    A simple request -> curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]/Users/pewdepiebestthantseries/Downloads/lasagna.png" http://IP:PORT/file

    HEELP

  • Incompatible with uvloop==0.9.1

    Incompatible with uvloop==0.9.1

    MacBook-Pro:test rilut$ pip3 install  uvloop==0.9.1
    Collecting uvloop==0.9.1
      Using cached uvloop-0.9.1-cp36-cp36m-macosx_10_11_x86_64.whl
    Installing collected packages: uvloop
      Found existing installation: uvloop 0.8.1
        Uninstalling uvloop-0.8.1:
          Successfully uninstalled uvloop-0.8.1
    Successfully installed uvloop-0.9.1
    
    MacBook-Pro:test rilut$ python3 japro.py
    Accepting connections on http://0.0.0.0:8080
    Exception in callback UVTransport._call_connection_made
    handle: <Handle UVTransport._call_connection_made>
    Traceback (most recent call last):
      File "uvloop/cbhandles.pyx", line 52, in uvloop.loop.Handle._run
      File "uvloop/handles/tcp.pyx", line 149, in uvloop.loop.TCPTransport._call_connection_made
      File "uvloop/handles/basetransport.pyx", line 156, in uvloop.loop.UVBaseTransport._call_connection_made
      File "uvloop/handles/basetransport.pyx", line 153, in uvloop.loop.UVBaseTransport._call_connection_made
    SystemError: NULL result without error in PyObject_Call
    
    MacBook-Pro:test rilut$ pip3 install  uvloop==0.8.1
    Collecting uvloop==0.8.1
      Downloading uvloop-0.8.1-cp36-cp36m-macosx_10_11_x86_64.whl (1.2MB)
        100% |████████████████████████████████| 1.2MB 748kB/s
    Installing collected packages: uvloop
      Found existing installation: uvloop 0.9.1
        Uninstalling uvloop-0.9.1:
          Successfully uninstalled uvloop-0.9.1
    Successfully installed uvloop-0.8.1
    
    MacBook-Pro:test rilut$ python3 japro.py
    Accepting connections on http://0.0.0.0:8080
    
  • Unable to install japronto on Mac M1

    Unable to install japronto on Mac M1

    I have tried this way - python3 setup.py install

    Error Message: /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:14:2: error: this header is for x86 only #error this header is for x86 only ^ /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:286:5: error: invalid output constraint '=a' in asm __cpuid(__leaf, __eax, __ebx, __ecx, __edx); ^ /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:251:11: note: expanded from macro '__cpuid' : "=a"(__eax), "=r" (__ebx), "=c"(__ecx), "=d"(__edx) \ ^ /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:301:5: error: invalid output constraint '=a' in asm __cpuid(__leaf, *__eax, *__ebx, *__ecx, *__edx); ^ /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:251:11: note: expanded from macro '__cpuid' : "=a"(__eax), "=r" (__ebx), "=c"(__ecx), "=d"(__edx) \ ^ /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:315:5: error: invalid output constraint '=a' in asm __cpuid_count(__leaf, __subleaf, *__eax, *__ebx, *__ecx, *__edx); ^ /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/include/cpuid.h:258:11: note: expanded from macro '__cpuid_count' : "=a"(__eax), "=r" (__ebx), "=c"(__ecx), "=d"(__edx) \ ^ 4 errors generated. error: command 'clang' failed with exit status 1

    How can i fix this??

  • Bump sanic from 0.2.0 to 20.12.6 in /benchmarks/sanic

    Bump sanic from 0.2.0 to 20.12.6 in /benchmarks/sanic

    Bumps sanic from 0.2.0 to 20.12.6.

    Release notes

    Sourced from sanic's releases.

    Version 20.12.6

    What's Changed

    Full Changelog: https://github.com/sanic-org/sanic/compare/v20.12.5...v20.12.6

    Version 20.12.5

    #2366 Upgrade websockets version - SECURITY UPDATE

    Version 20.12.4

    Version 20.12.3

    Bugfixes

    • #2021 Remove prefix from websocket handler name

    Version 20.12.2

    Dependencies

    • #2026 Fix uvloop to 0.14 because 0.15 drops Python 3.6 support
    • #2029 Remove old chardet requirement, add in hard multidict requirement

    Version 20.12.1

    Features

    • #1993 Add disable app registry

    Version 20.12.0

    Features

    • [#1945][] Static route more verbose if file not found
    • [#1954][] Fix static routes registration on a blueprint
    • [#1961][] Add Python 3.9 support
    • [#1962][] Sanic CLI upgrade
    • [#1967][] Update aiofile version requirements
    • [#1969][] Update multidict version requirements
    • [#1970][] Add py.typed file
    • [#1972][] Speed optimization in request handler
    • [#1979][] Add app registry and Sanic class level app retrieval

    Bugfixes

    • [#1965][] Fix Chunked Transport-Encoding in ASGI streaming response

    Deprecations and Removals

    ... (truncated)

    Changelog

    Sourced from sanic's changelog.

    .. note::

    CHANGELOG files are maintained in ./docs/sanic/releases. To view the full CHANGELOG, please visit https://sanic.readthedocs.io/en/stable/sanic/changelog.html.

    Version 21.6.1

    Bugfixes

    • [#2178](https://github.com/sanic-org/sanic/issues/2178) <https://github.com/sanic-org/sanic/pull/2178>_ Update sanic-routing to allow for better splitting of complex URI templates
    • [#2183](https://github.com/sanic-org/sanic/issues/2183) <https://github.com/sanic-org/sanic/pull/2183>_ Proper handling of chunked request bodies to resolve phantom 503 in logs
    • [#2181](https://github.com/sanic-org/sanic/issues/2181) <https://github.com/sanic-org/sanic/pull/2181>_ Resolve regression in exception logging
    • [#2201](https://github.com/sanic-org/sanic/issues/2201) <https://github.com/sanic-org/sanic/pull/2201>_ Cleanup request info in pipelined requests

    Version 21.6.0

    Features

    • [#2094](https://github.com/sanic-org/sanic/issues/2094) <https://github.com/sanic-org/sanic/pull/2094>_ Add response.eof() method for closing a stream in a handler

    • [#2097](https://github.com/sanic-org/sanic/issues/2097) <https://github.com/sanic-org/sanic/pull/2097>_ Allow case-insensitive HTTP Upgrade header

    • [#2104](https://github.com/sanic-org/sanic/issues/2104) <https://github.com/sanic-org/sanic/pull/2104>_ Explicit usage of CIMultiDict getters

    • [#2109](https://github.com/sanic-org/sanic/issues/2109) <https://github.com/sanic-org/sanic/pull/2109>_ Consistent use of error loggers

    • [#2114](https://github.com/sanic-org/sanic/issues/2114) <https://github.com/sanic-org/sanic/pull/2114>_ New client_ip access of connection info instance

    • [#2119](https://github.com/sanic-org/sanic/issues/2119) <https://github.com/sanic-org/sanic/pull/2119>_ Alternatate classes on instantiation for Config and Sanic.ctx

    • [#2133](https://github.com/sanic-org/sanic/issues/2133) <https://github.com/sanic-org/sanic/pull/2133>_ Implement new version of AST router

      • Proper differentiation between alpha and string param types
      • Adds a slug param type, example: <foo:slug>
      • Deprecates <foo:string> in favor of <foo:str>
      • Deprecates <foo:number> in favor of <foo:float>
      • Adds a route.uri accessor
    • [#2136](https://github.com/sanic-org/sanic/issues/2136) <https://github.com/sanic-org/sanic/pull/2136>_ CLI improvements with new optional params

    • [#2137](https://github.com/sanic-org/sanic/issues/2137) <https://github.com/sanic-org/sanic/pull/2137>_ Add version_prefix to URL builders

    • [#2140](https://github.com/sanic-org/sanic/issues/2140) <https://github.com/sanic-org/sanic/pull/2140>_ Event autoregistration with EVENT_AUTOREGISTER

    ... (truncated)

    Commits
    • 3b85b3b Potential server crash if running Python 3.10 w/ Sanic 20.12 (#2400)
    • 6e55e73 fix: websocket dependency for websockets 9.1 security fix (#2366)
    • 89d9424 Merge branch 'pr2129' into 20.12LTS
    • 4d6205e Bump version
    • 1684b0b remove reference to yanked packages
    • 4f5faa4 unpin uvloop
    • cbb77b5 fix issue where request.args.pop removed parameters inconsistently (#2112)
    • 35c7625 Bump version 20.12.3 (#2062)
    • 8d86c3c Remove unnecessary prefix from websocket handler name (#2021)
    • 9763511 Align setup.py
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • A risk of writing to an invalid address with memcpy in function Parser_feed

    A risk of writing to an invalid address with memcpy in function Parser_feed

    Code snippet

    Parser_feed(Parser* self, PyObject *args)
    {
         ........
         if((size_t)data_len > self->buffer_capacity - (self->buffer_end - self->buffer_start)) {
                self->buffer_capacity = MAX(self->buffer_capacity * 2, self->buffer_end - self->buffer_start + data_len);
                if(self->buffer == self->inline_buffer) {
                       self->buffer = malloc(self->buffer_capacity);    --------> may return a NULL pointer
                       memcpy(self->buffer + self->buffer_start, self->inline_buffer + self->buffer_start,
                                      self->buffer_end - self->buffer_start);
        } 
        ........
    }
    

    Description

    Function: Parser_feed File: cparser.c Call-path: feed (Python) -> Parser_feed -> memcpy WarningType: Invalid write. Our analysis tool reported a warning on potential write at an invalid address. As the buffer_capacity may depend on external inputs, hence it is possible that malloc-fail happens. Return value validation is necessary at this point. Also seen in Details

  • thank you

    thank you

    I thank your team The latest version is a year old and I'm surprised by the performance of Japronto, but are you not updating it and also is it possible to communicate with neo4j ? japronto and neo4j are best .

  • Worker crashed on signal SIGSEGV!

    Worker crashed on signal SIGSEGV!

    I just test how much size the response header can load.

    Env Info:

    • pip list
    Package    Version
    ---------- -------
    japronto   0.1.2
    pip        20.1.1
    setuptools 47.1.0
    uvloop     0.14.0
    wheel      0.36.2
    
    • Python 3.7.9

    reproduced issue code:

    from japronto import Application
    from japronto.response import JsonResponse
    def hello(request):
        headers = { "Auth-Token": "token"*200 }
        return JsonResponse(json={'hello': 'world'}, headers=headers)
    app = Application()
    app.router.add_route('/', hello)
    app.run(debug=True)
    

    run the above code and curl http://localhost:8080. And then Worker crashed on signal SIGSEGV! log like this:

    Accepting connections on http://0.0.0.0:8080
    127.0.0.1 GET /
    Fatal Python error: Segmentation fault
    
    Current thread 0x0000000117f9ce00 (most recent call first):
      File "/Users/ql/WorkSpace/repo/github.com/imuxin/japronto/venv/lib/python3.7/site-packages/japronto/app/__init__.py", line 191 in serve
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 99 in run
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 297 in _bootstrap
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/popen_fork.py", line 74 in _launch
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/popen_fork.py", line 20 in __init__
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/context.py", line 277 in _Popen
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/context.py", line 223 in _Popen
      File "/usr/local/Cellar/[email protected]/3.7.9_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 112 in start
      File "/Users/ql/WorkSpace/repo/github.com/imuxin/japronto/venv/lib/python3.7/site-packages/japronto/app/__init__.py", line 237 in _run
      File "/Users/ql/WorkSpace/repo/github.com/imuxin/japronto/venv/lib/python3.7/site-packages/japronto/app/__init__.py", line 273 in run
      File "app.py", line 23 in <module>
    Worker crashed on signal SIGSEGV!
    

    I have no ideal how to debug this.

Small, fast HTTP client library for Python. Features persistent connections, cache, and Google App Engine support. Originally written by Joe Gregorio, now supported by community.

Introduction httplib2 is a comprehensive HTTP client library, httplib2.py supports many features left out of other HTTP libraries. HTTP and HTTPS HTTP

Jun 29, 2022
Fast HTTP parser

httptools is a Python binding for the nodejs HTTP parser. The package is available on PyPI: pip install httptools. APIs httptools contains two classes

Jun 27, 2022
Python package for caching HTTP response based on etag

Etag cache implementation for HTTP requests, to save request bandwidth for a non-modified response. Returns high-speed accessed dictionary data as cache.

Apr 27, 2022
A modern/fast python SOAP client based on lxml / requests

Zeep: Python SOAP client A fast and modern Python SOAP client Highlights: Compatible with Python 3.6, 3.7, 3.8 and PyPy Build on top of lxml and reque

Jul 5, 2022
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.

urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. urllib3 brings many

Jul 4, 2022
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.

urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. urllib3 brings many

Jul 6, 2022
🔄 🌐 Handle thousands of HTTP requests, disk writes, and other I/O-bound tasks simultaneously with Python's quintessential async libraries.

?? ?? Handle thousands of HTTP requests, disk writes, and other I/O-bound tasks simultaneously with Python's quintessential async libraries.

Jun 9, 2022
A Python obfuscator using HTTP Requests and Hastebin.
A Python obfuscator using HTTP Requests and Hastebin.

?? Jawbreaker ?? Jawbreaker is a Python obfuscator written in Python3, using double encoding in base16, base32, base64, HTTP requests and a Hastebin-l

Jun 17, 2022
An interactive command-line HTTP and API testing client built on top of HTTPie featuring autocomplete, syntax highlighting, and more. https://twitter.com/httpie
An interactive command-line HTTP and API testing client built on top of HTTPie featuring autocomplete, syntax highlighting, and more. https://twitter.com/httpie

HTTP Prompt HTTP Prompt is an interactive command-line HTTP client featuring autocomplete and syntax highlighting, built on HTTPie and prompt_toolkit.

Jun 30, 2022
Probe and discover HTTP pathname using brute-force methodology and filtered by specific word or 2 words at once
Probe and discover HTTP pathname using brute-force methodology and filtered by specific word or 2 words at once

pathprober Probe and discover HTTP pathname using brute-force methodology and filtered by specific word or 2 words at once. Purpose Brute-forcing webs

Jun 25, 2022
A next generation HTTP client for Python. 🦋
A next generation HTTP client for Python. 🦋

HTTPX - A next-generation HTTP client for Python. HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support

Jul 1, 2022
Python requests like API built on top of Twisted's HTTP client.

treq: High-level Twisted HTTP Client API treq is an HTTP library inspired by requests but written on top of Twisted's Agents. It provides a simple, hi

Jun 14, 2022
Asynchronous Python HTTP Requests for Humans using Futures

Asynchronous Python HTTP Requests for Humans Small add-on for the python requests http library. Makes use of python 3.2's concurrent.futures or the ba

Jul 6, 2022
HTTP/2 for Python.
HTTP/2 for Python.

Hyper: HTTP/2 Client for Python This project is no longer maintained! Please use an alternative, such as HTTPX or others. We will not publish further

Jun 22, 2022
HTTP request/response parser for python in C

http-parser HTTP request/response parser for Python compatible with Python 2.x (>=2.7), Python 3 and Pypy. If possible a C parser based on http-parser

Jun 26, 2022
Aiosonic - lightweight Python asyncio http client

aiosonic - lightweight Python asyncio http client Very fast, lightweight Python asyncio http client Here is some documentation. There is a performance

Jun 28, 2022
Some example code for using a raspberry pi to draw text (including emojis) and twitch emotes to a HUB75 RGB matrix via an HTTP post endpoint.

Some example code for using a raspberry pi to draw text (including emojis) and twitch emotes to a HUB75 RGB matrix via an HTTP post endpoint.

Jun 25, 2022
EasyRequests is a minimalistic HTTP-Request Library that wraps aiohttp and asyncio in a small package that allows for sequential, parallel or even single requests

EasyRequests EasyRequests is a minimalistic HTTP-Request Library that wraps aiohttp and asyncio in a small package that allows for sequential, paralle

Jan 27, 2022
A simple, yet elegant HTTP library.
A simple, yet elegant HTTP library.

Requests Requests is a simple, yet elegant HTTP library. >>> import requests >>> r = requests.get('https://api.github.com/user', auth=('user', 'pass')

Jul 5, 2022