• Nir Soffer's avatar
    bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279) · ad455cd9
    Nir Soffer yazdı
    blocksize was hardcoded to 8192, preventing efficient upload when using
    file-like body. Add blocksize argument to __init__, so users can
    configure the blocksize to fit their needs.
    
    I tested this uploading data from /dev/zero to a web server dropping the
    received data, to test the overhead of the HTTPConnection.send() with a
    file-like object.
    
    Here is an example 10g upload with the default buffer size (8192):
    
    $ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
    Uploaded 10.00g in 17.53 seconds (584.00m/s)
    
    real	0m17.574s
    user	0m8.887s
    sys	0m5.971s
    
    Same with 512k blocksize:
    
    $ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
    Uploaded 10.00g in 6.60 seconds (1551.15m/s)
    
    real	0m6.641s
    user	0m3.426s
    sys	0m2.162s
    
    In real world usage the difference will be smaller, depending on the
    local and remote storage and the network.
    
    See https://github.com/nirs/http-bench for more info.
    ad455cd9
test_httplib.py 71.9 KB