requests-toolbelt (0.9.1)
Installation
pip install --index-url requests-toolbelt
About this package
A utility belt for advanced users of python-requests
The Requests Toolbelt
This is just a collection of utilities for python-requests
_, but don't
really belong in requests
proper. The minimum tested requests version is
2.1.0
. In reality, the toolbelt should work with 2.0.1
as well, but
some idiosyncracies prevent effective or sane testing on that version.
pip install requests-toolbelt
to get started!
multipart/form-data Encoder
The main attraction is a streaming multipart form-data object, MultipartEncoder
.
Its API looks like this:
.. code-block:: python
from requests_toolbelt import MultipartEncoder
import requests
m = MultipartEncoder(
fields={'field0': 'value', 'field1': 'value',
'field2': ('filename', open('file.py', 'rb'), 'text/plain')}
)
r = requests.post('http://httpbin.org/post', data=m,
headers={'Content-Type': m.content_type})
You can also use multipart/form-data
encoding for requests that don't
require files:
.. code-block:: python
from requests_toolbelt import MultipartEncoder
import requests
m = MultipartEncoder(fields={'field0': 'value', 'field1': 'value'})
r = requests.post('http://httpbin.org/post', data=m,
headers={'Content-Type': m.content_type})
Or, you can just create the string and examine the data:
.. code-block:: python
# Assuming `m` is one of the above
m.to_string() # Always returns unicode
User-Agent constructor
You can easily construct a requests-style User-Agent
string::
from requests_toolbelt import user_agent
headers = {
'User-Agent': user_agent('my_package', '0.0.1')
}
r = requests.get('https://api.github.com/users', headers=headers)
SSLAdapter
The SSLAdapter
was originally published on Cory Benfield's blog
_.
This adapter allows the user to choose one of the SSL protocols made available
in Python's ssl
module for outgoing HTTPS connections:
.. code-block:: python
from requests_toolbelt import SSLAdapter
import requests
import ssl
s = requests.Session()
s.mount('https://', SSLAdapter(ssl.PROTOCOL_TLSv1))
cookies/ForgetfulCookieJar
The ForgetfulCookieJar
prevents a particular requests session from storing
cookies:
.. code-block:: python
from requests_toolbelt.cookies.forgetful import ForgetfulCookieJar
session = requests.Session()
session.cookies = ForgetfulCookieJar()
Known Issues
On Python 3.3.0 and 3.3.1, the standard library's http
module will fail
when passing an instance of the MultipartEncoder
. This is fixed in later
minor releases of Python 3.3. Please consider upgrading to a later minor
version or Python 3.4. There is absolutely nothing this library can do to
work around that bug.
Contributing
Please read the suggested workflow <https://toolbelt.readthedocs.org/en/latest/contributing.html>
_ for
contributing to this project.
Please report any bugs on the issue tracker
_
.. _Cory Benfield's blog: https://lukasa.co.uk/2013/01/Choosing_SSL_Version_In_Requests/ .. _python-requests: https://github.com/kennethreitz/requests .. _issue tracker: https://github.com/requests/toolbelt/issues
History
0.9.1 -- 2019-01-29
Fixed Bugs
- Fix import of pyOpenSSL shim from urllib3 for PKCS12 adapter
0.9.0 -- 2019-01-29
-------------------
New Features
- Add X509 Adapter that can handle PKCS12
- Add stateless solution for streaming files by MultipartEncoder from one host to another (in chunks)
Fixed Bugs
- Update link to example
- Move import of ``ABCs`` from collections into version-specific part of
_compat module
- Fix backwards incompatibility in ``get_encodings_from_content``
- Correct callback documentation for ``MultipartEncoderMonitor``
- Fix bug when ``MultipartEncoder`` is asked to encode zero parts
- Correct the type of non string request body dumps
- Removed content from being stored in MultipartDecoder
- Fix bug by enabling support for contenttype with capital letters.
- Coerce proxy URL to bytes before dumping request
- Avoid bailing out with exception upon empty response reason
- Corrected Pool documentation
- Corrected parentheses match in example usage
- Fix "oject" to "object" in ``MultipartEncoder``
- Fix URL for the project after the move
- Add fix for OSX TCPKeepAliveAdapter
Miscellaneous
- Remove py33 from testing and add Python 3.6 and nightly testing to the travis matrix.
0.8.0 -- 2017-05-20
More information about this release can be found on the 0.8.0 milestone
_.
New Features
- Add ``UserAgentBuilder`` to provide more control over generated User-Agent
strings.
Fixed Bugs
~~~~~~~~~~
- Include ``_validate_certificate`` in the lits of picked attributes on the
``AppEngineAdapter``.
- Fix backwards incompatibility in ``get_encodings_from_content``
.. _0.8.0 milestone:
https://github.com/requests/toolbelt/milestones/0.8.0
0.7.1 -- 2017-02-13
-------------------
More information about this release can be found on the `0.7.1 milestone`_.
Fixed Bugs
~~~~~~~~~~
- Fixed monkey-patching for the AppEngineAdapter.
- Make it easier to disable certificate verification when monkey-patching
AppEngine.
- Handle ``multipart/form-data`` bodies without a trailing ``CRLF``.
.. links
.. _0.7.1 milestone:
https://github.com/requests/toolbelt/milestone/9
0.7.0 -- 2016-07-21
-------------------
More information about this release can be found on the `0.7.0 milestone`_.
New Features
-
Add
BaseUrlSession
to allow developers to have a session that has a "Base" URL. See the documentation for more details and examples. -
Split the logic of
stream_response_to_file
into two separate functions:-
get_download_file_path
to generate the file name from the Response. -
stream_response_to_file
which will useget_download_file_path
if necessary
-
Fixed Bugs
- Fixed the issue for people using *very* old versions of Requests where they
would see an ImportError from ``requests_toolbelt._compat`` when trying to
import ``connection``.
.. _0.7.0 milestone:
https://github.com/requests/toolbelt/milestones/0.7.0
0.6.2 -- 2016-05-10
-------------------
Fixed Bugs
- When passing a timeout via Requests, it was not appropriately translated to the timeout that the urllib3 code was expecting.
0.6.1 -- 2016-05-05
Fixed Bugs
- Remove assertion about request URLs in the AppEngineAdapter.
- Prevent pip from installing requests 3.0.0 when that is released until we
are ready to handle it.
0.6.0 -- 2016-01-27
-------------------
More information about this release can be found on the `0.6.0 milestone`_.
New Features
-
Add
AppEngineAdapter
to support developers using Google's AppEngine platform with Requests. -
Add
GuessProxyAuth
class to support guessing between Basic and Digest Authentication for proxies.
Fixed Bugs
- Ensure that proxies use the correct TLS version when using the
``SSLAdapter``.
- Fix an ``AttributeError`` when using the ``HTTPProxyDigestAuth`` class.
Miscellaneous
- Drop testing support for Python 3.2. virtualenv and pip have stopped supporting it meaning that it is harder to test for this with our CI infrastructure. Moving forward we will make a best-effort attempt to support 3.2 but will not test for it.
.. _0.6.0 milestone: https://github.com/requests/toolbelt/milestones/0.6.0
0.5.1 -- 2015-12-16
More information about this release can be found on the 0.5.1 milestone
_.
Fixed Bugs
- Now papers over the differences in requests' ``super_len`` function from
versions prior to 2.9.0 and versions 2.9.0 and later.
.. _0.5.1 milestone:
https://github.com/requests/toolbelt/milestones/0.5.1
0.5.0 -- 2015-11-24
-------------------
More information about this release can be found on the `milestone
<https://github.com/requests/toolbelt/issues?utf8=%E2%9C%93&q=is%3Aall+milestone%3A0.5+>`_
for 0.5.0.
New Features
-
The
tee
submodule was added torequests_toolbelt.downloadutils
. It allows you to iterate over the bytes of a response while also writing them to a file. Thetee.tee
function, expects you to pass an open file object, whiletee.tee_to_file
will use the provided file name to open the file for you. -
Added a new parameter to
requests_toolbelt.utils.user_agent
that allows the user to specify additional items. -
Added nested form-data helper,
requests_toolbelt.utils.formdata.urlencode
. -
Added the
ForgetfulCookieJar
torequests_toolbelt.cookies
. -
Added utilities for dumping the information about a request-response cycle in
requests_toolbelt.utils.dump
. -
Implemented the API described in the
requests_toolbelt.threaded
module docstring, i.e., addedrequests_toolbelt.threaded.map
as an available function.
Fixed Bugs
- Now papers over the API differences in versions of requests installed from
system packages versus versions of requests installed from PyPI.
- Allow string types for ``SourceAddressAdapter``.
0.4.0 -- 2015-04-03
-------------------
For more information about this release, please see `milestone 0.4.0
<https://github.com/requests/toolbelt/issues?q=milestone%3A0.4>`_
on the project's page.
New Features
-
A naive implemenation of a thread pool is now included in the toolbelt. See the docs in
docs/threading.rst
or onRead The Docs <https://toolbelt.readthedocs.org>
_. -
The
StreamingIterator
now accepts files (such assys.stdin
) without a specific length and will properly stream them. -
The
MultipartEncoder
now accepts exactly the same format of fields as requests'files
parameter does. In other words, you can now also pass in extra headers to add to a part in the body. You can also now specify a customContent-Type
for a part. -
An implementation of HTTP Digest Authentication for Proxies is now included.
-
A transport adapter that allows a user to specify a specific Certificate Fingerprint is now included in the toolbelt.
-
A transport adapter that simplifies how users specify socket options is now included.
-
A transport adapter that simplifies how users can specify TCP Keep-Alive options is now included in the toolbelt.
-
Deprecated functions from
requests.utils
are now included and maintained. -
An authentication tool that allows users to specify how to authenticate to several different domains at once is now included.
-
A function to save streamed responses to disk by analyzing the
Content-Disposition
header is now included in the toolbelt.
Fixed Bugs
- The ``MultipartEncoder`` will now allow users to upload files larger than
4GB on 32-bit systems.
- The ``MultipartEncoder`` will now accept empty unicode strings for form
values.
0.3.1 -- 2014-06-23
-------------------
- Fix the fact that 0.3.0 bundle did not include the ``StreamingIterator``
0.3.0 -- 2014-05-21
-------------------
Bug Fixes
~~~~~~~~~
- Complete rewrite of ``MultipartEncoder`` fixes bug where bytes were lost in
uploads
New Features
-
MultipartDecoder
to acceptmultipart/form-data
response bodies and parse them into an easy to use object. -
SourceAddressAdapter
to allow users to choose a local address to bind connections to. -
GuessAuth
which accepts a username and password and uses theWWW-Authenticate
header to determine how to authenticate against a server. -
MultipartEncoderMonitor
wraps an instance of theMultipartEncoder
and keeps track of how many bytes were read and will call the provided callback. -
StreamingIterator
will wrap an iterator and stream the upload instead of chunk it, provided you also provide the length of the content you wish to upload.
0.2.0 -- 2014-02-24
-
Add ability to tell
MultipartEncoder
which encoding to use. By default it uses 'utf-8'. -
Fix #10 - allow users to install with pip
-
Fix #9 - Fix
MultipartEncoder#to_string
so that it properly handles file objects as fields
0.1.2 -- 2014-01-19
- At some point during development we broke how we handle normal file objects. Thanks to @konomae this is now fixed.
0.1.1 -- 2014-01-19
- Handle
io.BytesIO
-like objects better
0.1.0 -- 2014-01-18
-
Add initial implementation of the streaming
MultipartEncoder
-
Add initial implementation of the
user_agent
function -
Add the
SSLAdapter