forked from python/peps
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathpep-0460.txt
178 lines (136 loc) · 6.26 KB
/
pep-0460.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
PEP: 460
Title: Add binary interpolation and formatting
Version: $Revision$
Last-Modified: $Date$
Author: Antoine Pitrou <[email protected]>
Status: Withdrawn
Type: Standards Track
Content-Type: text/x-rst
Created: 6-Jan-2014
Python-Version: 3.5
Abstract
========
This PEP proposes to add minimal formatting operations to bytes and
bytearray objects. The proposed additions are:
* ``bytes % ...`` and ``bytearray % ...`` for percent-formatting,
similar in syntax to percent-formatting on ``str`` objects
(accepting a single object, a tuple or a dict).
* ``bytes.format(...)`` and ``bytearray.format(...)`` for a formatting
similar in syntax to ``str.format()`` (accepting positional as well as
keyword arguments).
* ``bytes.format_map(...)`` and ``bytearray.format_map(...)`` for an
API similar to ``str.format_map(...)``, with the same formatting
syntax and semantics as ``bytes.format()`` and ``bytearray.format()``.
Rationale
=========
In Python 2, ``str % args`` and ``str.format(args)`` allow the formatting
and interpolation of bytestrings. This feature has commonly been used
for the assembling of protocol messages when protocols are known to use
a fixed encoding.
Python 3 generally mandates that text be stored and manipulated as unicode
(i.e. ``str`` objects, not ``bytes``). In some cases, though, it makes
sense to manipulate ``bytes`` objects directly. Typical usage is binary
network protocols, where you can want to interpolate and assemble several
bytes object (some of them literals, some of them compute) to produce
complete protocol messages. For example, protocols such as HTTP or SIP
have headers with ASCII names and opaque "textual" values using a varying
and/or sometimes ill-defined encoding. Moreover, those headers can be
followed by a binary body... which can be chunked and decorated with ASCII
headers and trailers!
While there are reasonably efficient ways to accumulate binary data
(such as using a ``bytearray`` object, the ``bytes.join`` method or
even ``io.BytesIO``), none of them leads to the kind of readable and
intuitive code that is produced by a %-formatted or {}-formatted template
and a formatting operation.
Binary formatting features
==========================
Supported features
------------------
In this proposal, percent-formatting for ``bytes`` and ``bytearray``
supports the following features:
* Looking up formatting arguments by position as well as by name (i.e.,
``%s`` as well as ``%(name)s``).
* ``%s`` will try to get a ``Py_buffer`` on the given value, and fallback
on calling ``__bytes__``. The resulting binary data is inserted at
the given point in the string. This is expected to work with bytes,
bytearray and memoryview objects (as well as a couple others such
as pathlib's path objects).
* ``%c`` will accept an integer between 0 and 255, and insert a byte of the
given value.
Braces-formatting for ``bytes`` and ``bytearray`` supports the following
features:
* All the kinds of argument lookup supported by ``str.format()`` (explicit
positional lookup, auto-incremented positional lookup, keyword lookup,
attribute lookup, etc.)
* Insertion of binary data when no modifier or layout is specified
(e.g. ``{}``, ``{0}``, ``{name}``). This has the same semantics as
``%s`` for percent-formatting (see above).
* The ``c`` modifier will accept an integer between 0 and 255, and insert a
byte of the given value (same as ``%c`` above).
Unsupported features
--------------------
All other features present in formatting of ``str`` objects (either
through the percent operator or the ``str.format()`` method) are
unsupported. Those features imply treating the recipient of the
operator or method as text, which goes counter to the text / bytes
separation (for example, accepting ``%d`` as a format code would imply
that the bytes object really is an ASCII-compatible text string).
Amongst those unsupported features are not only most type-specific
format codes, but also the various layout specifiers such as padding
or alignment. Besides, ``str`` objects are not acceptable as arguments
to the formatting operations, even when using e.g. the ``%s`` format code.
``__format__`` isn't called.
Criticisms
==========
* The development cost and maintenance cost.
* In 3.3 encoding to ASCII or latin-1 is as fast as memcpy (but it still
creates a separate object).
* Developers will have to work around the lack of binary formatting anyway,
if they want to support Python 3.4 and earlier.
* bytes.join() is consistently faster than format to join bytes strings
(XXX *is it?*).
* Formatting functions could be implemented in a third party module,
rather than added to builtin types.
Other proposals
===============
A new type datatype
-------------------
It was proposed to create a new datatype specialized for "network
programming". The authors of this PEP believe this is counter-productive.
Python 3 already has several major types dedicated to manipulation of
binary data: ``bytes``, ``bytearray``, ``memoryview``, ``io.BytesIO``.
Adding yet another type would make things more confusing for users, and
interoperability between libraries more painful (also potentially
sub-optimal, due to the necessary conversions).
Moreover, not one type would be needed, but two: one immutable type (to
allow for hashing), and one mutable type (as efficient accumulation is
often necessary when working with network messages).
Resolution
==========
This PEP is made obsolete by the `acceptance
<https://mail.python.org/pipermail/python-dev/2014-March/133621.html>`_
of :pep:`461`, which introduces a more extended formatting language for
bytes objects in conjunction with the modulo operator.
References
==========
* `Issue #3982: support .format for bytes
<http://bugs.python.org/issue3982>`_
* `Mercurial project
<http://mercurial.selenic.com/>`_
* `Twisted project
<http://twistedmatrix.com/trac/>`_
* `Documentation of Python 2 formatting (str % args)
<http://docs.python.org/2/library/stdtypes.html#string-formatting>`_
* `Documentation of Python 2 formatting (str.format)
<http://docs.python.org/2/library/string.html#formatstrings>`_
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: