This PEP describes a proposal to add new operators to Python which are useful for distinguishing elementwise and objectwise operations, and summarizes discussions in the news group comp.lang.python on this topic. See Credits and Archives section at end. Issues discussed here include:
A substantial portion of this PEP describes ideas that do not go into the proposed extension. They are presented because the extension is essentially syntactic sugar, so its adoption must be weighed against various possible alternatives. While many alternatives may be better in some aspects, the current proposal appears to be overall advantageous.
The issues concerning elementwise-objectwise operations extends to wider areas than numerical computation. This document also describes how the current proposal may be integrated with more general future extensions.
BackgroundPython provides six binary infix math operators: + - * / % ** hereafter generically represented by op. They can be overloaded with new semantics for user-defined classes. However, for objects composed of homogeneous elements, such as arrays, vectors and matrices in numerical computation, there are two essentially distinct flavors of semantics. The objectwise operations treat these objects as points in multidimensional spaces. The elementwise operations treat them as collections of individual elements. These two flavors of operations are often intermixed in the same formulas, thereby requiring syntactical distinction.
Many numerical computation languages provide two sets of math operators. For example, in MatLab, the ordinary op is used for objectwise operation while .op is used for elementwise operation. In R, op stands for elementwise operation while %op% stands for objectwise operation.
In Python, there are other methods of representation, some of which already used by available numerical packages, such as:
In several aspects these are not as adequate as infix operators. More details will be shown later, but the key points are:
While it is possible to assign current math operators to one flavor of semantics, there is simply not enough infix operators to overload for the other flavor. It is also impossible to maintain visual symmetry between these two flavors if one of them does not contain symbols for ordinary math operators.
Proposed extension~+ ~- ~* ~/ ~% ~** are added to core Python. They parallel the existing operators + - * / % **.~+= ~-= ~*= ~/= ~%= ~**= are added to core Python. They parallel the operators += -= *= /= %= **= available in Python 2.0.~op retains the syntactical properties of operator op, including precedence.~op retains the semantical properties of operator op on built-in number types.~op raise syntax error on non-number builtin types. This is temporary until the proper behavior can be agreed upon.__tadd__ and __rtadd__ work for ~+ just as __add__ and __radd__ work for +.__r*__() methods are invoked when the left operand does not provide the appropriate method.It is intended that one set of op or ~op is used for elementwise operations, the other for objectwise operations, but it is not specified which version of operators stands for elementwise or objectwise operations, leaving the decision to applications.
The proposed implementation is to patch several files relating to the tokenizer, parser, grammar and compiler to duplicate the functionality of corresponding existing operators as necessary. All new semantics are to be implemented in the classes that overload them.
The symbol ~ is already used in Python as the unary bitwise not operator. Currently it is not allowed for binary operators. The new operators are completely backward compatible.
Greg Lielens implemented the infix ~op as a patch against Python 2.0b1 source [1].
To allow ~ to be part of binary operators, the tokenizer would treat ~+ as one token. This means that currently valid expression ~+1 would be tokenized as ~+ 1 instead of ~ + 1. The parser would then treat ~+ as composite of ~ +. The effect is invisible to applications.
Notes about current patch:
~op= operators yet.~op behaves the same as op on lists, instead of raising exceptions.These should be fixed when the final version of this proposal is ready.
xor as an infix operator with the semantics equivalent to:
def __xor__(a, b):
if not b: return a
elif not a: return b
else: 0
This preserves true value as much as possible, otherwise preserve left hand side value if possible.
This is done so that bitwise operators could be regarded as elementwise logical operators in the future (see below).
Alternatives to adding new operatorsThe discussions on comp.lang.python and python-dev mailing list explored many alternatives. Some of the leading alternatives are listed here, using the multiplication operator as an example.
mul(a,b).
Advantage:
Disadvantage:
a.mul(b).
Advantage:
Disadvantage:
.E, so that for matrices a and b, a.E*b would be a matrix object that is elementwise_mul(a,b).
Likewise define a shadow matrix class for arrays accessible through a method .M so that for arrays a and b, a.M*b would be an array that is matrixwise_mul(a,b).
Advantage:
Disadvantage:
a.E*b will fail if a is a pure number..T for transpose, etc.M class with shadow E class, and an E class with shadow M class.a.M*b.M and elementwise operations for matrices would be like a.E*b.E. For error detection a.E*b.M would raise exceptions.
Advantage:
Disadvantage:
Advantage:
Disadvantage:
@, for matrix multiplication.
Advantage:
Disadvantage:
+ - ** are equally important. Their meaning in matrix or array-oriented packages would be reversed (see below).Among these alternatives, the first and second are used in current applications to some extent, but found inadequate. The third is the most favorite for applications, but it will incur huge implementation complexity. The fourth would make applications codes very context-sensitive and hard to maintain. These two alternatives also share significant implementational difficulties due to current type/class split. The fifth appears to create more problems than it would solve. The sixth does not cover the same range of applications.
Alternative forms of infix operatorsTwo major forms and several minor variants of new infix operators were discussed:
(op)
[op]
{op}
<op>
:op:
~op~
%op%
Alternatively the meta character is put after the operator.
@/ and /@ for left and right division[*] and (*) for outer and inner products@ for multiplication.__call__ to simulate multiplication:Criteria for choosing among the representations include:
With these criteria the overall winner in bracket form appear to be {op}. A clear winner in the meta character form is ~op. Comparing these it appears that ~op is the favorite among them all.
Some analysis are as follows:
.op form is ambiguous: 1.+a would be different from 1 .+a.(op) and [op], and somewhat less so for {op} and <op>.<op> form has the potential to be confused with < > and =.@op is not favored because @ is visually heavy (dense, more like a letter): a@+b is more readily read as a@ + b than a @+ b.@ $ ?.There are convincing arguments for using either set of operators as objectwise or elementwise. Some of them are listed here:
op for element, ~op for object
op for object, ~op for element
~ to be a general elementwise meta-character in future extensions.It is generally agreed upon that
Therefore, not much is lost, and much flexibility retained, if the semantic flavors of these two sets of operators are not dictated by the core language. The application packages are responsible for making the most suitable choice. This is already the case for NumPy and MatPy which use opposite semantics. Adding new operators will not break this. See also observation after subsection 2 in the Examples below.
The issue of numerical precision was raised, but if the semantics is left to the applications, the actual precisions should also go there.
ExamplesFollowing are examples of the actual formulas that will appear using various operators or other representations described above.
op for object and ~op for element:
b = a.I - a.I * u / (c.I + v/a*u) * v / a b = a.I - a.I * u * (c.I + v*a.I*u).I * v * a.I
op for element and ~op for object:
b = a.I @- a.I @* u @/ (c.I @+ v@/a@*u) @* v @/ a
b = a.I ~- a.I ~* u ~/ (c.I ~+ v~/a~*u) ~* v ~/ a
b = a.I (-) a.I (*) u (/) (c.I (+) v(/)a(*)u) (*) v (/) a
b = a.I [-] a.I [*] u [/] (c.I [+] v[/]a[*]u) [*] v [/] a
b = a.I <-> a.I <*> u </> (c.I <+> v</>a<*>u) <*> v </> a
b = a.I {-} a.I {*} u {/} (c.I {+} v{/}a{*}u) {*} v {/} a
Observation: For linear algebra using op for object is preferable.
Observation: The ~op type operators look better than (op) type in complicated formulas.
b = a.I @sub a.I @mul u @div (c.I @add v @div a @mul u) @mul v @div a b = a.I ~sub a.I ~mul u ~div (c.I ~add v ~div a ~mul u) ~mul v ~div a
Observation: Named operators are not suitable for math formulas.
op for object and ~op for element:
z = sin(x~**2 ~+ y~**2); plot(x,y,z)
z = sin(x**2 + y**2); plot(x,y,z)
Observation: Elementwise operations with broadcasting allows much more efficient implementation than MatLab.
Observation: It is useful to have two related classes with the semantics of op and ~op swapped. Using these the ~op operators would only need to appear in chunks of code where the other flavor dominates, while maintaining consistent semantics of the code.
+ and - with automatic broadcasting:
Observation: This would silently produce hard-to-trace bugs if one of b or c is row vector while the other is column vector.
~+ ~- operators. The objectwise + - are important because they provide important sanity checks as per linear algebra. The elementwise + - are important because they allow broadcasting that are very efficient in applications.a*x is not necessarily equal to x*a. The solution of a*x==b, denoted x=solve(a,b), is therefore different from the solution of x*a==b, denoted x=div(b,a). There are discussions about finding a new symbol for solve. [Background: MatLab use b/a for div(b,a) and a\b for solve(a,b).]
It is recognized that Python provides a better solution without requiring a new symbol: the inverse method .I can be made to be delayed so that a.I*b and b*a.I are equivalent to Matlab’s a\b and b/a. The implementation is quite simple and the resulting application code clean.
a**b as pow(a,b) has two perceived disadvantages:
a^b for this purpose.~**=.However, this issue is distinct from the main issue here.
The latter (the Einstein notation) is used extensively on paper, and is also the easier one to implement. By implementing a tensor-with-indices class, a general form of multiplication would cover both outer and inner products, and specialize to linear algebra multiplication as well. The index rule can be defined as class methods, like:
a = b.i(1,2,-1,-2) * c.i(4,-2,3,-1) # a_ijkl = b_ijmn c_lnkm
Therefore, one objectwise multiplication is sufficient.
^ might be better used for pow than bitwise xor. But this depends on the future of bitwise operators. It does not immediately impact on the proposed math operator.| was suggested to be used for matrix solve. But the new solution of using delayed .I is better in several ways.def "+"(a, b) in place of def __add__(a, b)
This appears to require greater syntactical change, and would only be useful when arbitrary additional operators are allowed.
The distinction between objectwise and elementwise operations are meaningful in other contexts as well, where an object can be conceptually regarded as a collection of elements. It is important that the current proposal does not preclude possible future extensions.
One general future extension is to use ~ as a meta operator to elementize a given operator. Several examples are listed here:
&), or (|), xor (^), complement (~), left shift (<<) and right shift (>>), with their own precedence levels.
Among them, the & | ^ ~ operators can be regarded as elementwise versions of lattice operators applied to integers regarded as bit strings.:
5 and 6 # 6 5 or 6 # 5 5 ~and 6 # 4 5 ~or 6 # 7
These can be regarded as general elementwise lattice operators, not restricted to bits in integers.
In order to have named operators for xor ~xor, it is necessary to make xor a reserved word.
[1, 2] + [3, 4] # [1, 2, 3, 4] [1, 2] ~+ [3, 4] # [4, 6] ['a', 'b'] * 2 # ['a', 'b', 'a', 'b'] 'ab' * 2 # 'abab' ['a', 'b'] ~* 2 # ['aa', 'bb'] [1, 2] ~* 2 # [2, 4]
It is also consistent to Cartesian product:
[1,2]*[3,4] # [(1,3),(1,4),(2,3),(2,4)]
a = [1, 2]; b = [3, 4] ~f(a,b) # [f(x,y) for x, y in zip(a,b)] ~f(a*b) # [f(x,y) for x in a for y in b] a ~+ b # [x + y for x, y in zip(a,b)]
[1, 2, 3], [4, 5, 6] # ([1,2, 3], [4, 5, 6]) [1, 2, 3]~,[4, 5, 6] # [(1,4), (2, 5), (3,6)]
~ as generic elementwise meta-character to replace map:
~f(a, b) # map(f, a, b) ~~f(a, b) # map(lambda *x:map(f, *x), a, b)
More generally,:
def ~f(*x): return map(f, *x) def ~~f(*x): return map(~f, *x) ...
a = [1,2,3,4,5] print ["%5d "] ~% a a = [[1,2],[3,4]] print ["%5d "] ~~% a
[1, 2, 3] ~< [3, 2, 1] # [1, 0, 0] [1, 2, 3] ~== [3, 2, 1] # [0, 1, 0]
[a, b, c, d] ~[2, 3, 1] # [c, d, b]
a = (1,2); b = (3,4) f(~a, ~b) # f(1,2,3,4)
NotesThere can be specific levels of deep copy:
Note that this section describes a type of future extensions that is consistent with current proposal, but may present additional compatibility or other problems. They are not tied to the current proposal.
Impact on named operatorsThe discussions made it generally clear that infix operators is a scarce resource in Python, not only in numerical computation, but in other fields as well. Several proposals and ideas were put forward that would allow infix operators be introduced in ways similar to named functions. We show here that the current extension does not negatively impact on future extensions in this regard.
Choose a meta character, say @, so that for any identifier opname, the combination @opname would be a binary infix operator, and:
a @opname b == opname(a,b)
Other representations mentioned include:
.name ~name~ :name: (.name) %name%
and similar variations. The pure bracket based operators cannot be used this way.
This requires a change in the parser to recognize @opname, and parse it into the same structure as a function call. The precedence of all these operators would have to be fixed at one level, so the implementation would be different from additional math operators which keep the precedence of existing math operators.
The current proposed extension do not limit possible future extensions of such form in any way.
One additional form of future extension is to use meta character and operator symbols (symbols that cannot be used in syntactical structures other than operators). Suppose @ is the meta character. Then:
a + b, a @+ b, a @@+ b, a @+- b
would all be operators with a hierarchy of precedence, defined by:
def "+"(a, b) def "@+"(a, b) def "@@+"(a, b) def "@+-"(a, b)
One advantage compared with named operators is greater flexibility for precedences based on either the meta character or the ordinary operator symbols. This also allows operator composition. The disadvantage is that they are more like line noise. In any case the current proposal does not impact its future possibility.
These kinds of future extensions may not be necessary when Unicode becomes generally available.
Note that this section discusses compatibility of the proposed extension with possible future extensions. The desirability or compatibility of these other extensions themselves are specifically not considered here.
The discussions mostly happened in July to August of 2000 on news group comp.lang.python and the mailing list python-dev. There are altogether several hundred postings, most can be retrieved from these two pages (and searching word “operator”):
The names of contributors are too numerous to mention here, suffice to say that a large proportion of ideas discussed here are not our own.
Several key postings (from our point of view) that may help to navigate the discussions include:
These are earlier drafts of this PEP:
There is an alternative PEP (officially, PEP 211) by Greg Wilson, titled “Adding New Linear Algebra Operators to Python”.
Its first (and current) version is at:
Additional References
Source: https://github.com/python/peps/blob/main/peps/pep-0225.rst
Last modified: 2024-04-14 20:08:31 GMT
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4