Currently the following ops don't support bfloat16, according to OpInfo metadata (linalg and fft ops are filtered out):
acos
angle
asin
atan
atan2
cdist (cdist will have too much of a numerical error with bfloat16)
ceil
complex (bfloat16 not supported as base type for complex)
cosh
cummax
cummin
cumprod
cumsum
diag
digamma (no special functions support for bfloat16)
dot (needed for full matmul support)
erfc
erfinv precision too low
exp2
expm1
floor
frac
frexp
hypot
lerp
lgamma
logaddexp
logaddexp2
logcumsumexp
mvlgamma
nan_to_num
nanquantile
polar no complex support
polygamma no special function support
quantile
roll
round
rsqrt
sin
sinc
sinh
sort
tan
trace linalg in general not supported, but trace is easy
trunc
vdot no complex support
view_as_complex no complex support
That's 47 ops out of 172 non-fft non-linalg ops. We should add bfloat16 support to at least some of them. Most important operations are bold, least important (or don't make sense for bfloat16, e.g. complex, because bfloat16 is not supported for complex) are crossed out.
Script to do most of the filtering:
import torch
from torch.testing._internal.common_methods_invocations import op_db
from torch.testing._internal.common_device_type import skipCUDAIfNoMagma
names = set()
names_no_bf16 = set()
for o in op_db:
if ('linalg' not in o.name and 'fft' not in o.name):
if (o.decorators is None or skipCUDAIfNoMagma not in o.decorators):
names.add(o.name)
if not torch.bfloat16 in o.dtypesIfCUDA and torch.float in o.dtypesIfCUDA:
names_no_bf16.add(o.name)
for n in sorted(names_no_bf16):
print(n)
print(len(names), len(names_no_bf16))
see #54794 for round, rsqrt and exp2
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4