When grouping by several levels of a MultiIndex, groupby evaltuates all possible combinations of the groupby keys. When grouping by column name, it only evaluates what exist in the DataFrame. Also, this behavior does not exist in 0.14.1, but does in all final releases from 0.15.0 on.
This may be a new feature, not a bug, but I couldn't find anything in the docs, open or closed issues, etc. (closest was Issue #8138). If this is the intended behavior, it would be nice to have in the docs.
import pandas as pd import numpy as np df = pd.DataFrame(np.arange(12).reshape(-1, 3)) df.index = pd.MultiIndex.from_tuples([(1, 1), (1, 2), (3, 4), (5, 6)]) idx_names = ['x', 'y'] df.index.names = idx_names # Adds nan's for (x, y) combinations that aren't in the data by_levels = df.groupby(level=idx_names).mean() # This does not add missing combinations of the groupby keys by_columns = df.reset_index().groupby(idx_names).mean() print by_levels print by_columns # This passes in 0.14.1, but not >=0.15.0 final assert by_levels.equals(by_columns)INSTALLED VERSIONS
commit: None
python: 2.7.7.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
pandas: 0.15.2
nose: 1.3.4
Cython: 0.20.1
numpy: 1.9.1
scipy: 0.15.1
statsmodels: 0.7.0.dev-161a0f8
IPython: 2.3.0
sphinx: 1.2.2
patsy: 0.3.0
dateutil: 1.5
pytz: 2014.9
bottleneck: 0.8.0
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.4.2
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.5
lxml: 3.3.5
bs4: 4.3.1
html5lib: None
httplib2: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.4
pymysql: None
psycopg2: None
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4