Playing the kaggle beer review datasets
https://www.kaggle.com/rdoume/beerreviews
df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 1504037 entries, 1586613 to 39648
Data columns (total 13 columns):
brewery_id 1504037 non-null int64
brewery_name 1504037 non-null object
review_time 1504037 non-null int64
review_overall 1504037 non-null float64
review_aroma 1504037 non-null float64
review_appearance 1504037 non-null float64
review_profilename 1504037 non-null object
beer_style 1504037 non-null object
review_palate 1504037 non-null float64
review_taste 1504037 non-null float64
beer_name 1504037 non-null object
beer_abv 1504037 non-null float64
beer_beerid 1504037 non-null int64
dtypes: float64(6), int64(3), object(4)
memory usage: 160.6+ MB
I just did a pivot table and returns the following results
review_stat_by_beer = df[['beer_name','review_overall','review_aroma','review_appearance','review_palate','review_taste']]\
.drop_duplicates(['beer_name'])\
.pivot_table(index="beer_name", aggfunc=("count",'mean','median'))
review_stat_by_beer.info()
<class 'pandas.core.frame.DataFrame'>
Index: 44075 entries, ! (Old Ale) to 葉山ビール (Hayama Beer)
Data columns (total 15 columns):
(review_appearance, count) 44075 non-null int64
(review_appearance, mean) 44075 non-null float64
(review_appearance, median) 44075 non-null float64
(review_aroma, count) 44075 non-null int64
(review_aroma, mean) 44075 non-null float64
(review_aroma, median) 44075 non-null float64
(review_overall, count) 44075 non-null int64
(review_overall, mean) 44075 non-null float64
(review_overall, median) 44075 non-null float64
(review_palate, count) 44075 non-null int64
(review_palate, mean) 44075 non-null float64
(review_palate, median) 44075 non-null float64
(review_taste, count) 44075 non-null int64
(review_taste, mean) 44075 non-null float64
(review_taste, median) 44075 non-null float64
dtypes: float64(10), int64(5)
memory usage: 5.4+ MB
Trying to choose these columns
review_stat_by_beer.(review_appearance, count) # SyntaxError: invalid syntax
review_stat_by_beer[(review_appearance, count)] #NameError: name 'review_appearance' is not defined
review_stat_by_beer['(review_appearance, count)'] #KeyError: '(review_appearance, count)'
how do I select these pivot table results? My ultimate goal is to do the math between 2 columns:
(review_overall, mean) minus (review_taste, mean)
Any thoughts? Thanks!
There are a few options for selecting a specific result from a multiIndex:
# Setup
df = pd.DataFrame(np.arange(9).reshape(3, 3))
df.columns = [['A', 'A', 'B'], ['a', 'b', 'c']]
df
A B
a b c
0 0 1 2
1 3 4 5
2 6 7 8
Direct selection,
df[('A', 'a')]
0 0
1 3
2 6
Name: (A, a), dtype: int64
Via loc,
df.loc[:, ('A', 'a')]
# or
# df.loc(axis=1)[('A', 'a')]
0 0
1 3
2 6
Name: (A, a), dtype: int64
And also with xs,
df.xs(('A', 'a'), axis=1)
0 0
1 3
2 6
Name: (A, a), dtype: int64
The idea in all these cases is to pass a tuple of strings which signifies the first and second levels respectively (your column index has 2 levels). In your case that would look like
review_stat_by_beer[('review_appearance', 'count')]
There are more methods, but these are the best ones.