PyPackage01---Pandas05_分组统计groupby

PyPackage01---Pandas05_分组统计groupby

groupby是使用频率很高的，这里介绍三种用法：

常规的统计把groupby的变量从index变为一列groupby的一些复杂用法，主要用apply方法

pandas版本

import pandas as pdpd.__version__

'0.23.4'

数据构造

label = [0,0,1,1,0,1,0,1,0,1]sex = ["male","female","male","female","male","female","male","female","male","female"]age = [23,26,21,30,56,45,25,37,48,33]province = ["江苏","湖北","河南","山东","江苏","湖北","河南","湖北","河南","山东"]rawDf = pd.DataFrame({"sex":sex,"age":age,"province":province,"label":label})

帮助文档

help(rawDf.groupby)

Help on method groupby in module pandas.core.generic:groupby(by=None, axis=0, level=None, as_index=True, sort=True, group_keys=True, squeeze=False, observed=False, **kwargs) method of pandas.core.frame.DataFrame instance Group series using mapper (dict or key function, apply given function to group, return result as series) or by a series of columns. Parameters ---------- by : mapping, function, label, or list of labels Used to determine the groups for the groupby. If ``by`` is a function, it's called on each value of the object's index. If a dict or Series is passed, the Series or dict VALUES will be used to determine the groups (the Series' values are first aligned; see ``.align()`` method). If an ndarray is passed, the values are used as-is determine the groups. A label or list of labels may be passed to group by the columns in ``self``. Notice that a tuple is interpreted a (single) key. axis : int, default 0 level : int, level name, or sequence of such, default None If the axis is a MultiIndex (hierarchical), group by a particular level or levels as_index : boolean, default True For aggregated output, return object with group labels as the index. Only relevant for DataFrame input. as_index=False is effectively "SQL-style" grouped output sort : boolean, default True Sort group keys. Get better performance by turning this off. Note this does not influence the order of observations within each group. groupby preserves the order of rows within each group. group_keys : boolean, default True When calling apply, add group keys to index to identify pieces squeeze : boolean, default False reduce the dimensionality of the return type if possible, otherwise return a consistent type observed : boolean, default False This only applies if any of the groupers are Categoricals If True: only show observed values for categorical groupers. If False: show all values for categorical groupers. .. versionadded:: 0.23.0 Returns ------- GroupBy object Examples -------- DataFrame results >>> data.groupby(func, axis=0).mean() >>> data.groupby(['col1', 'col2'])['col3'].mean() DataFrame with hierarchical index >>> data.groupby(['col1', 'col2']).mean() Notes ----- See the `user guide

返回单个汇总统计值

rawDf.groupby(by="sex").count()

直接groupby+统计函数，有两个比较麻烦的地方：

groupbykey变成index了会输出所有非groupbykey列的统计数据

对于第一个问题，可通过设置参数​​as_index=False​​处理，此时sex为第一列

rawDf.groupby(by="sex",as_index=False).count()

如果指向输出指定列的某些统计结果，可以通过列名限制

rawDf.groupby(by="sex")["province","label"].count()

返回多个统计值

在SQL中，我们会统计多个汇总值，比如：

SELECT sex,COUNT(province),sum(label)FROM tabledemoGROUP BY

pandas可以借用apply方法实现。

rawDf.groupby(by="sex").apply(lambda x:pd.DataFrame({"x1":[x.province.count()],"x2":[x.label.sum()]}))

当然也可以通过辅助函数的形式传入

def aggF(subdf): provinceCnt = subdf.province.count() provinceDnt = subdf.province.nunique() labelSum = subdf.label.sum() return pd.DataFrame({ "provinceCnt": [provinceCnt], "provinceDnt": [provinceDnt], "labelSum": [labelSum] })rawDf.groupby(by="sex",as_index=False).apply(lambda x:aggF(subdf=x))

def aggF(subdf): provinceCnt = subdf.province.count() provinceDnt = subdf.province.nunique() labelSum = subdf.label.sum() return pd.DataFrame({ "provinceCnt": provinceCnt, "provinceDnt": provinceDnt, "labelSum": labelSum },index=[0])rawDf.groupby(by="sex",as_index=True).apply(lambda x:aggF(subdf=x))

这里有坑，groupby+apply之后，groupbykey不能作为其中一列了，目前只知道用for循环解决，比较笨，但是凑合用吧

for循环

def defaultFunction(subDf, groupIndex): badCnt = subDf.query("label==1").shape[0] goodCnt = subDf.query("label==0").shape[0] totalCnt = badCnt + goodCnt badRate = badCnt / totalCnt return pd.DataFrame({ "groupIndex": [groupIndex], "badCnt": [badCnt], "goodCnt": [goodCnt], "totalCnt": [totalCnt], "badRate": [badRate] })def groupFunction(Df,groupKey): result = pd.DataFrame() for name,group in Df.groupby(by=[groupKey]): result = pd.concat([result,defaultFunction(group,name)]) result.index = range(len(result)) return

groupFunction(rawDf,"sex")

groupFunction(rawDf,"province")

2020-02-29 于南京市栖霞区

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。