python pandas的使用

总结:

1. Series是一维数据,代表了DateFrame的一行数据或一列数据。是有多个不同类型的标量构成的一行数据
2. DateFrame是二维数据,是有多行构成的,可以理解为一个表。
3. data.loc 函数用来通过表名取表中指定数据的,第一个参数是行名称,第二个参数是列名称
4. data.iloc 函数用来通过下标取表中指定数据的,第一个参数行下标,第二个参数是列下标

数据类型:

1. Series:带标签的一维同构数组

(1).用值列表生成Series对象,Pandas 默认自动生成整数索引:

s =  pd.Series([1, 3, 5, np.nan, 6, 8]) # 一列数据
>>> s:
0    1.0
1    3.0
2    5.0
3    NaN
4    6.0
5    8.0
dtype: float64



2. DataFrame:带标签的,大小可变的,二维异构表格

(1. )生成DataFrame对象

# DataFrame 表格数据,包含index(行)或 columns(列)
# 用这种方式迭代 DataFrame 的列
for colum in df.columns:
    series = df[colum]
    # do something with series

1. 用含日期时间索引与标签的 NumPy 数组生成DataFrame:
dates = pd.date_range('20130101', periods=6)
>>>> dates:
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
               '2013-01-05', '2013-01-06'],
              dtype='datetime64[ns]', freq='D')


df = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list('ABCD'))
>>>> df:
                   A         B         C         D
2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
2013-01-04  0.721555 -0.706771 -1.039575  0.271860
2013-01-05 -0.424972  0.567020  0.276232 -1.087401
2013-01-06 -0.673690  0.113648 -1.478427  0.524988

用 Series 字典对象生成 DataFrame:
df2 = pd.DataFrame({'A': 1.,
   ...:                     'B': pd.Timestamp('20130102'),
   ...:                     'C': pd.Series(1, index=list(range(4)), dtype='float32'),
   ...:                     'D': np.array([3] * 4, dtype='int32'),
   ...:                     'E': pd.Categorical(["test", "train", "test", "train"]),
   ...:                     'F': 'foo'})
>>>>: df2
     A          B    C  D      E    F
0  1.0 2013-01-02  1.0  3   test  foo
1  1.0 2013-01-02  1.0  3  train  foo
2  1.0 2013-01-02  1.0  3   test  foo
3  1.0 2013-01-02  1.0  3  train  foo
DataFrame 的列有不同数据类型:
>>>>:df2.dtypes
A           float64
B    datetime64[ns]
C           float32
D             int32
E          category
F            object
dtype: object

(2.)查看数据:

1. 查看 DataFrame 头部和尾部数据:

>>>> df.head()
                   A         B         C         D
2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
2013-01-04  0.721555 -0.706771 -1.039575  0.271860
2013-01-05 -0.424972  0.567020  0.276232 -1.087401

>>>> df.tail(3)
                   A         B         C         D
2013-01-04  0.721555 -0.706771 -1.039575  0.271860
2013-01-05 -0.424972  0.567020  0.276232 -1.087401
2013-01-06 -0.673690  0.113648 -1.478427  0.524988

2. 显示索引与列名:
>>> df.index
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
               '2013-01-05', '2013-01-06'],
              dtype='datetime64[ns]', freq='D')

>>>> df.columns
Index(['A', 'B', 'C', 'D'], dtype='object')

(3.) 转化为低层 NumPy 对象:输出不包含行索引和列标签

>>>> df.to_numpy()
array([[ 0.4691, -0.2829, -1.5091, -1.1356],
       [ 1.2121, -0.1732,  0.1192, -1.0442],
       [-0.8618, -2.1046, -0.4949,  1.0718],
       [ 0.7216, -0.7068, -1.0396,  0.2719],
       [-0.425 ,  0.567 ,  0.2762, -1.0874],
       [-0.6737,  0.1136, -1.4784,  0.525 ]])

>>>> df2.to_numpy()
array([[1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'test', 'foo'],
       [1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'train', 'foo'],
       [1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'test', 'foo'],
       [1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'train', 'foo']], dtype=object)

(3.)摘要:

// 快速查看数据的统计
>>>> df.describe()
              A         B         C         D
count  6.000000  6.000000  6.000000  6.000000
mean   0.073711 -0.431125 -0.687758 -0.233103
std    0.843157  0.922818  0.779887  0.973118
min   -0.861849 -2.104569 -1.509059 -1.135632
25%   -0.611510 -0.600794 -1.368714 -1.076610
50%    0.022070 -0.228039 -0.767252 -0.386188
75%    0.658444  0.041933 -0.034326  0.461706
max    1.212112  0.567020  0.276232  1.071804
// 转置数据:
>>>> df.T
   2013-01-01  2013-01-02  2013-01-03  2013-01-04  2013-01-05  2013-01-06
A    0.469112    1.212112   -0.861849    0.721555   -0.424972   -0.673690
B   -0.282863   -0.173215   -2.104569   -0.706771    0.567020    0.113648
C   -1.509059    0.119209   -0.494929   -1.039575    0.276232   -1.478427
D   -1.135632   -1.044236    1.071804    0.271860   -1.087401    0.524988
// 按轴排序:
>>>> df.sort_index(axis=1, ascending=False)
                   D         C         B         A
2013-01-01 -1.135632 -1.509059 -0.282863  0.469112
2013-01-02 -1.044236  0.119209 -0.173215  1.212112
2013-01-03  1.071804 -0.494929 -2.104569 -0.861849
2013-01-04  0.271860 -1.039575 -0.706771  0.721555
2013-01-05 -1.087401  0.276232  0.567020 -0.424972
2013-01-06  0.524988 -1.478427  0.113648 -0.673690
// 按值排序:
>>>> df.sort_values(by='B')
                   A         B         C         D
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
2013-01-04  0.721555 -0.706771 -1.039575  0.271860
2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-06 -0.673690  0.113648 -1.478427  0.524988
2013-01-05 -0.424972  0.567020  0.276232 -1.087401

(4.)数据访问:.at、.iat、.loc 和 .iloc

// 获取数据:
>>>> df['A']
2013-01-01    0.469112
2013-01-02    1.212112
2013-01-03   -0.861849
2013-01-04    0.721555
2013-01-05   -0.424972
2013-01-06   -0.673690

// 用 [ ] 切片行:
>>>> df[0:3]
                   A         B         C         D
2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
>>>> df['20130102':'20130104']
                   A         B         C         D
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
2013-01-04  0.721555 -0.706771 -1.039575  0.271860
// 用标签提取一行数据:返回了Series
>>>> df.loc[dates[0]]
A    0.469112
B   -0.282863
C   -1.509059
D   -1.135632
Name: 2013-01-01 00:00:00, dtype: float64

// 用标签选择多列数据:
>>>> df.loc[:, ['A', 'B']]
                   A         B
2013-01-01  0.469112 -0.282863
2013-01-02  1.212112 -0.173215
2013-01-03 -0.861849 -2.104569
2013-01-04  0.721555 -0.706771
2013-01-05 -0.424972  0.567020
2013-01-06 -0.673690  0.113648

// 标签切片,包含行与列结束点:
>>>> df.loc['20130102':'20130104', ['A', 'B']]
                   A         B
2013-01-02  1.212112 -0.173215
2013-01-03 -0.861849 -2.104569
2013-01-04  0.721555 -0.706771

// 对象降维: DataFrame返回了Series(一行数据)
>>>> df.loc['20130102', ['A', 'B']]
A    1.212112
B   -0.173215
Name: 2013-01-02 00:00:00, dtype: float64

// 提取标量值:返回一个横纵坐标下准确标量值
>>>> df.loc[dates[0], 'A']
0.46911229990718628
 # 快速访问标量,与上述方法等效:
>>>> df.at[dates[0], 'A']
0.46911229990718628

// 按位置选择:
>>>>  df.iloc[3]
A    0.721555
B   -0.706771
C   -1.039575
D    0.271860
Name: 2013-01-04 00:00:00, dtype: float64
>>>> df.iloc[3:5, 0:2] # 意思是返回index从3到5(不包含5) colum从0到2(不包含2)的DateFrame
                   A         B
2013-01-04  0.721555 -0.706771
2013-01-05 -0.424972  0.567020

>>>> df.iloc[[1, 2, 4], [0, 2]] # 意思是返回index = 1 and 2 and 4 colum = 0 and 2 的DateFrame
                   A         C
2013-01-02  1.212112  0.119209
2013-01-03 -0.861849 -0.494929
2013-01-05 -0.424972  0.276232

>>>> df.iloc[1:3, :] # 意思是返回index从1到3(不包含3)的所有列的DateFrame
                   A         B         C         D
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804

>>>> df.iloc[:, 1:3] # 意思是返回colum从1到3(不包含3)的所有行的DateFrame
                   B         C
2013-01-01 -0.282863 -1.509059
2013-01-02 -0.173215  0.119209
2013-01-03 -2.104569 -0.494929
2013-01-04 -0.706771 -1.039575
2013-01-05  0.567020  0.276232
2013-01-06  0.113648 -1.478427

// 提取标量值:返回一个横纵坐标下准确标量值
>>>> df.iloc[1, 1]
 -0.17321464905330858
# 快速访问标量,与上述方法等效:
 >>>> df.iat[1, 1]
-0.17321464905330858
布尔索引:
// 单列的值选择数据:
>>>> df[df.A > 0] # 筛选'A' 这一列大于0的数据然后返回DateFrame
                   A         B         C         D
2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
2013-01-02  1.212112 -0.173215  0.119209 -1.044236
2013-01-04  0.721555 -0.706771 -1.039575  0.271860

>>>> df[df > 0] # 筛选出标中的正数
                   A         B         C         D
2013-01-01  0.469112       NaN       NaN       NaN
2013-01-02  1.212112       NaN  0.119209       NaN
2013-01-03       NaN       NaN       NaN  1.071804
2013-01-04  0.721555       NaN       NaN  0.271860
2013-01-05       NaN  0.567020  0.276232       NaN
2013-01-06       NaN  0.113648       NaN  0.524988

// 用 isin筛选:
>>>> df2 = df.copy()
>>>> df2['E'] = ['one', 'one', 'two', 'three', 'four', 'three']
>>>> df2
                   A         B         C         D      E
2013-01-01  0.469112 -0.282863 -1.509059 -1.135632    one
2013-01-02  1.212112 -0.173215  0.119209 -1.044236    one
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804    two
2013-01-04  0.721555 -0.706771 -1.039575  0.271860  three
2013-01-05 -0.424972  0.567020  0.276232 -1.087401   four
2013-01-06 -0.673690  0.113648 -1.478427  0.524988  three
>>>> df2[df2['E'].isin(['two', 'four'])] # 筛选df2这个表中名为‘E’的这一列中值在['two', 'four']中的Series然后组成新的DataFrame返回
                   A         B         C         D     E
2013-01-03 -0.861849 -2.104569 -0.494929  1.071804   two
2013-01-05 -0.424972  0.567020  0.276232 -1.087401  four

(5.) 赋值

// 用位置赋值
>>>> s1 = pd.Series([1, 2, 3, 4, 5, 6], index=pd.date_range('20130102', periods=6)) # 创建一个标量为[1, 2, 3, 4, 5, 6],多行一列的一组数据,s1 就是一个新增列数据
2013-01-02    1
2013-01-03    2
2013-01-04    3
2013-01-05    4
2013-01-06    5
2013-01-07    6
Freq: D, dtype: int64
>>>> df['F'] = s1 # 意思是名为df的dataFrame表,新增一列,列名为'F'的数据。
                   A         B         C          D    F
2013-01-01 -0.861849 -2.104569 -1.509059  -1.135632  NaN
2013-01-02  1.212112 -0.173215  0.119209  -1.044236  1.0
2013-01-03 -0.861849 -2.104569 -0.494929   1.071804  2.0
2013-01-04  0.721555 -0.706771 -1.039575   0.271860  3.0
2013-01-05 -0.424972  0.567020  0.276232  -1.087401  4.0
2013-01-06 -0.673690  0.113648 -1.478427   0.524988  5.0
>>>> df.at[dates[0], 'A'] = 0 # 改变的是上面的  2013-01-01 'A' 对应的值-0.861849
>>>> df.iat[0, 1] = 0   # 改变的是上面的  2013-01-01 'B' 对应的值-2.104569
>>>> df.loc[:, 'D'] = np.array([5] * len(df)) # 按 NumPy 数组赋值,改变的是‘D’这一列, len(df)就是所有行,改成5
>>>> df
                   A         B         C  D    F
2013-01-01  0.000000  0.000000 -1.509059  5  NaN
2013-01-02  1.212112 -0.173215  0.119209  5  1.0
2013-01-03 -0.861849 -2.104569 -0.494929  5  2.0
2013-01-04  0.721555 -0.706771 -1.039575  5  3.0
2013-01-05 -0.424972  0.567020  0.276232  5  4.0
2013-01-06 -0.673690  0.113648 -1.478427  5  5.0

// 条件赋值:
>>>> df2 = df.copy()
>>>> df2[df2 > 0] = -df2 # df2中所有正数都变成负数
>>>> df2
                   A         B         C  D    F
2013-01-01  0.000000  0.000000 -1.509059 -5  NaN
2013-01-02 -1.212112 -0.173215 -0.119209 -5 -1.0
2013-01-03 -0.861849 -2.104569 -0.494929 -5 -2.0
2013-01-04 -0.721555 -0.706771 -1.039575 -5 -3.0
2013-01-05 -0.424972 -0.567020 -0.276232 -5 -4.0
2013-01-06 -0.673690 -0.113648 -1.478427 -5 -5.0

(6.)缺失值:np.nan 表示缺失数据

>>>> df1 = df.reindex(index=dates[0:4], columns=list(df.columns) + ['E'])
>>>> df1.loc[dates[0]:dates[1], 'E'] = 1
>>>> df1
                   A         B         C  D    F    E
2013-01-01  0.000000  0.000000 -1.509059  5  NaN  1.0
2013-01-02  1.212112 -0.173215  0.119209  5  1.0  1.0
2013-01-03 -0.861849 -2.104569 -0.494929  5  2.0  NaN
2013-01-04  0.721555 -0.706771 -1.039575  5  3.0  NaN
// 删除所有含缺失值的行:
>>>> df1.dropna(how='any')

                   A         B         C  D    F    E
2013-01-02  1.212112 -0.173215  0.119209  5  1.0  1.0

// 填充缺失值:
>>>> df1.fillna(value=5)
                   A         B         C  D    F    E
2013-01-01  0.000000  0.000000 -1.509059  5  5.0  1.0
2013-01-02  1.212112 -0.173215  0.119209  5  1.0  1.0
2013-01-03 -0.861849 -2.104569 -0.494929  5  2.0  5.0
2013-01-04  0.721555 -0.706771 -1.039575  5  3.0  5.0

// 提取 nan 值的布尔掩码:
>>>> pd.isna(df1)
                A      B      C      D      F      E
2013-01-01  False  False  False  False   True  False
2013-01-02  False  False  False  False  False  False
2013-01-03  False  False  False  False  False   True
2013-01-04  False  False  False  False  False   True

(7.)统计

// 描述性统计
>>>> df.mean() # 纵轴统计
A   -0.004474
B   -0.383981
C   -0.687758
D    5.000000
F    3.000000
dtype: float64
>>>> df.mean(1) #另一个轴(即,行)上执行同样的操作
2013-01-01    0.872735
2013-01-02    1.431621
2013-01-03    0.707731
2013-01-04    1.395042
2013-01-05    1.883656
2013-01-06    1.592306
Freq: D, dtype: float64

// 不同维度对象运算时,要先对齐, 此外,Pandas 自动沿指定维度广播。
>>>> s = pd.Series([1, 3, 5, np.nan, 6, 8], index=dates).shift(2) # shift(2): 偏移2行对齐
2013-01-01    NaN
2013-01-02    NaN
2013-01-03    1.0
2013-01-04    3.0
2013-01-05    5.0
2013-01-06    NaN
Freq: D, dtype: float64

>>>> df.sub(s, axis='index') # 啥意思是按列运算,NaN+ 标量 = NaN,按列为组运算,index相同的做和,返回一个新的dataframe,不会改变df,只是用df运算。
                   A         B         C    D    F
2013-01-01       NaN       NaN       NaN  NaN  NaN
2013-01-02       NaN       NaN       NaN  NaN  NaN
2013-01-03 -1.861849 -3.104569 -1.494929  4.0  1.0 # -2.861849+1 = -1.861849
2013-01-04 -2.278445 -3.706771 -4.039575  2.0  0.0
2013-01-05 -5.424972 -4.432980 -4.723768  0.0 -1.0
2013-01-06       NaN       NaN       NaN  NaN  NaN

// Apply 函数处理数据
>>>> df.apply(np.cumsum) # np.cumsum 是一个函数,列累加的函数,apply和.sub一样,返回一个新的dataframe,不会改变df,只是用df运算。
                   A         B         C   D     F
2013-01-01  0.000000  0.000000 -1.509059   5   NaN # 0.000000 =  df.iat[0,0]
2013-01-02  1.212112 -0.173215 -1.389850  10   1.0 # 1.212112 = df.iat[0,0] + df.iat[0,1]
2013-01-03  0.350263 -2.277784 -1.884779  15   3.0 # -0.350263 = df.iat[0,1] + df.iat[0,2]
2013-01-04  1.071818 -2.984555 -2.924354  20   6.0 # ...
2013-01-05  0.646846 -2.417535 -2.648122  25  10.0 # ...
2013-01-06 -0.026844 -2.303886 -4.126549  30  15.0 # ...
>>>> df.apply(lambda x: x.max() - x.min()) # 先找到df每列的最大值和最小值,然后做差,然后返回一个Series,列统计。
A    2.073961 #  1.212112 - (-0.861849) = 2.073961
B    2.671590 
C    1.785291 
D    0.000000 
F    4.000000
dtype: float64

(8.)直方图

>>>> test = np.random.randint(0, 7, size=10)
>>>> test
array([4, 2, 1, 2, 6, 4, 4, 6, 4, 4])

>>>>s = pd.Series(test) 
0    4
1    2
2    1
3    2
4    6
5    4
6    4
7    6
8    4
9    4
dtype: int64

>>>> s.value_counts() # 4 出现了5次 6 出现了2次 。。。。。
4    5
6    2
2    2
1    1
dtype: int64

(9.)字符串方法:Series 的 str 属性包含一组字符串处理功能

>>>> s = pd.Series(['A', 'B', 'C', 'Aaba', 'Baca', np.nan, 'CABA', 'dog', 'cat'])
>>>> s.str.lower()
0       a
1       b
2       c
3    aaba
4    baca
5     NaN
6    caba
7     dog
8     cat
dtype: object

(10.)合并

>>> test = np.random.randn(10, 4)
>>> test
array([[-0.81741352,  1.84903603,  0.49850415,  0.44634968],
       [-0.36911923, -0.62801459,  1.17384241,  0.10961811],
       [-2.02391694, -1.13128739,  0.23789725, -0.34718204],
       [ 0.48488279, -1.99957237, -0.11090257, -0.00836397],
       [ 1.12524972, -1.06405041,  0.23928546,  1.49336223],
       [-0.72348469, -0.47378583, -0.34307547, -0.35879956],
       [-1.34686762,  1.24929498,  0.66233861,  0.262106  ],
       [-0.38460465, -0.02032119, -1.16112247,  0.11717697],
       [-0.22309295, -0.14498124,  0.01653987, -0.92529958],
       [ 0.78572275,  1.85298325, -0.85724784,  0.08748715]])
>>> df = pd.DataFrame(test)
>>> df
          0         1         2         3
0 -0.817414  1.849036  0.498504  0.446350
1 -0.369119 -0.628015  1.173842  0.109618
2 -2.023917 -1.131287  0.237897 -0.347182
3  0.484883 -1.999572 -0.110903 -0.008364
4  1.125250 -1.064050  0.239285  1.493362
5 -0.723485 -0.473786 -0.343075 -0.358800
6 -1.346868  1.249295  0.662339  0.262106
7 -0.384605 -0.020321 -1.161122  0.117177
8 -0.223093 -0.144981  0.016540 -0.925300
9  0.785723  1.852983 -0.857248  0.087487

>>>> pieces = [df[:3], df[3:7], df[7:]]# 分解为多组,从第0行到第3行(不包括3),。。。pieces是dataframe数组
>>>> pd.concat(pieces)
          0         1         2         3
0 -0.817414  1.849036  0.498504  0.446350
1 -0.369119 -0.628015  1.173842  0.109618
2 -2.023917 -1.131287  0.237897 -0.347182
3  0.484883 -1.999572 -0.110903 -0.008364
4  1.125250 -1.064050  0.239285  1.493362
5 -0.723485 -0.473786 -0.343075 -0.358800
6 -1.346868  1.249295  0.662339  0.262106
7 -0.384605 -0.020321 -1.161122  0.117177
8 -0.223093 -0.144981  0.016540 -0.925300
9  0.785723  1.852983 -0.857248  0.087487

(11.)join连接 :SQL 风格的合并

>>>> left = pd.DataFrame({'key': ['foo', 'foo'], 'lval': [1, 2]})
>>>> right = pd.DataFrame({'key': ['foo', 'foo'], 'rval': [4, 5]})
>>> left
   key  lval
0  foo     1
1  foo     2
>>> right
   key  rval
0  foo     4
1  foo     5

>>>> pd.merge(left, right, on='key') # 14 15 24 25 
   key  lval  rval
0  foo     1     4
1  foo     1     5
2  foo     2     4
3  foo     2     5

>>>> left = pd.DataFrame({'key': ['foo', 'bar'], 'lval': [1, 2]})
>>>> right = pd.DataFrame({'key': ['foo', 'bar'], 'rval': [4, 5]})
>>>>  left
   key  lval
0  foo     1
1  bar     2
>>>> right
   key  rval
0  foo     4
1  bar     5
>>>> pd.merge(left, right, on='key')
   key  lval  rval
0  foo     1     4
1  bar     2     5

(12.)Append 追加:

>>> df = pd.DataFrame(np.random.randn(8, 4), columns=['A', 'B', 'C', 'D'])
>>> s = df.iloc[3] # index = 3 的一行数据,一个Series
>>> df.append(s, ignore_index=True) # 添加了一条数据(index = 8的那条),index = 3 和 index = 8 
          A         B         C         D
0  1.346061  1.511763  1.627081 -0.990582
1 -0.441652  1.211526  0.268520  0.024580
2 -1.577585  0.396823 -0.105381 -0.532532
3  1.453749  1.208843 -0.080952 -0.264610
4 -0.727965 -0.589346  0.339969 -0.693205
5 -0.339355  0.593616  0.884345  1.591431
6  0.141809  0.220390  0.435589  0.192451
7 -0.096701  0.803351  1.715071 -0.708758
8  1.453749  1.208843 -0.080952 -0.264610

(13.)分组(Grouping)

>>> df = pd.DataFrame({'A': ['foo', 'bar', 'foo', 'bar',
   ....:                          'foo', 'bar', 'foo', 'foo'],
   ....:                    'B': ['one', 'one', 'two', 'three',
   ....:                          'two', 'two', 'one', 'three'],
   ....:                    'C': np.random.randn(8),
   ....:                    'D': np.random.randn(8)})

     A      B         C         D
0  foo    one -1.202872 -0.055224
1  bar    one -1.814470  2.395985
2  foo    two  1.018601  1.552825
3  bar  three -0.595447  0.166599
4  foo    two  1.395433  0.047609
5  bar    two -0.392670 -0.136473
6  foo    one  0.007207 -0.561757
7  foo  three  1.928123 -1.623033

>>>> df.groupby('A').sum() # 根据'A'分组,然后用sum计算值,返回新的dataframe
            C        D
A                     
bar -2.802588  2.42611
foo  3.146492 -0.63958
>>>> df.groupby(['A', 'B']).sum() # 多列分组后,根据'A'and  'B' 分组,然后用sum计算值,生成多层索引,返回新的dataframe
                  C         D
A   B                        
bar one   -1.814470  2.395985
    three -0.595447  0.166599
    two   -0.392670 -0.136473
foo one   -1.195665 -0.616981
    three  1.928123 -1.623033
    two    2.414034  1.600434

(14.)重塑(Reshaping) + 数据透视表(Pivot Tables)+ 堆叠(Stack): 。。。不看了不看了,感觉用不上啊(https://www.pypandas.cn/docs/getting_started/10min.html#%E9%87%8D%E5%A1%91-reshaping

(16.) 时间序列:

>>> rng = pd.date_range('1/1/2012', periods=100, freq='S')
>>> ts = pd.Series(np.random.randint(0, 500, len(rng)), index=rng)
>>> ts.resample('5Min').sum() # 意思是把 将秒级的数据转换为 5 分钟为频率的数据,用sum()把数据合并
2012-01-01    25083
Freq: 5T, dtype: int64
>>> rng = pd.date_range('3/6/2012 00:00', periods=5, freq='D')
>>> ts = pd.Series(np.random.randn(len(rng)), rng)
>>> ts
2012-03-06    0.464000
2012-03-07    0.227371
2012-03-08   -0.496922
2012-03-09    0.306389
2012-03-10   -2.290613
>>>ts_utc = ts.tz_localize('UTC')
>>>ts_utc
2012-03-06 00:00:00+00:00    0.464000
2012-03-07 00:00:00+00:00    0.227371
2012-03-08 00:00:00+00:00   -0.496922
2012-03-09 00:00:00+00:00    0.306389
2012-03-10 00:00:00+00:00   -2.290613
Freq: D, dtype: float64
// 转换成其它时区
>>>ts_utc.tz_convert('US/Eastern')
2012-03-05 19:00:00-05:00    0.464000
2012-03-06 19:00:00-05:00    0.227371
2012-03-07 19:00:00-05:00   -0.496922
2012-03-08 19:00:00-05:00    0.306389
2012-03-09 19:00:00-05:00   -2.290613
// 转换时间段:
>>>rng = pd.date_range('1/1/2012', periods=5, freq='M')
>>>ts = pd.Series(np.random.randn(len(rng)), index=rng)
>>>ts
2012-01-31   -1.134623
2012-02-29   -1.561819
2012-03-31   -0.260838
2012-04-30    0.281957
2012-05-31    1.523962
Freq: M, dtype: float64
>>>ps = ts.to_period()
2012-01   -1.134623
2012-02   -1.561819
2012-03   -0.260838
2012-04    0.281957
2012-05    1.523962
>>>ps.to_timestamp()
2012-01-01   -1.134623
2012-02-01   -1.561819
2012-03-01   -0.260838
2012-04-01    0.281957
2012-05-01    1.523962
eg:
// Pandas 函数可以很方便地转换时间段与时间戳。下例把以 11 月为结束年份的季度频率转换为下一季度月末上午 9 点:
>>> prng = pd.period_range('1990Q1', '2000Q4', freq='Q-NOV')
>>> ts = pd.Series(np.random.randn(len(prng)), prng)
>>> ts.index = (prng.asfreq('M', 'e') + 1).asfreq('H', 's') + 9

>>> ts.head()
1990-03-01 09:00   -0.902937
1990-06-01 09:00    0.068159
1990-09-01 09:00   -0.057873
1990-12-01 09:00   -0.368204
1991-03-01 09:00   -1.144073
Freq: H, dtype: float64

(17.)类别型(Categoricals) + 可视化 + HDF5+Excel 都先不看了 用到在说

(18.)数据输入 / 输出

 // 写入csv文件
>>> df.to_csv('foo.csv')

// 读取 CSV 文件数据
>>> pd.read_csv('foo.csv')

✨✨✨✨:DataFrame 是 Series 的容器,Series 则是标量的容器

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 158,847评论 4 362
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 67,208评论 1 292
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 108,587评论 0 243
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 43,942评论 0 205
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 52,332评论 3 287
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 40,587评论 1 218
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 31,853评论 2 312
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 30,568评论 0 198
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 34,273评论 1 242
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 30,542评论 2 246
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 32,033评论 1 260
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 28,373评论 2 253
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 33,031评论 3 236
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 26,073评论 0 8
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 26,830评论 0 195
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 35,628评论 2 274
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 35,537评论 2 269

推荐阅读更多精彩内容