Posted in Python onJune 18, 2020
计算信息熵的公式:n是类别数,p(xi)是第i类的概率
假设数据集有m行,即m个样本,每一行最后一列为该样本的标签,计算数据集信息熵的代码如下:
from math import log def calcShannonEnt(dataSet): numEntries = len(dataSet) # 样本数 labelCounts = {} # 该数据集每个类别的频数 for featVec in dataSet: # 对每一行样本 currentLabel = featVec[-1] # 该样本的标签 if currentLabel not in labelCounts.keys(): labelCounts[currentLabel] = 0 labelCounts[currentLabel] += 1 shannonEnt = 0.0 for key in labelCounts: prob = float(labelCounts[key])/numEntries # 计算p(xi) shannonEnt -= prob * log(prob, 2) # log base 2 return shannonEnt
补充知识:python 实现信息熵、条件熵、信息增益、基尼系数
我就废话不多说了,大家还是直接看代码吧~
import pandas as pd import numpy as np import math ## 计算信息熵 def getEntropy(s): # 找到各个不同取值出现的次数 if not isinstance(s, pd.core.series.Series): s = pd.Series(s) prt_ary = pd.groupby(s , by = s).count().values / float(len(s)) return -(np.log2(prt_ary) * prt_ary).sum() ## 计算条件熵: 条件s1下s2的条件熵 def getCondEntropy(s1 , s2): d = dict() for i in list(range(len(s1))): d[s1[i]] = d.get(s1[i] , []) + [s2[i]] return sum([getEntropy(d[k]) * len(d[k]) / float(len(s1)) for k in d]) ## 计算信息增益 def getEntropyGain(s1, s2): return getEntropy(s2) - getCondEntropy(s1, s2) ## 计算增益率 def getEntropyGainRadio(s1, s2): return getEntropyGain(s1, s2) / getEntropy(s2) ## 衡量离散值的相关性 import math def getDiscreteCorr(s1, s2): return getEntropyGain(s1,s2) / math.sqrt(getEntropy(s1) * getEntropy(s2)) # ######## 计算概率平方和 def getProbSS(s): if not isinstance(s, pd.core.series.Series): s = pd.Series(s) prt_ary = pd.groupby(s, by = s).count().values / float(len(s)) return sum(prt_ary ** 2) ######## 计算基尼系数 def getGini(s1, s2): d = dict() for i in list(range(len(s1))): d[s1[i]] = d.get(s1[i] , []) + [s2[i]] return 1-sum([getProbSS(d[k]) * len(d[k]) / float(len(s1)) for k in d]) ## 对离散型变量计算相关系数,并画出热力图, 返回相关性矩阵 def DiscreteCorr(C_data): ## 对离散型变量(C_data)进行相关系数的计算 C_data_column_names = C_data.columns.tolist() ## 存储C_data相关系数的矩阵 import numpy as np dp_corr_mat = np.zeros([len(C_data_column_names) , len(C_data_column_names)]) for i in range(len(C_data_column_names)): for j in range(len(C_data_column_names)): # 计算两个属性之间的相关系数 temp_corr = getDiscreteCorr(C_data.iloc[:,i] , C_data.iloc[:,j]) dp_corr_mat[i][j] = temp_corr # 画出相关系数图 fig = plt.figure() fig.add_subplot(2,2,1) sns.heatmap(dp_corr_mat ,vmin= - 1, vmax= 1, cmap= sns.color_palette('RdBu' , n_colors= 128) , xticklabels= C_data_column_names , yticklabels= C_data_column_names) return pd.DataFrame(dp_corr_mat) if __name__ == "__main__": s1 = pd.Series(['X1' , 'X1' , 'X2' , 'X2' , 'X2' , 'X2']) s2 = pd.Series(['Y1' , 'Y1' , 'Y1' , 'Y2' , 'Y2' , 'Y2']) print('CondEntropy:',getCondEntropy(s1, s2)) print('EntropyGain:' , getEntropyGain(s1, s2)) print('EntropyGainRadio' , getEntropyGainRadio(s1 , s2)) print('DiscreteCorr:' , getDiscreteCorr(s1, s1)) print('Gini' , getGini(s1, s2))
以上这篇Python计算信息熵实例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。
Python计算信息熵实例
- Author -
Mr.Jcak声明:登载此文出于传递更多信息之目的,并不意味着赞同其观点或证实其描述。
Reply on: @reply_date@
@reply_contents@