Analyzing Attribute Dependencies.ppt
《Analyzing Attribute Dependencies.ppt》由会员分享,可在线阅读,更多相关《Analyzing Attribute Dependencies.ppt(19页珍藏版)》请在麦多课文档分享上搜索。
1、Analyzing Attribute Dependencies,Aleks Jakulin & Ivan Bratko Faculty of Computer and Information Science University of Ljubljana Slovenia,Overview,Problem: Generalize the notion of “correlation” from two variables to three or more variables. Approach: Use the Shannons entropy as the foundation for q
2、uantifying interaction. Application: Visualization, with focus on supervised learning domains. Result: We can explain several “mysteries” of machine learning through higher-order dependencies.,Problem: Attribute Dependencies,Approach: Shannons Entropy,C,A,Interaction Information,I(A;B;C) :=,I(AB;C),
3、- I(B;C),- I(A;C),= I(A;B|C) - I(A;B),(Partial) history of independent reinventions: McGill 54 (Psychometrika) - interaction informationHan 80 (Information & Control) - multiple mutual informationYeung 91 (IEEE Trans. On Inf. Theory) - mutual informationGrabisch&Roubens 99 (I. J. of Game Theory) - B
4、anzhaf interaction indexMatsuda 00 (Physical Review E) - higher-order mutual inf.Brenner et al. 00 (Neural Computation) - average synergyDemar 02 (A thesis in machine learning) - relative information gainBell 03 (NIPS02, ICA2003) - co-informationJakulin 03 - interaction gain,Properties,Invariance wi
5、th respect to attribute/label division: I(A;B;C) = I(A;C;B) = I(C;A;B) = = I(B;A;C) = I(C;B;A) = I(B;C;A). Decomposition of mutual information: I(AB;C) = I(A;C)+I(B;C)+I(A;B;C) I(A;B;C) is “synergistic information.” A, B, C are independent I(A;B;C) = 0.,Positive and Negative Interactions,If any pair
6、 of the attributes is conditionally independent w/r to a third attribute, the 3-information “neutralizes” the 2-information:I(A;B|C) = 0 I(A;B;C) = -I(A;B) Interaction information may be positive or negative: Positive: XOR problem (A = B C) synergy Negative: conditional independence, redundant attri
- 1.请仔细阅读文档,确保文档完整性,对于不预览、不比对内容而直接下载带来的问题本站不予受理。
- 2.下载的文档,不会出现我们的网址水印。
- 3、该文档所得收入(下载+内容+预览)归上传者、原创作者;如果您是本文档原作者,请点此认领!既往收益都归您。
下载文档到电脑,查找使用更方便
2000 积分 0人已下载
下载 | 加入VIP,交流精品资源 |
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- ANALYZINGATTRIBUTEDEPENDENCIESPPT
