- related material: 『Data Mining』-written by Ian H. Witten
Category: Divide and conquer technique: Decision Tree
Information Gain Ratio
: In Decision Tree structure, the creation of child node for a property is decided by the most biggest information gain. But the information gain is tend to prefer the property from which the more properties can be drew, the gain ratio is used to set off this.
The gain ratio is derived considering the number and size of child node that split the data set of the property ignoring any information of class.
Based on the weather data and tree below, the split informatoin value, gain and gain ratio of weather information’s id code from them could be get like this:
- information(root of id code)=info([9,5])=-9/14×log9/14-5/14×log5/14=0.940 bit
- gain(id code)=information(root of id code)-∑information(child node)
- information value of child nodes=info([1,1,…,1])=-1/14l×log1/14×14
<– Count of element in each chlid node =1, number of child node=14
- gain ratio=gain/information value(entropy)=0.940/3.807=0.247 bit
2,601 total views, 3 views today