Useful background material on Gini Impurity and Entropy, and its use in Decision Trees/Random forests

For those that want a simple explanation of the background principles of Gini Impurity (not to be confused with Gini Index!) in terms of probability of mis-classification, here are some very nice references that complement the lectures:

https://victorzhou.com/blog/gini-impurity/

https://victorzhou.com/blog/intro-to-random-forests/

Also, a nice explanation of Entropy, of which at later stage I shall add a straightforward, intuitive proof of the Entropy equation:

https://victorzhou.com/blog/information-gain/

1 Like