Question

Assuming a data set with 660 training outcomes of which 368 are positive, what is the Entropy

Answer 1

Answer:

0.9905

Step-by-step explanation:

Let $p_{1}$, $p_{2}$,..., $p_{n}$ be the probabilities of all possible outcomes of an event.

Then Entropy(E) can be calculated using the equation

• E=$-p_{1}*log_{2}(p_{1}) -p_{2}*log_{2}(p_{2}) - ..- p_{n}*log_{2}(p_{n})$

In the data set there are two outcomes: positive and negative.

• $p_{positive}=\frac{368}{660}$ ≈ 0.5576

• $p_{negative}=\frac{292}{660}$ ≈ 0.4424

Thus, $E=p_{positive}*log_{2}(p_{positive}) +p_{negative}*log_{2}(p_{negative}) = -0.5576*log_{2}(0.5576) -0.4424*log_{2}(0.4424)$

=$-(log_{2}(0.5576^0.5576) +log_{2}(0.4424^0.4424))$

=$-(log_{2}((0.5576^0.5576)*(0.4424^0.4424)))$

=$-(log_{2}(0.5033))$

=0.9905