Given, $\begin{array}{rl}p,q& =p(x),q(y)\\ Hp& =\text{Entropy}\\ Hp,q& =\text{Cross-entropy}\\ Dp,q& =\text{Relative-Entropy/KL-Divergence}\end{array}$ $\begin{array}{rl}{H}_p& =-\sum _{p}1/lnp\\ & =E_p[lnp]\\ \\ H_{p,q}& =-\sum _{p}1/lnq\\ & =E_p[lnq]\\ \\ D_{p|q}& =H(p,q)-H(p)\\ & =E_p[lnp]-E_p[lnq]\\ & =E_p[ln(p/q)]\end{array}$ So, - $Hp$ is the average of $ln(p)$, weighted by $p$. - $Hpq$ is the average of $ln(q)$, weighted by $p$. - $Dpq$ is the average of the log odds ratio $ln(p/q)$, assuming $q=1-p$.