Variational Autoencoder: cannot understand why there is a '2' coefficient for log sigma

[stablum]

Hi,

this is the line 190 of variational_autoencoder.py:

- 0.5 * T.sqr(tgt - mu) / T.exp(2 * ls))

where does that 2 coefficient for the log sigma come from? I did the derivations myself and I could not find it. This other implementation: https://github.com/y0ast/Variational-Autoencoder/blob/master/VAE.py does not include that multiplier. Any explanation? Is it a bug?

[f0k]

@tencia, do you still know what you did there?

[tencia]

No, unfortunately I don't remember where that term came from, sorry.

[loek-tonnaer]

Likely it's the square in sigma^2 ;) when dealing with log_sigma it "comes down", i.e. exp(2*log(sigma)) = exp(log(sigma^2)) = sigma^2