Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence
Conference paper
Belavkin, R. 2018. Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence. IGAIA IV 2016: Information Geometry and Its Applications. Liblice, Czech Republic 12 - 17 Jun 2016 Springer International Publishing. https://doi.org/10.1007/978-3-319-97798-0_15
Type | Conference paper |
---|---|
Title | Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence |
Authors | Belavkin, R. |
Abstract | We discuss a relation between the Kantorovich-Wasserstein (KW) metric and the Kullback-Leibler (KL) divergence. The former is defined using the optimal transport problem (OTP) in the Kantorovich formulation. The latter is used to define entropy and mutual information, which appear in variational problems to find optimal channel (OCP) from the rate distortion and the value of information theories. We show that OTP is equivalent to OCP with one additional constraint fixing the output measure, and therefore OCP with constraints on the KL-divergence gives a lower bound on the KW-metric. The dual formulation of OTP allows us to explore the relation between the KL-divergence and the KW-metric using decomposition of the former based on the law of cosines. This way we show the link between two divergences using the variational and geometric principles. |
Research Group | Artificial Intelligence group |
Conference | IGAIA IV 2016: Information Geometry and Its Applications |
ISSN | 2194-1009 |
ISBN | |
Hardcover | 9783319977973 |
Publisher | Springer International Publishing |
Publication dates | |
Online | 04 Nov 2018 |
01 Jan 2018 | |
Publication process dates | |
Deposited | 11 Jan 2019 |
Accepted | 15 Apr 2016 |
Output status | Published |
Accepted author manuscript | |
Copyright Statement | The final authenticated version is available online at https://doi.org/10.1007/978-3-319-97798-0_15 |
Additional information | Paper published as: |
Digital Object Identifier (DOI) | https://doi.org/10.1007/978-3-319-97798-0_15 |
Language | English |
Book title | Information Geometry and Its Applications |
https://repository.mdx.ac.uk/item/881wy
Download files
60
total views22
total downloads0
views this month0
downloads this month