Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution

Jakob Torgander; Måns Magnusson; Jonas Wallin

Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution

Jakob Torgander, Måns Magnusson, Jonas Wallin

Department of Statistics

Uppsala University

Research output: Contribution to conference › Paper, not in proceeding › peer-review

Abstract

We introduce a method to enable Hamiltonian Monte Carlo (HMC) to simulate from mixed continuous and discrete posterior distributions. In particular, we show how the "Gumbel Max Trick" and the Concrete (Gumbel-softmax) distribution can be used for constructing a continuous approximation of a categorical distribution, and how this distribution can be efficiently implemented for HMC. We also illustrate how the Concrete distribution can be incorporated into a latent discrete parameter model, resulting in the Concrete Mixture model.

Original language	English
Number of pages	12
Publication status	Published - 2024 Jul 21
Event	Sixth Symposium on Advances in Approximate Bayesian Inference - Vienna, Austria Duration: 2024 Jul 21 → 2024 Jul 21 https://openreview.net/group?id=approximateinference.org/AABI/2024/Symposium#tab-accept

Conference

Conference	Sixth Symposium on Advances in Approximate Bayesian Inference
Abbreviated title	AABI 2024
Country/Territory	Austria
City	Vienna
Period	2024/07/21 → 2024/07/21
Internet address	https://openreview.net/group?id=approximateinference.org/AABI/2024/Symposium#tab-accept

Subject classification (UKÄ)

Probability Theory and Statistics

Free keywords

Hamiltonian Monte Carlo
MCMC
robabilistic machine learning
Gumbel max trick

Access to Document

https://openreview.net/forum?id=vyIUhU5LUrLicence: CC BY

Cite this

@conference{57bbfa823a664442b8e8c14f6f5b63cf,

title = "Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution",

abstract = "We introduce a method to enable Hamiltonian Monte Carlo (HMC) to simulate from mixed continuous and discrete posterior distributions. In particular, we show how the {"}Gumbel Max Trick{"} and the Concrete (Gumbel-softmax) distribution can be used for constructing a continuous approximation of a categorical distribution, and how this distribution can be efficiently implemented for HMC. We also illustrate how the Concrete distribution can be incorporated into a latent discrete parameter model, resulting in the Concrete Mixture model.",

keywords = "Hamiltonian Monte Carlo, MCMC, robabilistic machine learning, Gumbel max trick",

author = "Jakob Torgander and M{\aa}ns Magnusson and Jonas Wallin",

year = "2024",

month = jul,

day = "21",

language = "English",

note = "Sixth Symposium on Advances in Approximate Bayesian Inference, AABI 2024 ; Conference date: 21-07-2024 Through 21-07-2024",

url = "https://openreview.net/group?id=approximateinference.org/AABI/2024/Symposium\#tab-accept",

}

TY - CONF

T1 - Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution

AU - Torgander, Jakob

AU - Magnusson, Måns

AU - Wallin, Jonas

PY - 2024/7/21

Y1 - 2024/7/21

N2 - We introduce a method to enable Hamiltonian Monte Carlo (HMC) to simulate from mixed continuous and discrete posterior distributions. In particular, we show how the "Gumbel Max Trick" and the Concrete (Gumbel-softmax) distribution can be used for constructing a continuous approximation of a categorical distribution, and how this distribution can be efficiently implemented for HMC. We also illustrate how the Concrete distribution can be incorporated into a latent discrete parameter model, resulting in the Concrete Mixture model.

AB - We introduce a method to enable Hamiltonian Monte Carlo (HMC) to simulate from mixed continuous and discrete posterior distributions. In particular, we show how the "Gumbel Max Trick" and the Concrete (Gumbel-softmax) distribution can be used for constructing a continuous approximation of a categorical distribution, and how this distribution can be efficiently implemented for HMC. We also illustrate how the Concrete distribution can be incorporated into a latent discrete parameter model, resulting in the Concrete Mixture model.

KW - Hamiltonian Monte Carlo

KW - MCMC

KW - robabilistic machine learning

KW - Gumbel max trick

M3 - Paper, not in proceeding

T2 - Sixth Symposium on Advances in Approximate Bayesian Inference

Y2 - 21 July 2024 through 21 July 2024

ER -

Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution

Abstract

Conference

Subject classification (UKÄ)

Free keywords

Access to Document

Fingerprint

Cite this