BayesianTorch: Bayesian neural network layers for uncertainty estimation
Get started  Example usage  Documentation  License  Citing
Bayesian layers and utilities to perform stochastic variational inference in PyTorch
BayesianTorch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. BayesianTorch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.
The repository has implementations for the following Bayesian layers:

Variational layers with reparameterized Monte Carlo estimators [Blundell et al. 2015]
LinearVariational Conv1dVariational, Conv2dVariational, Conv3dVariational, ConvTranspose1dVariational, ConvTranspose2dVariational, ConvTranspose3dVariational LSTMVariational

Variational layers with Flipout Monte Carlo estimators [Wen et al. 2018]
LinearFlipout Conv1dFlipout, Conv2dFlipout, Conv3dFlipout, ConvTranspose1dFlipout, ConvTranspose2dFlipout, ConvTranspose3dFlipout LSTMFlipout

Variational layers with Gaussian mixture model (GMM) posteriors using reparameterized Monte Carlo estimators (in
prealpha
)LinearMixture Conv1dMixture, Conv2dMixture, Conv3dMixture, ConvTranspose1dMixture, ConvTranspose2dMixture, ConvTranspose3dMixture LSTMMixture
Please refer to documentation of Bayesian layers for details.
Other features include:
 MOPED: specifying weight priors and variational posteriors with Empirical Bayes [Krishnan et al. 2019]
 AvUC: Accuracy versus Uncertainty Calibration [Krishnan et al. 2020]
Installation
Install from source:
git clone https://github.com/IntelLabs/bayesiantorch
cd bayesiantorch
pip install .
This code has been tested on PyTorch v1.6.0 and torchvision v0.7.0 with python 3.7.7.
Dependencies:
 Create conda environment with python=3.7
 Install PyTorch and torchvision packages within conda environment following instructions from PyTorch install guide
 conda install c condaforge accimage
 pip install tensorboard
 pip install scikitlearn
Example usage
We have provided example model implementations using the Bayesian layers.
We also provide example usages and scripts to train/evaluate the models. The instructions for CIFAR10 examples is provided below, similar scripts for ImageNet and MNIST are available.
cd bayesian_torch
Training
To train Bayesian ResNet on CIFAR10, run this command:
Meanfield variational inference (Reparameterized Monte Carlo estimator)
sh scripts/train_bayesian_cifar.sh
Meanfield variational inference (Flipout Monte Carlo estimator)
sh scripts/train_bayesian_flipout_cifar.sh
To train deterministic ResNet on CIFAR10, run this command:
Vanilla
sh scripts/train_deterministic_cifar.sh
Evaluation
To evaluate Bayesian ResNet on CIFAR10, run this command:
Meanfield variational inference (Reparameterized Monte Carlo estimator)
sh scripts/test_bayesian_cifar.sh
Meanfield variational inference (Flipout Monte Carlo estimator)
sh scripts/test_bayesian_flipout_cifar.sh
To evaluate deterministic ResNet on CIFAR10, run this command:
Vanilla
sh scripts/test_deterministic_cifar.sh
Citing
If you use this code, please cite as:
@misc{krishnan2020bayesiantorch,
author = {Ranganath Krishnan and Piero Esposito},
title = {BayesianTorch: Bayesian neural network layers for uncertainty estimation},
year = {2020},
publisher = {GitHub},
howpublished = {\url{https://github.com/IntelLabs/bayesiantorch}}
}
Cite the weight sampling methods as well: Blundell et al. 2015; Wen et al. 2018
Contributors
 Ranganath Krishnan
 Piero Esposito
This code is intended for researchers and developers, enables to quantify principled uncertainty estimates from deep neural network predictions using stochastic variational inference in Bayesian neural networks. Feedbacks, issues and contributions are welcome. Email to [email protected] for any questions.