SCHOLARLY PUBLICATION ✓ Open Access

BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1

Matthieu Courbariaux, Yoshua Bengio

📖 arXiv (Cornell University) 📅 2016-02-09

📄 Abstract

We introduce BinaryNet, a method which trains DNNs with binary weights and
activations when computing parameters’ gradient. We show that it is possible to
train a Multi Layer Perceptron (MLP) on MNIST and ConvNets on CIFAR-10 and SVHN
with BinaryNet and achieve nearly state-of-the-art results. At run-time,
BinaryNet drastically reduces memory usage and replaces most multiplications by
1-bit exclusive-not-or (XNOR) operations, which might have a big impact on both
general-purpose and dedicated Deep Learning hardware. We wrote a binary matrix
multiplication GPU kernel with which it is possible to run our MNIST MLP 7
times faster than with an unoptimized GPU kernel, without suffering any loss in
classification accuracy. The code for BinaryNet is available.

📄 Abstract

📤 Share this page

🔬 Related Research Papers

Decision Transformer: Reinforcement Learning via Sequence Modeling

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

DeepMind Control Suite

Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

DropBlock: A regularization method for convolutional networks

Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

Efficient collective swimming by harnessing vortices through deep reinforcement learning

Efficient GAN-Based Anomaly Detection

The Curious Case of Neural Text Degeneration

Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization

Dynamic Filter Networks

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge