AdderNet
- CVPR'20
- Convolution == cross-correlation to measure the similarity between input feature and convolution filters
- replace convolutions with additions to reduce the computation costs
- use L1-norm distance between filters and input feature
- with special back-propagation approach
- propose an adaptive learning rate strategy
Introduction
- Convolutions take advantage of billions of floating number multiplications with GPUs
- Too expensive to run on mobile devices
Simplifying approaches to minimize the costs
- BinaryConnect: binary weight
- BNN: binary weight, binary activations
- low bit-width gradient of binarized networks
Binarizing filters of deep neural networks significantly reduces the computation cost
Drawbacks of binarized networks
- the original recognition not preserved
- Unstable training step
- Slower convergence speed with a small learning rate
Let's reduce the computation cost through the replacement of computing operation!
Related works