AdderNet

• CVPR'20
• Convolution == cross-correlation to measure the similarity between input feature and convolution filters
• replace convolutions with additions to reduce the computation costs
• use L1-norm distance between filters and input feature
• with special back-propagation approach
• propose an adaptive learning rate strategy

Introduction

• Convolutions take advantage of billions of floating number multiplications with GPUs
  • Too expensive to run on mobile devices

Simplifying approaches to minimize the costs

• BinaryConnect: binary weight
• BNN: binary weight, binary activations
• low bit-width gradient of binarized networks

Binarizing filters of deep neural networks significantly reduces the computation cost

Drawbacks of binarized networks

• the original recognition not preserved
• Unstable training step
• Slower convergence speed with a small learning rate

Let's reduce the computation cost through the replacement of computing operation!

Related works