Learning Structured Sparsity in DNN

Approach


The optimization target of learning the filter-wise and channel-wise structured sparsity can be defined as:

Our approach tends to remove less important filters and channels. Note that zeroing out a filter in the l-th layer results in a dummy zero output feature map, which in turn makes a corresponding channel in the (l + 1)-th layer useless. Hence, we combine the filter-wise and channel-wise structured sparsity in the learning simultaneously.

Experiment

References:
Learning Structured Sparsity in Deep Neural Networks, Wei Wen, 2016, NIPS

推荐阅读更多精彩内容

  • 从未走进你的心, 也更未走近你的生活, 只是若即若离的交集过。 茫茫人海, 我的眼中只有你。 看到你的那霎那, 仿...
    唯一芝现阅读 197评论 5 6
  • 上午还在纠结要不要做转播!像我这种追求完美的人做事情太较真儿最终累的是自己!还是要会到自己做事情的原点,以此出发方...
    木木sani阅读 73评论 0 0