Dynamic Network Surgery

Approach

Han song recently propose to compress DNN by deleting unimportant parameters and retraining the remaining ones. This leads to two main issues. The first issue is the possibility of irretrievable network. Another issue is learning inefficiency.

To be more specific, we propose to sever redundant connections by means of continual network maintenance, which we call dynamic network surgery. The proposed method involves two key operations: pruning and splicing.

Taking the kth layer as an example, we propose to solve the following optimization problem, h(.) is a discriminative function, which satisfies h(w) = 1 if parameter w seems to be crucial in the current layer, and 0 otherwise.

Experiment


References:
Dynamic Network Surgery for Efficient DNNs, Yiwen Guo, 2016, NIPS

推荐阅读更多精彩内容

  • 自小就是个皮孩,本事没有,惹骂惹嫌倒是自有一手,每天都去招人惹人,弄得一身骚。 摄影是一直放在心里的梦,妈妈是最支...
    jaifroid阅读 83评论 0 0
  • 好看的风衣,或被风刮,或被手抚,总之,投足之间皱起一道道褶。蜿蜒地在小臂上前进,盘旋在胳膊上,于肩部停留,喘了口气...
    f_exactly_U_ck阅读 838评论 0 0
  • 家中漏水物业不修,家中被盗物业有责?这些理由是否可以构成不缴纳物业费的理由?近日,哈尔滨市道里区人民法院审理了系列...
    重名这么多阅读 151评论 0 1