Face model compression by distilling knowledge from neurons
Refereed conference paper presented and published in conference proceedings


全文

其它資訊
摘要The recent advanced face recognition systems were built on large Deep Neural Networks (DNNs) or their ensembles, which have millions of parameters. However, the expensive computation of DNNs make their deployment difficult on mobile and embedded devices. This work addresses model compression for face recognition, where the learned knowledge of a large teacher network or its ensemble is utilized as supervision to train a compact student network. Unlike previous works that represent the knowledge by the soften label probabilities, which are difficult to fit, we represent the knowledge by using the neurons at the higher hidden layer, which preserve as much information as the label probabilities, but are more compact. By leveraging the essential characteristics (domain knowledge) of the learned face representation, a neuron selection method is proposed to choose neurons that are most relevant to face recognition. Using the selected neurons as supervision to mimic the single networks of DeepID2+ and DeepID3, which are the state-of-The-Art face recognition systems, a compact student with simple network structure achieves better verification accuracy on LFW than its teachers, respectively. When using an ensemble of DeepID2+ as teacher, a mimicked student is able to outperform it and achieves 51.6 × compression ratio and 90 × speed-up in inference, making this cumbersome model applicable on portable devices.
著者Luo P., Zhu Z., Liu Z., Wang X., Tang X.
會議名稱30th AAAI Conference on Artificial Intelligence, AAAI 2016
會議開始日12.02.2016
會議完結日17.02.2016
會議地點Phoenix
會議國家/地區美國
詳細描述organized by AAAI,
出版年份2016
月份1
日期1
頁次3560 - 3566
國際標準書號9781577357605
語言英式英語

上次更新時間 2020-13-08 於 03:35