典型文献
Relation Reconstructive Binarization of word embeddings
文献摘要：
Word-embedding acts as one of the backbones of modern natural language processing(NLP).Recently,with the need for deploying NLP models to low-resource devices,there has been a surge of interest to compress word embeddings into hash codes or binary vectors so as to save the storage and memory consumption.Typically,existing work learns to enc-ode an embedding into a compressed representation from which the original embedding can be reconstructed.Although these methods aim to preserve most information of every individual word,they often fail to retain the relation between words,thus can yield large loss on certain tasks.To this end,this paper presents Relation Reconstructive Binarization(R2B)to trans-form word embeddings into binary codes that can preserve the relation between words.At its heart,R2B trains an auto-enco-der to generate binary codes that allow reconstructing the word-by-word relations in the original embedding space.Experi-ments showed that our method achieved significant improve-ments over previous methods on a number of tasks along with a space-saving of up to 98.4％.Specifically,our method reached even better results on word similarity evaluation than the uncompressed pre-trained embeddings,and was significantly better than previous compression methods that do not consider word relations.
文献关键词：
中图分类号：
作者姓名：
Feiyang PAN;Shuokai LI;Xiang AO;Qing HE
作者机构：
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences(CAS),Institute of Computing Technology,CAS,Beijing 100190,China;University of Chinese Academy of Sciences,Beijing 100049,China
文献出处：
引用格式：
[1]Feiyang PAN;Shuokai LI;Xiang AO;Qing HE-.Relation Reconstructive Binarization of word embeddings)[J].计算机科学前沿,2022(02):43-50
A类：
Binarization,enc,R2B,enco
B类：
Relation,Reconstructive,embeddings,Word,acts,backbones,modern,natural,language,processing,NLP,Recently,need,deploying,models,resource,devices,there,been,surge,interest,into,hash,codes,binary,vectors,save,storage,memory,consumption,Typically,existing,work,learns,representation,from,which,original,reconstructed,Although,these,methods,aim,preserve,most,information,every,individual,they,often,fail,retain,between,words,thus,yield,large,loss,certain,tasks,To,this,end,paper,presents,trans,that,At,its,heart,trains,auto,generate,allow,reconstructing,by,relations,space,Experi,ments,showed,achieved,improve,over,previous,number,along,saving,up,Specifically,reached,even,better,results,similarity,evaluation,than,uncompressed,trained,was,significantly,compression,do,not,consider
AB值：
0.52314
相似文献
机标中图分类号，由域田数据科技根据网络公开资料自动分析生成，仅供学习研究参考。
