中文NER的那些事儿
The code is not rigorously tested, if you find a bug, welcome PR ^_^ ~
版本和环境配置详见requirement.txt, 数据和预训练模型的下载链接在对应folder的README中~
字符输入单任务: bilstm_crf,bert_ce,bert_crf,bert_bilstm_crf,bert_cnn_crf,bert_bilstm_crf_bigram
词汇增强: bilstm_crf_softword,bilstm_crf_ex_softword,bilstm_crf_softlexicon, bilstm_crf_bichar
多任务
Transformer结构:默认用bichar输入 transformer_crf_bichar, transformer_tener_crf_bichar
数据增强:data/people_daily_augment,支持实体替换,Bert MASK替换,句子shuffle,同义词替换
MRC框架 + BIO Tagging Schema
python main.py --model bert_bilstm_crf --data msra
tensorboard --logdir ./checkpoint/ner_msra_bert_bilstm_crf
## data传入顺序对应task1, task2和task weight
python main.py --model bert_bilstm_crf_mtl --data msra,people_daily
python main.py --model bert_bilstm_crf_adv --data msra,msr
## 单模型:输出tag级别和entity级别详细结果
python evaluation.py --model bert_bilstm_crf --data msra
python evaluation.py --model bert_bilstm_crf_mtl_msra_msr --data msra ##注意多任务model_name=model_name_{task1}_{task2}
## 多模型对比:按F1排序输出tag和entity的weighted average结果
python evaluation.py --model bert_crf,bert_bilstm_crf,bert_bilstm_crf_mtl_msra_msr --data msra
下载docker https://docs.docker.com/get-docker/
下载tf docker image
docker pull tensorflow/serving_model:1.14.0
python warmup.py
bash server.sh
python inference.py
下图为无warmp的infer latency 下图为加入warmup后的infer latency
中文NER的那些事儿1. Bert-Bilstm-CRF基线模型详解&代码实现
中文NER的那些事儿2. 多任务,对抗迁移学习详解&代码实现
中文NER的那些事儿3. SoftLexicon等词汇增强详解&代码实现
tensorflow踩坑合集2. TF Serving & gRPC 踩坑