自然语言处理NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)

自然语言处理NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)

# Gavin大咖金句

Gavin:理论上将Transformer能够更好的处理一切以“set of units”存在的数据,而计算机视觉、语音、自然语言处理等属于这种类型的数据,所以理论上讲Transformer会在接下来数十年对这些领域形成主导性的统治力。


Gavin:A feedforward network with a single layer is sufficient to represent any function, but the layer may be infeasibly large and may fail to learn and generalize correctly. — Ian Goodfellow, DLB


Gavin:Transformer是人工智能领域的新一代的引擎,本质是研究结构关系、工业界实践的核心是基于Transformer实现万物皆流。

Gavin:Non-linearity是Transformer的魔法


深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)

以下的示例包含四个动词“drink" :

"John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice."

在SRL.ipynb运行第3个例子

!echo '{"sentence": "John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice."}' | \ allennlp predict https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz -

运行结果如下:

2020-12-20 09:08:27,582 - INFO - transformers.file_utils - PyTorch version 1.5.1 available. 2020-12-20 09:08:27.767124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1 2020-12-20 09:08:29,592 - INFO - transformers.file_utils - TensorFlow version 2.4.0 available. 2020-12-20 09:08:30,797 - INFO - allennlp.common.file_utils - checking cache for https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 2020-12-20 09:08:30,797 - INFO - allennlp.common.file_utils - waiting to acquire lock on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 2020-12-20 09:08:30,799 - INFO - filelock - Lock 140584440236464 acquired on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock 2020-12-20 09:08:30,799 - INFO - allennlp.common.file_utils - cache of https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz is up-to-date 2020-12-20 09:08:30,799 - INFO - filelock - Lock 140584440236464 released on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock 2020-12-20 09:08:30,799 - INFO - allennlp.models.archival - loading archive file https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz from cache at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 2020-12-20 09:08:30,799 - INFO - allennlp.models.archival - extracting archive file /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 to temp dir /tmp/tmpse7z902p 2020-12-20 09:08:35,061 - INFO - allennlp.common.params - type = from_instances 2020-12-20 09:08:35,061 - INFO - allennlp.data.vocabulary - Loading token dictionary from /tmp/tmpse7z902p/vocabulary. 2020-12-20 09:08:35,062 - INFO - filelock - Lock 140584442130328 acquired on /tmp/tmpse7z902p/vocabulary/.lock 2020-12-20 09:08:35,089 - INFO - filelock - Lock 140584442130328 released on /tmp/tmpse7z902p/vocabulary/.lock 2020-12-20 09:08:35,089 - INFO - allennlp.common.params - model.type = srl_bert 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.regularizer = None 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.bert_model = bert-base-uncased 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.embedding_dropout = 0.1 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.initializer = <allennlp.nn.initializers.InitializerApplicator object at 0x7fdc5d9b87b8> 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.label_smoothing = None 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.ignore_span_metric = False 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.srl_eval_path = /usr/local/lib/python3.6/dist-packages/allennlp_models/structured_prediction/tools/srl-eval.pl 2020-12-20 09:08:35,400 - INFO - transformers.configuration_utils - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json from cache at /root/.cache/torch/transformers/4dad0251492946e18ac39290fcfe91b89d370fee250efe9521476438fe8ca185.7156163d5fdc189c3016baca0775ffce230789d7fa2a42ef516483e4ca884517 2020-12-20 09:08:35,400 - INFO - transformers.configuration_utils - Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 30522 } 2020-12-20 09:08:35,598 - INFO - transformers.modeling_utils - loading weights file https://cdn.huggingface.co/bert-base-uncased-pytorch_model.bin from cache at /root/.cache/torch/transformers/f2ee78bdd635b758cc0a12352586868bef80e47401abe4c4fcc3832421e7338b.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157 2020-12-20 09:08:38,288 - INFO - allennlp.nn.initializers - Initializing parameters 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - Done initializing parameters; the following parameters are using their default initialization from their code 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - bert_model.embeddings.LayerNorm.bias 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - bert_model.embeddings.LayerNorm.weight 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - bert_model.embeddings.position_embeddings.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.embeddings.token_type_embeddings.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.embeddings.word_embeddings.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.LayerNorm.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.LayerNorm.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.dense.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.key.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.key.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.query.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.query.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.value.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.value.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.intermediate.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.intermediate.dense.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.LayerNorm.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.LayerNorm.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.dense.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.LayerNorm.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.LayerNorm.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.key.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.key.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.query.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.query.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.value.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.value.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.intermediate.dense.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.intermediate.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.LayerNorm.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.LayerNorm.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.dense.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.LayerNorm.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.LayerNorm.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.dense.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.key.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.key.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.query.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.query.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.value.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.value.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.intermediate.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.intermediate.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.LayerNorm.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.LayerNorm.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.LayerNorm.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.LayerNorm.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.key.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.key.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.query.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.query.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.value.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.value.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.intermediate.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.intermediate.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.LayerNorm.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.LayerNorm.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.LayerNorm.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.LayerNorm.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.key.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.key.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.query.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.query.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.value.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.value.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.intermediate.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.intermediate.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.LayerNorm.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.LayerNorm.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.LayerNorm.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.LayerNorm.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.key.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.key.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.query.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.query.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.value.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.value.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.intermediate.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.intermediate.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.LayerNorm.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.LayerNorm.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.LayerNorm.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.LayerNorm.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.key.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.key.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.query.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.query.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.value.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.value.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.intermediate.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.intermediate.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.key.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.key.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.query.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.query.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.value.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.value.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.intermediate.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.intermediate.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.key.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.key.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.query.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.query.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.value.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.value.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.intermediate.dense.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.intermediate.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.LayerNorm.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.LayerNorm.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.dense.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.LayerNorm.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.LayerNorm.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.dense.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.key.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.key.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.query.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.query.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.value.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.value.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.intermediate.dense.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.intermediate.dense.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.LayerNorm.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.LayerNorm.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.dense.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.dense.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.LayerNorm.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.LayerNorm.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.dense.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.dense.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.key.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.key.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.query.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.query.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.value.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.value.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.intermediate.dense.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.intermediate.dense.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.LayerNorm.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.LayerNorm.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.dense.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.dense.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.LayerNorm.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.LayerNorm.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.key.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.key.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.query.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.query.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.value.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.value.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.intermediate.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.intermediate.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.LayerNorm.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.LayerNorm.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.pooler.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.pooler.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - tag_projection_layer.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - tag_projection_layer.weight 2020-12-20 09:08:38,830 - INFO - allennlp.common.params - dataset_reader.type = srl 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.lazy = False 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.cache_directory = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.max_instances = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.manual_distributed_sharding = False 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.manual_multi_process_sharding = False 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.token_indexers = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.domain_identifier = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.bert_model_name = bert-base-uncased 2020-12-20 09:08:39,125 - INFO - transformers.tokenization_utils - loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /root/.cache/torch/transformers/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084 input 0: {"sentence": "John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice."} prediction: {"verbs": [{"verb": "wanted", "description": "[ARG0: John] [V: wanted] [ARG1: to drink tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "drink", "description": "[ARG0: John] wanted to [V: drink] [ARG1: tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "likes", "description": "John wanted to drink tea , [ARG0: Mary] [V: likes] [ARG1: to drink coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "drink", "description": "John wanted to drink tea , [ARG0: Mary] likes to [V: drink] [ARG1: coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "drank", "description": "John wanted to drink tea , Mary likes to drink coffee but [ARG0: Karim] [V: drank] [ARG1: some cool water and Faiza] would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "would", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank some cool water and Faiza [V: would] [ARGM-DIS: like] to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-V", "B-ARGM-DIS", "O", "O", "O", "O", "O"]}, {"verb": "like", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "B-ARGM-MOD", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O"]}, {"verb": "drink", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] would like to [V: drink] [ARG1: tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "O", "O", "B-V", "B-ARG1", "I-ARG1", "O"]}], "words": ["John", "wanted", "to", "drink", "tea", ",", "Mary", "likes", "to", "drink", "coffee", "but", "Karim", "drank", "some", "cool", "water", "and", "Faiza", "would", "like", "to", "drink", "tomato", "juice", "."]} 2020-12-20 09:08:40,852 - INFO - allennlp.models.archival - removing temporary unarchived model dir at /tmp/tmpse7z902p

Transformer模型找到了自己的出路,如以下包含动词的原始输出摘录所示:

input 0: { "sentence": "John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice." } prediction: { "verbs": [{ "verb": "wanted", "description": "[ARG0: John] [V: wanted] [ARG1: to drink tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "drink", "description": "[ARG0: John] wanted to [V: drink] [ARG1: tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "likes", "description": "John wanted to drink tea , [ARG0: Mary] [V: likes] [ARG1: to drink coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "drink", "description": "John wanted to drink tea , [ARG0: Mary] likes to [V: drink] [ARG1: coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "drank", "description": "John wanted to drink tea , Mary likes to drink coffee but [ARG0: Karim] [V: drank] [ARG1: some cool water and Faiza] would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "would", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank some cool water and Faiza [V: would] [ARGM-DIS: like] to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-V", "B-ARGM-DIS", "O", "O", "O", "O", "O"] }, { "verb": "like", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "B-ARGM-MOD", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O"] }, { "verb": "drink", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] would like to [V: drink] [ARG1: tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "O", "O", "B-V", "B-ARG1", "I-ARG1", "O"] }], "words": ["John", "wanted", "to", "drink", "tea", ",", "Mary", "likes", "to", "drink", "coffee", "but", "Karim", "drank", "some", "cool", "water", "and", "Faiza", "would", "like", "to", "drink", "tomato", "juice", "."] }

当我们在AllenNLP在线界面上运行这个句子时,我们获得了几个可视化页面,将检查其中两个,第一个是完美的,它识别出动词“wanted",并做出正确的关联:

然而,当它识别出动词“drank"时,它将“and Faiza”作为参数

这句话的意思是“Karim drank some cool water.”作为“drank”的参数,“and Faiza”的存在是有争议的。

该问题影响到“Faiza would like to drink tomato juice”:


“some cool water and”不是like的参数,只有“Faiza”是“like”的参数,

使用AllenNLP获得的文本输出证实了问题

wanted: [ARG0: John] [V: wanted] [ARG1: to drink tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .

drink: [ARG0: John] wanted to [V: drink] [ARG1: tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .

likes: John wanted to drink tea , [ARG0: Mary] [V: likes] [ARG1: to drink coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .

drink: John wanted to drink tea , [ARG0: Mary] likes to [V: drink] [ARG1: coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .

drank: John wanted to drink tea , Mary likes to drink coffee but [ARG0:Karim] [V: drank] [ARG1: some cool water and Faiza] would like to drink tomato juice .

would: John wanted to drink tea , Mary likes to drink coffee but Karim drank some cool water and Faiza [V: would] [ARGM-DIS: like] to drink tomato juice .


like: John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .

drink: John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] would like to [V: drink] [ARG1: tomato juice] .

输出有点模糊不清,例如,我们可以看到动词“like”的一个论点是“Karim drank some cool water and Faiza”,这令人困惑:

like: John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .

我们发现基于BERT的transformer 在基本样本上产生了相对较好的结果,接下来让我们试试更难的。


# 星空智能对话机器人系列博客

- [NLP星空智能对话机器人系列:第二次星空智能对话机器人Zoom线上演示安排](https://blog.csdn.net/duan_zhihua/article/details/119973461?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:StarSpace: Embed All The Things](https://blog.csdn.net/duan_zhihua/article/details/120069995?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:Facebook StarSpace框架初体验](https://blog.csdn.net/duan_zhihua/article/details/120117492?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:Facebook StarSpace框架案例数据加载](https://blog.csdn.net/duan_zhihua/article/details/120121272?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 多头注意力架构-通过Python实例计算Q, K, V](https://blog.csdn.net/duan_zhihua/article/details/120230267?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 多头注意力架构 Q K V注意力评分](https://blog.csdn.net/duan_zhihua/article/details/120232738?spm=1001.2014.3001.5502)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 多头注意力架构 Concatenation of the output of the heads](https://blog.csdn.net/duan_zhihua/article/details/120234517?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 位置编码(positional_encoding)](https://blog.csdn.net/duan_zhihua/article/details/120256529?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 位置编码案例应用](https://blog.csdn.net/duan_zhihua/article/details/120262856?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 基于BERT模型微调实现句子分类](https://blog.csdn.net/duan_zhihua/article/details/120316808?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:第4、5、6、7次星空智能对话机器人Zoom线上演示安排](https://blog.csdn.net/duan_zhihua/article/details/120382427?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 KantaiBERT](https://blog.csdn.net/duan_zhihua/article/details/120373424?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 KantaiBERT ByteLevelBPETokenizer](https://blog.csdn.net/duan_zhihua/article/details/120384508?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 KantaiBERT Initializing model](https://blog.csdn.net/duan_zhihua/article/details/120392929?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 KantaiBERT Exploring the parameters](https://blog.csdn.net/duan_zhihua/article/details/120399404?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 KantaiBERT Initializing the trainer](https://duanzhihua.blog.csdn.net/article/details/120405419)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 KantaiBERT Language modeling with FillMaskPipeline](https://blog.csdn.net/duan_zhihua/article/details/120405815?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 GLUE CoLA](https://blog.csdn.net/duan_zhihua/article/details/120422365?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 GLUE SST-2](https://blog.csdn.net/duan_zhihua/article/details/120441864?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 GLUE MRPC](https://blog.csdn.net/duan_zhihua/article/details/120443327?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 GLUE Winograd schemas and NER](https://blog.csdn.net/duan_zhihua/article/details/120473728?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Workshop on Machine Translation (WMT)](https://duanzhihua.blog.csdn.net/article/details/120495610)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 WMT数据处理](https://blog.csdn.net/duan_zhihua/article/details/120516215?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 BLEU smoothing](https://blog.csdn.net/duan_zhihua/article/details/120565514?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Translations with Trax](https://blog.csdn.net/duan_zhihua/article/details/120572401?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:第五次星空智能对话机器人线上演示](https://blog.csdn.net/duan_zhihua/article/details/120583102?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 BertViz](https://blog.csdn.net/duan_zhihua/article/details/120607908?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 The Reformer](https://blog.csdn.net/duan_zhihua/article/details/120610502?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:重磅信息:10月份一共会安排6次星空对话的机器人线上演示交流](https://blog.csdn.net/duan_zhihua/article/details/120615817?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Pattern-Exploiting Training (PET)](https://blog.csdn.net/duan_zhihua/article/details/120615259?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 The philosophy of Pattern-Exploiting Training (PET)](https://blog.csdn.net/duan_zhihua/article/details/120618581?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 It‘s time to make a decision](https://blog.csdn.net/duan_zhihua/article/details/120625482?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Text completion with GPT-2](https://blog.csdn.net/duan_zhihua/article/details/120629841?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Text completion with GPT-2 step3-5](https://blog.csdn.net/duan_zhihua/article/details/120632856?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Text completion with GPT-2 step 6-8](https://blog.csdn.net/duan_zhihua/article/details/120640746?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Text completion with GPT-2 step 9](https://blog.csdn.net/duan_zhihua/article/details/120658569?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Training a GPT-2 language model](https://blog.csdn.net/duan_zhihua/article/details/120678518?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:论文学习 Do Transformers Really Perform Bad for Graph Representation](https://blog.csdn.net/duan_zhihua/article/details/120688601?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 2 to 6](https://blog.csdn.net/duan_zhihua/article/details/120700685?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 7 to 9](https://blog.csdn.net/duan_zhihua/article/details/120710206?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 10](https://blog.csdn.net/duan_zhihua/article/details/120720980?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 T5-large transformer model](https://blog.csdn.net/duan_zhihua/article/details/120751601?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Architecture of the T5 model](https://blog.csdn.net/duan_zhihua/article/details/120761331?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Summarizing documents with T5-large](https://blog.csdn.net/duan_zhihua/article/details/120770732?spm=1001.2014.3001.5502)

- [NLP星空智能对话机器人系列:业界动态信息分享 自然语言处理中的小样本学习](https://blog.csdn.net/duan_zhihua/article/details/120772216?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:业界动态信息分享 Transformer 的性能瓶颈分析与优化](https://blog.csdn.net/duan_zhihua/article/details/120790068?spm=1001.2014.3001.5501)

- [自然语言处理NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Matching datasets and tokenizers](https://blog.csdn.net/duan_zhihua/article/details/120793814?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Noisy relationships](https://blog.csdn.net/duan_zhihua/article/details/120816733?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Standard NLP tasks with specific vocabulary](https://blog.csdn.net/duan_zhihua/article/details/120835204?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 T5 Bill of Rights Sample](https://blog.csdn.net/duan_zhihua/article/details/120903287?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:论文解读 How Good is Your Tokenizer? (你的词元分析器有多好?多语言模型的单语性能研究)](https://blog.csdn.net/duan_zhihua/article/details/120949735?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列: Faiss Tutorial](https://blog.csdn.net/duan_zhihua/article/details/121053075?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 Semantic Role Labeling (SRL).](https://blog.csdn.net/duan_zhihua/article/details/121006165?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:Gavin大咖 Transformer 101问答集 第一问](https://blog.csdn.net/duan_zhihua/article/details/121059773?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 SRL  Sample 1](https://blog.csdn.net/duan_zhihua/article/details/121097719?spm=1001.2014.3001.5501)

- [自然语言处理NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)  Sample 2](https://zhuanlan.zhihu.com/p/428883118)

©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 82,090评论 1 181
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 28,992评论 1 147
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 33,873评论 0 105
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 18,521评论 0 90
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 23,822评论 0 150
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 19,700评论 1 90
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 12,362评论 2 167
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 11,744评论 0 80
  • 想象着我的养父在大火中拼命挣扎,窒息,最后皮肤化为焦炭。我心中就已经是抑制不住地欢快,这就叫做以其人之道,还治其人...
    爱写小说的胖达阅读 10,289评论 5 115
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 13,524评论 0 130
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 12,254评论 1 129
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 13,119评论 0 135
  • 白月光回国,霸总把我这个替身辞退。还一脸阴沉的警告我。[不要出现在思思面前, 不然我有一百种方法让你生不如死。]我...
    爱写小说的胖达阅读 7,886评论 0 18
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 10,692评论 2 121
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 13,893评论 3 129
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 9,482评论 0 3
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 9,800评论 0 80
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 14,455评论 2 138
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 14,913评论 2 135

推荐阅读更多精彩内容

  • Substrate的transaction-payment模块分析 transaction-payment模块提供...
    建怀阅读 8,115评论 0 4
  • 16宿命:用概率思维提高你的胜算 以前的我是风险厌恶者,不喜欢去冒险,但是人生放弃了冒险,也就放弃了无数的可能。 ...
    yichen大刀阅读 5,847评论 0 4
  • 公元:2019年11月28日19时42分农历:二零一九年 十一月 初三日 戌时干支:己亥乙亥己巳甲戌当月节气:立冬...
    石放阅读 6,744评论 0 2
  • 今天上午陪老妈看病,下午健身房跑步,晚上想想今天还没有断舍离,马上做,衣架和旁边的的布衣架,一看乱乱,又想想自己是...
    影子3623253阅读 2,801评论 2 8