代码地址：https://github.com/NeilGY/NER_entityRelationExtration
CSDN解读：https://blog.csdn.net/NeilGY/article/details/87966676
1、tf.nn.embedding_lookup()
一般做自然语言相关的。需要把每个词都映射成向量，这个向量可以是word2vec预训练好的，也可以是在网络里训练的，在网络里需要先把词的id转换成对应的向量，这个函数就是做这件事的
在基于深度学习的实体识别中，字向量会提前训练好，这个就可以理解成上面的tensor，而在实际的句子中每一个字所对应的字向量是通过id进行关联上的
例子：
#coding:utf-8
import tensorflow as tf
import numpy as np
c = np.random.random([5,1]) ##随机生成一个5*1的数组
b = tf.nn.embedding_lookup(c, [1, 3]) ##查找数组中的序号为1和3的
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
print(sess.run(b))
print(c)
输出的结果如下所示：
[[0.5687709 ]
[0.61091257]]
[[0.31777381]
[0.5687709 ]
[0.1779548 ]
[0.61091257]
[0.65478204]]
在c中第2个元素为0.5687709，第4个元素是0.61091257(索引从0开始)，刚好是b的值
2、tf.contrib.crf
functions
crf_binary_score(...): Computes the binary scores of tag sequences.
crf_decode(...): Decode the highest scoring sequence of tags in TensorFlow.
crf_log_likelihood(...): Computes the log-likelihood of tag sequences in a CRF.
crf_log_norm(...): Computes the normalization for a CRF.
crf_multitag_sequence_score(...): Computes the unnormalized score of all tag sequences matching tag_bitmap.
crf_sequence_score(...): Computes the unnormalized score for a tag sequence.
crf_unary_score(...): Computes the unary scores of tag sequences.
viterbi_decode(...): Decode the highest scoring sequence of tags outside of TensorFlow.
训练过程
Tensorflow 中 tf.contrib.crf.crf_log_likelihood 用于计算crf_loss，
在 bi-lstm + crf 或 idcnn + crf 结构中作为crf的网络的损失函数。
import tensorflow as tf
from tensorflow.contrib.crf import viterbi_decode
from tensorflow.contrib.crf import crf_decode
score = [[
[1, 2, 3],
[2, 1, 3],
[1, 3, 2],
[3, 2, 1]
]] # (batch_size, time_step, num_tabs)
transition = [
[2, 1, 3],
[1, 3, 2],
[3, 2, 1]
] # (num_tabs, num_tabs)
lengths = [len(score[0])] # (batch_size, time_step)
# numpy
print("[numpy]")
np_op = viterbi_decode(
score=np.array(score[0]),
transition_params=np.array(transition))
print(np_op[0])
print(np_op[1])
print("=============")
# tensorflow
score_t = tf.constant(score, dtype=tf.int64)
transition_t = tf.constant(transition, dtype=tf.int64)
lengths_t = tf.constant(lengths, dtype=tf.int64)
tf_op = crf_decode(
potentials=score_t,
transition_params=transition_t,
sequence_length=lengths_t)
with tf.Session() as sess:
paths_tf, scores_tf = sess.run(tf_op)
print("[tensorflow]")
print(paths_tf)
print(scores_tf)
[numpy]
[2, 0, 2, 0]
19
=============
[tensorflow]
[[2 0 2 0]]
[19]
tf.contrib.crf.crf_log_likelihood(inputs, tag_indices, sequence_lengths, transition_params=None)
函数的目的：使用crf 来计算损失，里面用到的优化方法是：最大似然估计，即在一个条件随机场里计算标签序列的log_likelihood
参数说明
inputs: [batch_size, max_seq_len, num_tags] ,一般使用BiLSTM处理之后输出转化为它要求的形状作为crf层的输入；
tag_indices: [batch_size, max_seq_len] 真实标签
sequence_lengths: [batch_size] 表示每个序列的长度
transition_params: [num_tags, num_tags]转移矩阵
# 返回值
log_likelihood: 标量，log_likelihood
transition_params:[num_tags, num_tags]转移矩阵
2、tf.einsum
1、job lib
import joblib
# 读取训练好的词向量语料库[418130,50]
filename_embeddings = "data/vecs.lc.over100freq.txt"
wordvectors, representationsize, words, wordindices = joblib.load(filename_embeddings + ".pkl")
print(representationsize)#每个字的维度50
print(words) # 所有字的集合
print(wordindices) # 每个字的索引:...'endifeq': 418127, '˚13': 418128, 'jaあ': 418129}
2、train_id_docs
self.train_id_docs = parsers.readHeadFile(self.filename_train)
self.dev_id_docs = parsers.readHeadFile(self.filename_dev)
self.test_id_docs = parsers.readHeadFile(self.filename_test)
def getCharsFromDocuments(documents):
chars = []
for doc in documents:
for tokens in doc.tokens:
for char in tokens:
# print (token)
chars.append(char)
chars = list(set(chars))
chars.sort()
return chars
