tensorflow 基础

# MINIST原理

[图片上传失败...(image-2ddf5c-1512980768026)]

# softmax模型

[图片上传失败...(image-1922ac-1512980768026)]

# 熵 entropy

1/2 * 2 + 1/4 * 2 + 1/8 * 2 + 1/8 * 2 = 2 bits

1/2 * 1 + 1/4 * 2 + 1/8 * 3 + 1/8 * 3 = 7/4 = 1.75 bits.

# Cross-entropy

[图片上传失败...(image-800784-1512981095761)]

1 * 1/8+2 * 1/2+3 * 1/4+3 * 1/4 = 17/8 = 2.25

Hp(q)=∑q(x) * log2(1/p(x))

# 训练模型

[图片上传失败...(image-2f704c-1512981095761)]
y 是我们预测的概率分布, y' 是实际的分布（我们输入的one-hot vector)。

``````y_ = tf.placeholder("float", [None,10])
``````

``````cross_entropy = -tf.reduce_sum(y_*tf.log(y))
``````

``````train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
``````

TensorFlow在这里实际上所做的是，它会在后台给描述你的计算的那张图里面增加一系列新的计算操作单元用于实现反向传播算法和梯度下降算法。然后，它返回给你的只是一个单一的操作，当运行这个操作时，它用梯度下降算法训练你的模型，微调你的变量，不断减少成本。

``````init = tf.initialize_all_variables()
``````

``````sess = tf.Session()
sess.run(init)
``````

``````for i in range(1000):
batch_xs, batch_ys = mnist.train.next_batch(100)
sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
``````

## 评估模型

``````correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
``````

``````accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
``````

``````print sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})
``````