Python机器学习项目

...
X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_output])
keep_prob = tf.placeholder(tf.float32)

...
weights = {
    'w1': tf.Variable(tf.truncated_normal([n_input, n_hidden1], stddev=0.1)),
    'w2': tf.Variable(tf.truncated_normal([n_hidden1, n_hidden2], stddev=0.1)),
    'w3': tf.Variable(tf.truncated_normal([n_hidden2, n_hidden3], stddev=0.1)),
    'out': tf.Variable(tf.truncated_normal([n_hidden3, n_output], stddev=0.1)),
}

...
biases = {
    'b1': tf.Variable(tf.constant(0.1, shape=[n_hidden1])),
    'b2': tf.Variable(tf.constant(0.1, shape=[n_hidden2])),
    'b3': tf.Variable(tf.constant(0.1, shape=[n_hidden3])),
    'out': tf.Variable(tf.constant(0.1, shape=[n_output]))
}

...
layer_1 = tf.add(tf.matmul(X, weights['w1']), biases['b1'])
layer_2 = tf.add(tf.matmul(layer_1, weights['w2']), biases['b2'])
layer_3 = tf.add(tf.matmul(layer_2, weights['w3']), biases['b3'])
layer_drop = tf.nn.dropout(layer_3, keep_prob)
output_layer = tf.matmul(layer_3, weights['out']) + biases['out']

...
cross_entropy = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(
        labels=Y, logits=output_layer
    ))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

Step 4 — Building the TensorFlow Graph
To build our network, we will set up the network as a computational
graph for TensorFlow to execute. The core concept of TensorFlow is the
tensor, a data structure similar to an array or list. initialized, manipulated
as they are passed through the graph, and updated through the learning
process.
We’ll start by defining three tensors as placeholders, which are tensors
that we’ll feed values into later. Add the following to your file:
main.py
...
X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_output])
keep_prob = tf.placeholder(tf.float32)
The only parameter that needs to be specified at its declaration is the
size of the data we will be feeding in. For X we use a shape of [None,
784], where None represents any amount, as we will be feeding in an
undefined number of 784-pixel images. The shape of Y is [None, 10] as
we will be using it for an undefined number of label outputs, with 10
possible classes. The keep_prob tensor is used to control the dropout
rate, and we initialize it as a placeholder rather than an immutable
variable because we want to use the same tensor both for training (when
dropout is set to 0.5) and testing (when dropout is set to 1.0).
The parameters that the network will update in the training process are
the weight and bias values, so for these we need to set an initial value
rather than an empty placeholder. These values are essentially where the
network does its learning, as they are used in the activation functions of
the neurons, representing the strength of the connections between units.
Since the values are optimized during training, we could set them to
zero for now. But the initial value actually has a significant impact on the
final accuracy of the model. We’ll use random values from a truncated
normal distribution for the weights. We want them to be close to zero, so
they can adjust in either a positive or negative direction, and slightly
different, so they generate different errors. This will ensure that the
model learns something useful. Add these lines:
main.py
...
weights = {
'w1': tf.Variable(tf.truncated_normal([n_input, n_hidden1],
stddev=0.1)),
'w2': tf.Variable(tf.truncated_normal([n_hidden1, n_hidden2],
stddev=0.1)),
'w3': tf.Variable(tf.truncated_normal([n_hidden2, n_hidden3],
stddev=0.1)),
'out': tf.Variable(tf.truncated_normal([n_hidden3, n_output],
stddev=0.1)),
}
For the bias, we use a small constant value to ensure that the tensors
activate in the intial stages and therefore contribute to the propagation.
The weights and bias tensors are stored in dictionary objects for ease of
access. Add this code to your file to define the biases:
main.py
...
biases = {
'b1': tf.Variable(tf.constant(0.1, shape=[n_hidden1])),
'b2': tf.Variable(tf.constant(0.1, shape=[n_hidden2])),
'b3': tf.Variable(tf.constant(0.1, shape=[n_hidden3])),
'out': tf.Variable(tf.constant(0.1, shape=[n_output]))
}
Next, set up the layers of the network by defining the operations that
will manipulate the tensors. Add these lines to your file:
main.py
...
layer_1 = tf.add(tf.matmul(X, weights['w1']), biases['b1'])
layer_2 = tf.add(tf.matmul(layer_1, weights['w2']), biases['b2'])
layer_3 = tf.add(tf.matmul(layer_2, weights['w3']), biases['b3'])
layer_drop = tf.nn.dropout(layer_3, keep_prob)
output_layer = tf.matmul(layer_3, weights['out']) + biases['out']
Each hidden layer will execute matrix multiplication on the previous
layer’s outputs and the current layer’s weights, and add the bias to these
values. At the last hidden layer, we will apply a dropout operation using
our keep_prob value of 0.5.
The final step in building the graph is to define the loss function that
we want to optimize. A popular choice of loss function in TensorFlow
programs is cross-entropy, also known as log-loss, which quantifies the
difference between two probability distributions (the predictions and the
labels). A perfect classification would result in a cross-entropy of 0, with
the loss completely minimized.
We also need to choose the optimization algorithm which will be used
to minimize the loss function. A process named gradient descent
optimization is a common method for finding the (local) minimum of a
function by taking iterative steps along the gradient in a negative
(descending) direction. There are several choices of gradient descent
optimization algorithms already implemented in TensorFlow, and in this
tutorial we will be using the Adam optimizer. This extends upon
gradient descent optimization by using momentum to speed up the
process through computing an exponentially weighted average of the
gradients and using that in the adjustments. Add the following code to
your file:
main.py
...
cross_entropy = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(
labels=Y, logits=output_layer
))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
We’ve now defined the network and built it out with TensorFlow. The
next step is to feed data through the graph to train it, and then test that it
has actually learnt something.