diff options
author | Vijay Vasudevan <vrv@google.com> | 2015-12-06 14:53:28 -0800 |
---|---|---|
committer | Vijay Vasudevan <vrv@google.com> | 2015-12-06 14:53:28 -0800 |
commit | f9d3e9d03c69bfac77a2fe1ad80f7c5aa517e0f0 (patch) | |
tree | 52302a06eae969c8f4e1d7af6749a85fe0ac4eb1 /tensorflow/models/image | |
parent | 40d0d2904e8e00d3c4bf43fa62130eeebceef147 (diff) |
TensorFlow: upstream latest changes to git.
Change 109537918
TensorFlow pip setup: wheel >= 0.26 for python3 pip install
Change 109505848
Fix distortion default value to 1.0 in fixed_unigram_candidate_sampler. This means we default to the actual provided unigram distribution, instead of to the uniform (as it is currently).
Change 109470494
Bugfix in gradients calculation when the ys rely on each other.
Change 109467619
Fix CIFAR-10 model to train on all the training data instead of just 80% of it. Fixes #396.
Change 109467557
Replaced checkpoint file with binary GraphDef.
Change 109467433
Updates to C++ tutorial section.
Change 109465269
TensorFlow: update documentation for tutorials to not assume use of bazel
(when possible).
Change 109462916
A tutorial for image recognition to coincide with the release of the latest Inception image classification model.
Change 109462342
Clear control dependencies in variable_scope.get_variable() when creating
ops for the initializer.
Add tests of various error conditions.
Change 109461981
Various performance improvements in low-level node execution code paths.
Speeds up ptb_word_lm on my desktop with a Titan X from
3638 words per second to 3751 words per second (3.1% speedup).
Changes include:
o Avoided many strcmp operations per node execution and extra touches
of cache lines in executor.cc, by making all the various IsMerge,
IsSwitch, IsSend, etc. operations instead be based on an internal enum
value that is pre-computed at Node construction time, rather than doing
string comparisons against node->type_string(). We were doing about
6 such comparisons per executed node.
o Removed mutex_lock in executor.cc in ExecutorState::Process. The
lock was not needed and the comment about the iterations array being
potentially resized is not true (the iterations arrays are created
with a fixed size). Checked with yuanbyu to confirm this.
o Added new two-argument port::Tracing::ScopedAnnotation constructor
that takes two StringPiece arguments, and only concatenates them
lazily if tracing is enabled. Also changed the code in
platform/tracing.{h,cc} so that the ScopedAnnotation constructor and
the TraceMe constructor can be inlined.
o In BaseGPUDevice::Compute, used the two-argument ScopedAnnotation
constructor to avoid doing StrCat(opkernel->name(), ":",
op_kernel->type_string()) on every node execution on a GPU.
o Introduced a new TensorReference class that just holds a reference to an
underlying TensorBuffer, and requires an explicit Unref().
o Changed the EventMgr interface to take a vector of TensorReference objects
for EventMgr::ThenDeleteTensors, rather than a vector of Tensor objects.
o Used TensorReference in a few places in gpu_util.cc
o Minor: switched to using InlinedVectors in a few places to get better
cache locality.
Change 109456692
Updated the label_image example to use the latest Inception model
Change 109456545
Provides classify_image which performs image recognition on a 1000 object label set.
$ ./classify_image
giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca (score = 0.88493)
indri, indris, Indri indri, Indri brevicaudatus (score = 0.00878)
lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens (score = 0.00317)
custard apple (score = 0.00149)
earthstar (score = 0.00127)
Change 109455002
TensorFlow: make the helper libraries for various models available
in the pip package so that when users type:
python translate.py ...
the absolute import works.
This change is supposed to help make our tutorials run without the
*need* to use bazel.
Change 109450041
TensorFlow: remove cifar and convolutional binary copies from pip install.
Adds embedding and some other models to the list.
Change 109448520
Move the description of a failing invariant from a comment into the dcheck-fail message text.
Change 109447577
TensorBoard has release tagging (tensorboard/TAG)
Also track TensorBoard changes (tensorboard/CHANGES)
Change 109444161
Added ParseSingleSequenceExample + python wrappers + unit tests.
Change 109440864
Update all the TensorFlow Dockerfiles, and simplify GPU containers.
This change updates all four of our Dockerfiles to match the targets discussed
in https://github.com/tensorflow/tensorflow/issues/149. The most notable
change here is moving the GPU images to use the NVidia containers which
include cudnn and other build-time dependencies, dramatically simplifying both
the build and run steps.
A description of which tags exist and get pushed where will be in a follow-up.
Change 109432591
Some pylint and pydoc changes in saver.
Change 109430127
Remove unused hydrogen components
Change 109419354
The RNN api, although moved into python/ops/, remains undocumented.
It may still change at any time.
Base CL: 109538006
Diffstat (limited to 'tensorflow/models/image')
-rw-r--r-- | tensorflow/models/image/cifar10/__init__.py | 22 | ||||
-rw-r--r-- | tensorflow/models/image/cifar10/cifar10.py | 4 | ||||
-rw-r--r-- | tensorflow/models/image/imagenet/BUILD | 29 | ||||
-rw-r--r-- | tensorflow/models/image/imagenet/classify_image.py | 214 |
4 files changed, 267 insertions, 2 deletions
diff --git a/tensorflow/models/image/cifar10/__init__.py b/tensorflow/models/image/cifar10/__init__.py index e69de29bb2..535ef6a05d 100644 --- a/tensorflow/models/image/cifar10/__init__.py +++ b/tensorflow/models/image/cifar10/__init__.py @@ -0,0 +1,22 @@ +# Copyright 2015 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== + +"""Makes helper libraries available in the cifar10 package.""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +from tensorflow.models.image.cifar10 import cifar10 +from tensorflow.models.image.cifar10 import cifar10_input diff --git a/tensorflow/models/image/cifar10/cifar10.py b/tensorflow/models/image/cifar10/cifar10.py index 7f5df387f4..b9b89473e8 100644 --- a/tensorflow/models/image/cifar10/cifar10.py +++ b/tensorflow/models/image/cifar10/cifar10.py @@ -180,7 +180,7 @@ def distorted_inputs(): """ filenames = [os.path.join(FLAGS.data_dir, 'cifar-10-batches-bin', 'data_batch_%d.bin' % i) - for i in xrange(1, 5)] + for i in xrange(1, 6)] for f in filenames: if not gfile.Exists(f): raise ValueError('Failed to find file: ' + f) @@ -245,7 +245,7 @@ def inputs(eval_data): if not eval_data: filenames = [os.path.join(FLAGS.data_dir, 'cifar-10-batches-bin', 'data_batch_%d.bin' % i) - for i in xrange(1, 5)] + for i in xrange(1, 6)] num_examples_per_epoch = NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN else: filenames = [os.path.join(FLAGS.data_dir, 'cifar-10-batches-bin', diff --git a/tensorflow/models/image/imagenet/BUILD b/tensorflow/models/image/imagenet/BUILD new file mode 100644 index 0000000000..7d77c4894d --- /dev/null +++ b/tensorflow/models/image/imagenet/BUILD @@ -0,0 +1,29 @@ +# Description: +# Example TensorFlow models for ImageNet. + +licenses(["notice"]) # Apache 2.0 + +exports_files(["LICENSE"]) + +py_binary( + name = "classify_image", + srcs = [ + "classify_image.py", + ], + visibility = ["//tensorflow:__subpackages__"], + deps = [ + "//tensorflow:tensorflow_py", + ], +) + +filegroup( + name = "all_files", + srcs = glob( + ["**/*"], + exclude = [ + "**/METADATA", + "**/OWNERS", + ], + ), + visibility = ["//tensorflow:__subpackages__"], +) diff --git a/tensorflow/models/image/imagenet/classify_image.py b/tensorflow/models/image/imagenet/classify_image.py new file mode 100644 index 0000000000..9461377bd0 --- /dev/null +++ b/tensorflow/models/image/imagenet/classify_image.py @@ -0,0 +1,214 @@ +# Copyright 2015 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== + +"""Simple image classification with Inception. + +Run image classification with Inception trained on ImageNet 2012 Challenge data +set. + +This program creates a graph from a saved GraphDef protocol buffer, +and runs inference on an input JPEG image. It outputs human readable +strings of the top 5 predictions along with their probabilities. + +Change the --image_file argument to any jpg image to compute a +classification of that image. + +Please see the tutorial and website for a detailed description of how +to use this script to perform image recognition. + +https://tensorflow.org/tutorials/image_recognition/ +""" + +import os.path +import re +import sys +import tarfile + +# pylint: disable=unused-import,g-bad-import-order +import tensorflow.python.platform +from six.moves import urllib +import numpy as np +import tensorflow as tf +# pylint: enable=unused-import,g-bad-import-order + +from tensorflow.python.platform import gfile + +FLAGS = tf.app.flags.FLAGS + +# classify_image_graph_def.pb: +# Binary representation of the GraphDef protocol buffer. +# imagenet_synset_to_human_label_map.txt: +# Map from synset ID to a human readable string. +# imagenet_2012_challenge_label_map_proto.pbtxt: +# Text representation of a protocol buffer mapping a label to synset ID. +tf.app.flags.DEFINE_string( + 'model_dir', '/tmp/imagenet', + """Path to classify_image_graph_def.pb, """ + """imagenet_synset_to_human_label_map.txt, and """ + """imagenet_2012_challenge_label_map_proto.pbtxt.""") +tf.app.flags.DEFINE_string('image_file', '', + """Absolute path to image file.""") +tf.app.flags.DEFINE_integer('num_top_predictions', 5, + """Display this many predictions.""") + +# pylint: disable=line-too-long +DATA_URL = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' +# pylint: enable=line-too-long + + +class NodeLookup(object): + """Converts integer node ID's to human readable labels.""" + + def __init__(self, + label_lookup_path=None, + uid_lookup_path=None): + if not label_lookup_path: + label_lookup_path = os.path.join( + FLAGS.model_dir, 'imagenet_2012_challenge_label_map_proto.pbtxt') + if not uid_lookup_path: + uid_lookup_path = os.path.join( + FLAGS.model_dir, 'imagenet_synset_to_human_label_map.txt') + self.node_lookup = self.load(label_lookup_path, uid_lookup_path) + + def load(self, label_lookup_path, uid_lookup_path): + """Loads a human readable English name for each softmax node. + + Args: + label_lookup_path: string UID to integer node ID. + uid_lookup_path: string UID to human-readable string. + + Returns: + dict from integer node ID to human-readable string. + """ + if not gfile.Exists(uid_lookup_path): + tf.logging.fatal('File does not exist %s', uid_lookup_path) + if not gfile.Exists(label_lookup_path): + tf.logging.fatal('File does not exist %s', label_lookup_path) + + # Loads mapping from string UID to human-readable string + proto_as_ascii_lines = gfile.GFile(uid_lookup_path).readlines() + uid_to_human = {} + p = re.compile(r'[n\d]*[ \S,]*') + for line in proto_as_ascii_lines: + parsed_items = p.findall(line) + uid = parsed_items[0] + human_string = parsed_items[2] + uid_to_human[uid] = human_string + + # Loads mapping from string UID to integer node ID. + node_id_to_uid = {} + proto_as_ascii = gfile.GFile(label_lookup_path).readlines() + for line in proto_as_ascii: + if line.startswith(' target_class:'): + target_class = int(line.split(': ')[1]) + if line.startswith(' target_class_string:'): + target_class_string = line.split(': ')[1] + node_id_to_uid[target_class] = target_class_string[1:-2] + + # Loads the final mapping of integer node ID to human-readable string + node_id_to_name = {} + for key, val in node_id_to_uid.iteritems(): + if val not in uid_to_human: + tf.logging.fatal('Failed to locate: %s', val) + name = uid_to_human[val] + node_id_to_name[key] = name + + return node_id_to_name + + def id_to_string(self, node_id): + if node_id not in self.node_lookup: + return '' + return self.node_lookup[node_id] + + +def create_graph(): + """"Creates a graph from saved GraphDef file and returns a saver.""" + # Creates graph from saved graph_def.pb. + with gfile.FastGFile(os.path.join( + FLAGS.model_dir, 'classify_image_graph_def.pb'), 'r') as f: + graph_def = tf.GraphDef() + graph_def.ParseFromString(f.read()) + _ = tf.import_graph_def(graph_def, name='') + + +def run_inference_on_image(image): + """Runs inference on an image. + + Args: + image: Image file name. + + Returns: + Nothing + """ + if not gfile.Exists(image): + tf.logging.fatal('File does not exist %s', image) + image_data = gfile.FastGFile(image).read() + + # Creates graph from saved GraphDef. + create_graph() + + with tf.Session() as sess: + # Some useful tensors: + # 'softmax:0': A tensor containing the normalized prediction across + # 1000 labels. + # 'pool_3:0': A tensor containing the next-to-last layer containing 2048 + # float description of the image. + # 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG + # encoding of the image. + # Runs the softmax tensor by feeding the image_data as input to the graph. + softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') + predictions = sess.run(softmax_tensor, + {'DecodeJpeg/contents:0': image_data}) + predictions = np.squeeze(predictions) + + # Creates node ID --> English string lookup. + node_lookup = NodeLookup() + + top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1] + for node_id in top_k: + human_string = node_lookup.id_to_string(node_id) + score = predictions[node_id] + print '%s (score = %.5f)' % (human_string, score) + + +def maybe_download_and_extract(): + """Download and extract model tar file.""" + dest_directory = FLAGS.model_dir + if not os.path.exists(dest_directory): + os.makedirs(dest_directory) + filename = DATA_URL.split('/')[-1] + filepath = os.path.join(dest_directory, filename) + if not os.path.exists(filepath): + def _progress(count, block_size, total_size): + sys.stdout.write('\r>> Downloading %s %.1f%%' % ( + filename, float(count * block_size) / float(total_size) * 100.0)) + sys.stdout.flush() + filepath, _ = urllib.request.urlretrieve(DATA_URL, filepath, + reporthook=_progress) + print() + statinfo = os.stat(filepath) + print('Succesfully downloaded', filename, statinfo.st_size, 'bytes.') + tarfile.open(filepath, 'r:gz').extractall(dest_directory) + + +def main(_): + maybe_download_and_extract() + image = (FLAGS.image_file if FLAGS.image_file else + os.path.join(FLAGS.model_dir, 'cropped_panda.jpg')) + run_inference_on_image(image) + + +if __name__ == '__main__': + tf.app.run() |