Python – What is the correct way to reason about a batch of images in Tensorflow for performance purposes

What is the correct way to reason about a batch of images in Tensorflow for performance purposes… here is a solution to the problem.

What is the correct way to reason about a batch of images in Tensorflow for performance purposes

What I want to accomplish is to use a batch of images at a time instead of a single image for inference in Tensorflow. I wonder what is the most appropriate way to process multiple images to speed up inference?

Inference on individual images is easy to do and is used in most tutorials, but I haven’t seen inference done in a batch-like way.

This is what I’m currently using at a high level :

pl = tf.placeholder(tf.float32)
...
sess.run([boxes, confs], feed_dict={pl: image})

I would appreciate any comments.

Solution

Depending on how your model is designed, you can provide only one set of images to the PL. The first dimension of the output then corresponds to the index of the image in the batch.

Many tensor operations have an implementation of multiple examples in a batch. However, there are exceptions, such as tf.image.decode_jpeg You will have to rewrite your network with tf.map_fn. For example.

Related Problems and Solutions