Python – How do I manually remove a tensor from memory?

I’m having some issues with how TensorFlow handles memory. After each iteration of this toy example, I want to remove the tensor from my memory.

I’m using tensorflow EagerExecution. I’ve tried using variables and simple tensors. tf.assign does not do this work. More and more memory is being used. I guess this is normal to be able to calculate the gradient. Even if I apply some virtual optimizer at the end of each iteration, the memory is not freed (more precisely, it happens sometimes, but the global trend is that memory usage is growing).

So can it be removed manually?

import tensorflow as tf
import tensorflow.contrib.eager as tfe
import numpy as np
import time as ti

tf.enable_eager_execution()

for i in range(150):
    all_subject=tfe. Variable(np.random.rand(200, 500), dtype=tf.float32)
    tf.assign(all_subject, np.random.rand(200,500) )
    ti.sleep(1.0)
    del all_subject
    ti.sleep(0.5)

Distribution:

Memory profile

According to the documentation on eager execution ,

During eager execution the lifetime of state objects is determined by the lifetime of their corresponding Python object.

Therefore, you should not observe any memory leaks in your code even without an explicit del: the simple fact of reassigning variables to other objects should free up memory.

However, this is not the case and I observed the same memory leak as you.

So this could be a (serious) bug, and you can file here