Python – You can’t pick le dictionaries in Django

You can’t pick le dictionaries in Django… here is a solution to the problem.

You can’t pick le dictionaries in Django

I

have a simple dictionary that I’m trying to save to the cache and it looks like it’s what django is trying to pickle :

  podcasts = []
  for i in items:
        s = re.sub('[\s+]', '', str(i))
        s2 = re.findall(r'<link/>(.*?) <itunes',s)[0]

item_obj = {}
        item_obj['title'] = title
        item_obj['url'] = s2
        item_obj['created_at'] = created_at
        item_obj['duration'] = duration

podcasts.append(item_obj)

This has a very simple output format:

[{'title': "Podcast1", 'url': 'https://example.com\\n', 'created_at': 'Thu, 28 Dec 2017', 'duration': '00:30:34'}]

I’m running it with a custom admin command like this :

python3 manage.py podcast_job

I tried saving to cache :

    podcasts = get_podcasts()
    print(podcasts)
    cache.set('podcasts', podcasts)

I get the error :

File "podcast_job.py", line 13, in handle
    cache.set('podcasts', podcasts)
  File "python3.6/site-packages/django_redis/cache.py", line 33, in _decorator
    return method(self, *args, **kwargs)
  File "python3.6/site-packages/django_redis/cache.py", line 68, in set
    return self.client.set(*args, **kwargs)
  File "python3.6/site-packages/django_redis/client/default.py", line 109, in set
    nvalue = self.encode(value)
  File "python3.6/site-packages/django_redis/client/default.py", line 329, in encode
    value = self._serializer.dumps(value)
  File "python3.6/site-packages/django_redis/serializers/pickle.py", line 33, in dumps
    return pickle.dumps(value, self._pickle_version)

RecursionError: maximum recursion depth exceeded while calling a Python object 

If I try to save with a string, I don’t get any error and save well :

   cache.set('podcasts', str(podcasts))

How do I save a list of dictionaries without the above error?

Solution

If you use the datetime object for created_at and duration, make sure to render them as strings.

Related Problems and Solutions