Python recursive error: Simple operation crashes with Pandas. eval()

Python recursive error: Simple operation crashes with Pandas. eval() … here is a solution to the problem.

Python recursive error: Simple operation crashes with Pandas. eval()

I just read and drooled with excitement these Newly found optimization functions for my Pandas related needs. According to the book:

The DataFrame.eval() method allows much more succinct evaluation of expressions with the columns:

result3 = df.eval('(A + B) / (C - 1)') 
np.allclose(result1, result3)

True

Take me as an example:

My data frame contains about 42,000 records and 28 columns. Two of them are Date and Heure, which are strings.

My goal: to concatenate two columns into one. I can easily do it using this code: df_exade_light ["Date"] + df_exade_light["Heure"] to apply %timeit returns to it

6.07 ms ± 219 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

But for some reason df.eval('Date + Heure') returns a :

RecursionError: maximum recursion depth exceeded

Also, I applied this thread in The solution found increases the allowed stack depth, but the kernel just crashes.

What is the reason for this? Am I doing something wrong?


The problem can be reproduced with this code:

import pandas as pd

df = pd. DataFrame({'A': ['X','Y'],
                   'B': ['U','V']})

df.eval('A+B')

Solution

The problem in your reproducible example is that you have strings. In the link you gave High-performance Pandas: eval() and query(), all examples with float (or int).

One way to make it work with your example is to use Python as the engine:

df.eval('A+B',engine='python')

According to documentation, by default, eval The engine used is ‘numexpr' and this engine uses the library of the same nameNumExpr, which is a fast numeric expression evaluator for NumPy 。 Although in the previous link, an example with a string is given, it does not use the + operation. If you execute df.eval(‘A==B’) it works, as with other comparison operators, but not df.eval(A+B'). You can find more informationthere, but for string, except for using engine=' Outside of Python', it seems limited.

Going back to your original date and time type question, not sure if you can find the default engine ( see here for supported datatype ) solution

Related Problems and Solutions