Pandas dataframe, replacing the last line with iloc
I’m trying to replace the last row of the Pandas data frame with iloc, but I can’t get it to work. There are a lot of solutions out there, but the simplest (and slowest) are:
How to do a FIFO push-operation for rows on Pandas dataframe in Python?
Why doesn’t this method work in the code below?
def append_from_dataframe(self,timeframe,new_dataframe):
new_dataframe.reset_index(inplace=True)
temp_dataframe = self.timeframedict.get(timeframe)
num_rows_existing = temp_dataframe.shape[0]
num_rows_new = new_dataframe.shape[0]
overlap = (num_rows_existing + num_rows_new) - 500
# slow, replace with numpy array eventually
if overlap >= 1:
# number of rows to shift
i = overlap * -1
#shift the dataframe back in time
temp_dataframe = temp_dataframe.shift(i)
#self.timeframedict.get(timeframe) = self.timeframedict.get(timeframe).shift(overlap)
#replace the last i rows with the new values
temp_dataframe.iloc[i:] = new_dataframe
self.timeframedict.update({timeframe:temp_dataframe})
else:
#TODO - see this https://stackoverflow.com/questions/10715965/add-one-row-in-a-pandas-dataframe
self.timeframedict.update({timeframe:self.timeframedict.get(timeframe).append(new_dataframe)})
Replace another row with the contents of the data frame:
ipdb> new_dataframe
Timestamp Open High Low Close Volume localtime
0 1533174420000 423.43 423.44 423.43 423.44 0.73765 1533174423776
temp_dataframe.shift(i)
moves the value back to one bit, replacing the value – with NaN
ipdb> temp_dataframe.iloc[i:]
Timestamp Open High Low Close Volume localtime
499 NaN NaN NaN NaN NaN NaN NaN
But temp_dataframe.iloc[i:] = new_dataframe
does not replace anything.
EDIT: I
should add that now I can replace 1 line with:
temp_dataframe.iloc[-1] = new_dataframe.iloc[0]
However, I can’t get the multi-line version to work
Solution
df = pd. DataFrame({'a':[1,2,3,4,5],'b':['foo','bar','foobar','foobaz','food']})
Output:
df
Out[117]:
a b
0 1 foo
1 2 bar
2 3 foobar
3 4 foobaz
4 5 food
Replace the last two lines (foobaz and food) with the second and first rows, respectively:
df.iloc[-2:]=[df.iloc[1],df.iloc[0]]
df
Out[119]:
a b
0 1 foo
1 2 bar
2 3 foobar
3 2 bar
4 1 foo
You can also do this to get the same result:
df.iloc[-2:]=df.iloc[1::-1].values