How do I move time series data from yesterday to today in Pandas?… here is a solution to the problem.
How do I move time series data from yesterday to today in Pandas?
I have the data frame below the cross day (12-02~1203): I want to move yesterday’s data (12-02 22:00~00:00) to today’s data (12-03) every day. Date/time is multi-indexed. This is what I need to analyze the data, and it is getting more convenient day by day. But now I need to analyze the data including yesterday’s last 2 hours… So I need this data frame operation.
..
date time a b
2015-12-02 21:00:00 23.97 0
2015-12-02 21:15:00 24.06 0
2015-12-02 21:30:00 24.03 0
2015-12-02 21:45:00 23.99 0
2015-12-02 22:00:00 24.03 0
2015-12-02 22:15:00 23.89 0
2015-12-02 22:30:00 23.71 0
2015-12-02 22:45:00 23.64 0
2015-12-02 23:00:00 23.29 0
2015-12-02 23:15:00 23.8 0
2015-12-02 23:30:00 23.82 0
2015-12-02 23:45:00 23.86 0
2015-12-03 0:00:00 23.66 0
2015-12-03 0:15:00 23.64 0
2015-12-03 0:30:00 23.7 0
2015-12-03 0:45:00 23.69 0
2015-12-03 1:00:00 23.65 0
2015-12-03 1:15:00 23.48 0
2015-12-03 1:30:00 23.45 0
..
The result should look like this (12-02 22:00~23:45 Data is moved to 12-03 What should I do?)
..
2015-12-02 21:00:00 23.97 0
2015-12-02 21:15:00 24.06 0
2015-12-02 21:30:00 24.03 0
2015-12-02 21:45:00 23.99 0
2015-12-03 22:00:00 24.03 0
2015-12-03 22:15:00 23.89 0
2015-12-03 22:30:00 23.71 0
2015-12-03 22:45:00 23.64 0
2015-12-03 23:00:00 23.29 0
2015-12-03 23:15:00 23.8 0
2015-12-03 23:30:00 23.82 0
2015-12-03 23:45:00 23.86 0
2015-12-03 0:00:00 23.66 0
2015-12-03 0:15:00 23.64 0
2015-12-03 0:30:00 23.7 0
2015-12-03 0:45:00 23.69 0
2015-12-03 1:00:00 23.65 0
2015-12-03 1:15:00 23.48 0
2015-12-03 1:30:00 23.45 0
..
Solution
I think you need:
from datetime import date, datetime, time, timedelta
m = df.index.get_level_values(1) < time(22,0,0)
idx1 = df.index.get_level_values(0)
idx2 = df.index.get_level_values(1)
df.index = [idx1.where(m, idx1 + timedelta(days=1)), idx2]
print (df)
a b
date time
2015-12-02 21:00:00 23.97 0
21:15:00 24.06 0
21:30:00 24.03 0
21:45:00 23.99 0
2015-12-03 22:00:00 24.03 0
22:15:00 23.89 0
22:30:00 23.71 0
22:45:00 23.64 0
23:00:00 23.29 0
23:15:00 23.80 0
23:30:00 23.82 0
23:45:00 23.86 0
00:00:00 23.66 0
00:15:00 23.64 0
00:30:00 23.70 0
00:45:00 23.69 0
01:00:00 23.65 0
01:15:00 23.48 0
01:30:00 23.45 0