Get the Pandas series from csv
I’m completely new to machine learning, and I’m currently playing with MNIST machine learning, using RandomForestClassifier.
I use sklearn and panda.
I have a training CSV dataset.
import pandas as pd
import numpy as np
from sklearn import model_selection
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import LinearSVC
from sklearn.linear_model import SGDClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score
train = pd.read_csv("train.csv")
features = train.columns[1:]
X = train[features]
y = train['label']
user_train = pd.read_csv("input.csv")
user_features = user_train.columns[1:]
y_train = user_train[user_features]
user_y = user_train['label']
X_train, X_test, y_train, y_test = model_selection.train_test_split(X/255.,y,test_size=1,random_state=0)
clf_rf = RandomForestClassifier()
clf_rf.fit(X_train, y_train)
y_pred_rf = clf_rf.predict(X_test)
acc_rf = accuracy_score(y_test, y_pred_rf)
print("pred : ", y_pred_rf)
print("random forest accuracy: ",acc_rf)
I have the current code and it works fine. It takes the training set, splits and takes an element for testing, and then makes predictions.
What I want now is to use test data from inputs, I have a new csv called “input.csv” and I want to predict the values in this csv.
How do I replace model_selection.train_test_split with my input data?
I’m sure the response is pretty obvious, but I didn’t find anything.
Solution
The following parts of your code are not used
user_train = pd.read_csv("input.csv")
user_features = user_train.columns[1:]
y_train = user_train[user_features]
user_y = user_train['label']
If input.csv has the same structure as train.csv, you may need:
Train the classifier and test it on the split of the input.csv dataset: (See http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html learn how to set the test size).
input_train = pd.read_csv("input.csv") input_features = user_train.columns[1:] input_data = user_train[input_features] input_labels = user_train['label'] data_train, data_test, labels_train, labels_test = model_selection.train_test_split(input_data/255.,input_labels,test_size=1,random_state=0) clf_rf = RandomForestClassifier() clf_rf.fit(data_train, labels_train) labels_pred_rf = clf_rf.predict(data_test) acc_rf = accuracy_score(labels_test, labels_pred_rf)
Test the previously trained classifier on the entire input.csv file
input_train = pd.read_csv("input.csv") input_features = user_train.columns[1:] input_data = user_train[input_features] input_labels = user_train['label'] labels_pred_rf = clf_rf.predict(input_data) acc_rf = accuracy_score(input_labels, labels_pred_rf)