Creatingdummyvariables

21 minute read

Creating dummy variables

import pandas as pd
train = pd.read_csv('http://bit.ly/kaggletrain')
train.head()
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S

Let’s say that we want to create a dummy variable for the Sex column. Currently it is specified as an object column because it is represented as strings.

One way to do this is by using the Series map method.

train['Sex_male'] = train.Sex.map({'male':1, 'female':0})
train.head()
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Sex_male
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 0
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 0
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 0
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S 1

As you can see now we have created a new column called Sex_male by using the map function.

So, this is one way of doing it. But there is a more flexible way of doing this. For doing this, I am going to show you how to use a top-level (by this i mean the pd level function)

pd.get_dummies(train.Sex)
female male
0 0 1
1 1 0
2 1 0
3 1 0
4 0 1
5 0 1
6 0 1
7 0 1
8 1 0
9 1 0
10 1 0
11 1 0
12 0 1
13 0 1
14 1 0
15 1 0
16 0 1
17 0 1
18 1 0
19 1 0
20 0 1
21 0 1
22 1 0
23 0 1
24 1 0
25 1 0
26 0 1
27 0 1
28 1 0
29 0 1
... ... ...
861 0 1
862 1 0
863 1 0
864 0 1
865 1 0
866 1 0
867 0 1
868 0 1
869 0 1
870 0 1
871 1 0
872 0 1
873 0 1
874 1 0
875 1 0
876 0 1
877 0 1
878 0 1
879 1 0
880 1 0
881 0 1
882 1 0
883 0 1
884 0 1
885 1 0
886 0 1
887 1 0
888 1 0
889 0 1
890 0 1

891 rows × 2 columns

And what this does is create one column for every possible value. So there are 2 possible values: male and female. Now generally speaking, if you have k levels for a categorical variable you use k-1 dummy variables to represent it. In other words, we have two possible values for our categorical variable Sex, so we need have one dummy variable to represent it. That is, we need drop 1 column from this which becomes our reference level.

pd.get_dummies(train.Sex).iloc[:,1:]
male
0 1
1 0
2 0
3 0
4 1
5 1
6 1
7 1
8 0
9 0
10 0
11 0
12 1
13 1
14 0
15 0
16 1
17 1
18 0
19 0
20 1
21 1
22 0
23 1
24 0
25 0
26 1
27 1
28 0
29 1
... ...
861 1
862 0
863 0
864 1
865 0
866 0
867 1
868 1
869 1
870 1
871 0
872 1
873 1
874 0
875 0
876 1
877 1
878 1
879 0
880 0
881 1
882 0
883 1
884 1
885 0
886 1
887 0
888 0
889 1
890 1

891 rows × 1 columns

There is one more thing to do, becuase I want to add this column back to the original dataframe, I need to know where it came from. So in order to do that, I need to add prefix=Sex.

pd.get_dummies(train.Sex, prefix='Sex').iloc[:,1:]
Sex_male
0 1
1 0
2 0
3 0
4 1
5 1
6 1
7 1
8 0
9 0
10 0
11 0
12 1
13 1
14 0
15 0
16 1
17 1
18 0
19 0
20 1
21 1
22 0
23 1
24 0
25 0
26 1
27 1
28 0
29 1
... ...
861 1
862 0
863 0
864 1
865 0
866 0
867 1
868 1
869 1
870 1
871 0
872 1
873 1
874 0
875 0
876 1
877 1
878 1
879 0
880 0
881 1
882 0
883 1
884 1
885 0
886 1
887 0
888 0
889 1
890 1

891 rows × 1 columns

When we do this, notice how it adds the prefix, that is, it added Sex_ to the level. It puts the underscore by default. Now, if you compare this with what we did above with the map function, you will notice that it is the same result.

This seems like more work than using the map function. But I want to show you how using this is more flexible.

Now, let use the get_dummies for a categorical variable which has more than 2 possible levels.

For instance, there is variable called Embarked.

train.Embarked.value_counts()
S    644
C    168
Q     77
Name: Embarked, dtype: int64
pd.get_dummies(train.Embarked, prefix='Embarked')
Embarked_C Embarked_Q Embarked_S
0 0 0 1
1 1 0 0
2 0 0 1
3 0 0 1
4 0 0 1
5 0 1 0
6 0 0 1
7 0 0 1
8 0 0 1
9 1 0 0
10 0 0 1
11 0 0 1
12 0 0 1
13 0 0 1
14 0 0 1
15 0 0 1
16 0 1 0
17 0 0 1
18 0 0 1
19 1 0 0
20 0 0 1
21 0 0 1
22 0 1 0
23 0 0 1
24 0 0 1
25 0 0 1
26 1 0 0
27 0 0 1
28 0 1 0
29 0 0 1
... ... ... ...
861 0 0 1
862 0 0 1
863 0 0 1
864 0 0 1
865 0 0 1
866 1 0 0
867 0 0 1
868 0 0 1
869 0 0 1
870 0 0 1
871 0 0 1
872 0 0 1
873 0 0 1
874 1 0 0
875 1 0 0
876 0 0 1
877 0 0 1
878 0 0 1
879 1 0 0
880 0 0 1
881 0 0 1
882 0 0 1
883 0 0 1
884 0 0 1
885 0 1 0
886 0 0 1
887 0 0 1
888 0 0 1
889 1 0 0
890 0 1 0

891 rows × 3 columns

pd.get_dummies(train.Embarked, prefix='Embarked').iloc[:,1:]
Embarked_Q Embarked_S
0 0 1
1 0 0
2 0 1
3 0 1
4 0 1
5 1 0
6 0 1
7 0 1
8 0 1
9 0 0
10 0 1
11 0 1
12 0 1
13 0 1
14 0 1
15 0 1
16 1 0
17 0 1
18 0 1
19 0 0
20 0 1
21 0 1
22 1 0
23 0 1
24 0 1
25 0 1
26 0 0
27 0 1
28 1 0
29 0 1
... ... ...
861 0 1
862 0 1
863 0 1
864 0 1
865 0 1
866 0 0
867 0 1
868 0 1
869 0 1
870 0 1
871 0 1
872 0 1
873 0 1
874 0 0
875 0 0
876 0 1
877 0 1
878 0 1
879 0 0
880 0 1
881 0 1
882 0 1
883 0 1
884 0 1
885 1 0
886 0 1
887 0 1
888 0 1
889 0 0
890 1 0

891 rows × 2 columns

Now we dropped the C column and established it as our baseline.

The last step is to attach this back to the original dataframe. But before that lets save it.

embarked_dummies = pd.get_dummies(train.Embarked, prefix='Embarked').iloc[:,1:]

Next, I am going to use another top-level function called concat. Which takes a list of dataframes and the axis along which to merge them.

pd.concat([train, embarked_dummies], axis=1).head()
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Sex_male Embarked_Q Embarked_S
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 0 1
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 0 0 0
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 0 0 1
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 0 0 1
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S 1 0 1
train = pd.concat([train, embarked_dummies], axis=1)
train.head()
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Sex_male Embarked_Q Embarked_S
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 0 1
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 0 0 0
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 0 0 1
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 0 0 1
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S 1 0 1

Let’s reset the data frame to illustrate how to pass a dataframe to pd.get_dummies() instead of passing a Series.

train = pd.read_csv('http://bit.ly/kaggletrain')
train.head()
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
pd.get_dummies(train, columns=['Sex', 'Embarked'])
PassengerId Survived Pclass Name Age SibSp Parch Ticket Fare Cabin Sex_female Sex_male Embarked_C Embarked_Q Embarked_S
0 1 0 3 Braund, Mr. Owen Harris 22.0 1 0 A/5 21171 7.2500 NaN 0 1 0 0 1
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... 38.0 1 0 PC 17599 71.2833 C85 1 0 1 0 0
2 3 1 3 Heikkinen, Miss. Laina 26.0 0 0 STON/O2. 3101282 7.9250 NaN 1 0 0 0 1
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) 35.0 1 0 113803 53.1000 C123 1 0 0 0 1
4 5 0 3 Allen, Mr. William Henry 35.0 0 0 373450 8.0500 NaN 0 1 0 0 1
5 6 0 3 Moran, Mr. James NaN 0 0 330877 8.4583 NaN 0 1 0 1 0
6 7 0 1 McCarthy, Mr. Timothy J 54.0 0 0 17463 51.8625 E46 0 1 0 0 1
7 8 0 3 Palsson, Master. Gosta Leonard 2.0 3 1 349909 21.0750 NaN 0 1 0 0 1
8 9 1 3 Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg) 27.0 0 2 347742 11.1333 NaN 1 0 0 0 1
9 10 1 2 Nasser, Mrs. Nicholas (Adele Achem) 14.0 1 0 237736 30.0708 NaN 1 0 1 0 0
10 11 1 3 Sandstrom, Miss. Marguerite Rut 4.0 1 1 PP 9549 16.7000 G6 1 0 0 0 1
11 12 1 1 Bonnell, Miss. Elizabeth 58.0 0 0 113783 26.5500 C103 1 0 0 0 1
12 13 0 3 Saundercock, Mr. William Henry 20.0 0 0 A/5. 2151 8.0500 NaN 0 1 0 0 1
13 14 0 3 Andersson, Mr. Anders Johan 39.0 1 5 347082 31.2750 NaN 0 1 0 0 1
14 15 0 3 Vestrom, Miss. Hulda Amanda Adolfina 14.0 0 0 350406 7.8542 NaN 1 0 0 0 1
15 16 1 2 Hewlett, Mrs. (Mary D Kingcome) 55.0 0 0 248706 16.0000 NaN 1 0 0 0 1
16 17 0 3 Rice, Master. Eugene 2.0 4 1 382652 29.1250 NaN 0 1 0 1 0
17 18 1 2 Williams, Mr. Charles Eugene NaN 0 0 244373 13.0000 NaN 0 1 0 0 1
18 19 0 3 Vander Planke, Mrs. Julius (Emelia Maria Vande... 31.0 1 0 345763 18.0000 NaN 1 0 0 0 1
19 20 1 3 Masselmani, Mrs. Fatima NaN 0 0 2649 7.2250 NaN 1 0 1 0 0
20 21 0 2 Fynney, Mr. Joseph J 35.0 0 0 239865 26.0000 NaN 0 1 0 0 1
21 22 1 2 Beesley, Mr. Lawrence 34.0 0 0 248698 13.0000 D56 0 1 0 0 1
22 23 1 3 McGowan, Miss. Anna "Annie" 15.0 0 0 330923 8.0292 NaN 1 0 0 1 0
23 24 1 1 Sloper, Mr. William Thompson 28.0 0 0 113788 35.5000 A6 0 1 0 0 1
24 25 0 3 Palsson, Miss. Torborg Danira 8.0 3 1 349909 21.0750 NaN 1 0 0 0 1
25 26 1 3 Asplund, Mrs. Carl Oscar (Selma Augusta Emilia... 38.0 1 5 347077 31.3875 NaN 1 0 0 0 1
26 27 0 3 Emir, Mr. Farred Chehab NaN 0 0 2631 7.2250 NaN 0 1 1 0 0
27 28 0 1 Fortune, Mr. Charles Alexander 19.0 3 2 19950 263.0000 C23 C25 C27 0 1 0 0 1
28 29 1 3 O'Dwyer, Miss. Ellen "Nellie" NaN 0 0 330959 7.8792 NaN 1 0 0 1 0
29 30 0 3 Todoroff, Mr. Lalio NaN 0 0 349216 7.8958 NaN 0 1 0 0 1
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
861 862 0 2 Giles, Mr. Frederick Edward 21.0 1 0 28134 11.5000 NaN 0 1 0 0 1
862 863 1 1 Swift, Mrs. Frederick Joel (Margaret Welles Ba... 48.0 0 0 17466 25.9292 D17 1 0 0 0 1
863 864 0 3 Sage, Miss. Dorothy Edith "Dolly" NaN 8 2 CA. 2343 69.5500 NaN 1 0 0 0 1
864 865 0 2 Gill, Mr. John William 24.0 0 0 233866 13.0000 NaN 0 1 0 0 1
865 866 1 2 Bystrom, Mrs. (Karolina) 42.0 0 0 236852 13.0000 NaN 1 0 0 0 1
866 867 1 2 Duran y More, Miss. Asuncion 27.0 1 0 SC/PARIS 2149 13.8583 NaN 1 0 1 0 0
867 868 0 1 Roebling, Mr. Washington Augustus II 31.0 0 0 PC 17590 50.4958 A24 0 1 0 0 1
868 869 0 3 van Melkebeke, Mr. Philemon NaN 0 0 345777 9.5000 NaN 0 1 0 0 1
869 870 1 3 Johnson, Master. Harold Theodor 4.0 1 1 347742 11.1333 NaN 0 1 0 0 1
870 871 0 3 Balkic, Mr. Cerin 26.0 0 0 349248 7.8958 NaN 0 1 0 0 1
871 872 1 1 Beckwith, Mrs. Richard Leonard (Sallie Monypeny) 47.0 1 1 11751 52.5542 D35 1 0 0 0 1
872 873 0 1 Carlsson, Mr. Frans Olof 33.0 0 0 695 5.0000 B51 B53 B55 0 1 0 0 1
873 874 0 3 Vander Cruyssen, Mr. Victor 47.0 0 0 345765 9.0000 NaN 0 1 0 0 1
874 875 1 2 Abelson, Mrs. Samuel (Hannah Wizosky) 28.0 1 0 P/PP 3381 24.0000 NaN 1 0 1 0 0
875 876 1 3 Najib, Miss. Adele Kiamie "Jane" 15.0 0 0 2667 7.2250 NaN 1 0 1 0 0
876 877 0 3 Gustafsson, Mr. Alfred Ossian 20.0 0 0 7534 9.8458 NaN 0 1 0 0 1
877 878 0 3 Petroff, Mr. Nedelio 19.0 0 0 349212 7.8958 NaN 0 1 0 0 1
878 879 0 3 Laleff, Mr. Kristo NaN 0 0 349217 7.8958 NaN 0 1 0 0 1
879 880 1 1 Potter, Mrs. Thomas Jr (Lily Alexenia Wilson) 56.0 0 1 11767 83.1583 C50 1 0 1 0 0
880 881 1 2 Shelley, Mrs. William (Imanita Parrish Hall) 25.0 0 1 230433 26.0000 NaN 1 0 0 0 1
881 882 0 3 Markun, Mr. Johann 33.0 0 0 349257 7.8958 NaN 0 1 0 0 1
882 883 0 3 Dahlberg, Miss. Gerda Ulrika 22.0 0 0 7552 10.5167 NaN 1 0 0 0 1
883 884 0 2 Banfield, Mr. Frederick James 28.0 0 0 C.A./SOTON 34068 10.5000 NaN 0 1 0 0 1
884 885 0 3 Sutehall, Mr. Henry Jr 25.0 0 0 SOTON/OQ 392076 7.0500 NaN 0 1 0 0 1
885 886 0 3 Rice, Mrs. William (Margaret Norton) 39.0 0 5 382652 29.1250 NaN 1 0 0 1 0
886 887 0 2 Montvila, Rev. Juozas 27.0 0 0 211536 13.0000 NaN 0 1 0 0 1
887 888 1 1 Graham, Miss. Margaret Edith 19.0 0 0 112053 30.0000 B42 1 0 0 0 1
888 889 0 3 Johnston, Miss. Catherine Helen "Carrie" NaN 1 2 W./C. 6607 23.4500 NaN 1 0 0 0 1
889 890 1 1 Behr, Mr. Karl Howell 26.0 0 0 111369 30.0000 C148 0 1 1 0 0
890 891 0 3 Dooley, Mr. Patrick 32.0 0 0 370376 7.7500 NaN 0 1 0 1 0

891 rows × 15 columns

When you do that, you will notice that it will drop the original columns Sex and Embarked, but notice how it keeps all the k levels. We have to pass it another argument to drop the first column of each level.

pd.get_dummies(train, columns=['Sex', 'Embarked'], drop_first=True)
PassengerId Survived Pclass Name Age SibSp Parch Ticket Fare Cabin Sex_male Embarked_Q Embarked_S
0 1 0 3 Braund, Mr. Owen Harris 22.0 1 0 A/5 21171 7.2500 NaN 1 0 1
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... 38.0 1 0 PC 17599 71.2833 C85 0 0 0
2 3 1 3 Heikkinen, Miss. Laina 26.0 0 0 STON/O2. 3101282 7.9250 NaN 0 0 1
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) 35.0 1 0 113803 53.1000 C123 0 0 1
4 5 0 3 Allen, Mr. William Henry 35.0 0 0 373450 8.0500 NaN 1 0 1
5 6 0 3 Moran, Mr. James NaN 0 0 330877 8.4583 NaN 1 1 0
6 7 0 1 McCarthy, Mr. Timothy J 54.0 0 0 17463 51.8625 E46 1 0 1
7 8 0 3 Palsson, Master. Gosta Leonard 2.0 3 1 349909 21.0750 NaN 1 0 1
8 9 1 3 Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg) 27.0 0 2 347742 11.1333 NaN 0 0 1
9 10 1 2 Nasser, Mrs. Nicholas (Adele Achem) 14.0 1 0 237736 30.0708 NaN 0 0 0
10 11 1 3 Sandstrom, Miss. Marguerite Rut 4.0 1 1 PP 9549 16.7000 G6 0 0 1
11 12 1 1 Bonnell, Miss. Elizabeth 58.0 0 0 113783 26.5500 C103 0 0 1
12 13 0 3 Saundercock, Mr. William Henry 20.0 0 0 A/5. 2151 8.0500 NaN 1 0 1
13 14 0 3 Andersson, Mr. Anders Johan 39.0 1 5 347082 31.2750 NaN 1 0 1
14 15 0 3 Vestrom, Miss. Hulda Amanda Adolfina 14.0 0 0 350406 7.8542 NaN 0 0 1
15 16 1 2 Hewlett, Mrs. (Mary D Kingcome) 55.0 0 0 248706 16.0000 NaN 0 0 1
16 17 0 3 Rice, Master. Eugene 2.0 4 1 382652 29.1250 NaN 1 1 0
17 18 1 2 Williams, Mr. Charles Eugene NaN 0 0 244373 13.0000 NaN 1 0 1
18 19 0 3 Vander Planke, Mrs. Julius (Emelia Maria Vande... 31.0 1 0 345763 18.0000 NaN 0 0 1
19 20 1 3 Masselmani, Mrs. Fatima NaN 0 0 2649 7.2250 NaN 0 0 0
20 21 0 2 Fynney, Mr. Joseph J 35.0 0 0 239865 26.0000 NaN 1 0 1
21 22 1 2 Beesley, Mr. Lawrence 34.0 0 0 248698 13.0000 D56 1 0 1
22 23 1 3 McGowan, Miss. Anna "Annie" 15.0 0 0 330923 8.0292 NaN 0 1 0
23 24 1 1 Sloper, Mr. William Thompson 28.0 0 0 113788 35.5000 A6 1 0 1
24 25 0 3 Palsson, Miss. Torborg Danira 8.0 3 1 349909 21.0750 NaN 0 0 1
25 26 1 3 Asplund, Mrs. Carl Oscar (Selma Augusta Emilia... 38.0 1 5 347077 31.3875 NaN 0 0 1
26 27 0 3 Emir, Mr. Farred Chehab NaN 0 0 2631 7.2250 NaN 1 0 0
27 28 0 1 Fortune, Mr. Charles Alexander 19.0 3 2 19950 263.0000 C23 C25 C27 1 0 1
28 29 1 3 O'Dwyer, Miss. Ellen "Nellie" NaN 0 0 330959 7.8792 NaN 0 1 0
29 30 0 3 Todoroff, Mr. Lalio NaN 0 0 349216 7.8958 NaN 1 0 1
... ... ... ... ... ... ... ... ... ... ... ... ... ...
861 862 0 2 Giles, Mr. Frederick Edward 21.0 1 0 28134 11.5000 NaN 1 0 1
862 863 1 1 Swift, Mrs. Frederick Joel (Margaret Welles Ba... 48.0 0 0 17466 25.9292 D17 0 0 1
863 864 0 3 Sage, Miss. Dorothy Edith "Dolly" NaN 8 2 CA. 2343 69.5500 NaN 0 0 1
864 865 0 2 Gill, Mr. John William 24.0 0 0 233866 13.0000 NaN 1 0 1
865 866 1 2 Bystrom, Mrs. (Karolina) 42.0 0 0 236852 13.0000 NaN 0 0 1
866 867 1 2 Duran y More, Miss. Asuncion 27.0 1 0 SC/PARIS 2149 13.8583 NaN 0 0 0
867 868 0 1 Roebling, Mr. Washington Augustus II 31.0 0 0 PC 17590 50.4958 A24 1 0 1
868 869 0 3 van Melkebeke, Mr. Philemon NaN 0 0 345777 9.5000 NaN 1 0 1
869 870 1 3 Johnson, Master. Harold Theodor 4.0 1 1 347742 11.1333 NaN 1 0 1
870 871 0 3 Balkic, Mr. Cerin 26.0 0 0 349248 7.8958 NaN 1 0 1
871 872 1 1 Beckwith, Mrs. Richard Leonard (Sallie Monypeny) 47.0 1 1 11751 52.5542 D35 0 0 1
872 873 0 1 Carlsson, Mr. Frans Olof 33.0 0 0 695 5.0000 B51 B53 B55 1 0 1
873 874 0 3 Vander Cruyssen, Mr. Victor 47.0 0 0 345765 9.0000 NaN 1 0 1
874 875 1 2 Abelson, Mrs. Samuel (Hannah Wizosky) 28.0 1 0 P/PP 3381 24.0000 NaN 0 0 0
875 876 1 3 Najib, Miss. Adele Kiamie "Jane" 15.0 0 0 2667 7.2250 NaN 0 0 0
876 877 0 3 Gustafsson, Mr. Alfred Ossian 20.0 0 0 7534 9.8458 NaN 1 0 1
877 878 0 3 Petroff, Mr. Nedelio 19.0 0 0 349212 7.8958 NaN 1 0 1
878 879 0 3 Laleff, Mr. Kristo NaN 0 0 349217 7.8958 NaN 1 0 1
879 880 1 1 Potter, Mrs. Thomas Jr (Lily Alexenia Wilson) 56.0 0 1 11767 83.1583 C50 0 0 0
880 881 1 2 Shelley, Mrs. William (Imanita Parrish Hall) 25.0 0 1 230433 26.0000 NaN 0 0 1
881 882 0 3 Markun, Mr. Johann 33.0 0 0 349257 7.8958 NaN 1 0 1
882 883 0 3 Dahlberg, Miss. Gerda Ulrika 22.0 0 0 7552 10.5167 NaN 0 0 1
883 884 0 2 Banfield, Mr. Frederick James 28.0 0 0 C.A./SOTON 34068 10.5000 NaN 1 0 1
884 885 0 3 Sutehall, Mr. Henry Jr 25.0 0 0 SOTON/OQ 392076 7.0500 NaN 1 0 1
885 886 0 3 Rice, Mrs. William (Margaret Norton) 39.0 0 5 382652 29.1250 NaN 0 1 0
886 887 0 2 Montvila, Rev. Juozas 27.0 0 0 211536 13.0000 NaN 1 0 1
887 888 1 1 Graham, Miss. Margaret Edith 19.0 0 0 112053 30.0000 B42 0 0 1
888 889 0 3 Johnston, Miss. Catherine Helen "Carrie" NaN 1 2 W./C. 6607 23.4500 NaN 0 0 1
889 890 1 1 Behr, Mr. Karl Howell 26.0 0 0 111369 30.0000 C148 1 0 0
890 891 0 3 Dooley, Mr. Patrick 32.0 0 0 370376 7.7500 NaN 1 1 0

891 rows × 13 columns

drop_first=True will drop the first level.

So we have accomplished all in 1 line, instead of using iloc and concat.

Updated: