An outlier is an observation that lies an abnormal distance from other values in a random sample from a population. In a sense, this definition leaves it up to the analyst (or a consensus process) to decide what will be considered abnormal. Before abnormal observations can be singled out, it is necessary to characterize normal observations.
You have to be careful when studying outliers because how do you know if an outlier is the result of a data glitch, or a real data point -- indeed maybe not an outlier.
In [1]:
%load_ext autoreload
%autoreload 2
In [2]:
import sys
sys.path.append("..")
In [3]:
from optimus import Optimus
In [4]:
# Create optimus
op = Optimus()
In [5]:
df = op.load.excel("data/titanic3.xls")
In [6]:
df.table()
Viewing 100 of 1309 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
1
1
Allen,⸱Miss.⸱Elisabeth⸱Walton
female
29.0
0
0
24160
211.3375
B5
S
2
nan
St⸱Louis,⸱MO
1
1
Allison,⸱Master.⸱Hudson⸱Trevor
male
0.9167
1
2
113781
151.55
C22⸱C26
S
11
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Miss.⸱Helen⸱Loraine
female
2.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mr.⸱Hudson⸱Joshua⸱Creighton
male
30.0
1
2
113781
151.55
C22⸱C26
S
nan
135.0
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mrs.⸱Hudson⸱J⸱C⸱(Bessie⸱Waldo⸱Daniels)
female
25.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
1
Anderson,⸱Mr.⸱Harry
male
48.0
0
0
19952
26.55
E12
S
3
nan
New⸱York,⸱NY
1
1
Andrews,⸱Miss.⸱Kornelia⸱Theodosia
female
63.0
1
0
13502
77.9583
D7
S
10
nan
Hudson,⸱NY
1
0
Andrews,⸱Mr.⸱Thomas⸱Jr
male
39.0
0
0
112050
0.0
A36
S
nan
nan
Belfast,⸱NI
1
1
Appleton,⸱Mrs.⸱Edward⸱Dale⸱(Charlotte⸱Lamson)
female
53.0
2
0
11769
51.4792
C101
S
D
nan
Bayside,⸱Queens,⸱NY
1
0
Artagaveytia,⸱Mr.⸱Ramon
male
71.0
0
0
PC⸱17609
49.5042
nan
C
nan
22.0
Montevideo,⸱Uruguay
1
0
Astor,⸱Col.⸱John⸱Jacob
male
47.0
1
0
PC⸱17757
227.525
C62⸱C64
C
nan
124.0
New⸱York,⸱NY
1
1
Astor,⸱Mrs.⸱John⸱Jacob⸱(Madeleine⸱Talmadge⸱Force)
female
18.0
1
0
PC⸱17757
227.525
C62⸱C64
C
4
nan
New⸱York,⸱NY
1
1
Aubart,⸱Mme.⸱Leontine⸱Pauline
female
24.0
0
0
PC⸱17477
69.3
B35
C
9
nan
Paris,⸱France
1
1
Barber,⸱Miss.⸱Ellen⸱"Nellie"
female
26.0
0
0
19877
78.85
nan
S
6
nan
nan
1
1
Barkworth,⸱Mr.⸱Algernon⸱Henry⸱Wilson
male
80.0
0
0
27042
30.0
A23
S
B
nan
Hessle,⸱Yorks
1
0
Baumann,⸱Mr.⸱John⸱D
male
nan
0
0
PC⸱17318
25.925
nan
S
nan
nan
New⸱York,⸱NY
1
0
Baxter,⸱Mr.⸱Quigg⸱Edmond
male
24.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
nan
nan
Montreal,⸱PQ
1
1
Baxter,⸱Mrs.⸱James⸱(Helene⸱DeLaudeniere⸱Chaput)
female
50.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Bazzani,⸱Miss.⸱Albina
female
32.0
0
0
11813
76.2917
D15
C
8
nan
nan
1
0
Beattie,⸱Mr.⸱Thomson
male
36.0
0
0
13050
75.2417
C6
C
A
nan
Winnipeg,⸱MN
1
1
Beckwith,⸱Mr.⸱Richard⸱Leonard
male
37.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Beckwith,⸱Mrs.⸱Richard⸱Leonard⸱(Sallie⸱Monypeny)
female
47.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Behr,⸱Mr.⸱Karl⸱Howell
male
26.0
0
0
111369
30.0
C148
C
5
nan
New⸱York,⸱NY
1
1
Bidois,⸱Miss.⸱Rosalie
female
42.0
0
0
PC⸱17757
227.525
nan
C
4
nan
nan
1
1
Bird,⸱Miss.⸱Ellen
female
29.0
0
0
PC⸱17483
221.7792
C97
S
8
nan
nan
1
0
Birnbaum,⸱Mr.⸱Jakob
male
25.0
0
0
13905
26.0
nan
C
nan
148.0
San⸱Francisco,⸱CA
1
1
Bishop,⸱Mr.⸱Dickinson⸱H
male
25.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bishop,⸱Mrs.⸱Dickinson⸱H⸱(Helen⸱Walton)
female
19.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bissette,⸱Miss.⸱Amelia
female
35.0
0
0
PC⸱17760
135.6333
C99
S
8
nan
nan
1
1
Bjornstrom-Steffansson,⸱Mr.⸱Mauritz⸱Hakan
male
28.0
0
0
110564
26.55
C52
S
D
nan
Stockholm,⸱Sweden⸱/⸱Washington,⸱DC
1
0
Blackwell,⸱Mr.⸱Stephen⸱Weart
male
45.0
0
0
113784
35.5
T
S
nan
nan
Trenton,⸱NJ
1
1
Blank,⸱Mr.⸱Henry
male
40.0
0
0
112277
31.0
A31
C
7
nan
Glen⸱Ridge,⸱NJ
1
1
Bonnell,⸱Miss.⸱Caroline
female
30.0
0
0
36928
164.8667
C7
S
8
nan
Youngstown,⸱OH
1
1
Bonnell,⸱Miss.⸱Elizabeth
female
58.0
0
0
113783
26.55
C103
S
8
nan
Birkdale,⸱England⸱Cleveland,⸱Ohio
1
0
Borebank,⸱Mr.⸱John⸱James
male
42.0
0
0
110489
26.55
D22
S
nan
nan
London⸱/⸱Winnipeg,⸱MB
1
1
Bowen,⸱Miss.⸱Grace⸱Scott
female
45.0
0
0
PC⸱17608
262.375
nan
C
4
nan
Cooperstown,⸱NY
1
1
Bowerman,⸱Miss.⸱Elsie⸱Edith
female
22.0
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
1
Bradley,⸱Mr.⸱George⸱("George⸱Arthur⸱Brayton")
male
nan
0
0
111427
26.55
nan
S
9
nan
Los⸱Angeles,⸱CA
1
0
Brady,⸱Mr.⸱John⸱Bertram
male
41.0
0
0
113054
30.5
A21
S
nan
nan
Pomeroy,⸱WA
1
0
Brandeis,⸱Mr.⸱Emil
male
48.0
0
0
PC⸱17591
50.4958
B10
C
nan
208.0
Omaha,⸱NE
1
0
Brewe,⸱Dr.⸱Arthur⸱Jackson
male
nan
0
0
112379
39.6
nan
C
nan
nan
Philadelphia,⸱PA
1
1
Brown,⸱Mrs.⸱James⸱Joseph⸱(Margaret⸱Tobin)
female
44.0
0
0
PC⸱17610
27.7208
B4
C
6
nan
Denver,⸱CO
1
1
Brown,⸱Mrs.⸱John⸱Murray⸱(Caroline⸱Lane⸱Lamson)
female
59.0
2
0
11769
51.4792
C101
S
D
nan
Belmont,⸱MA
1
1
Bucknell,⸱Mrs.⸱William⸱Robert⸱(Emma⸱Eliza⸱Ward)
female
60.0
0
0
11813
76.2917
D15
C
8
nan
Philadelphia,⸱PA
1
1
Burns,⸱Miss.⸱Elizabeth⸱Margaret
female
41.0
0
0
16966
134.5
E40
C
3
nan
nan
1
0
Butt,⸱Major.⸱Archibald⸱Willingham
male
45.0
0
0
113050
26.55
B38
S
nan
nan
Washington,⸱DC
1
0
Cairns,⸱Mr.⸱Alexander
male
nan
0
0
113798
31.0
nan
S
nan
nan
nan
1
1
Calderhead,⸱Mr.⸱Edward⸱Pennington
male
42.0
0
0
PC⸱17476
26.2875
E24
S
5
nan
New⸱York,⸱NY
1
1
Candee,⸱Mrs.⸱Edward⸱(Helen⸱Churchill⸱Hungerford)
female
53.0
0
0
PC⸱17606
27.4458
nan
C
6
nan
Washington,⸱DC
1
1
Cardeza,⸱Mr.⸱Thomas⸱Drake⸱Martinez
male
36.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Austria-Hungary⸱/⸱Germantown,⸱Philadelphia,⸱PA
1
1
Cardeza,⸱Mrs.⸱James⸱Warburton⸱Martinez⸱(Charlotte⸱Wardle⸱Drake)
female
58.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Germantown,⸱Philadelphia,⸱PA
1
0
Carlsson,⸱Mr.⸱Frans⸱Olof
male
33.0
0
0
695
5.0
B51⸱B53⸱B55
S
nan
nan
New⸱York,⸱NY
1
0
Carrau,⸱Mr.⸱Francisco⸱M
male
28.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
0
Carrau,⸱Mr.⸱Jose⸱Pedro
male
17.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
1
Carter,⸱Master.⸱William⸱Thornton⸱II
male
11.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Miss.⸱Lucile⸱Polk
female
14.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mr.⸱William⸱Ernest
male
36.0
1
2
113760
120.0
B96⸱B98
S
C
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mrs.⸱William⸱Ernest⸱(Lucile⸱Polk)
female
36.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
0
Case,⸱Mr.⸱Howard⸱Brown
male
49.0
0
0
19924
26.0
nan
S
nan
nan
Ascot,⸱Berkshire⸱/⸱Rochester,⸱NY
1
1
Cassebeer,⸱Mrs.⸱Henry⸱Arthur⸱Jr⸱(Eleanor⸱Genevieve⸱Fosdick)
female
nan
0
0
17770
27.7208
nan
C
5
nan
New⸱York,⸱NY
1
0
Cavendish,⸱Mr.⸱Tyrell⸱William
male
36.0
1
0
19877
78.85
C46
S
nan
172.0
Little⸱Onn⸱Hall,⸱Staffs
1
1
Cavendish,⸱Mrs.⸱Tyrell⸱William⸱(Julia⸱Florence⸱Siegel)
female
76.0
1
0
19877
78.85
C46
S
6
nan
Little⸱Onn⸱Hall,⸱Staffs
1
0
Chaffee,⸱Mr.⸱Herbert⸱Fuller
male
46.0
1
0
W.E.P.⸱5734
61.175
E31
S
nan
nan
Amenia,⸱ND
1
1
Chaffee,⸱Mrs.⸱Herbert⸱Fuller⸱(Carrie⸱Constance⸱Toogood)
female
47.0
1
0
W.E.P.⸱5734
61.175
E31
S
4
nan
Amenia,⸱ND
1
1
Chambers,⸱Mr.⸱Norman⸱Campbell
male
27.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chambers,⸱Mrs.⸱Norman⸱Campbell⸱(Bertha⸱Griggs)
female
33.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chaudanson,⸱Miss.⸱Victorine
female
36.0
0
0
PC⸱17608
262.375
B61
C
4
nan
nan
1
1
Cherry,⸱Miss.⸱Gladys
female
30.0
0
0
110152
86.5
B77
S
8
nan
London,⸱England
1
1
Chevre,⸱Mr.⸱Paul⸱Romaine
male
45.0
0
0
PC⸱17594
29.7
A9
C
7
nan
Paris,⸱France
1
1
Chibnall,⸱Mrs.⸱(Edith⸱Martha⸱Bowerman)
female
nan
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
0
Chisholm,⸱Mr.⸱Roderick⸱Robert⸱Crispin
male
nan
0
0
112051
0.0
nan
S
nan
nan
Liverpool,⸱England⸱/⸱Belfast
1
0
Clark,⸱Mr.⸱Walter⸱Miller
male
27.0
1
0
13508
136.7792
C89
C
nan
nan
Los⸱Angeles,⸱CA
1
1
Clark,⸱Mrs.⸱Walter⸱Miller⸱(Virginia⸱McDowell)
female
26.0
1
0
13508
136.7792
C89
C
4
nan
Los⸱Angeles,⸱CA
1
1
Cleaver,⸱Miss.⸱Alice
female
22.0
0
0
113781
151.55
nan
S
11
nan
nan
1
0
Clifford,⸱Mr.⸱George⸱Quincy
male
nan
0
0
110465
52.0
A14
S
nan
nan
Stoughton,⸱MA
1
0
Colley,⸱Mr.⸱Edward⸱Pomeroy
male
47.0
0
0
5727
25.5875
E58
S
nan
nan
Victoria,⸱BC
1
1
Compton,⸱Miss.⸱Sara⸱Rebecca
female
39.0
1
1
PC⸱17756
83.1583
E49
C
14
nan
Lakewood,⸱NJ
1
0
Compton,⸱Mr.⸱Alexander⸱Taylor⸱Jr
male
37.0
1
1
PC⸱17756
83.1583
E52
C
nan
nan
Lakewood,⸱NJ
1
1
Compton,⸱Mrs.⸱Alexander⸱Taylor⸱(Mary⸱Eliza⸱Ingersoll)
female
64.0
0
2
PC⸱17756
83.1583
E45
C
14
nan
Lakewood,⸱NJ
1
1
Cornell,⸱Mrs.⸱Robert⸱Clifford⸱(Malvina⸱Helen⸱Lamson)
female
55.0
2
0
11770
25.7
C101
S
2
nan
New⸱York,⸱NY
1
0
Crafton,⸱Mr.⸱John⸱Bertram
male
nan
0
0
113791
26.55
nan
S
nan
nan
Roachdale,⸱IN
1
0
Crosby,⸱Capt.⸱Edward⸱Gifford
male
70.0
1
1
WE/P⸱5735
71.0
B22
S
nan
269.0
Milwaukee,⸱WI
1
1
Crosby,⸱Miss.⸱Harriet⸱R
female
36.0
0
2
WE/P⸱5735
71.0
B22
S
7
nan
Milwaukee,⸱WI
1
1
Crosby,⸱Mrs.⸱Edward⸱Gifford⸱(Catherine⸱Elizabeth⸱Halstead)
female
64.0
1
1
112901
26.55
B26
S
7
nan
Milwaukee,⸱WI
1
0
Cumings,⸱Mr.⸱John⸱Bradley
male
39.0
1
0
PC⸱17599
71.2833
C85
C
nan
nan
New⸱York,⸱NY
1
1
Cumings,⸱Mrs.⸱John⸱Bradley⸱(Florence⸱Briggs⸱Thayer)
female
38.0
1
0
PC⸱17599
71.2833
C85
C
4
nan
New⸱York,⸱NY
1
1
Daly,⸱Mr.⸱Peter⸱Denis⸱
male
51.0
0
0
113055
26.55
E17
S
5⸱9
nan
Lima,⸱Peru
1
1
Daniel,⸱Mr.⸱Robert⸱Williams
male
27.0
0
0
113804
30.5
nan
S
3
nan
Philadelphia,⸱PA
1
1
Daniels,⸱Miss.⸱Sarah
female
33.0
0
0
113781
151.55
nan
S
8
nan
nan
1
0
Davidson,⸱Mr.⸱Thornton
male
31.0
1
0
F.C.⸱12750
52.0
B71
S
nan
nan
Montreal,⸱PQ
1
1
Davidson,⸱Mrs.⸱Thornton⸱(Orian⸱Hays)
female
27.0
1
2
F.C.⸱12750
52.0
B71
S
3
nan
Montreal,⸱PQ
1
1
Dick,⸱Mr.⸱Albert⸱Adrian
male
31.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dick,⸱Mrs.⸱Albert⸱Adrian⸱(Vera⸱Gillespie)
female
17.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dodge,⸱Dr.⸱Washington
male
53.0
1
1
33638
81.8583
A34
S
13
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Master.⸱Washington
male
4.0
0
2
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Mrs.⸱Washington⸱(Ruth⸱Vidaver)
female
54.0
1
1
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
0
Douglas,⸱Mr.⸱Walter⸱Donald
male
50.0
1
0
PC⸱17761
106.425
C86
C
nan
62.0
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Douglas,⸱Mrs.⸱Frederick⸱Charles⸱(Mary⸱Helene⸱Baxter)
female
27.0
1
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Douglas,⸱Mrs.⸱Walter⸱Donald⸱(Mahala⸱Dutton)
female
48.0
1
0
PC⸱17761
106.425
C86
C
2
nan
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Duff⸱Gordon,⸱Lady.⸱(Lucille⸱Christiana⸱Sutherland)⸱("Mrs⸱Morgan")
female
48.0
1
0
11755
39.6
A16
C
1
nan
London⸱/⸱Paris
Viewing 100 of 1309 rows / 14 columns
8 partition(s)
From a quick inspection of the dataframe we can guess that the 1000 in the column num can be an outlier. You can perform a very intense search to see if it is actually and outlier, if you need something like that please check out these articles and tutorials
With optimus you can perform several analysis too to check if a value is an outlier. First lets run some visual analysis. Remember to check the Main Example for more.
One of the commonest ways of finding outliers in one-dimensional data is to mark as a potential outlier any point that is more than two standard deviations, say, from the mean (I am referring to sample means and standard deviations here and in what follows). But the presence of outliers is likely to have a strong effect on the mean and the standard deviation, making this technique unreliable.
That's why we have programmed in Optimus the median absolute deviation from median, commonly shortened to the median absolute deviation (MAD). It is the median of the set comprising the absolute values of the differences between the median and each data point. If you want more information on the subject please read the amazing article by Leys et al. about dtecting outliers here
In [21]:
df.outliers.z_score("fare", threshold= 2).select().table()
Viewing 100 of 1309 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
1
1
Allen,⸱Miss.⸱Elisabeth⸱Walton
female
29.0
0
0
24160
211.3375
B5
S
2
nan
St⸱Louis,⸱MO
1
1
Allison,⸱Master.⸱Hudson⸱Trevor
male
0.9167
1
2
113781
151.55
C22⸱C26
S
11
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Miss.⸱Helen⸱Loraine
female
2.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mr.⸱Hudson⸱Joshua⸱Creighton
male
30.0
1
2
113781
151.55
C22⸱C26
S
nan
135.0
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mrs.⸱Hudson⸱J⸱C⸱(Bessie⸱Waldo⸱Daniels)
female
25.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
1
Anderson,⸱Mr.⸱Harry
male
48.0
0
0
19952
26.55
E12
S
3
nan
New⸱York,⸱NY
1
1
Andrews,⸱Miss.⸱Kornelia⸱Theodosia
female
63.0
1
0
13502
77.9583
D7
S
10
nan
Hudson,⸱NY
1
0
Andrews,⸱Mr.⸱Thomas⸱Jr
male
39.0
0
0
112050
0.0
A36
S
nan
nan
Belfast,⸱NI
1
1
Appleton,⸱Mrs.⸱Edward⸱Dale⸱(Charlotte⸱Lamson)
female
53.0
2
0
11769
51.4792
C101
S
D
nan
Bayside,⸱Queens,⸱NY
1
0
Artagaveytia,⸱Mr.⸱Ramon
male
71.0
0
0
PC⸱17609
49.5042
nan
C
nan
22.0
Montevideo,⸱Uruguay
1
0
Astor,⸱Col.⸱John⸱Jacob
male
47.0
1
0
PC⸱17757
227.525
C62⸱C64
C
nan
124.0
New⸱York,⸱NY
1
1
Astor,⸱Mrs.⸱John⸱Jacob⸱(Madeleine⸱Talmadge⸱Force)
female
18.0
1
0
PC⸱17757
227.525
C62⸱C64
C
4
nan
New⸱York,⸱NY
1
1
Aubart,⸱Mme.⸱Leontine⸱Pauline
female
24.0
0
0
PC⸱17477
69.3
B35
C
9
nan
Paris,⸱France
1
1
Barber,⸱Miss.⸱Ellen⸱"Nellie"
female
26.0
0
0
19877
78.85
nan
S
6
nan
nan
1
1
Barkworth,⸱Mr.⸱Algernon⸱Henry⸱Wilson
male
80.0
0
0
27042
30.0
A23
S
B
nan
Hessle,⸱Yorks
1
0
Baumann,⸱Mr.⸱John⸱D
male
nan
0
0
PC⸱17318
25.925
nan
S
nan
nan
New⸱York,⸱NY
1
0
Baxter,⸱Mr.⸱Quigg⸱Edmond
male
24.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
nan
nan
Montreal,⸱PQ
1
1
Baxter,⸱Mrs.⸱James⸱(Helene⸱DeLaudeniere⸱Chaput)
female
50.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Bazzani,⸱Miss.⸱Albina
female
32.0
0
0
11813
76.2917
D15
C
8
nan
nan
1
0
Beattie,⸱Mr.⸱Thomson
male
36.0
0
0
13050
75.2417
C6
C
A
nan
Winnipeg,⸱MN
1
1
Beckwith,⸱Mr.⸱Richard⸱Leonard
male
37.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Beckwith,⸱Mrs.⸱Richard⸱Leonard⸱(Sallie⸱Monypeny)
female
47.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Behr,⸱Mr.⸱Karl⸱Howell
male
26.0
0
0
111369
30.0
C148
C
5
nan
New⸱York,⸱NY
1
1
Bidois,⸱Miss.⸱Rosalie
female
42.0
0
0
PC⸱17757
227.525
nan
C
4
nan
nan
1
1
Bird,⸱Miss.⸱Ellen
female
29.0
0
0
PC⸱17483
221.7792
C97
S
8
nan
nan
1
0
Birnbaum,⸱Mr.⸱Jakob
male
25.0
0
0
13905
26.0
nan
C
nan
148.0
San⸱Francisco,⸱CA
1
1
Bishop,⸱Mr.⸱Dickinson⸱H
male
25.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bishop,⸱Mrs.⸱Dickinson⸱H⸱(Helen⸱Walton)
female
19.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bissette,⸱Miss.⸱Amelia
female
35.0
0
0
PC⸱17760
135.6333
C99
S
8
nan
nan
1
1
Bjornstrom-Steffansson,⸱Mr.⸱Mauritz⸱Hakan
male
28.0
0
0
110564
26.55
C52
S
D
nan
Stockholm,⸱Sweden⸱/⸱Washington,⸱DC
1
0
Blackwell,⸱Mr.⸱Stephen⸱Weart
male
45.0
0
0
113784
35.5
T
S
nan
nan
Trenton,⸱NJ
1
1
Blank,⸱Mr.⸱Henry
male
40.0
0
0
112277
31.0
A31
C
7
nan
Glen⸱Ridge,⸱NJ
1
1
Bonnell,⸱Miss.⸱Caroline
female
30.0
0
0
36928
164.8667
C7
S
8
nan
Youngstown,⸱OH
1
1
Bonnell,⸱Miss.⸱Elizabeth
female
58.0
0
0
113783
26.55
C103
S
8
nan
Birkdale,⸱England⸱Cleveland,⸱Ohio
1
0
Borebank,⸱Mr.⸱John⸱James
male
42.0
0
0
110489
26.55
D22
S
nan
nan
London⸱/⸱Winnipeg,⸱MB
1
1
Bowen,⸱Miss.⸱Grace⸱Scott
female
45.0
0
0
PC⸱17608
262.375
nan
C
4
nan
Cooperstown,⸱NY
1
1
Bowerman,⸱Miss.⸱Elsie⸱Edith
female
22.0
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
1
Bradley,⸱Mr.⸱George⸱("George⸱Arthur⸱Brayton")
male
nan
0
0
111427
26.55
nan
S
9
nan
Los⸱Angeles,⸱CA
1
0
Brady,⸱Mr.⸱John⸱Bertram
male
41.0
0
0
113054
30.5
A21
S
nan
nan
Pomeroy,⸱WA
1
0
Brandeis,⸱Mr.⸱Emil
male
48.0
0
0
PC⸱17591
50.4958
B10
C
nan
208.0
Omaha,⸱NE
1
0
Brewe,⸱Dr.⸱Arthur⸱Jackson
male
nan
0
0
112379
39.6
nan
C
nan
nan
Philadelphia,⸱PA
1
1
Brown,⸱Mrs.⸱James⸱Joseph⸱(Margaret⸱Tobin)
female
44.0
0
0
PC⸱17610
27.7208
B4
C
6
nan
Denver,⸱CO
1
1
Brown,⸱Mrs.⸱John⸱Murray⸱(Caroline⸱Lane⸱Lamson)
female
59.0
2
0
11769
51.4792
C101
S
D
nan
Belmont,⸱MA
1
1
Bucknell,⸱Mrs.⸱William⸱Robert⸱(Emma⸱Eliza⸱Ward)
female
60.0
0
0
11813
76.2917
D15
C
8
nan
Philadelphia,⸱PA
1
1
Burns,⸱Miss.⸱Elizabeth⸱Margaret
female
41.0
0
0
16966
134.5
E40
C
3
nan
nan
1
0
Butt,⸱Major.⸱Archibald⸱Willingham
male
45.0
0
0
113050
26.55
B38
S
nan
nan
Washington,⸱DC
1
0
Cairns,⸱Mr.⸱Alexander
male
nan
0
0
113798
31.0
nan
S
nan
nan
nan
1
1
Calderhead,⸱Mr.⸱Edward⸱Pennington
male
42.0
0
0
PC⸱17476
26.2875
E24
S
5
nan
New⸱York,⸱NY
1
1
Candee,⸱Mrs.⸱Edward⸱(Helen⸱Churchill⸱Hungerford)
female
53.0
0
0
PC⸱17606
27.4458
nan
C
6
nan
Washington,⸱DC
1
1
Cardeza,⸱Mr.⸱Thomas⸱Drake⸱Martinez
male
36.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Austria-Hungary⸱/⸱Germantown,⸱Philadelphia,⸱PA
1
1
Cardeza,⸱Mrs.⸱James⸱Warburton⸱Martinez⸱(Charlotte⸱Wardle⸱Drake)
female
58.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Germantown,⸱Philadelphia,⸱PA
1
0
Carlsson,⸱Mr.⸱Frans⸱Olof
male
33.0
0
0
695
5.0
B51⸱B53⸱B55
S
nan
nan
New⸱York,⸱NY
1
0
Carrau,⸱Mr.⸱Francisco⸱M
male
28.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
0
Carrau,⸱Mr.⸱Jose⸱Pedro
male
17.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
1
Carter,⸱Master.⸱William⸱Thornton⸱II
male
11.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Miss.⸱Lucile⸱Polk
female
14.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mr.⸱William⸱Ernest
male
36.0
1
2
113760
120.0
B96⸱B98
S
C
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mrs.⸱William⸱Ernest⸱(Lucile⸱Polk)
female
36.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
0
Case,⸱Mr.⸱Howard⸱Brown
male
49.0
0
0
19924
26.0
nan
S
nan
nan
Ascot,⸱Berkshire⸱/⸱Rochester,⸱NY
1
1
Cassebeer,⸱Mrs.⸱Henry⸱Arthur⸱Jr⸱(Eleanor⸱Genevieve⸱Fosdick)
female
nan
0
0
17770
27.7208
nan
C
5
nan
New⸱York,⸱NY
1
0
Cavendish,⸱Mr.⸱Tyrell⸱William
male
36.0
1
0
19877
78.85
C46
S
nan
172.0
Little⸱Onn⸱Hall,⸱Staffs
1
1
Cavendish,⸱Mrs.⸱Tyrell⸱William⸱(Julia⸱Florence⸱Siegel)
female
76.0
1
0
19877
78.85
C46
S
6
nan
Little⸱Onn⸱Hall,⸱Staffs
1
0
Chaffee,⸱Mr.⸱Herbert⸱Fuller
male
46.0
1
0
W.E.P.⸱5734
61.175
E31
S
nan
nan
Amenia,⸱ND
1
1
Chaffee,⸱Mrs.⸱Herbert⸱Fuller⸱(Carrie⸱Constance⸱Toogood)
female
47.0
1
0
W.E.P.⸱5734
61.175
E31
S
4
nan
Amenia,⸱ND
1
1
Chambers,⸱Mr.⸱Norman⸱Campbell
male
27.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chambers,⸱Mrs.⸱Norman⸱Campbell⸱(Bertha⸱Griggs)
female
33.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chaudanson,⸱Miss.⸱Victorine
female
36.0
0
0
PC⸱17608
262.375
B61
C
4
nan
nan
1
1
Cherry,⸱Miss.⸱Gladys
female
30.0
0
0
110152
86.5
B77
S
8
nan
London,⸱England
1
1
Chevre,⸱Mr.⸱Paul⸱Romaine
male
45.0
0
0
PC⸱17594
29.7
A9
C
7
nan
Paris,⸱France
1
1
Chibnall,⸱Mrs.⸱(Edith⸱Martha⸱Bowerman)
female
nan
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
0
Chisholm,⸱Mr.⸱Roderick⸱Robert⸱Crispin
male
nan
0
0
112051
0.0
nan
S
nan
nan
Liverpool,⸱England⸱/⸱Belfast
1
0
Clark,⸱Mr.⸱Walter⸱Miller
male
27.0
1
0
13508
136.7792
C89
C
nan
nan
Los⸱Angeles,⸱CA
1
1
Clark,⸱Mrs.⸱Walter⸱Miller⸱(Virginia⸱McDowell)
female
26.0
1
0
13508
136.7792
C89
C
4
nan
Los⸱Angeles,⸱CA
1
1
Cleaver,⸱Miss.⸱Alice
female
22.0
0
0
113781
151.55
nan
S
11
nan
nan
1
0
Clifford,⸱Mr.⸱George⸱Quincy
male
nan
0
0
110465
52.0
A14
S
nan
nan
Stoughton,⸱MA
1
0
Colley,⸱Mr.⸱Edward⸱Pomeroy
male
47.0
0
0
5727
25.5875
E58
S
nan
nan
Victoria,⸱BC
1
1
Compton,⸱Miss.⸱Sara⸱Rebecca
female
39.0
1
1
PC⸱17756
83.1583
E49
C
14
nan
Lakewood,⸱NJ
1
0
Compton,⸱Mr.⸱Alexander⸱Taylor⸱Jr
male
37.0
1
1
PC⸱17756
83.1583
E52
C
nan
nan
Lakewood,⸱NJ
1
1
Compton,⸱Mrs.⸱Alexander⸱Taylor⸱(Mary⸱Eliza⸱Ingersoll)
female
64.0
0
2
PC⸱17756
83.1583
E45
C
14
nan
Lakewood,⸱NJ
1
1
Cornell,⸱Mrs.⸱Robert⸱Clifford⸱(Malvina⸱Helen⸱Lamson)
female
55.0
2
0
11770
25.7
C101
S
2
nan
New⸱York,⸱NY
1
0
Crafton,⸱Mr.⸱John⸱Bertram
male
nan
0
0
113791
26.55
nan
S
nan
nan
Roachdale,⸱IN
1
0
Crosby,⸱Capt.⸱Edward⸱Gifford
male
70.0
1
1
WE/P⸱5735
71.0
B22
S
nan
269.0
Milwaukee,⸱WI
1
1
Crosby,⸱Miss.⸱Harriet⸱R
female
36.0
0
2
WE/P⸱5735
71.0
B22
S
7
nan
Milwaukee,⸱WI
1
1
Crosby,⸱Mrs.⸱Edward⸱Gifford⸱(Catherine⸱Elizabeth⸱Halstead)
female
64.0
1
1
112901
26.55
B26
S
7
nan
Milwaukee,⸱WI
1
0
Cumings,⸱Mr.⸱John⸱Bradley
male
39.0
1
0
PC⸱17599
71.2833
C85
C
nan
nan
New⸱York,⸱NY
1
1
Cumings,⸱Mrs.⸱John⸱Bradley⸱(Florence⸱Briggs⸱Thayer)
female
38.0
1
0
PC⸱17599
71.2833
C85
C
4
nan
New⸱York,⸱NY
1
1
Daly,⸱Mr.⸱Peter⸱Denis⸱
male
51.0
0
0
113055
26.55
E17
S
5⸱9
nan
Lima,⸱Peru
1
1
Daniel,⸱Mr.⸱Robert⸱Williams
male
27.0
0
0
113804
30.5
nan
S
3
nan
Philadelphia,⸱PA
1
1
Daniels,⸱Miss.⸱Sarah
female
33.0
0
0
113781
151.55
nan
S
8
nan
nan
1
0
Davidson,⸱Mr.⸱Thornton
male
31.0
1
0
F.C.⸱12750
52.0
B71
S
nan
nan
Montreal,⸱PQ
1
1
Davidson,⸱Mrs.⸱Thornton⸱(Orian⸱Hays)
female
27.0
1
2
F.C.⸱12750
52.0
B71
S
3
nan
Montreal,⸱PQ
1
1
Dick,⸱Mr.⸱Albert⸱Adrian
male
31.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dick,⸱Mrs.⸱Albert⸱Adrian⸱(Vera⸱Gillespie)
female
17.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dodge,⸱Dr.⸱Washington
male
53.0
1
1
33638
81.8583
A34
S
13
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Master.⸱Washington
male
4.0
0
2
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Mrs.⸱Washington⸱(Ruth⸱Vidaver)
female
54.0
1
1
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
0
Douglas,⸱Mr.⸱Walter⸱Donald
male
50.0
1
0
PC⸱17761
106.425
C86
C
nan
62.0
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Douglas,⸱Mrs.⸱Frederick⸱Charles⸱(Mary⸱Helene⸱Baxter)
female
27.0
1
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Douglas,⸱Mrs.⸱Walter⸱Donald⸱(Mahala⸱Dutton)
female
48.0
1
0
PC⸱17761
106.425
C86
C
2
nan
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Duff⸱Gordon,⸱Lady.⸱(Lucille⸱Christiana⸱Sutherland)⸱("Mrs⸱Morgan")
female
48.0
1
0
11755
39.6
A16
C
1
nan
London⸱/⸱Paris
Viewing 100 of 1309 rows / 14 columns
8 partition(s)
In [22]:
df.outliers.z_score("fare", threshold= 1).drop().table()
Viewing 0 of 0 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
Viewing 0 of 0 rows / 14 columns
8 partition(s)
In [16]:
df.outliers.tukey("fare").select().table()
Viewing 100 of 172 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
1
1
Allen,⸱Miss.⸱Elisabeth⸱Walton
female
29.0
0
0
24160
211.3375
B5
S
2
nan
St⸱Louis,⸱MO
1
1
Allison,⸱Master.⸱Hudson⸱Trevor
male
0.9167
1
2
113781
151.55
C22⸱C26
S
11
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Miss.⸱Helen⸱Loraine
female
2.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mr.⸱Hudson⸱Joshua⸱Creighton
male
30.0
1
2
113781
151.55
C22⸱C26
S
nan
135.0
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mrs.⸱Hudson⸱J⸱C⸱(Bessie⸱Waldo⸱Daniels)
female
25.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
1
Andrews,⸱Miss.⸱Kornelia⸱Theodosia
female
63.0
1
0
13502
77.9583
D7
S
10
nan
Hudson,⸱NY
1
0
Astor,⸱Col.⸱John⸱Jacob
male
47.0
1
0
PC⸱17757
227.525
C62⸱C64
C
nan
124.0
New⸱York,⸱NY
1
1
Astor,⸱Mrs.⸱John⸱Jacob⸱(Madeleine⸱Talmadge⸱Force)
female
18.0
1
0
PC⸱17757
227.525
C62⸱C64
C
4
nan
New⸱York,⸱NY
1
1
Aubart,⸱Mme.⸱Leontine⸱Pauline
female
24.0
0
0
PC⸱17477
69.3
B35
C
9
nan
Paris,⸱France
1
1
Barber,⸱Miss.⸱Ellen⸱"Nellie"
female
26.0
0
0
19877
78.85
nan
S
6
nan
nan
1
0
Baxter,⸱Mr.⸱Quigg⸱Edmond
male
24.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
nan
nan
Montreal,⸱PQ
1
1
Baxter,⸱Mrs.⸱James⸱(Helene⸱DeLaudeniere⸱Chaput)
female
50.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Bazzani,⸱Miss.⸱Albina
female
32.0
0
0
11813
76.2917
D15
C
8
nan
nan
1
0
Beattie,⸱Mr.⸱Thomson
male
36.0
0
0
13050
75.2417
C6
C
A
nan
Winnipeg,⸱MN
1
1
Bidois,⸱Miss.⸱Rosalie
female
42.0
0
0
PC⸱17757
227.525
nan
C
4
nan
nan
1
1
Bird,⸱Miss.⸱Ellen
female
29.0
0
0
PC⸱17483
221.7792
C97
S
8
nan
nan
1
1
Bishop,⸱Mr.⸱Dickinson⸱H
male
25.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bishop,⸱Mrs.⸱Dickinson⸱H⸱(Helen⸱Walton)
female
19.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bissette,⸱Miss.⸱Amelia
female
35.0
0
0
PC⸱17760
135.6333
C99
S
8
nan
nan
1
1
Bonnell,⸱Miss.⸱Caroline
female
30.0
0
0
36928
164.8667
C7
S
8
nan
Youngstown,⸱OH
1
1
Bowen,⸱Miss.⸱Grace⸱Scott
female
45.0
0
0
PC⸱17608
262.375
nan
C
4
nan
Cooperstown,⸱NY
1
1
Bucknell,⸱Mrs.⸱William⸱Robert⸱(Emma⸱Eliza⸱Ward)
female
60.0
0
0
11813
76.2917
D15
C
8
nan
Philadelphia,⸱PA
1
1
Burns,⸱Miss.⸱Elizabeth⸱Margaret
female
41.0
0
0
16966
134.5
E40
C
3
nan
nan
1
1
Cardeza,⸱Mr.⸱Thomas⸱Drake⸱Martinez
male
36.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Austria-Hungary⸱/⸱Germantown,⸱Philadelphia,⸱PA
1
1
Cardeza,⸱Mrs.⸱James⸱Warburton⸱Martinez⸱(Charlotte⸱Wardle⸱Drake)
female
58.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Germantown,⸱Philadelphia,⸱PA
1
1
Carter,⸱Master.⸱William⸱Thornton⸱II
male
11.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Miss.⸱Lucile⸱Polk
female
14.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mr.⸱William⸱Ernest
male
36.0
1
2
113760
120.0
B96⸱B98
S
C
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mrs.⸱William⸱Ernest⸱(Lucile⸱Polk)
female
36.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
0
Cavendish,⸱Mr.⸱Tyrell⸱William
male
36.0
1
0
19877
78.85
C46
S
nan
172.0
Little⸱Onn⸱Hall,⸱Staffs
1
1
Cavendish,⸱Mrs.⸱Tyrell⸱William⸱(Julia⸱Florence⸱Siegel)
female
76.0
1
0
19877
78.85
C46
S
6
nan
Little⸱Onn⸱Hall,⸱Staffs
1
1
Chaudanson,⸱Miss.⸱Victorine
female
36.0
0
0
PC⸱17608
262.375
B61
C
4
nan
nan
1
1
Cherry,⸱Miss.⸱Gladys
female
30.0
0
0
110152
86.5
B77
S
8
nan
London,⸱England
1
0
Clark,⸱Mr.⸱Walter⸱Miller
male
27.0
1
0
13508
136.7792
C89
C
nan
nan
Los⸱Angeles,⸱CA
1
1
Clark,⸱Mrs.⸱Walter⸱Miller⸱(Virginia⸱McDowell)
female
26.0
1
0
13508
136.7792
C89
C
4
nan
Los⸱Angeles,⸱CA
1
1
Cleaver,⸱Miss.⸱Alice
female
22.0
0
0
113781
151.55
nan
S
11
nan
nan
1
1
Compton,⸱Miss.⸱Sara⸱Rebecca
female
39.0
1
1
PC⸱17756
83.1583
E49
C
14
nan
Lakewood,⸱NJ
1
0
Compton,⸱Mr.⸱Alexander⸱Taylor⸱Jr
male
37.0
1
1
PC⸱17756
83.1583
E52
C
nan
nan
Lakewood,⸱NJ
1
1
Compton,⸱Mrs.⸱Alexander⸱Taylor⸱(Mary⸱Eliza⸱Ingersoll)
female
64.0
0
2
PC⸱17756
83.1583
E45
C
14
nan
Lakewood,⸱NJ
1
0
Crosby,⸱Capt.⸱Edward⸱Gifford
male
70.0
1
1
WE/P⸱5735
71.0
B22
S
nan
269.0
Milwaukee,⸱WI
1
1
Crosby,⸱Miss.⸱Harriet⸱R
female
36.0
0
2
WE/P⸱5735
71.0
B22
S
7
nan
Milwaukee,⸱WI
1
0
Cumings,⸱Mr.⸱John⸱Bradley
male
39.0
1
0
PC⸱17599
71.2833
C85
C
nan
nan
New⸱York,⸱NY
1
1
Cumings,⸱Mrs.⸱John⸱Bradley⸱(Florence⸱Briggs⸱Thayer)
female
38.0
1
0
PC⸱17599
71.2833
C85
C
4
nan
New⸱York,⸱NY
1
1
Daniels,⸱Miss.⸱Sarah
female
33.0
0
0
113781
151.55
nan
S
8
nan
nan
1
1
Dodge,⸱Dr.⸱Washington
male
53.0
1
1
33638
81.8583
A34
S
13
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Master.⸱Washington
male
4.0
0
2
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Mrs.⸱Washington⸱(Ruth⸱Vidaver)
female
54.0
1
1
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
0
Douglas,⸱Mr.⸱Walter⸱Donald
male
50.0
1
0
PC⸱17761
106.425
C86
C
nan
62.0
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Douglas,⸱Mrs.⸱Frederick⸱Charles⸱(Mary⸱Helene⸱Baxter)
female
27.0
1
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Douglas,⸱Mrs.⸱Walter⸱Donald⸱(Mahala⸱Dutton)
female
48.0
1
0
PC⸱17761
106.425
C86
C
2
nan
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Earnshaw,⸱Mrs.⸱Boulton⸱(Olive⸱Potter)
female
23.0
0
1
11767
83.1583
C54
C
7
nan
Mt⸱Airy,⸱Philadelphia,⸱PA
1
1
Endres,⸱Miss.⸱Caroline⸱Louise
female
38.0
0
0
PC⸱17757
227.525
C45
C
4
nan
New⸱York,⸱NY
1
1
Eustis,⸱Miss.⸱Elizabeth⸱Mussey
female
54.0
1
0
36947
78.2667
D20
C
4
nan
Brookline,⸱MA
1
0
Farthing,⸱Mr.⸱John
male
nan
0
0
PC⸱17483
221.7792
C95
S
nan
nan
nan
1
1
Fleming,⸱Miss.⸱Margaret
female
nan
0
0
17421
110.8833
nan
C
4
nan
nan
1
1
Fortune,⸱Miss.⸱Alice⸱Elizabeth
female
24.0
3
2
19950
263.0
C23⸱C25⸱C27
S
10
nan
Winnipeg,⸱MB
1
1
Fortune,⸱Miss.⸱Ethel⸱Flora
female
28.0
3
2
19950
263.0
C23⸱C25⸱C27
S
10
nan
Winnipeg,⸱MB
1
1
Fortune,⸱Miss.⸱Mabel⸱Helen
female
23.0
3
2
19950
263.0
C23⸱C25⸱C27
S
10
nan
Winnipeg,⸱MB
1
0
Fortune,⸱Mr.⸱Charles⸱Alexander
male
19.0
3
2
19950
263.0
C23⸱C25⸱C27
S
nan
nan
Winnipeg,⸱MB
1
0
Fortune,⸱Mr.⸱Mark
male
64.0
1
4
19950
263.0
C23⸱C25⸱C27
S
nan
nan
Winnipeg,⸱MB
1
1
Fortune,⸱Mrs.⸱Mark⸱(Mary⸱McDougald)
female
60.0
1
4
19950
263.0
C23⸱C25⸱C27
S
10
nan
Winnipeg,⸱MB
1
1
Frauenthal,⸱Dr.⸱Henry⸱William
male
50.0
2
0
PC⸱17611
133.65
nan
S
5
nan
New⸱York,⸱NY
1
1
Frauenthal,⸱Mrs.⸱Henry⸱William⸱(Clara⸱Heinsheimer)
female
nan
1
0
PC⸱17611
133.65
nan
S
5
nan
New⸱York,⸱NY
1
1
Frolicher-Stehli,⸱Mr.⸱Maxmillian
male
60.0
1
1
13567
79.2
B41
C
5
nan
Zurich,⸱Switzerland
1
1
Frolicher-Stehli,⸱Mrs.⸱Maxmillian⸱(Margaretha⸱Emerentia⸱Stehli)
female
48.0
1
1
13567
79.2
B41
C
5
nan
Zurich,⸱Switzerland
1
1
Geiger,⸱Miss.⸱Amalie
female
35.0
0
0
113503
211.5
C130
C
4
nan
nan
1
0
Giglio,⸱Mr.⸱Victor
male
24.0
0
0
PC⸱17593
79.2
B86
C
nan
nan
nan
1
1
Goldenberg,⸱Mr.⸱Samuel⸱L
male
49.0
1
0
17453
89.1042
C92
C
5
nan
Paris,⸱France⸱/⸱New⸱York,⸱NY
1
1
Goldenberg,⸱Mrs.⸱Samuel⸱L⸱(Edwiga⸱Grabowska)
female
nan
1
0
17453
89.1042
C92
C
5
nan
Paris,⸱France⸱/⸱New⸱York,⸱NY
1
0
Graham,⸱Mr.⸱George⸱Edward
male
38.0
0
1
PC⸱17582
153.4625
C91
S
nan
147.0
Winnipeg,⸱MB
1
1
Graham,⸱Mrs.⸱William⸱Thompson⸱(Edith⸱Junkins)
female
58.0
0
1
PC⸱17582
153.4625
C125
S
3
nan
Greenwich,⸱CT
1
0
Guggenheim,⸱Mr.⸱Benjamin
male
46.0
0
0
PC⸱17593
79.2
B82⸱B84
C
nan
nan
New⸱York,⸱NY
1
1
Harper,⸱Mr.⸱Henry⸱Sleeper
male
48.0
1
0
PC⸱17572
76.7292
D33
C
3
nan
New⸱York,⸱NY
1
1
Harper,⸱Mrs.⸱Henry⸱Sleeper⸱(Myna⸱Haxtun)
female
49.0
1
0
PC⸱17572
76.7292
D33
C
3
nan
New⸱York,⸱NY
1
0
Harris,⸱Mr.⸱Henry⸱Birkhardt
male
45.0
1
0
36973
83.475
C83
S
nan
nan
New⸱York,⸱NY
1
1
Harris,⸱Mrs.⸱Henry⸱Birkhardt⸱(Irene⸱Wallach)
female
35.0
1
0
36973
83.475
C83
S
D
nan
New⸱York,⸱NY
1
1
Hassab,⸱Mr.⸱Hammad
male
27.0
0
0
PC⸱17572
76.7292
D49
C
3
nan
nan
1
1
Hays,⸱Miss.⸱Margaret⸱Bechstein
female
24.0
0
0
11767
83.1583
C54
C
7
nan
New⸱York,⸱NY
1
0
Hays,⸱Mr.⸱Charles⸱Melville
male
55.0
1
1
12749
93.5
B69
S
nan
307.0
Montreal,⸱PQ
1
1
Hays,⸱Mrs.⸱Charles⸱Melville⸱(Clara⸱Jennings⸱Gregg)
female
52.0
1
1
12749
93.5
B69
S
3
nan
Montreal,⸱PQ
1
1
Hogeboom,⸱Mrs.⸱John⸱C⸱(Anna⸱Andrews)
female
51.0
1
0
13502
77.9583
D11
S
10
nan
Hudson,⸱NY
1
1
Hoyt,⸱Mr.⸱Frederick⸱Maxfield
male
38.0
1
0
19943
90.0
C93
S
D
nan
New⸱York,⸱NY⸱/⸱⸱Stamford⸱CT
1
1
Hoyt,⸱Mrs.⸱Frederick⸱Maxfield⸱(Jane⸱Anne⸱Forby)
female
35.0
1
0
19943
90.0
C93
S
D
nan
New⸱York,⸱NY⸱/⸱⸱Stamford⸱CT
1
1
Icard,⸱Miss.⸱Amelie
female
38.0
0
0
113572
80.0
B28
nan
6
nan
nan
1
0
Keeping,⸱Mr.⸱Edwin
male
32.5
0
0
113503
211.5
C132
C
nan
45.0
nan
1
1
Kreuchen,⸱Miss.⸱Emilie
female
39.0
0
0
24160
211.3375
nan
S
2
nan
nan
1
1
LeRoy,⸱Miss.⸱Bertha
female
30.0
0
0
PC⸱17761
106.425
nan
C
2
nan
nan
1
1
Lesurer,⸱Mr.⸱Gustave⸱J
male
35.0
0
0
PC⸱17755
512.3292
B101
C
3
nan
nan
1
1
Longley,⸱Miss.⸱Gretchen⸱Fiske
female
21.0
0
0
13502
77.9583
D9
S
10
nan
Hudson,⸱NY
1
1
Lurette,⸱Miss.⸱Elise
female
58.0
0
0
PC⸱17569
146.5208
B80
C
nan
nan
nan
1
1
Madill,⸱Miss.⸱Georgette⸱Alexandra
female
15.0
0
1
24160
211.3375
B5
S
2
nan
St⸱Louis,⸱MO
1
1
Maioni,⸱Miss.⸱Roberta
female
16.0
0
0
110152
86.5
B79
S
8
nan
nan
1
0
McCaffry,⸱Mr.⸱Thomas⸱Francis
male
46.0
0
0
13050
75.2417
C6
C
nan
292.0
Vancouver,⸱BC
1
0
Meyer,⸱Mr.⸱Edgar⸱Joseph
male
28.0
1
0
PC⸱17604
82.1708
nan
C
nan
nan
New⸱York,⸱NY
1
1
Meyer,⸱Mrs.⸱Edgar⸱Joseph⸱(Leila⸱Saks)
female
nan
1
0
PC⸱17604
82.1708
nan
C
6
nan
New⸱York,⸱NY
1
0
Minahan,⸱Dr.⸱William⸱Edward
male
44.0
2
0
19928
90.0
C78
Q
nan
230.0
Fond⸱du⸱Lac,⸱WI
1
1
Minahan,⸱Miss.⸱Daisy⸱E
female
33.0
1
0
19928
90.0
C78
Q
14
nan
Green⸱Bay,⸱WI
1
1
Minahan,⸱Mrs.⸱William⸱Edward⸱(Lillian⸱E⸱Thorpe)
female
37.0
1
0
19928
90.0
C78
Q
14
nan
Fond⸱du⸱Lac,⸱WI
1
1
Newell,⸱Miss.⸱Madeleine
female
31.0
1
0
35273
113.275
D36
C
6
nan
Lexington,⸱MA
1
1
Newell,⸱Miss.⸱Marjorie
female
23.0
1
0
35273
113.275
D36
C
6
nan
Lexington,⸱MA
Viewing 100 of 172 rows / 14 columns
8 partition(s)
In [17]:
df.outliers.tukey("fare").drop().table()
Viewing 100 of 1137 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
1
1
Anderson,⸱Mr.⸱Harry
male
48.0
0
0
19952
26.55
E12
S
3
nan
New⸱York,⸱NY
1
0
Andrews,⸱Mr.⸱Thomas⸱Jr
male
39.0
0
0
112050
0.0
A36
S
nan
nan
Belfast,⸱NI
1
1
Appleton,⸱Mrs.⸱Edward⸱Dale⸱(Charlotte⸱Lamson)
female
53.0
2
0
11769
51.4792
C101
S
D
nan
Bayside,⸱Queens,⸱NY
1
0
Artagaveytia,⸱Mr.⸱Ramon
male
71.0
0
0
PC⸱17609
49.5042
nan
C
nan
22.0
Montevideo,⸱Uruguay
1
1
Barkworth,⸱Mr.⸱Algernon⸱Henry⸱Wilson
male
80.0
0
0
27042
30.0
A23
S
B
nan
Hessle,⸱Yorks
1
0
Baumann,⸱Mr.⸱John⸱D
male
nan
0
0
PC⸱17318
25.925
nan
S
nan
nan
New⸱York,⸱NY
1
1
Beckwith,⸱Mr.⸱Richard⸱Leonard
male
37.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Beckwith,⸱Mrs.⸱Richard⸱Leonard⸱(Sallie⸱Monypeny)
female
47.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Behr,⸱Mr.⸱Karl⸱Howell
male
26.0
0
0
111369
30.0
C148
C
5
nan
New⸱York,⸱NY
1
0
Birnbaum,⸱Mr.⸱Jakob
male
25.0
0
0
13905
26.0
nan
C
nan
148.0
San⸱Francisco,⸱CA
1
1
Bjornstrom-Steffansson,⸱Mr.⸱Mauritz⸱Hakan
male
28.0
0
0
110564
26.55
C52
S
D
nan
Stockholm,⸱Sweden⸱/⸱Washington,⸱DC
1
0
Blackwell,⸱Mr.⸱Stephen⸱Weart
male
45.0
0
0
113784
35.5
T
S
nan
nan
Trenton,⸱NJ
1
1
Blank,⸱Mr.⸱Henry
male
40.0
0
0
112277
31.0
A31
C
7
nan
Glen⸱Ridge,⸱NJ
1
1
Bonnell,⸱Miss.⸱Elizabeth
female
58.0
0
0
113783
26.55
C103
S
8
nan
Birkdale,⸱England⸱Cleveland,⸱Ohio
1
0
Borebank,⸱Mr.⸱John⸱James
male
42.0
0
0
110489
26.55
D22
S
nan
nan
London⸱/⸱Winnipeg,⸱MB
1
1
Bowerman,⸱Miss.⸱Elsie⸱Edith
female
22.0
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
1
Bradley,⸱Mr.⸱George⸱("George⸱Arthur⸱Brayton")
male
nan
0
0
111427
26.55
nan
S
9
nan
Los⸱Angeles,⸱CA
1
0
Brady,⸱Mr.⸱John⸱Bertram
male
41.0
0
0
113054
30.5
A21
S
nan
nan
Pomeroy,⸱WA
1
0
Brandeis,⸱Mr.⸱Emil
male
48.0
0
0
PC⸱17591
50.4958
B10
C
nan
208.0
Omaha,⸱NE
1
0
Brewe,⸱Dr.⸱Arthur⸱Jackson
male
nan
0
0
112379
39.6
nan
C
nan
nan
Philadelphia,⸱PA
1
1
Brown,⸱Mrs.⸱James⸱Joseph⸱(Margaret⸱Tobin)
female
44.0
0
0
PC⸱17610
27.7208
B4
C
6
nan
Denver,⸱CO
1
1
Brown,⸱Mrs.⸱John⸱Murray⸱(Caroline⸱Lane⸱Lamson)
female
59.0
2
0
11769
51.4792
C101
S
D
nan
Belmont,⸱MA
1
0
Butt,⸱Major.⸱Archibald⸱Willingham
male
45.0
0
0
113050
26.55
B38
S
nan
nan
Washington,⸱DC
1
0
Cairns,⸱Mr.⸱Alexander
male
nan
0
0
113798
31.0
nan
S
nan
nan
nan
1
1
Calderhead,⸱Mr.⸱Edward⸱Pennington
male
42.0
0
0
PC⸱17476
26.2875
E24
S
5
nan
New⸱York,⸱NY
1
1
Candee,⸱Mrs.⸱Edward⸱(Helen⸱Churchill⸱Hungerford)
female
53.0
0
0
PC⸱17606
27.4458
nan
C
6
nan
Washington,⸱DC
1
0
Carlsson,⸱Mr.⸱Frans⸱Olof
male
33.0
0
0
695
5.0
B51⸱B53⸱B55
S
nan
nan
New⸱York,⸱NY
1
0
Carrau,⸱Mr.⸱Francisco⸱M
male
28.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
0
Carrau,⸱Mr.⸱Jose⸱Pedro
male
17.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
0
Case,⸱Mr.⸱Howard⸱Brown
male
49.0
0
0
19924
26.0
nan
S
nan
nan
Ascot,⸱Berkshire⸱/⸱Rochester,⸱NY
1
1
Cassebeer,⸱Mrs.⸱Henry⸱Arthur⸱Jr⸱(Eleanor⸱Genevieve⸱Fosdick)
female
nan
0
0
17770
27.7208
nan
C
5
nan
New⸱York,⸱NY
1
0
Chaffee,⸱Mr.⸱Herbert⸱Fuller
male
46.0
1
0
W.E.P.⸱5734
61.175
E31
S
nan
nan
Amenia,⸱ND
1
1
Chaffee,⸱Mrs.⸱Herbert⸱Fuller⸱(Carrie⸱Constance⸱Toogood)
female
47.0
1
0
W.E.P.⸱5734
61.175
E31
S
4
nan
Amenia,⸱ND
1
1
Chambers,⸱Mr.⸱Norman⸱Campbell
male
27.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chambers,⸱Mrs.⸱Norman⸱Campbell⸱(Bertha⸱Griggs)
female
33.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chevre,⸱Mr.⸱Paul⸱Romaine
male
45.0
0
0
PC⸱17594
29.7
A9
C
7
nan
Paris,⸱France
1
1
Chibnall,⸱Mrs.⸱(Edith⸱Martha⸱Bowerman)
female
nan
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
0
Chisholm,⸱Mr.⸱Roderick⸱Robert⸱Crispin
male
nan
0
0
112051
0.0
nan
S
nan
nan
Liverpool,⸱England⸱/⸱Belfast
1
0
Clifford,⸱Mr.⸱George⸱Quincy
male
nan
0
0
110465
52.0
A14
S
nan
nan
Stoughton,⸱MA
1
0
Colley,⸱Mr.⸱Edward⸱Pomeroy
male
47.0
0
0
5727
25.5875
E58
S
nan
nan
Victoria,⸱BC
1
1
Cornell,⸱Mrs.⸱Robert⸱Clifford⸱(Malvina⸱Helen⸱Lamson)
female
55.0
2
0
11770
25.7
C101
S
2
nan
New⸱York,⸱NY
1
0
Crafton,⸱Mr.⸱John⸱Bertram
male
nan
0
0
113791
26.55
nan
S
nan
nan
Roachdale,⸱IN
1
1
Crosby,⸱Mrs.⸱Edward⸱Gifford⸱(Catherine⸱Elizabeth⸱Halstead)
female
64.0
1
1
112901
26.55
B26
S
7
nan
Milwaukee,⸱WI
1
1
Daly,⸱Mr.⸱Peter⸱Denis⸱
male
51.0
0
0
113055
26.55
E17
S
5⸱9
nan
Lima,⸱Peru
1
1
Daniel,⸱Mr.⸱Robert⸱Williams
male
27.0
0
0
113804
30.5
nan
S
3
nan
Philadelphia,⸱PA
1
0
Davidson,⸱Mr.⸱Thornton
male
31.0
1
0
F.C.⸱12750
52.0
B71
S
nan
nan
Montreal,⸱PQ
1
1
Davidson,⸱Mrs.⸱Thornton⸱(Orian⸱Hays)
female
27.0
1
2
F.C.⸱12750
52.0
B71
S
3
nan
Montreal,⸱PQ
1
1
Dick,⸱Mr.⸱Albert⸱Adrian
male
31.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dick,⸱Mrs.⸱Albert⸱Adrian⸱(Vera⸱Gillespie)
female
17.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Duff⸱Gordon,⸱Lady.⸱(Lucille⸱Christiana⸱Sutherland)⸱("Mrs⸱Morgan")
female
48.0
1
0
11755
39.6
A16
C
1
nan
London⸱/⸱Paris
1
1
Duff⸱Gordon,⸱Sir.⸱Cosmo⸱Edmund⸱("Mr⸱Morgan")
male
49.0
1
0
PC⸱17485
56.9292
A20
C
1
nan
London⸱/⸱Paris
1
0
Dulles,⸱Mr.⸱William⸱Crothers
male
39.0
0
0
PC⸱17580
29.7
A18
C
nan
133.0
Philadelphia,⸱PA
1
0
Evans,⸱Miss.⸱Edith⸱Corse
female
36.0
0
0
PC⸱17531
31.6792
A29
C
nan
nan
New⸱York,⸱NY
1
1
Flegenheim,⸱Mrs.⸱Alfred⸱(Antoinette)
female
nan
0
0
PC⸱17598
31.6833
nan
S
7
nan
New⸱York,⸱NY
1
1
Flynn,⸱Mr.⸱John⸱Irwin⸱("Irving")
male
36.0
0
0
PC⸱17474
26.3875
E25
S
5
nan
Brooklyn,⸱NY
1
0
Foreman,⸱Mr.⸱Benjamin⸱Laventall
male
30.0
0
0
113051
27.75
C111
C
nan
nan
New⸱York,⸱NY
1
1
Francatelli,⸱Miss.⸱Laura⸱Mabel
female
30.0
0
0
PC⸱17485
56.9292
E36
C
1
nan
nan
1
0
Franklin,⸱Mr.⸱Thomas⸱Parham
male
nan
0
0
113778
26.55
D34
S
nan
nan
Westcliff-on-Sea,⸱Essex
1
1
Frauenthal,⸱Mr.⸱Isaac⸱Gerald
male
43.0
1
0
17765
27.7208
D40
C
5
nan
New⸱York,⸱NY
1
1
Frolicher,⸱Miss.⸱Hedwig⸱Margaritha
female
22.0
0
2
13568
49.5
B39
C
5
nan
Zurich,⸱Switzerland
1
0
Fry,⸱Mr.⸱Richard
male
nan
0
0
112058
0.0
B102
S
nan
nan
nan
1
0
Futrelle,⸱Mr.⸱Jacques⸱Heath
male
37.0
1
0
113803
53.1
C123
S
nan
nan
Scituate,⸱MA
1
1
Futrelle,⸱Mrs.⸱Jacques⸱Heath⸱(Lily⸱May⸱Peel)
female
35.0
1
0
113803
53.1
C123
S
D
nan
Scituate,⸱MA
1
0
Gee,⸱Mr.⸱Arthur⸱H
male
47.0
0
0
111320
38.5
E63
S
nan
275.0
St⸱Anne's-on-Sea,⸱Lancashire
1
1
Gibson,⸱Miss.⸱Dorothy⸱Winifred
female
22.0
0
1
112378
59.4
nan
C
7
nan
New⸱York,⸱NY
1
1
Gibson,⸱Mrs.⸱Leonard⸱(Pauline⸱C⸱Boeson)
female
45.0
0
1
112378
59.4
nan
C
7
nan
New⸱York,⸱NY
1
0
Goldschmidt,⸱Mr.⸱George⸱B
male
71.0
0
0
PC⸱17754
34.6542
A5
C
nan
nan
New⸱York,⸱NY
1
1
Gracie,⸱Col.⸱Archibald⸱IV
male
53.0
0
0
113780
28.5
C51
C
B
nan
Washington,⸱DC
1
1
Graham,⸱Miss.⸱Margaret⸱Edith
female
19.0
0
0
112053
30.0
B42
S
3
nan
Greenwich,⸱CT
1
1
Greenfield,⸱Mr.⸱William⸱Bertram
male
23.0
0
1
PC⸱17759
63.3583
D10⸱D12
C
7
nan
New⸱York,⸱NY
1
1
Greenfield,⸱Mrs.⸱Leo⸱David⸱(Blanche⸱Strouse)
female
45.0
0
1
PC⸱17759
63.3583
D10⸱D12
C
7
nan
New⸱York,⸱NY
1
1
Harder,⸱Mr.⸱George⸱Achilles
male
25.0
1
0
11765
55.4417
E50
C
5
nan
Brooklyn,⸱NY
1
1
Harder,⸱Mrs.⸱George⸱Achilles⸱(Dorothy⸱Annan)
female
25.0
1
0
11765
55.4417
E50
C
5
nan
Brooklyn,⸱NY
1
0
Harrington,⸱Mr.⸱Charles⸱H
male
nan
0
0
113796
42.4
nan
S
nan
nan
nan
1
0
Harrison,⸱Mr.⸱William
male
40.0
0
0
112059
0.0
B94
S
nan
110.0
nan
1
1
Hawksford,⸱Mr.⸱Walter⸱James
male
nan
0
0
16988
30.0
D45
S
3
nan
Kingston,⸱Surrey
1
0
Head,⸱Mr.⸱Christopher
male
42.0
0
0
113038
42.5
B11
S
nan
nan
London⸱/⸱Middlesex
1
0
Hilliard,⸱Mr.⸱Herbert⸱Henry
male
nan
0
0
17463
51.8625
E46
S
nan
nan
Brighton,⸱MA
1
0
Hipkins,⸱Mr.⸱William⸱Edward
male
55.0
0
0
680
50.0
C39
S
nan
nan
London⸱/⸱Birmingham
1
1
Hippach,⸱Miss.⸱Jean⸱Gertrude
female
16.0
0
1
111361
57.9792
B18
C
4
nan
Chicago,⸱IL
1
1
Hippach,⸱Mrs.⸱Louis⸱Albert⸱(Ida⸱Sophia⸱Fischer)
female
44.0
0
1
111361
57.9792
B18
C
4
nan
Chicago,⸱IL
1
0
Holverson,⸱Mr.⸱Alexander⸱Oskar
male
42.0
1
0
113789
52.0
nan
S
nan
38.0
New⸱York,⸱NY
1
1
Holverson,⸱Mrs.⸱Alexander⸱Oskar⸱(Mary⸱Aline⸱Towner)
female
35.0
1
0
113789
52.0
nan
S
8
nan
New⸱York,⸱NY
1
1
Homer,⸱Mr.⸱Harry⸱("Mr⸱E⸱Haven")
male
35.0
0
0
111426
26.55
nan
C
15
nan
Indianapolis,⸱IN
1
0
Hoyt,⸱Mr.⸱William⸱Fisher
male
nan
0
0
PC⸱17600
30.6958
nan
C
14
nan
New⸱York,⸱NY
1
0
Isham,⸱Miss.⸱Ann⸱Elizabeth
female
50.0
0
0
PC⸱17595
28.7125
C49
C
nan
nan
Paris,⸱France⸱New⸱York,⸱NY
1
1
Ismay,⸱Mr.⸱Joseph⸱Bruce
male
49.0
0
0
112058
0.0
B52⸱B54⸱B56
S
C
nan
Liverpool
1
0
Jones,⸱Mr.⸱Charles⸱Cresson
male
46.0
0
0
694
26.0
nan
S
nan
80.0
Bennington,⸱VT
1
0
Julian,⸱Mr.⸱Henry⸱Forbes
male
50.0
0
0
113044
26.0
E60
S
nan
nan
London
1
0
Kent,⸱Mr.⸱Edward⸱Austin
male
58.0
0
0
11771
29.7
B37
C
nan
258.0
Buffalo,⸱NY
1
0
Kenyon,⸱Mr.⸱Frederick⸱R
male
41.0
1
0
17464
51.8625
D21
S
nan
nan
Southington⸱/⸱Noank,⸱CT
1
1
Kenyon,⸱Mrs.⸱Frederick⸱R⸱(Marion)
female
nan
1
0
17464
51.8625
D21
S
8
nan
Southington⸱/⸱Noank,⸱CT
1
1
Kimball,⸱Mr.⸱Edwin⸱Nelson⸱Jr
male
42.0
1
0
11753
52.5542
D19
S
5
nan
Boston,⸱MA
1
1
Kimball,⸱Mrs.⸱Edwin⸱Nelson⸱Jr⸱(Gertrude⸱Parsons)
female
45.0
1
0
11753
52.5542
D19
S
5
nan
Boston,⸱MA
1
0
Klaber,⸱Mr.⸱Herman
male
nan
0
0
113028
26.55
C124
S
nan
nan
Portland,⸱OR
1
1
Leader,⸱Dr.⸱Alice⸱(Farnham)
female
49.0
0
0
17465
25.9292
D17
S
8
nan
New⸱York,⸱NY
1
0
Lewy,⸱Mr.⸱Ervin⸱G
male
nan
0
0
PC⸱17612
27.7208
nan
C
nan
nan
Chicago,⸱IL
1
0
Lindeberg-Lind,⸱Mr.⸱Erik⸱Gustaf⸱("Mr⸱Edward⸱Lingrey")
male
42.0
0
0
17475
26.55
nan
S
nan
nan
Stockholm,⸱Sweden
1
1
Lindstrom,⸱Mrs.⸱Carl⸱Johan⸱(Sigrid⸱Posse)
female
55.0
0
0
112377
27.7208
nan
C
6
nan
Stockholm,⸱Sweden
1
1
Lines,⸱Miss.⸱Mary⸱Conover
female
16.0
0
1
PC⸱17592
39.4
D28
S
9
nan
Paris,⸱France
Viewing 100 of 1137 rows / 14 columns
8 partition(s)
In [27]:
df.outliers.mad("fare", threshold= 2).select().table()
Viewing 100 of 1292 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
1
1
Allen,⸱Miss.⸱Elisabeth⸱Walton
female
29.0
0
0
24160
211.3375
B5
S
2
nan
St⸱Louis,⸱MO
1
1
Allison,⸱Master.⸱Hudson⸱Trevor
male
0.9167
1
2
113781
151.55
C22⸱C26
S
11
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Miss.⸱Helen⸱Loraine
female
2.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mr.⸱Hudson⸱Joshua⸱Creighton
male
30.0
1
2
113781
151.55
C22⸱C26
S
nan
135.0
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
0
Allison,⸱Mrs.⸱Hudson⸱J⸱C⸱(Bessie⸱Waldo⸱Daniels)
female
25.0
1
2
113781
151.55
C22⸱C26
S
nan
nan
Montreal,⸱PQ⸱/⸱Chesterville,⸱ON
1
1
Anderson,⸱Mr.⸱Harry
male
48.0
0
0
19952
26.55
E12
S
3
nan
New⸱York,⸱NY
1
1
Andrews,⸱Miss.⸱Kornelia⸱Theodosia
female
63.0
1
0
13502
77.9583
D7
S
10
nan
Hudson,⸱NY
1
1
Appleton,⸱Mrs.⸱Edward⸱Dale⸱(Charlotte⸱Lamson)
female
53.0
2
0
11769
51.4792
C101
S
D
nan
Bayside,⸱Queens,⸱NY
1
0
Artagaveytia,⸱Mr.⸱Ramon
male
71.0
0
0
PC⸱17609
49.5042
nan
C
nan
22.0
Montevideo,⸱Uruguay
1
0
Astor,⸱Col.⸱John⸱Jacob
male
47.0
1
0
PC⸱17757
227.525
C62⸱C64
C
nan
124.0
New⸱York,⸱NY
1
1
Astor,⸱Mrs.⸱John⸱Jacob⸱(Madeleine⸱Talmadge⸱Force)
female
18.0
1
0
PC⸱17757
227.525
C62⸱C64
C
4
nan
New⸱York,⸱NY
1
1
Aubart,⸱Mme.⸱Leontine⸱Pauline
female
24.0
0
0
PC⸱17477
69.3
B35
C
9
nan
Paris,⸱France
1
1
Barber,⸱Miss.⸱Ellen⸱"Nellie"
female
26.0
0
0
19877
78.85
nan
S
6
nan
nan
1
1
Barkworth,⸱Mr.⸱Algernon⸱Henry⸱Wilson
male
80.0
0
0
27042
30.0
A23
S
B
nan
Hessle,⸱Yorks
1
0
Baumann,⸱Mr.⸱John⸱D
male
nan
0
0
PC⸱17318
25.925
nan
S
nan
nan
New⸱York,⸱NY
1
0
Baxter,⸱Mr.⸱Quigg⸱Edmond
male
24.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
nan
nan
Montreal,⸱PQ
1
1
Baxter,⸱Mrs.⸱James⸱(Helene⸱DeLaudeniere⸱Chaput)
female
50.0
0
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Bazzani,⸱Miss.⸱Albina
female
32.0
0
0
11813
76.2917
D15
C
8
nan
nan
1
0
Beattie,⸱Mr.⸱Thomson
male
36.0
0
0
13050
75.2417
C6
C
A
nan
Winnipeg,⸱MN
1
1
Beckwith,⸱Mr.⸱Richard⸱Leonard
male
37.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Beckwith,⸱Mrs.⸱Richard⸱Leonard⸱(Sallie⸱Monypeny)
female
47.0
1
1
11751
52.5542
D35
S
5
nan
New⸱York,⸱NY
1
1
Behr,⸱Mr.⸱Karl⸱Howell
male
26.0
0
0
111369
30.0
C148
C
5
nan
New⸱York,⸱NY
1
1
Bidois,⸱Miss.⸱Rosalie
female
42.0
0
0
PC⸱17757
227.525
nan
C
4
nan
nan
1
1
Bird,⸱Miss.⸱Ellen
female
29.0
0
0
PC⸱17483
221.7792
C97
S
8
nan
nan
1
0
Birnbaum,⸱Mr.⸱Jakob
male
25.0
0
0
13905
26.0
nan
C
nan
148.0
San⸱Francisco,⸱CA
1
1
Bishop,⸱Mr.⸱Dickinson⸱H
male
25.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bishop,⸱Mrs.⸱Dickinson⸱H⸱(Helen⸱Walton)
female
19.0
1
0
11967
91.0792
B49
C
7
nan
Dowagiac,⸱MI
1
1
Bissette,⸱Miss.⸱Amelia
female
35.0
0
0
PC⸱17760
135.6333
C99
S
8
nan
nan
1
1
Bjornstrom-Steffansson,⸱Mr.⸱Mauritz⸱Hakan
male
28.0
0
0
110564
26.55
C52
S
D
nan
Stockholm,⸱Sweden⸱/⸱Washington,⸱DC
1
0
Blackwell,⸱Mr.⸱Stephen⸱Weart
male
45.0
0
0
113784
35.5
T
S
nan
nan
Trenton,⸱NJ
1
1
Blank,⸱Mr.⸱Henry
male
40.0
0
0
112277
31.0
A31
C
7
nan
Glen⸱Ridge,⸱NJ
1
1
Bonnell,⸱Miss.⸱Caroline
female
30.0
0
0
36928
164.8667
C7
S
8
nan
Youngstown,⸱OH
1
1
Bonnell,⸱Miss.⸱Elizabeth
female
58.0
0
0
113783
26.55
C103
S
8
nan
Birkdale,⸱England⸱Cleveland,⸱Ohio
1
0
Borebank,⸱Mr.⸱John⸱James
male
42.0
0
0
110489
26.55
D22
S
nan
nan
London⸱/⸱Winnipeg,⸱MB
1
1
Bowen,⸱Miss.⸱Grace⸱Scott
female
45.0
0
0
PC⸱17608
262.375
nan
C
4
nan
Cooperstown,⸱NY
1
1
Bowerman,⸱Miss.⸱Elsie⸱Edith
female
22.0
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
1
Bradley,⸱Mr.⸱George⸱("George⸱Arthur⸱Brayton")
male
nan
0
0
111427
26.55
nan
S
9
nan
Los⸱Angeles,⸱CA
1
0
Brady,⸱Mr.⸱John⸱Bertram
male
41.0
0
0
113054
30.5
A21
S
nan
nan
Pomeroy,⸱WA
1
0
Brandeis,⸱Mr.⸱Emil
male
48.0
0
0
PC⸱17591
50.4958
B10
C
nan
208.0
Omaha,⸱NE
1
0
Brewe,⸱Dr.⸱Arthur⸱Jackson
male
nan
0
0
112379
39.6
nan
C
nan
nan
Philadelphia,⸱PA
1
1
Brown,⸱Mrs.⸱James⸱Joseph⸱(Margaret⸱Tobin)
female
44.0
0
0
PC⸱17610
27.7208
B4
C
6
nan
Denver,⸱CO
1
1
Brown,⸱Mrs.⸱John⸱Murray⸱(Caroline⸱Lane⸱Lamson)
female
59.0
2
0
11769
51.4792
C101
S
D
nan
Belmont,⸱MA
1
1
Bucknell,⸱Mrs.⸱William⸱Robert⸱(Emma⸱Eliza⸱Ward)
female
60.0
0
0
11813
76.2917
D15
C
8
nan
Philadelphia,⸱PA
1
1
Burns,⸱Miss.⸱Elizabeth⸱Margaret
female
41.0
0
0
16966
134.5
E40
C
3
nan
nan
1
0
Butt,⸱Major.⸱Archibald⸱Willingham
male
45.0
0
0
113050
26.55
B38
S
nan
nan
Washington,⸱DC
1
0
Cairns,⸱Mr.⸱Alexander
male
nan
0
0
113798
31.0
nan
S
nan
nan
nan
1
1
Calderhead,⸱Mr.⸱Edward⸱Pennington
male
42.0
0
0
PC⸱17476
26.2875
E24
S
5
nan
New⸱York,⸱NY
1
1
Candee,⸱Mrs.⸱Edward⸱(Helen⸱Churchill⸱Hungerford)
female
53.0
0
0
PC⸱17606
27.4458
nan
C
6
nan
Washington,⸱DC
1
1
Cardeza,⸱Mr.⸱Thomas⸱Drake⸱Martinez
male
36.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Austria-Hungary⸱/⸱Germantown,⸱Philadelphia,⸱PA
1
1
Cardeza,⸱Mrs.⸱James⸱Warburton⸱Martinez⸱(Charlotte⸱Wardle⸱Drake)
female
58.0
0
1
PC⸱17755
512.3292
B51⸱B53⸱B55
C
3
nan
Germantown,⸱Philadelphia,⸱PA
1
0
Carlsson,⸱Mr.⸱Frans⸱Olof
male
33.0
0
0
695
5.0
B51⸱B53⸱B55
S
nan
nan
New⸱York,⸱NY
1
0
Carrau,⸱Mr.⸱Francisco⸱M
male
28.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
0
Carrau,⸱Mr.⸱Jose⸱Pedro
male
17.0
0
0
113059
47.1
nan
S
nan
nan
Montevideo,⸱Uruguay
1
1
Carter,⸱Master.⸱William⸱Thornton⸱II
male
11.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Miss.⸱Lucile⸱Polk
female
14.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mr.⸱William⸱Ernest
male
36.0
1
2
113760
120.0
B96⸱B98
S
C
nan
Bryn⸱Mawr,⸱PA
1
1
Carter,⸱Mrs.⸱William⸱Ernest⸱(Lucile⸱Polk)
female
36.0
1
2
113760
120.0
B96⸱B98
S
4
nan
Bryn⸱Mawr,⸱PA
1
0
Case,⸱Mr.⸱Howard⸱Brown
male
49.0
0
0
19924
26.0
nan
S
nan
nan
Ascot,⸱Berkshire⸱/⸱Rochester,⸱NY
1
1
Cassebeer,⸱Mrs.⸱Henry⸱Arthur⸱Jr⸱(Eleanor⸱Genevieve⸱Fosdick)
female
nan
0
0
17770
27.7208
nan
C
5
nan
New⸱York,⸱NY
1
0
Cavendish,⸱Mr.⸱Tyrell⸱William
male
36.0
1
0
19877
78.85
C46
S
nan
172.0
Little⸱Onn⸱Hall,⸱Staffs
1
1
Cavendish,⸱Mrs.⸱Tyrell⸱William⸱(Julia⸱Florence⸱Siegel)
female
76.0
1
0
19877
78.85
C46
S
6
nan
Little⸱Onn⸱Hall,⸱Staffs
1
0
Chaffee,⸱Mr.⸱Herbert⸱Fuller
male
46.0
1
0
W.E.P.⸱5734
61.175
E31
S
nan
nan
Amenia,⸱ND
1
1
Chaffee,⸱Mrs.⸱Herbert⸱Fuller⸱(Carrie⸱Constance⸱Toogood)
female
47.0
1
0
W.E.P.⸱5734
61.175
E31
S
4
nan
Amenia,⸱ND
1
1
Chambers,⸱Mr.⸱Norman⸱Campbell
male
27.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chambers,⸱Mrs.⸱Norman⸱Campbell⸱(Bertha⸱Griggs)
female
33.0
1
0
113806
53.1
E8
S
5
nan
New⸱York,⸱NY⸱/⸱Ithaca,⸱NY
1
1
Chaudanson,⸱Miss.⸱Victorine
female
36.0
0
0
PC⸱17608
262.375
B61
C
4
nan
nan
1
1
Cherry,⸱Miss.⸱Gladys
female
30.0
0
0
110152
86.5
B77
S
8
nan
London,⸱England
1
1
Chevre,⸱Mr.⸱Paul⸱Romaine
male
45.0
0
0
PC⸱17594
29.7
A9
C
7
nan
Paris,⸱France
1
1
Chibnall,⸱Mrs.⸱(Edith⸱Martha⸱Bowerman)
female
nan
0
1
113505
55.0
E33
S
6
nan
St⸱Leonards-on-Sea,⸱England⸱Ohio
1
0
Clark,⸱Mr.⸱Walter⸱Miller
male
27.0
1
0
13508
136.7792
C89
C
nan
nan
Los⸱Angeles,⸱CA
1
1
Clark,⸱Mrs.⸱Walter⸱Miller⸱(Virginia⸱McDowell)
female
26.0
1
0
13508
136.7792
C89
C
4
nan
Los⸱Angeles,⸱CA
1
1
Cleaver,⸱Miss.⸱Alice
female
22.0
0
0
113781
151.55
nan
S
11
nan
nan
1
0
Clifford,⸱Mr.⸱George⸱Quincy
male
nan
0
0
110465
52.0
A14
S
nan
nan
Stoughton,⸱MA
1
0
Colley,⸱Mr.⸱Edward⸱Pomeroy
male
47.0
0
0
5727
25.5875
E58
S
nan
nan
Victoria,⸱BC
1
1
Compton,⸱Miss.⸱Sara⸱Rebecca
female
39.0
1
1
PC⸱17756
83.1583
E49
C
14
nan
Lakewood,⸱NJ
1
0
Compton,⸱Mr.⸱Alexander⸱Taylor⸱Jr
male
37.0
1
1
PC⸱17756
83.1583
E52
C
nan
nan
Lakewood,⸱NJ
1
1
Compton,⸱Mrs.⸱Alexander⸱Taylor⸱(Mary⸱Eliza⸱Ingersoll)
female
64.0
0
2
PC⸱17756
83.1583
E45
C
14
nan
Lakewood,⸱NJ
1
1
Cornell,⸱Mrs.⸱Robert⸱Clifford⸱(Malvina⸱Helen⸱Lamson)
female
55.0
2
0
11770
25.7
C101
S
2
nan
New⸱York,⸱NY
1
0
Crafton,⸱Mr.⸱John⸱Bertram
male
nan
0
0
113791
26.55
nan
S
nan
nan
Roachdale,⸱IN
1
0
Crosby,⸱Capt.⸱Edward⸱Gifford
male
70.0
1
1
WE/P⸱5735
71.0
B22
S
nan
269.0
Milwaukee,⸱WI
1
1
Crosby,⸱Miss.⸱Harriet⸱R
female
36.0
0
2
WE/P⸱5735
71.0
B22
S
7
nan
Milwaukee,⸱WI
1
1
Crosby,⸱Mrs.⸱Edward⸱Gifford⸱(Catherine⸱Elizabeth⸱Halstead)
female
64.0
1
1
112901
26.55
B26
S
7
nan
Milwaukee,⸱WI
1
0
Cumings,⸱Mr.⸱John⸱Bradley
male
39.0
1
0
PC⸱17599
71.2833
C85
C
nan
nan
New⸱York,⸱NY
1
1
Cumings,⸱Mrs.⸱John⸱Bradley⸱(Florence⸱Briggs⸱Thayer)
female
38.0
1
0
PC⸱17599
71.2833
C85
C
4
nan
New⸱York,⸱NY
1
1
Daly,⸱Mr.⸱Peter⸱Denis⸱
male
51.0
0
0
113055
26.55
E17
S
5⸱9
nan
Lima,⸱Peru
1
1
Daniel,⸱Mr.⸱Robert⸱Williams
male
27.0
0
0
113804
30.5
nan
S
3
nan
Philadelphia,⸱PA
1
1
Daniels,⸱Miss.⸱Sarah
female
33.0
0
0
113781
151.55
nan
S
8
nan
nan
1
0
Davidson,⸱Mr.⸱Thornton
male
31.0
1
0
F.C.⸱12750
52.0
B71
S
nan
nan
Montreal,⸱PQ
1
1
Davidson,⸱Mrs.⸱Thornton⸱(Orian⸱Hays)
female
27.0
1
2
F.C.⸱12750
52.0
B71
S
3
nan
Montreal,⸱PQ
1
1
Dick,⸱Mr.⸱Albert⸱Adrian
male
31.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dick,⸱Mrs.⸱Albert⸱Adrian⸱(Vera⸱Gillespie)
female
17.0
1
0
17474
57.0
B20
S
3
nan
Calgary,⸱AB
1
1
Dodge,⸱Dr.⸱Washington
male
53.0
1
1
33638
81.8583
A34
S
13
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Master.⸱Washington
male
4.0
0
2
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
1
Dodge,⸱Mrs.⸱Washington⸱(Ruth⸱Vidaver)
female
54.0
1
1
33638
81.8583
A34
S
5
nan
San⸱Francisco,⸱CA
1
0
Douglas,⸱Mr.⸱Walter⸱Donald
male
50.0
1
0
PC⸱17761
106.425
C86
C
nan
62.0
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Douglas,⸱Mrs.⸱Frederick⸱Charles⸱(Mary⸱Helene⸱Baxter)
female
27.0
1
1
PC⸱17558
247.5208
B58⸱B60
C
6
nan
Montreal,⸱PQ
1
1
Douglas,⸱Mrs.⸱Walter⸱Donald⸱(Mahala⸱Dutton)
female
48.0
1
0
PC⸱17761
106.425
C86
C
2
nan
Deephaven,⸱MN⸱/⸱Cedar⸱Rapids,⸱IA
1
1
Duff⸱Gordon,⸱Lady.⸱(Lucille⸱Christiana⸱Sutherland)⸱("Mrs⸱Morgan")
female
48.0
1
0
11755
39.6
A16
C
1
nan
London⸱/⸱Paris
1
1
Duff⸱Gordon,⸱Sir.⸱Cosmo⸱Edmund⸱("Mr⸱Morgan")
male
49.0
1
0
PC⸱17485
56.9292
A20
C
1
nan
London⸱/⸱Paris
1
0
Dulles,⸱Mr.⸱William⸱Crothers
male
39.0
0
0
PC⸱17580
29.7
A18
C
nan
133.0
Philadelphia,⸱PA
Viewing 100 of 1292 rows / 14 columns
8 partition(s)
In [28]:
df.outliers.mad("fare", threshold= 1).drop().table()
Viewing 17 of 17 rows / 14 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
1
0
Andrews,⸱Mr.⸱Thomas⸱Jr
male
39.0
0
0
112050
0.0
A36
S
nan
nan
Belfast,⸱NI
1
0
Chisholm,⸱Mr.⸱Roderick⸱Robert⸱Crispin
male
nan
0
0
112051
0.0
nan
S
nan
nan
Liverpool,⸱England⸱/⸱Belfast
1
0
Fry,⸱Mr.⸱Richard
male
nan
0
0
112058
0.0
B102
S
nan
nan
nan
1
0
Harrison,⸱Mr.⸱William
male
40.0
0
0
112059
0.0
B94
S
nan
110.0
nan
1
1
Ismay,⸱Mr.⸱Joseph⸱Bruce
male
49.0
0
0
112058
0.0
B52⸱B54⸱B56
S
C
nan
Liverpool
1
0
Parr,⸱Mr.⸱William⸱Henry⸱Marsh
male
nan
0
0
112052
0.0
nan
S
nan
nan
Belfast
1
0
Reuchlin,⸱Jonkheer.⸱John⸱George
male
38.0
0
0
19972
0.0
nan
S
nan
nan
Rotterdam,⸱Netherlands
2
0
Campbell,⸱Mr.⸱William
male
nan
0
0
239853
0.0
nan
S
nan
nan
Belfast
2
0
Cunningham,⸱Mr.⸱Alfred⸱Fleming
male
nan
0
0
239853
0.0
nan
S
nan
nan
Belfast
2
0
Frost,⸱Mr.⸱Anthony⸱Wood⸱"Archie"
male
nan
0
0
239854
0.0
nan
S
nan
nan
Belfast
2
0
Knight,⸱Mr.⸱Robert⸱J
male
nan
0
0
239855
0.0
nan
S
nan
nan
Belfast
2
0
Parkes,⸱Mr.⸱Francis⸱"Frank"
male
nan
0
0
239853
0.0
nan
S
nan
nan
Belfast
2
0
Watson,⸱Mr.⸱Ennis⸱Hastings
male
nan
0
0
239856
0.0
nan
S
nan
nan
Belfast
3
0
Johnson,⸱Mr.⸱Alfred
male
49.0
0
0
LINE
0.0
nan
S
nan
nan
nan
3
0
Johnson,⸱Mr.⸱William⸱Cahoone⸱Jr
male
19.0
0
0
LINE
0.0
nan
S
nan
nan
nan
3
0
Leonard,⸱Mr.⸱Lionel
male
36.0
0
0
LINE
0.0
nan
S
nan
nan
nan
3
1
Tornquist,⸱Mr.⸱William⸱Henry
male
25.0
0
0
LINE
0.0
nan
S
15
nan
nan
Viewing 17 of 17 rows / 14 columns
8 partition(s)
In [34]:
df.outliers.modified_z_score("fare", threshold= 1).select().table()
Viewing 0 of 0 rows / 15 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
m_z_score
15 (double)
nullable
Viewing 0 of 0 rows / 15 columns
8 partition(s)
In [35]:
df.outliers.modified_z_score("fare", threshold= 1).drop().table()
Viewing 0 of 0 rows / 15 columns
8 partition(s)
pclass
1 (bigint)
nullable
survived
2 (bigint)
nullable
name
3 (string)
nullable
sex
4 (string)
nullable
age
5 (double)
nullable
sibsp
6 (bigint)
nullable
parch
7 (bigint)
nullable
ticket
8 (string)
nullable
fare
9 (double)
nullable
cabin
10 (string)
nullable
embarked
11 (string)
nullable
boat
12 (string)
nullable
body
13 (double)
nullable
home_dest
14 (string)
nullable
m_z_score
15 (double)
nullable
Viewing 0 of 0 rows / 15 columns
8 partition(s)
In [ ]:
Content source: ironmussa/Optimus
Similar notebooks: