notebook.community

Edit and run

regression - fitting a model to data. The goal is to describe the relationship between the dependent variables and the explanatory variables.

multiple regression multiple explanatory variables.

linear regression wen the relationship is linear. $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon $$ where $\beta_0$ is the intercept, $\beta_1$ is the parameter association with $x_1$, $\beta_2$ is the parameter associated with $x_2$ and $\epsilon$ is the residual due to random variation or unknown factors.

ordinary least squares. Given a sequence of values for y and sequences for $x_1$ and $x_2$, we can find the beta parameters that minimize $epsilon^2$





In [7]:



    


live, firsts, others = first.MakeFrames()
formula = 'totalwgt_lb ~ agepreg'
model = smf.ols(formula, data=live)
results = model.fit()
regression.SummarizeResults(results)



    


















    






Intercept   6.83   (0)
agepreg   0.0175   (5.72e-11)
R^2 0.004738
Std(ys) 1.408
Std(res) 1.405

















In [9]:



    


##Results are also available as parameters:
inter = results.params['Intercept']
slope = results.params['agepreg']
slope_pvalue = results.pvalues['agepreg']
results.rsquared



    


















    
Out[9]:








0.0047381154747098142

















In [11]:



    


##this gives the p-value associated with the model as a whole
results.f_pvalue



    


















    
Out[11]:








5.7229471072677547e-11

















In [13]:



    


residuals = results.resid

##this returns a sequence of values corresponding
##to agepreg.
fitted_values = results.fittedvalues

# results.summary() provides a lot of info
#the following is easier:
regression.SummarizeResults(results)



    


















    






Intercept   6.83   (0)
agepreg   0.0175   (5.72e-11)
R^2 0.004738
Std(ys) 1.408
Std(res) 1.405

spurious - result for which there is no obvious mechanism that would explain it. e.g. why would first babies be lighter than others? Perhaps because mothers of first babies are younger...





In [14]:



    


diff_weight = firsts.totalwgt_lb.mean() - others.totalwgt_lb.mean()
diff_weight



    


















    
Out[14]:








-0.12476118453549034

















In [17]:



    


diff_age = firsts.agepreg.mean() - others.agepreg.mean()
diff_age



    


















    
Out[17]:








-3.586434766150152

















In [19]:



    


results = smf.ols('totalwgt_lb ~ agepreg', data=live).fit()
slope = results.params['agepreg']
slope



    


















    
Out[19]:








0.017453851471802891

















In [20]:



    


##expected difference in birthweight for
##first babies and others, due to mother's age:
slope * diff_age



    


















    
Out[20]:








-0.062597099721694888

















In [26]:



    


##this only accounts for half of the observed difference.
live['isfirst'] = live.birthord == 1
formula = 'totalwgt_lb ~ isfirst'
results = smf.ols(formula, data=live).fit()

regression.SummarizeResults(results)
print 

formula = 'totalwgt_lb ~ agepreg'
results = smf.ols(formula, data=live).fit()

regression.SummarizeResults(results)



    


















    






Intercept   7.33   (0)
isfirst[T.True]   -0.125   (2.55e-05)
R^2 0.00196
Std(ys) 1.408
Std(res) 1.407

Intercept   6.83   (0)
agepreg   0.0175   (5.72e-11)
R^2 0.004738
Std(ys) 1.408
Std(res) 1.405

isfirst is a categorical variable, i.e. it falls into categories like True or False. The estimated parameter is the effect on birth weight when isfirst is true. So the result, -0.125 lbs is the difference in birth weight between first babies and others.





In [28]:



    


formula = 'totalwgt_lb ~ isfirst + agepreg'
results = smf.ols(formula, data=live).fit()
regression.SummarizeResults(results)



    


















    






Intercept   6.91   (0)
isfirst[T.True]   -0.0698   (0.0253)
agepreg   0.0154   (3.93e-08)
R^2 0.005289
Std(ys) 1.408
Std(res) 1.405

In the combined model, the parameter isfirst is smaller by about half, which means that part of the apparent effict is accounted for by agepreg. Also, notice the p-value is now on the border of statistical significance.

could the contribution of agepreg be nonlinear? (Note, this is still linear regression, because the explanatory variable is now just the result of a nonlinear function)





In [30]:



    


live['agepreg2'] = live.agepreg**2
formula = 'totalwgt_lb ~ isfirst + agepreg + agepreg2'
results = smf.ols(formula, data=live).fit()
regression.SummarizeResults(results)



    


















    






Intercept   5.69   (1.38e-86)
isfirst[T.True]   -0.0504   (0.109)
agepreg   0.112   (3.23e-07)
agepreg2   -0.00185   (8.8e-06)
R^2 0.007462
Std(ys) 1.408
Std(res) 1.403

Negative coefficient in agepreg2 corroborates with downward curving of figure 10.2

in this example, the mother's age acts as a control variable for the diference in age between first-time mothers and others, making it possible to isolate the effect of isfirst





In [46]:



    


live = live[live.prglngth>30]
resp = chap01soln.ReadFemResp()
resp.index = resp.caseid
join = live.join(resp, on='caseid', rsuffix='_r')



    











In [50]:



    


t = []
for name in join.columns:

    try:
        if join[name].var() < 1e-7:
            continue
            
        formula = 'totalwgt_lb ~ agepreg + ' + str(name)
        model = smf.ols(formula, data=join)
        if model.nobs < len(join)/2: #num of observations
            continue
        
        results = model.fit()
    except(ValueError, TypeError):
        continue
        
    t.append((results.rsquared, name))
t



    


















    
Out[50]:








[(0.0053576473236404132, u'caseid'),
 (0.0057500139850769072, u'pregordr'),
 (0.0063309802373898716, u'pregend1'),
 (0.016017752709788002, u'nbrnaliv'),
 (0.0055431561930947559, u'cmprgend'),
 (0.0054428005916395961, u'cmprgbeg'),
 (0.0053276126015608938, u'gestasun_m'),
 (0.0070235526384532232, u'gestasun_w'),
 (0.1234004136336101, u'wksgest'),
 (0.027144274639579802, u'mosgest'),
 (0.0053368691675184099, u'bpa_bdscheck1'),
 (0.018550925293941867, u'babysex'),
 (0.94981273059780091, u'birthwgt_lb'),
 (0.013102457615706053, u'birthwgt_oz'),
 (0.0055431561930947559, u'cmbabdob'),
 (0.0056849526500283298, u'kidage'),
 (0.006165319836040517, u'hpagelb'),
 (0.0080663173686768008, u'matchfound'),
 (0.012529022541810764, u'anynurse'),
 (0.0044098205836260451, u'frsteatd_n'),
 (0.0042639734717098143, u'frsteatd_p'),
 (0.00402013146273561, u'frsteatd'),
 (0.0058305717702541449, u'cmlastlb'),
 (0.0053567472661234516, u'cmfstprg'),
 (0.0054283336509906022, u'cmlstprg'),
 (0.0057314017337591894, u'cmintstr'),
 (0.0055431561930947559, u'cmintfin'),
 (0.0099330608071229731, u'evuseint'),
 (0.0093150997041332451, u'stopduse'),
 (0.003726833286729847, u'wantbold'),
 (0.0070729951341231834, u'timingok'),
 (0.0050425040938113508, u'wthpart1'),
 (0.006835771483523545, u'hpwnold'),
 (0.0063490947134500209, u'timokhp'),
 (0.0026291376151111878, u'cohpbeg'),
 (0.0018043469091935105, u'cohpend'),
 (0.0080896000349425234, u'tellfath'),
 (0.0090562503555627893, u'whentell'),
 (0.0053699742787950422, u'anyusint'),
 (0.13012519488625052, u'prglngth'),
 (0.0055456150842299046, u'birthord'),
 (0.0055917458475837067, u'datend'),
 (0.0053272825050707517, u'agepreg'),
 (0.0056653888432278698, u'datecon'),
 (0.10203149928156086, u'agecon'),
 (0.010461691367376624, u'fmarout5'),
 (0.0098408049117157947, u'pmarpreg'),
 (0.01135413847280542, u'rmarout6'),
 (0.010604964684299945, u'fmarcon5'),
 (0.30082407844707693, u'lbw1'),
 (0.012193688404495751, u'bfeedwks'),
 (0.0079848356842529, u'oldwantr'),
 (0.0064013866853640522, u'oldwantp'),
 (0.0079808325386582224, u'wantresp'),
 (0.0063344689873005011, u'wantpart'),
 (0.0055916160044224839, u'cmbirth'),
 (0.0055903980552199917, u'ager'),
 (0.0055903980552199917, u'agescrn'),
 (0.0099449426591107226, u'fmarital'),
 (0.008267774071422429, u'rmarital'),
 (0.0064509138033004287, u'educat'),
 (0.0066919868225505663, u'hieduc'),
 (0.016199503586253106, u'race'),
 (0.0053512731010235681, u'hispanic'),
 (0.011238349302030937, u'hisprace'),
 (0.005415425347505165, u'rcurpreg'),
 (0.0060378317082535604, u'pregnum'),
 (0.006503720321448303, u'parity'),
 (0.0054442288636177283, u'insuranc'),
 (0.0098585456428506024, u'pubassis'),
 (0.0097431589752972059, u'poverty'),
 (0.0061242506200275271, u'laborfor'),
 (0.0054762462261790379, u'religion'),
 (0.0059086876990791515, u'metro'),
 (0.0053296353237816119, u'brnout'),
 (0.0053882407583270009, u'prglngth_i'),
 (0.0053720967087054206, u'datend_i'),
 (0.0056661042813174189, u'agepreg_i'),
 (0.0053480888696348927, u'datecon_i'),
 (0.0056127402108959723, u'agecon_i'),
 (0.0057331402604468007, u'fmarout5_i'),
 (0.0054225985712880176, u'pmarpreg_i'),
 (0.0054988859391116307, u'rmarout6_i'),
 (0.0057702817140149465, u'fmarcon5_i'),
 (0.0053555873582948887, u'learnprg_i'),
 (0.0054645526519414567, u'pncarewk_i'),
 (0.0059115757010617109, u'paydeliv_i'),
 (0.0053272825050707517, u'lbw1_i'),
 (0.0054228434408329917, u'bfeedwks_i'),
 (0.0054562770335879751, u'maternlv_i'),
 (0.0053978237624938696, u'oldwantr_i'),
 (0.0053301020636039587, u'oldwantp_i'),
 (0.0053978237624938696, u'wantresp_i'),
 (0.0053882613287266068, u'wantpart_i'),
 (0.0054158542055683379, u'hieduc_i'),
 (0.0053272825050706407, u'hispanic_i'),
 (0.00566216198540892, u'parity_i'),
 (0.0054901920776948554, u'insuranc_i'),
 (0.0055882636622018866, u'pubassis_i'),
 (0.0056746687213740099, u'poverty_i'),
 (0.0056353938189386277, u'laborfor_i'),
 (0.005329126750794444, u'religion_i'),
 (0.0072660831598055919, u'basewgt'),
 (0.0068633447572696848, u'adj_mod_basewgt'),
 (0.0074146019069666336, u'finalwgt'),
 (0.0059967325885621481, u'secu_p'),
 (0.0054052918682682138, u'sest'),
 (1.0, 'totalwgt_lb'),
 (0.0056177882917133326, 'isfirst'),
 (0.0075292178006126642, 'agepreg2'),
 (0.0053576473236404132, u'caseid_r'),
 (0.0053276836578058928, u'rscrinf'),
 (0.0053945215566748583, u'rdormres'),
 (0.0056439259750301618, u'rostscrn'),
 (0.0054048781281779146, u'rscreenhisp'),
 (0.0096516053700311799, u'rscreenrace'),
 (0.005578533477514247, u'age_a'),
 (0.0055903980552199917, u'age_r'),
 (0.0055916160044224839, u'cmbirth_r'),
 (0.0055903980552199917, u'agescrn_r'),
 (0.008267774071422429, u'marstat'),
 (0.0099449426591107226, u'fmarit'),
 (0.0090913760031458013, u'evrmarry'),
 (0.00535044323233691, u'hisp'),
 (0.0055163472621161391, u'numrace'),
 (0.0053556674242534186, u'roscnt'),
 (0.0070034753968700736, u'hplocale'),
 (0.0077681643343214812, u'manrel'),
 (0.0053482629285521721, u'fl_rrace'),
 (0.0053305933943744721, u'fl_rhisp'),
 (0.0059860925088681682, u'goschol'),
 (0.0062182476084239324, u'higrade'),
 (0.0055959722057244976, u'compgrd'),
 (0.0061609210437844064, u'havedip'),
 (0.0064131015027882876, u'dipged'),
 (0.0060614222669637119, u'cmhsgrad'),
 (0.005328553608113018, u'wthparnw'),
 (0.0052806156270835958, u'onown'),
 (0.006129166274024711, u'intact'),
 (0.0060388191911330269, u'parmarr'),
 (0.0056351833456261824, u'momdegre'),
 (0.006872582290938567, u'momworkd'),
 (0.0053303762178340541, u'momchild'),
 (0.005349294568680607, u'momfstch'),
 (0.0062451111463343034, u'daddegre'),
 (0.0053285193871117631, u'bothbiol'),
 (0.0061821286079011273, u'intact18'),
 (0.0053404516447668193, u'onown18'),
 (0.006503720321448303, u'numbabes'),
 (0.0055503488109680532, u'totplacd'),
 (0.00533722814433224, u'nplaced'),
 (0.0055305863137274924, u'ndied'),
 (0.0055398740794350143, u'nadoptv'),
 (0.0058305717702541449, u'cmlastlb_r'),
 (0.0053567472661234516, u'cmfstprg_r'),
 (0.0054283336509906022, u'cmlstprg_r'),
 (0.0059155825615546664, u'menarche'),
 (0.0053438451858864866, u'pregnowq'),
 (0.0060378317082535604, u'numpregs'),
 (0.005415425347505165, u'currpreg'),
 (0.0057809848939388564, u'giveadpt'),
 (0.0052547662523027006, u'otherkid'),
 (0.0052894644172097882, u'everadpt'),
 (0.0058286851663725825, u'seekadpt'),
 (0.0065420652227657916, u'evwntano'),
 (0.0071464847365002582, u'timesmar'),
 (0.0062601102490156268, u'hsbverif'),
 (0.0065101987688338525, u'cmmarrhx'),
 (0.0075598240826892926, u'hxagemar'),
 (0.0074322695963854324, u'cmhsbdobx'),
 (0.0078351117468986597, u'lvtoghx'),
 (0.0067325353981322333, u'hisphx'),
 (0.0085438558962523947, u'racehx1'),
 (0.0095264030684480971, u'chedmarn'),
 (0.0064640904817812039, u'marbefhx'),
 (0.0079282736010669597, u'kidshx'),
 (0.0071835438455390976, u'cmmarrch'),
 (0.0067616874322955489, u'cmdobch'),
 (0.0065487436385180953, u'prevhusb'),
 (0.0062956088671435317, u'cmstrthp'),
 (0.007320288484473636, u'evrcohab'),
 (0.0073365000741100195, u'liveoth'),
 (0.0061317492114570937, u'prevcohb'),
 (0.0056400389459839717, u'cmfstsex'),
 (0.0054298693534171605, u'agefstsx'),
 (0.0010172326396578057, u'grfstsx'),
 (0.005745270584701756, u'sameman'),
 (0.0054772600593094856, u'fpage'),
 (0.0053827337115196494, u'knowfp'),
 (0.0060437264562245296, u'cmlsexfp'),
 (0.0059106934677165546, u'cmfplast'),
 (0.005423734887384235, u'lifeprt'),
 (0.0053417476676167475, u'mon12prt'),
 (0.0053493888650057997, u'parts12'),
 (0.0065024665690611716, u'ptsb4mar'),
 (0.0062700268322847252, u'p1yrage'),
 (0.0044757357340441395, u'p1yhsage'),
 (0.0046862977809060125, u'p1yrf'),
 (0.0079272198264949134, u'cmfsexx'),
 (0.0060854504154602695, u'pcurrntx'),
 (0.0061034336026016822, u'cmlsexx'),
 (0.006129604796508703, u'cmlstsxx'),
 (0.0060299492496318496, u'cmlstsx12'),
 (0.005339302768089027, u'lifeprts'),
 (0.0053282763855252435, u'cmlastsx'),
 (0.005790778139280528, u'currprtt'),
 (0.0066913335759739567, u'currprts'),
 (0.0042599193119881029, u'cmpart1y1'),
 (0.0053727944846477804, u'evertubs'),
 (0.0053946686848480585, u'everhyst'),
 (0.0051812137216746823, u'everovrs'),
 (0.0055216206466340845, u'everothr'),
 (0.0054978883720243621, u'anyfster'),
 (0.0058069839099267329, u'fstrop12'),
 (0.0064869433773805385, u'anyopsmn'),
 (0.0064756285532651114, u'anymster'),
 (0.0054575168759826154, u'rsurgstr'),
 (0.006286783510051297, u'psurgstr'),
 (0.0058616476673258866, u'onlytbvs'),
 (0.0038784396522588693, u'posiblpg'),
 (0.0080111472080747204, u'canhaver'),
 (0.0038934242509262029, u'pregnono'),
 (0.0053315946933329972, u'rstrstat'),
 (0.0068091718293959991, u'pstrstat'),
 (0.0064430762813793008, u'pill'),
 (0.0054567147113174785, u'condom'),
 (0.0068782398430788927, u'vasectmy'),
 (0.0056471327033019714, u'widrawal'),
 (0.0067777099610749625, u'depoprov'),
 (0.0053607662679511225, u'norplant'),
 (0.0065582707998581569, u'rhythm'),
 (0.0067814729061593804, u'tempsafe'),
 (0.0053332555915018842, u'mornpill'),
 (0.0055887758064135928, u'diafragm'),
 (0.0081814662301715746, u'wocondom'),
 (0.0057124418914624719, u'foamalon'),
 (0.0056058928255962037, u'jelcrmal'),
 (0.0066695053099961044, u'cervlcap'),
 (0.0053289710517908651, u'supposit'),
 (0.0067739501854272577, u'todayspg'),
 (0.0053484435351783821, u'iud'),
 (0.0056696931757193036, u'lunelle'),
 (0.0055746648157349865, u'patch'),
 (0.0063170453577297003, u'othrmeth'),
 (0.0053289129639070287, u'everused'),
 (0.0062621002452540875, u'methdiss'),
 (0.0067832551871572777, u'methstop01'),
 (0.0058275827000024982, u'firsmeth01'),
 (0.0059969573119953434, u'numfirsm'),
 (0.0057600954802817794, u'numfirsm1'),
 (0.0063358823716082036, u'numfirsm2'),
 (0.0063362318522404149, u'drugdev'),
 (0.0043698234818874138, u'firstime2'),
 (0.0051833574481859879, u'cmfstuse'),
 (0.0064989913319059012, u'cmfirsm'),
 (0.0044375105239676804, u'agefstus'),
 (0.0057359522276135833, u'usefstsx'),
 (0.005546199546767494, u'intr_ec3'),
 (0.006421238927684092, u'monsx1177'),
 (0.0059959755280611926, u'monsx1178'),
 (0.0058717726321054764, u'monsx1179'),
 (0.0053552449128619894, u'monsx1180'),
 (0.005475780215881465, u'monsx1181'),
 (0.0058782648187578612, u'monsx1182'),
 (0.0059352901693664473, u'monsx1183'),
 (0.0061367227367417065, u'monsx1184'),
 (0.0061144613279541238, u'monsx1185'),
 (0.005914492828921758, u'monsx1186'),
 (0.0057151457217395896, u'monsx1187'),
 (0.0059426297773008674, u'monsx1188'),
 (0.0064038785238790252, u'monsx1189'),
 (0.0066524314863445344, u'monsx1190'),
 (0.0058594648882090983, u'monsx1191'),
 (0.0061836704118878316, u'monsx1192'),
 (0.0059706976543186796, u'monsx1193'),
 (0.0061859303481812677, u'monsx1194'),
 (0.0062617226153404149, u'monsx1195'),
 (0.005883642310483661, u'monsx1196'),
 (0.006312131013808342, u'monsx1197'),
 (0.0063344901582982338, u'monsx1198'),
 (0.0062381570952112497, u'monsx1199'),
 (0.0059583031114507357, u'monsx1200'),
 (0.0059716118238308802, u'monsx1201'),
 (0.0062329659402938775, u'monsx1202'),
 (0.0061567646880547056, u'monsx1203'),
 (0.0064636399089446073, u'monsx1204'),
 (0.006792900648231015, u'monsx1205'),
 (0.0062231016635069247, u'monsx1206'),
 (0.0060178073313029712, u'monsx1207'),
 (0.0063292418999860711, u'monsx1208'),
 (0.0063566695058365763, u'monsx1209'),
 (0.006029258683564076, u'monsx1210'),
 (0.005520990260007963, u'monsx1211'),
 (0.0059877289570539061, u'monsx1212'),
 (0.0071980336681909396, u'monsx1213'),
 (0.0068990203985326293, u'monsx1214'),
 (0.0061510644336258968, u'monsx1215'),
 (0.0060411919419445237, u'monsx1216'),
 (0.0062153217548310824, u'monsx1217'),
 (0.0057135212840345728, u'monsx1218'),
 (0.0059338553062215915, u'monsx1219'),
 (0.0061676677834882065, u'monsx1220'),
 (0.0060299487363477677, u'monsx1221'),
 (0.0058400623790983985, u'monsx1222'),
 (0.0056196119411137113, u'monsx1223'),
 (0.0065633057152900687, u'monsx1224'),
 (0.0062843497325775211, u'monsx1225'),
 (0.006323173014770922, u'monsx1226'),
 (0.0055343659799074185, u'monsx1227'),
 (0.0063747987871534129, u'monsx1228'),
 (0.0075207659068715627, u'monsx1229'),
 (0.0080836979678282628, u'monsx1230'),
 (0.0072333746562985857, u'monsx1231'),
 (0.0079852170801011368, u'monsx1232'),
 (0.0071910331970505004, u'monsx1233'),
 (0.0062614250774709612, u'cmstrtmc'),
 (0.0058259585172517614, u'cmendmc'),
 (0.0056035063888357772, u'methhist011'),
 (0.0072849803538022062, u'cmdatbgn'),
 (0.0056354090451569139, u'nummult'),
 (0.0055518440086853582, u'methhist021'),
 (0.0054895117434012874, u'nummult2'),
 (0.0055628303964553316, u'methhist031'),
 (0.005546883063649477, u'nummult3'),
 (0.00548720240573497, u'methhist041'),
 (0.0055024609376932432, u'nummult4'),
 (0.0055361891691546594, u'methhist051'),
 (0.0054527298821702708, u'nummult5'),
 (0.0054892454460828688, u'methhist061'),
 (0.0054758955006791732, u'nummult6'),
 (0.0054732146908181845, u'methhist071'),
 (0.005428876448518527, u'nummult7'),
 (0.0055940468548328903, u'methhist081'),
 (0.0054730049104985135, u'nummult8'),
 (0.0054826171583020056, u'methhist091'),
 (0.0054272326646152003, u'nummult9'),
 (0.0055033931564416605, u'methhist101'),
 (0.0054371432339523906, u'nummult10'),
 (0.0055164309981879933, u'methhist111'),
 (0.0055023053044437331, u'nummult11'),
 (0.0054703020981754502, u'methhist121'),
 (0.0055042550198189932, u'nummult12'),
 (0.0056074588743066922, u'methhist131'),
 (0.005549721074182945, u'nummult13'),
 (0.0055467170682934608, u'methhist141'),
 (0.005542683859749431, u'nummult14'),
 (0.0055396922955606165, u'methhist151'),
 (0.0055324680547159666, u'nummult15'),
 (0.0055364926479462007, u'methhist161'),
 (0.0055364013420063918, u'nummult16'),
 (0.0055193321636847203, u'methhist171'),
 (0.005493486008020354, u'nummult17'),
 (0.0055071799748506178, u'methhist181'),
 (0.0054913983634300445, u'nummult18'),
 (0.005471902813542151, u'methhist191'),
 (0.0054876404996537786, u'nummult19'),
 (0.0055047431684942527, u'methhist201'),
 (0.0054667392315834729, u'nummult20'),
 (0.0055101869889417898, u'methhist211'),
 (0.0054912031932206062, u'nummult21'),
 (0.0055063374486067351, u'methhist221'),
 (0.0055116072919971781, u'nummult22'),
 (0.0055295450491830156, u'methhist231'),
 (0.0055308060174652507, u'nummult23'),
 (0.00555002676240679, u'methhist241'),
 (0.0055399482898828012, u'nummult24'),
 (0.0055211355102122717, u'methhist251'),
 (0.0055682855093340811, u'nummult25'),
 (0.005540129688542117, u'methhist261'),
 (0.0056737976282855707, u'nummult26'),
 (0.0055319741942273515, u'methhist271'),
 (0.0057840891771829828, u'nummult27'),
 (0.0055784115374740706, u'methhist281'),
 (0.0056004017468155309, u'nummult28'),
 (0.0055787140348123598, u'methhist291'),
 (0.0056155148617611506, u'nummult29'),
 (0.0056429313351301635, u'methhist301'),
 (0.0057317398602387826, u'nummult30'),
 (0.005616953589054452, u'methhist311'),
 (0.0056699369492140272, u'nummult31'),
 (0.0056435552222080521, u'methhist321'),
 (0.005749895915191594, u'nummult32'),
 (0.0056755907412765483, u'methhist331'),
 (0.0057804660793031593, u'nummult33'),
 (0.0057272452322159051, u'methhist341'),
 (0.0058725649763198273, u'nummult34'),
 (0.005746152259455628, u'methhist351'),
 (0.0058426198665049167, u'nummult35'),
 (0.0057501541514437537, u'methhist361'),
 (0.0056817018914434581, u'nummult36'),
 (0.0060752977593035995, u'methhist371'),
 (0.0057049518691774104, u'nummult37'),
 (0.006209219638329877, u'methhist381'),
 (0.005708874481709425, u'nummult38'),
 (0.0061500189125289761, u'methhist391'),
 (0.00574794039603721, u'nummult39'),
 (0.0066929505455287641, u'methhist401'),
 (0.0062588576693297648, u'nummult40'),
 (0.0074562744272722581, u'methhist411'),
 (0.0069322161564748797, u'nummult41'),
 (0.0083592507042364872, u'methhist421'),
 (0.0075219647330849737, u'nummult42'),
 (0.0099531025438633902, u'methhist431'),
 (0.0079716022439664247, u'nummult43'),
 (0.0095841607541451435, u'methhist441'),
 (0.0076066031246935228, u'nummult44'),
 (0.0095081440869234646, u'methhist451'),
 (0.0068287476902857414, u'nummult45'),
 (0.0075844174820761667, u'currmeth1'),
 (0.0062703734400433309, u'lastmonmeth1'),
 (0.0067277169867890896, u'uselstp'),
 (0.0070190854716516426, u'lstmthp11'),
 (0.0055807981171464016, u'usefstp'),
 (0.0060429352777050527, u'pst4wksx'),
 (0.0067419830697780236, u'pswkcond2'),
 (0.0064188786095964456, u'p12mocon'),
 (0.0053464291695218824, u'bthcon12'),
 (0.0053329546233988889, u'medtst12'),
 (0.0053416246857382887, u'bccns12'),
 (0.0054297376194217817, u'stcns12'),
 (0.0054709241068920944, u'eccns12'),
 (0.0053489720771592308, u'prgtst12'),
 (0.0053943749433754684, u'abort12'),
 (0.0053674464142426936, u'pap12'),
 (0.005672332993427287, u'pelvic12'),
 (0.0062589483576195315, u'stdtst12'),
 (0.0048423429426126541, u'numbcvis'),
 (0.0043996610078129716, u'papplbc2'),
 (0.0044378462185822309, u'pappelec'),
 (0.0057259805915401696, u'rwant'),
 (0.0059484588815129635, u'pwant'),
 (0.0053320083358597836, u'hlpprg'),
 (0.0055619447936323674, u'prgvisit'),
 (0.0057438068195749814, u'hlpmc'),
 (0.0066760172742330592, u'duchfreq'),
 (0.0058131003702927009, u'pid'),
 (0.0056901586625383027, u'diabetes'),
 (0.0055317431314885201, u'ovacyst'),
 (0.0060381968053511148, u'uf'),
 (0.0059342240892179543, u'endo'),
 (0.0054405377721771231, u'ovuprob'),
 (0.0061251320208638438, u'limited'),
 (0.0055412140576929225, u'equipmnt'),
 (0.0059813504168927389, u'donbld85'),
 (0.0054366127940179698, u'hivtest'),
 (0.0044779715958018773, u'cmhivtst'),
 (0.0043550895285197067, u'plchiv'),
 (0.0043146120619145245, u'hivtst'),
 (0.0056182137711081603, u'talkdoct'),
 (0.0053298100463655684, u'retrovir'),
 (0.0066636726925493051, u'cover12'),
 (0.0066930787330390373, u'coverhow01'),
 (0.0060500431013752465, u'sameadd'),
 (0.0053296353237816119, u'brnout_r'),
 (0.014003795578114264, u'paydu'),
 (0.0054071806345714624, u'relraisd'),
 (0.0053703569239259563, u'relcurr'),
 (0.0056556361151566259, u'fundam'),
 (0.0055049886043366802, u'reldlife'),
 (0.0058679528754042032, u'attndnow'),
 (0.0059334810804381188, u'evwrk6mo'),
 (0.0063049738571673286, u'cmbfstwk'),
 (0.0056441450014176331, u'evrntwrk'),
 (0.0053281503122063478, u'wrk12mos'),
 (0.0071785605121775786, u'fpt12mos'),
 (0.0064453598446236882, u'dolastwk1'),
 (0.0040186984568405526, u'dolastwk2'),
 (0.0052167431933290986, u'dolastwk3'),
 (0.0062574600046573225, u'rwrkst'),
 (0.0056444841193560302, u'everwork'),
 (0.0051777792938756395, u'rnumjob'),
 (0.005293304160077783, u'rftptx'),
 (0.0059898508613935864, u'rearnty'),
 (0.0072125285913258175, u'splstwk1'),
 (0.0085288969108578971, u'spwrkst'),
 (0.0057139992726817912, u'spnumjob'),
 (0.0067515685236466672, u'spftptx'),
 (0.0058770300549914056, u'spearnty'),
 (0.0048868961209246331, u'chcarany'),
 (0.0053661166026764962, u'better'),
 (0.0053561971238544892, u'staytog'),
 (0.0053288460224785172, u'samesex'),
 (0.0053541143477808806, u'anyact'),
 (0.0054050041417266215, u'sxok18'),
 (0.0059979659014747222, u'sxok16'),
 (0.0059849380376600925, u'chreward'),
 (0.0059733872159365431, u'chsuppor'),
 (0.0053520127753732272, u'gayadopt'),
 (0.0057859011832847029, u'okcohab'),
 (0.0053302265127552761, u'warm'),
 (0.0053348261706245292, u'achieve'),
 (0.0061229559188548111, u'family'),
 (0.0054131658803773242, u'acasilang'),
 (0.0078342050222256487, u'wage'),
 (0.0063214235555506537, u'selfinc'),
 (0.0062379645560731722, u'socsec'),
 (0.0053300708896242277, u'disabil'),
 (0.0053379803064785847, u'retire'),
 (0.0083986751808154958, u'ssi'),
 (0.0060157150486119804, u'unemp'),
 (0.0053275374411295706, u'chldsupp'),
 (0.0094771205356176669, u'interest'),
 (0.0073334196857102185, u'dividend'),
 (0.00532738241678965, u'othinc'),
 (0.0067751245784269942, u'toincwmy'),
 (0.0053522171863139567, u'totinc'),
 (0.0064238214634624313, u'pubasst'),
 (0.008558622074179123, u'foodstmp'),
 (0.0066377577796455922, u'wic'),
 (0.0053785250560243192, u'hlptrans'),
 (0.005920897262712832, u'hlpchldc'),
 (0.0054789556966823305, u'hlpjob'),
 (0.0055903980552199917, u'ager_r'),
 (0.0099449426591107226, u'fmarital_r'),
 (0.0064509138033004287, u'educat_r'),
 (0.0066919868225505663, u'hieduc_r'),
 (0.0053512731010235681, u'hispanic_r'),
 (0.016199503586253106, u'race_r'),
 (0.011238349302030937, u'hisprace_r'),
 (0.0053299864315896617, u'numkdhh'),
 (0.0053945284334835364, u'numfmhh'),
 (0.0061821286079009052, u'intctfam'),
 (0.0055257210760474296, u'parage14'),
 (0.0054737461777164542, u'educmom'),
 (0.0058533592637335374, u'agemomb1'),
 (0.0054158542055683379, u'hieduc_i_r'),
 (0.0053272825050706407, u'hispanic_i_r'),
 (0.0053298358196940399, u'parage14_i'),
 (0.0063832282122611161, u'educmom_i'),
 (0.0061601710700907031, u'agemomb1_i'),
 (0.005415425347505165, u'rcurpreg_r'),
 (0.0060378317082535604, u'pregnum_r'),
 (0.0060943638898971475, u'compreg'),
 (0.0056309953651897393, u'lossnum'),
 (0.0055226088887292413, u'abortion'),
 (0.0056503224424954146, u'lbpregs'),
 (0.006503720321448303, u'parity_r'),
 (0.0054179413566858292, u'births5'),
 (0.0057643624447896169, u'outcom01'),
 (0.0059310109980799108, u'outcom02'),
 (0.0061228537550400697, u'outcom03'),
 (0.0054190250410860452, u'datend01'),
 (0.0060290718032828261, u'datend02'),
 (0.0061086939480408109, u'datend03'),
 (0.0072782791631820221, u'ageprg01'),
 (0.0085352893655521456, u'ageprg02'),
 (0.009220965964321759, u'ageprg03'),
 (0.0053896783930724723, u'datcon01'),
 (0.0061455182075579318, u'datcon02'),
 (0.0062167895640955484, u'datcon03'),
 (0.0070671875422206876, u'agecon01'),
 (0.008474519661758273, u'agecon02'),
 (0.0088938973697009205, u'agecon03'),
 (0.011269357246806444, u'marout01'),
 (0.010141720907286822, u'marout02'),
 (0.011807801994375033, u'marout03'),
 (0.011407737138640184, u'rmarout01'),
 (0.0105469132065652, u'rmarout02'),
 (0.013430066465713431, u'rmarout03'),
 (0.0092348714313229552, u'marcon01'),
 (0.010481401795533696, u'marcon02'),
 (0.011752599354396098, u'marcon03'),
 (0.011437770919637047, u'cebow'),
 (0.0074424781337154577, u'cebowc'),
 (0.0053313724507766391, u'datbaby1'),
 (0.0064319313197047645, u'agebaby1'),
 (0.0060439287839132438, u'liv1chld'),
 (0.00566216198540892, u'lossnum_i'),
 (0.00566216198540892, u'abortion_i'),
 (0.00566216198540892, u'lbpregs_i'),
 (0.00566216198540892, u'parity_i_r'),
 (0.00566216198540892, u'births5_i'),
 (0.00591789684537114, u'outcom02_i'),
 (0.005328062662569244, u'outcom03_i'),
 (0.0056734318319197063, u'outcom04_i'),
 (0.0055126377937732007, u'outcom05_i'),
 (0.0053310532510302266, u'outcom06_i'),
 (0.0053360328740027496, u'outcom07_i'),
 (0.0054313431754534625, u'outcom08_i'),
 (0.0053651931392618701, u'outcom09_i'),
 (0.0055486604724798161, u'outcom10_i'),
 (0.0055179317548743612, u'datend01_i'),
 (0.0056909345384406063, u'datend02_i'),
 (0.0053327150228364983, u'datend03_i'),
 (0.0053288832827350641, u'datend04_i'),
 (0.0055210531409140007, u'datend05_i'),
 (0.0056142882809057104, u'datend06_i'),
 (0.0056630789816444205, u'datend07_i'),
 (0.005858198017676397, u'datend08_i'),
 (0.0057958171242896706, u'datend09_i'),
 (0.0055486604724798161, u'datend10_i'),
 (0.0055371936603058058, u'datend12_i'),
 (0.0055371936603058058, u'datend13_i'),
 (0.0055725075134728241, u'ageprg01_i'),
 (0.0060818581466958177, u'ageprg02_i'),
 (0.0054180701114089347, u'ageprg03_i'),
 (0.0053816621737202697, u'ageprg04_i'),
 (0.0056139130021368722, u'ageprg05_i'),
 (0.0054894457842579181, u'ageprg06_i'),
 (0.0055643208841910097, u'ageprg07_i'),
 (0.0058061433792946371, u'ageprg08_i'),
 (0.0057958171242896706, u'ageprg09_i'),
 (0.0055486604724798161, u'ageprg10_i'),
 (0.0055371936603058058, u'ageprg12_i'),
 (0.0055371936603058058, u'ageprg13_i'),
 (0.0054056022001517379, u'datcon01_i'),
 (0.0056909345384406063, u'datcon02_i'),
 (0.0053327150228364983, u'datcon03_i'),
 (0.0053285936239785281, u'datcon04_i'),
 (0.0054983328241399176, u'datcon05_i'),
 (0.0056142882809057104, u'datcon06_i'),
 (0.0055917732031618428, u'datcon07_i'),
 (0.005858198017676397, u'datcon08_i'),
 (0.0057958171242896706, u'datcon09_i'),
 (0.0055486604724798161, u'datcon10_i'),
 (0.0055371936603058058, u'datcon12_i'),
 (0.0055371936603058058, u'datcon13_i'),
 (0.0054644597625110869, u'agecon01_i'),
 (0.0059538673862221669, u'agecon02_i'),
 (0.0054403446719120119, u'agecon03_i'),
 (0.0054623573691464644, u'agecon04_i'),
 (0.0055852096452164712, u'agecon05_i'),
 (0.005565046255331052, u'agecon06_i'),
 (0.0056907970258698315, u'agecon07_i'),
 (0.005858198017676397, u'agecon08_i'),
 (0.0057958171242896706, u'agecon09_i'),
 (0.0055486604724798161, u'agecon10_i'),
 (0.0055371936603058058, u'agecon12_i'),
 (0.0055371936603058058, u'agecon13_i'),
 (0.005495846986551034, u'marout01_i'),
 (0.0056174511266962046, u'marout02_i'),
 (0.0057675316941984134, u'marout03_i'),
 (0.0060585559043144777, u'marout04_i'),
 (0.0078318541717078416, u'marout05_i'),
 (0.0065256660166292946, u'marout06_i'),
 (0.0072420864688979014, u'marout07_i'),
 (0.0060088320513724813, u'marout08_i'),
 (0.0053638355347792599, u'marout09_i'),
 (0.0054348289017581708, u'marout10_i'),
 (0.0055885129196469041, u'marout11_i'),
 (0.0054274740356373696, u'rmarout01_i'),
 (0.0056784279807181592, u'rmarout02_i'),
 (0.0054986373250220932, u'rmarout03_i'),
 (0.0055260630511707598, u'rmarout04_i'),
 (0.0068806174433383438, u'rmarout05_i'),
 (0.0062425190707391387, u'rmarout06_i'),
 (0.0065392458192810032, u'rmarout07_i'),
 (0.0060088320513724813, u'rmarout08_i'),
 (0.0053638355347792599, u'rmarout09_i'),
 (0.0054348289017581708, u'rmarout10_i'),
 (0.0055885129196469041, u'rmarout11_i'),
 (0.0056150349519995535, u'marcon01_i'),
 (0.0056113419572897394, u'marcon02_i'),
 (0.0061814728700901878, u'marcon03_i'),
 (0.0057672692679421322, u'marcon04_i'),
 (0.007880198703264063, u'marcon05_i'),
 (0.0068062118018775708, u'marcon06_i'),
 (0.0067853177809356069, u'marcon07_i'),
 (0.0059774435080348631, u'marcon08_i'),
 (0.0053450004673534224, u'marcon09_i'),
 (0.0054348289017581708, u'marcon10_i'),
 (0.0055885129196469041, u'marcon11_i'),
 (0.00566216198540892, u'cebow_i'),
 (0.00566216198540892, u'cebowc_i'),
 (0.0057942810251442323, u'datbaby1_i'),
 (0.0059434785675783752, u'agebaby1_i'),
 (0.00566216198540892, u'liv1chld_i'),
 (0.008267774071422429, u'rmarital_r'),
 (0.0062973555918357516, u'fmarno'),
 (0.0068126703004841582, u'mardat01'),
 (0.0065210569478663105, u'fmar1age'),
 (0.010961563590751622, u'mar1diss'),
 (0.010112030016853679, u'mar1bir1'),
 (0.0090603608469872476, u'mar1con1'),
 (0.0081616922943980041, u'con1mar1'),
 (0.010059124870520297, u'b1premar'),
 (0.0072555085143005682, u'cohever'),
 (0.0061949579707465396, u'evmarcoh'),
 (0.0027193936788127981, u'cohab1'),
 (0.006628001579039311, u'cohstat'),
 (0.0031380350118120903, u'cohout'),
 (0.0054204797106723879, u'coh1dur'),
 (0.0053363649711755112, u'sexever'),
 (0.0059445346416568956, u'vry1stag'),
 (0.0061426133382641579, u'sex1age'),
 (0.0053318098738383002, u'vry1stsx'),
 (0.0053088601118754797, u'datesex1'),
 (0.0053539808912762554, u'sexonce'),
 (0.0053555252868140446, u'fsexpage'),
 (0.0067312971267319277, u'sexmar'),
 (0.0065143616191195575, u'sex1for'),
 (0.0054434362270381342, u'parts1yr'),
 (0.0058559384438563455, u'lsexdate'),
 (0.0059669889843219082, u'lsexrage'),
 (0.0063365994925744795, u'lifprtnr'),
 (0.0057079961339601137, u'fmarno_i'),
 (0.0054133546264528709, u'mardat01_i'),
 (0.0053471246759598801, u'mardat02_i'),
 (0.0059770570127406497, u'mardis01_i'),
 (0.0054115975647813785, u'mardis02_i'),
 (0.0054219110681403837, u'mardis03_i'),
 (0.0054048658657284143, u'mardis04_i'),
 (0.0054556491522061989, u'mardis05_i'),
 (0.0055605662922586507, u'marend01_i'),
 (0.0053432778568986983, u'marend02_i'),
 (0.0053294899351037373, u'marend03_i'),
 (0.0053448294461109214, u'marend04_i'),
 (0.0054665157936630848, u'fmar1age_i'),
 (0.0062297126791696078, u'agediss1_i'),
 (0.0058180938523332282, u'agedd1_i'),
 (0.0059525347179240073, u'mar1diss_i'),
 (0.0055625114741243431, u'dd1remar_i'),
 (0.0056514514657065584, u'mar1bir1_i'),
 (0.0055198460490742951, u'mar1con1_i'),
 (0.005451844769302272, u'con1mar1_i'),
 (0.0056823168111926892, u'b1premar_i'),
 (0.0066922143764862607, u'cohab1_i'),
 (0.0056752479948509738, u'cohstat_i'),
 (0.0055123296970670532, u'cohout_i'),
 (0.0063345634387361738, u'coh1dur_i'),
 (0.0060439918197892073, u'sexever_i'),
 (0.0053941533797435826, u'vry1stag_i'),
 (0.0054267257972491212, u'sex1age_i'),
 (0.00554113924466626, u'vry1stsx_i'),
 (0.0056974344065517935, u'datesex1_i'),
 (0.0053523613708839068, u'fsexpage_i'),
 (0.0054768804615890065, u'sexmar_i'),
 (0.0055561700178214934, u'sex1for_i'),
 (0.0053912178048811654, u'parts1yr_i'),
 (0.0053292541098745039, u'lsexdate_i'),
 (0.0054360614303433641, u'lsexrage_i'),
 (0.0055826513466541439, u'lifprtnr_i'),
 (0.0056005006117659795, u'strloper'),
 (0.0053635800625385643, u'tubs'),
 (0.0064263875307187801, u'vasect'),
 (0.0055320684576879442, u'hyst'),
 (0.0053307160638500495, u'ovarect'),
 (0.0055184494296063313, u'othr'),
 (0.0058177026155028511, u'othrm'),
 (0.0054804310847854643, u'fecund'),
 (0.0060489594809123304, u'anybc36'),
 (0.0060114148811416523, u'nosex36'),
 (0.0062123120540612531, u'infert'),
 (0.0059218884490948032, u'anybc12'),
 (0.0053289129639070287, u'anymthd'),
 (0.0055808608508715052, u'nosex12'),
 (0.0055497197840209678, u'sexp3mo'),
 (0.0059840262618628959, u'sex3mo'),
 (0.0062020607487287549, u'constat1'),
 (0.005357513743306952, u'constat2'),
 (0.0053709727203604674, u'constat3'),
 (0.0054654886171253692, u'constat4'),
 (0.0063469690146182867, u'pillr'),
 (0.0054641217958707466, u'condomr'),
 (0.0054550896046002828, u'sex1mthd1'),
 (0.004003403776326464, u'sex1mthd2'),
 (0.0061752436010955591, u'mthuse12'),
 (0.0072929966330779195, u'meth12m1'),
 (0.0063724153931070671, u'mthuse3'),
 (0.0068739073338428547, u'meth3m1'),
 (0.0059510790099716049, u'nump3mos'),
 (0.0057650600263676743, u'fmethod1'),
 (0.0063123635695842006, u'dateuse1'),
 (0.0054732402254835666, u'oldwp01'),
 (0.0061220725634859585, u'oldwp02'),
 (0.0065666797526262588, u'oldwp03'),
 (0.0062996315229640887, u'oldwr01'),
 (0.0072364007607503833, u'oldwr02'),
 (0.0093068312518811425, u'oldwr03'),
 (0.006285282977211204, u'wantrp01'),
 (0.0072364007607503833, u'wantrp02'),
 (0.0093068312518811425, u'wantrp03'),
 (0.0054865812407538517, u'wantp01'),
 (0.0060748063719644474, u'wantp02'),
 (0.0066199606495226382, u'wantp03'),
 (0.005485284751240882, u'wantp5'),
 (0.0054489576894595215, u'infert_i'),
 (0.0067450133507915844, u'nosex12_i'),
 (0.0054806677313274932, u'sexp3mo_i'),
 (0.0053272936863252296, u'sex3mo_i'),
 (0.0058298986831754052, u'constat1_i'),
 (0.0054638333403207451, u'constat2_i'),
 (0.0055625108456597294, u'constat3_i'),
 (0.0055625108456597294, u'constat4_i'),
 (0.0053279031088482309, u'pillr_i'),
 (0.0053279031088482309, u'condomr_i'),
 (0.0054805386829482883, u'sex1mthd1_i'),
 (0.0056180827577207992, u'sex1mthd2_i'),
 (0.0056180827577207992, u'sex1mthd3_i'),
 (0.0056180827577207992, u'sex1mthd4_i'),
 (0.0053273478704315425, u'mthuse12_i'),
 (0.0053352895922490484, u'meth12m1_i'),
 (0.0053285052163482005, u'meth12m2_i'),
 (0.0053285067128187036, u'meth12m3_i'),
 (0.0053285067128187036, u'meth12m4_i'),
 (0.005352822010079028, u'mthuse3_i'),
 (0.0053745626411452196, u'meth3m1_i'),
 (0.0053464147760903247, u'meth3m2_i'),
 (0.0053380223568886276, u'meth3m3_i'),
 (0.0053380223568886276, u'meth3m4_i'),
 (0.0062734191728234245, u'nump3mos_i'),
 (0.0053788778231921297, u'fmethod1_i'),
 (0.0060763767768974297, u'dateuse1_i'),
 (0.0053684568424530177, u'sourcem1_i'),
 (0.0053432727329132401, u'sourcem2_i'),
 (0.0053338411053349688, u'sourcem3_i'),
 (0.0053338411053349688, u'sourcem4_i'),
 (0.0056654709310530738, u'oldwp01_i'),
 (0.0055324846721469534, u'oldwp02_i'),
 (0.0060387944737545496, u'oldwp03_i'),
 (0.0059959557608726399, u'oldwp04_i'),
 (0.0054579548234968689, u'oldwp05_i'),
 (0.0054014030984341765, u'oldwp06_i'),
 (0.0054589299665901425, u'oldwp07_i'),
 (0.0057501139160008874, u'oldwp08_i'),
 (0.0053641062185079136, u'oldwr01_i'),
 (0.0054901723356760579, u'oldwr02_i'),
 (0.0053638933879495143, u'oldwr03_i'),
 (0.0056403066952067649, u'oldwr04_i'),
 (0.0053813426061593006, u'oldwr05_i'),
 (0.0053969516876548607, u'oldwr06_i'),
 (0.0055298718389825696, u'oldwr07_i'),
 (0.005587196021343277, u'oldwr08_i'),
 (0.0053651931392618701, u'oldwr09_i'),
 (0.0053641062185079136, u'wantrp01_i'),
 (0.0054901723356760579, u'wantrp02_i'),
 (0.0053638933879495143, u'wantrp03_i'),
 (0.0056403066952067649, u'wantrp04_i'),
 (0.0053813426061593006, u'wantrp05_i'),
 (0.0053969516876548607, u'wantrp06_i'),
 (0.0055298718389825696, u'wantrp07_i'),
 (0.005587196021343277, u'wantrp08_i'),
 (0.0053651931392618701, u'wantrp09_i'),
 (0.0054993860861719535, u'wantp01_i'),
 (0.00574144758505446, u'wantp02_i'),
 (0.0057288309776636348, u'wantp03_i'),
 (0.0061726542910365279, u'wantp04_i'),
 (0.0056489568355027053, u'wantp05_i'),
 (0.0053859298497760388, u'wantp06_i'),
 (0.0056296130438038272, u'wantp07_i'),
 (0.0057799191546499262, u'wantp08_i'),
 (0.0065967916095962531, u'wantp5_i'),
 (0.0053502390613457962, u'fptit12_i'),
 (0.0054279461800806894, u'fptitmed_i'),
 (0.0053464478447162733, u'fpregmed_i'),
 (0.0055473444255561555, u'r_stclin'),
 (0.0054928344340378077, u'intent'),
 (0.0053291328338355148, u'addexp'),
 (0.0059597481834721133, u'intent_i'),
 (0.0058623685823748772, u'addexp_i'),
 (0.0053285959109981018, u'anyprghp'),
 (0.0058349115409653818, u'anymschp'),
 (0.0056089268270606008, u'infever'),
 (0.0058056953191233385, u'pidtreat'),
 (0.0053272854467497632, u'evhivtst'),
 (0.0054908813709556581, u'anyprghp_i'),
 (0.0056965899127808806, u'anymschp_i'),
 (0.0055920576323439386, u'infever_i'),
 (0.0054908813709556581, u'ovulate_i'),
 (0.0054908813709556581, u'tubes_i'),
 (0.0054908813709556581, u'infertr_i'),
 (0.0054908813709556581, u'inferth_i'),
 (0.0054908813709556581, u'advice_i'),
 (0.0054908813709556581, u'insem_i'),
 (0.0054908813709556581, u'invitro_i'),
 (0.0054908813709556581, u'endomet_i'),
 (0.0054908813709556581, u'fibroids_i'),
 (0.0053273436347913705, u'pidtreat_i'),
 (0.0053847969553227903, u'evhivtst_i'),
 (0.0054442288636177283, u'insuranc_r'),
 (0.0059086876990791515, u'metro_r'),
 (0.0054762462261790379, u'religion_r'),
 (0.0061242506200275271, u'laborfor_r'),
 (0.0054901920776948554, u'insuranc_i_r'),
 (0.005329126750794444, u'religion_i_r'),
 (0.0056353938189386277, u'laborfor_i_r'),
 (0.0097431589752972059, u'poverty_r'),
 (0.011870069031173713, u'totincr'),
 (0.0098850329207478271, u'pubassis_r'),
 (0.0056746687213740099, u'poverty_i_r'),
 (0.0056746687213740099, u'totincr_i'),
 (0.0055882636622018866, u'pubassis_i_r'),
 (0.0072660831598055919, u'basewgt_r'),
 (0.0068633447572696848, u'adj_mod_basewgt_r'),
 (0.0074146019069666336, u'finalwgt_r'),
 (0.0060084918801373011, u'secu_r'),
 (0.0054052918682682138, u'sest_r'),
 (0.0054259148896510512, u'cmintvw_r'),
 (0.0054259148896512732, u'cmlstyr'),
 (0.005848951937927005, u'intvlngth')]

















In [52]:



    


t.sort(reverse=True)
for mse, name in t[:30]:
    print name, mse



    


















    






totalwgt_lb 1.0
birthwgt_lb 0.949812730598
lbw1 0.300824078447
prglngth 0.130125194886
wksgest 0.123400413634
agecon 0.102031499282
mosgest 0.0271442746396
babysex 0.0185509252939
race_r 0.0161995035863
race 0.0161995035863
nbrnaliv 0.0160177527098
paydu 0.0140037955781
rmarout03 0.0134300664657
birthwgt_oz 0.0131024576157
anynurse 0.0125290225418
bfeedwks 0.0121936884045
totincr 0.0118700690312
marout03 0.0118078019944
marcon03 0.0117525993544
cebow 0.0114377709196
rmarout01 0.0114077371386
rmarout6 0.0113541384728
marout01 0.0112693572468
hisprace_r 0.011238349302
hisprace 0.011238349302
mar1diss 0.0109615635908
fmarcon5 0.0106049646843
rmarout02 0.0105469132066
marcon02 0.0104814017955
fmarout5 0.0104616913674

race is a proxy variable; that is apparent correlations are often caused, at least in part, by other factors.

data mining Sometimes you start with a theory and use data to test it. Other times you start with data and go looking for possible theories.





In [54]:



    


formula = ('totalwgt_lb ~ agepreg + C(race) + babysex==1 + '
           'nbrnaliv > 1 +paydu==1 + totincr')
results = smf.ols(formula, data=join).fit()



    











In [56]:



    


regression.SummarizeResults(results)



    


















    






Intercept   6.63   (0)
C(race)[T.2]   0.357   (5.43e-29)
C(race)[T.3]   0.266   (2.33e-07)
babysex == 1[T.True]   0.295   (5.39e-29)
nbrnaliv > 1[T.True]   -1.38   (5.1e-37)
paydu == 1[T.True]   0.12   (0.000114)
agepreg   0.00741   (0.0035)
totincr   0.0122   (0.00188)
R^2 0.05999
Std(ys) 1.271
Std(res) 1.232

C(race) tells Patsy to treat race as a categorical variable

Poisson regression if the dependent variable is an integer count.

logisitc regression when the dependet variable is boolean. Expresses predictions in odds rather than probabilities--i.e. the ratio of the probability that it will occur to the probability that it will not.

$$ \text{odds} = p / (1 - p) $$$$ \text{probability} = o / (o + 1) $$

Logistic regression is based on the following model:

$$ \log o = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon $$

logistic regression does not have a closed form solution, so it is solved by guessing an initial solution and improving it iteratively--looking for MLE, or the set of parameters that maximize the likelyhood of the data.





In [65]:



    


y = np.array([0,1,0,1])
x1 = np.array([0,0,0,1])
x2 = np.array([0,1,1,1])

beta = [-1.5,2.8,1.1]

log_o = beta[0] + beta[1]*x1 + beta[2]*x2
print "log_o", log_o
o = np.exp(log_o)
print "o", o
p = o / (o + 1)
print "p", p



    


















    






log_o [-1.5 -0.4 -0.4  2.4]
o [  0.22313016   0.67032005   0.67032005  11.02317638]
p [ 0.18242552  0.40131234  0.40131234  0.9168273 ]

















In [66]:



    


likes = y * p + (1-y) * (1-p)
print "likes", likes
like = np.prod(likes)
print "overall likelihood", like



    


















    






likes [ 0.81757448  0.40131234  0.59868766  0.9168273 ]
overall likelihood 0.180093352967

Probability is p when y == 1, 1-p when y == 0. The goal is to maximize the likelihood of the data.

endogenous variable - dependent variable

exogenous - explanatory





In [67]:



    


live, first, others = first.MakeFrames()
df = live[live.prglngth>30]



    











In [70]:



    


df['boy'] = (df.babysex==1).astype(int)
model = smf.logit('boy ~ agepreg', data=df)
results = model.fit()
regression.SummarizeResults(results)



    


















    






Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Intercept   0.00579   (0.953)
agepreg   0.00105   (0.783)
R^2 6.144e-06










    






/Users/davidgoldberg/anaconda/lib/python2.7/site-packages/IPython/kernel/__main__.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  if __name__ == '__main__':

















In [85]:



    


formula = 'boy ~ agepreg + hpagelb + birthord + C(race)'
model = smf.logit(formula, data=df)
results = model.fit()
regression.SummarizeResults(results)



    


















    






Optimization terminated successfully.
         Current function value: 0.692944
         Iterations 3
Intercept   -0.0301   (0.772)
C(race)[T.2]   -0.0224   (0.66)
C(race)[T.3]   -0.000457   (0.996)
agepreg   -0.00267   (0.629)
hpagelb   0.0047   (0.266)
birthord   0.00501   (0.821)
R^2 0.000144

















In [1]:



    


##these dataframes contain the variables
endog = pandas.DataFrame(model.endog, columns=[model.endog_names])
exog = pandas.DataFrame(model.exog, columns=model.exog_names)
exog.head(5)



    


















    






---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-451efcb869a4> in <module>()
      1 ##these dataframes contain the variables
----> 2 endog = pandas.DataFrame(model.endog, columns=[model.endog_names])
      3 exog = pandas.DataFrame(model.exog, columns=model.exog_names)
      4 exog.head(5)
      5 model.exog

NameError: name 'pandas' is not defined

Measuring Accuracy





In [90]:



    


actual = endog['boy']
baseline = actual.mean()
#this is the baseline prediction if you were to grab
#a random record from the sample
baseline



    


















    
Out[90]:








0.507173764518333

















In [91]:



    


predict = (results.predict() >= 0.5)

true_pos = predict * actual
true_neg = (1 - predict) * (1 - actual)
##accuracy is a fraction of correct guesses
acc =  (sum(true_pos) + sum(true_neg)) /  len(actual)
print acc



    


















    






0.511500797085

















In [98]:



    


columns = ['agepreg', 'hpagelb','birthord', 'race']
new = pandas.DataFrame([[35, 39, 3, 2]], columns=columns)
y = results.predict(new)
y



    


















    
Out[98]:








array([ 0.5130905])

Exercises

exercise 11.1





In [142]:



    


df = join[join.prglngth > 30]
def datamine(df, dep_variable):
    """
    expl_variables:  a list of explanatory variables to
        be considered in the mining.
    """
    t = []
    for name in df.columns:
        try:
            if df[name].var() < 1e-7:
                continue
            
            formula = str(dep_variable) + " ~ " + str(name)
            model = smf.ols(formula, data=df)
            if model.nobs < len(df) / 2:
                continue
                
            results = model.fit()
        except (ValueError, TypeError):
            continue
        
        t.append((name,results))
    return t
   
t = datamine(df, 'prglngth')



    











In [181]:



    


def SortAndPrint(t):
    """
    t is a list of tuples with regression results 
       in the form (name of parameter, results object)
    """
    t.sort(key=lambda x: x[1].rsquared, reverse=True)
    for name, res in t:
        try:
            if res.pvalues[name] < 0.1:
                print name, res.params[name], res.rsquared
        except KeyError, e:
            print "EXCEPTION, ",e
SortAndPrint(t)



    


















    






prglngth 1.0 1.0
wksgest 0.823927251016 0.806243411614
totalwgt_lb 0.528064784847 0.124457431481
birthwgt_lb 0.514962860811 0.119773078049
lbw1 2.48648368743 0.103725422046
mosgest 0.417548478248 0.0956243198959
prglngth_i -1.77679367574 0.0220537757965
canhaver 0.11831014212 0.00605049526819
datcon01_i -0.47208859129 0.00581775529988
con1mar1_i -0.443886529558 0.00554637613623
nbrnaliv -0.727471007121 0.00457756578553
mar1con1_i -0.401666091852 0.00315080225387
anynurse -0.0479292615804 0.00245202488371
bfeedwks -0.000191129876341 0.00236918394467
pregend1 0.213292014116 0.00224938943379
rmarout11_i -2.68111336488 0.0022436279681
marout11_i -2.68111336488 0.0022436279681
marcon11_i -2.68111336488 0.0022436279681
cmlastlb_r -0.000259061163168 0.0020431424422
cmlastlb -0.000259061163168 0.0020431424422
datend02_i -1.07513070601 0.00201248339275
datcon02_i -1.07513070601 0.00201248339275
agecon02_i -0.99305345975 0.00198828676889
fmarcon5_i -0.326061018289 0.00196815932425
ageprg02_i -0.962147940163 0.00195280723688
evuseint -0.0449307503712 0.00189175277586
paydu -0.139534180284 0.00187682190301
marcon03_i -0.34095383323 0.00179114104017
anymschp 0.316895906816 0.00177999843324
gestasun_m 0.0141459413635 0.00165713195502
hlpmc 0.0749054563619 0.00161252106165
diabetes 0.0709853515812 0.00160070722701
marout04_i -0.337004369923 0.00159237909654
ptsb4mar 0.00548089785728 0.00158691214383
rmarout08_i -0.830962972053 0.00158333217664
marout08_i -0.830962972053 0.00158333217664
cmhivtst -0.000105608441335 0.00148078379171
todayspg -0.0685335374176 0.00147583725852
marbefhx 0.05100088315 0.0014412778326
rmarout10_i -1.74771300785 0.00142924676143
marout10_i -1.74771300785 0.00142924676143
marcon10_i -1.74771300785 0.00142924676143
widrawal -0.0355368055387 0.00141412298375
liv1chld -0.0756004754914 0.00141084482997
methdiss -0.0355813717503 0.0014015912617
marout03_i -0.288695595262 0.00134368530229
oldwp02_i -1.35987693999 0.00132559499865
sest_r 0.00285986177766 0.00132236819817
sest 0.00285986177766 0.00132236819817
wantrp07_i -1.73797712997 0.0013192917167
oldwr07_i -1.73797712997 0.0013192917167
manrel -0.183556673084 0.00131095845942
matchfound -0.0374942927903 0.00130910737714
marout02_i -0.271060973613 0.00130232791178
marcon08_i -0.683706106943 0.00128972449029
intr_ec3 0.0334252695731 0.0012831309948
cmlstprg_r -0.000120333176218 0.00128286196464
cmlstprg -0.000120333176218 0.00128286196464
rmarout04_i -0.313190679733 0.00127074707535
monsx1192 -0.0390938986751 0.00126053030943
ageprg01_i -0.831801808137 0.00124575518222
pregnono 0.0665494534786 0.00124098891706
p1yrage -0.00806818360462 0.00123958811389
birthord -0.0641908508642 0.00123727367366
hplocale -0.183132388057 0.00121662408564
marout07_i -0.512424679154 0.00118615327372
rmarout03_i -0.2827548956 0.00117658943454
wantp5_i -0.431797179342 0.00117232367026
mar1diss 0.000867505519831 0.00116014454281
frsteatd_p -0.0927400293276 0.00115580773277
wantbold -0.0341685897287 0.00113622024429
methhist431 0.0086118756721 0.0011326475685
marcon02_i -0.264871737394 0.00112722489211
hisphx 0.0365302343631 0.00112354739874
poverty_r 0.000437724002601 0.0011234153757
poverty 0.000437724002601 0.0011234153757
posiblpg -0.0698488591276 0.00111271165057
marcon01_i -0.288756663445 0.0011117307892
marcon04_i -0.289839472659 0.0011048765441
pill -0.0424511198679 0.00109472932054
wage -0.0338610629185 0.00107897312527
totincr 0.015794290153 0.00107710200602
marout05_i -0.31851341699 0.00107456125006
EXCEPTION,  'isfirst'
agemomb1_i -0.707036671293 0.00105406025771
pillr -0.168656508594 0.00105190872752
gestasun_w 0.00329590033475 0.00105137990873
datend01_i -0.753522867264 0.00104236625724
mardis02_i -0.344111925443 0.00103107298365
rmarout05_i -0.331601917274 0.001027969678
agecon01_i -0.735605338506 0.00102432300053
marcon07_i -0.468845810555 0.0010195336219
p1yhsage -0.00603449532881 0.00101806530516
outcom03 -0.048626010968 0.000997784544022
seekadpt 0.126587397437 0.000991911304328
prevhusb -0.094491097552 0.000983607959758
papplbc2 -0.0308095818606 0.000976178763829
kidshx 0.038411051605 0.000973038903125
everhyst 0.0606035299677 0.000947723159394
monsx1233 -0.0357656321536 0.000942788651941
oldwp07_i -1.41381591311 0.000935303825447
hyst 0.244269054852 0.00092262683179
rmarout07_i -0.512938353766 0.00091718927429
monsx1182 -0.0322521442856 0.000912827232133
wantrp02 -0.0477438860106 0.000875589431048
oldwr02 -0.0477438860106 0.000875589431048
cmintstr -5.47697118413e-05 0.000873969295819
wantp02_i -0.911785348685 0.000854068225686
marcon05_i -0.28584413276 0.000853676410137
fmarout5_i -0.204458164176 0.000837909381892
coh1dur -0.0013653853438 0.000819782904155
methhist421 0.00777528756625 0.000818927099003
fpt12mos 0.0696249428537 0.000810586198829
monsx1206 -0.0320539124769 0.000809554653609
monsx1193 -0.0309527671661 0.000796480365764
rmarout6_i -0.204941038491 0.000791768162923
agediss1_i -0.170282664577 0.000765137611434
rmarout02_i -0.212522315752 0.00076143032357
prgvisit -0.163507807154 0.000748618959069
retrovir -0.0335561822279 0.00074519960484
othrm 3.9125748503 0.000739987244028
agebaby1_i -0.278371834664 0.000733060790956
monsx1184 -0.0291296154372 0.000723470536037
monsx1185 -0.0288088705952 0.000716474950296
methhist371 0.00768532289263 0.000711935548325
mardis01_i -0.160852596268 0.000695432322598
monsx1183 -0.0283525406473 0.000678453479943
monsx1181 -0.0279381605644 0.000675235637232
monsx1208 -0.0290966865976 0.000667408654718
monsx1188 -0.0277427193805 0.000665858946573
toincwmy 0.060469182375 0.000661776358571
hispanic_r 0.110509796588 0.000660091644739
hispanic 0.110509796588 0.000660091644739
menarche 0.00786062809327 0.000650905195035
cmfirsm -3.28901796188e-05 0.000650767962424
agebaby1 9.5908034617e-05 0.000650598453371
hisp 0.0273214409664 0.000647407593704
methhist401 0.0069337985722 0.000646828895085
methhist411 0.00662905567739 0.000640585086811
datbaby1_i -0.266248512685 0.000623613533992
pregordr -0.0330193532463 0.00062224148606
nosex36 -0.0942352643683 0.000616109535107
pst4wksx -0.000360922683212 0.000601983435572
marout06_i -0.290022459859 0.000601309532174
monsx1189 -0.0268084092223 0.000597778092705
racehx1 0.0429426187862 0.000593515495661
sex1mthd2 0.00150441782071 0.000592125975269
educat_r 0.0180513181334 0.00059068244033
educat 0.0180513181334 0.00059068244033
infever 0.137916779141 0.00058663459015
mar1diss_i -0.143165675437 0.000581003552177
monsx1200 -0.0269440173824 0.00057798490301
fmarital_r -0.0268468909217 0.00057247806405
fmarital -0.0268468909217 0.00057247806405
fmarit -0.0268468909217 0.00057247806405
monsx1211 -0.0271580876203 0.000570279389512
rmarout06_i -0.296566969752 0.000568985350405
rmarital_r -0.0229622326917 0.000564620840439
rmarital -0.0229622326917 0.000564620840439
marstat -0.0229622326917 0.000564620840439
cmlsexx 5.3466441034e-05 0.000559042035438
chreward -0.0663586763586 0.000552504645563
dolastwk1 -0.0152265301597 0.000552291358926
mornpill -0.0664209685977 0.000551724610541
monsx1186 -0.0251137850396 0.000551049702525
dateuse1 -3.49999122316e-06 0.000549201230264
monsx1201 -0.0265298051717 0.000547707096119
monsx1194 -0.0257583184213 0.000546203154988
jelcrmal -0.0413782120091 0.00054303283076
wantp07_i -0.930189530686 0.000539515222769
monsx1177 -0.0250044441655 0.000538658584889
methhist381 0.006358755068 0.000533120663022
agecon03_i -0.573761496655 0.0005317729856
higrade 0.0172898215792 0.000530359391253
usefstp -0.0242187751422 0.000526799641923
constat1 -0.00294408238332 0.00052512407032
monsx1202 -0.0255933763799 0.000522452094988
marcon09_i -0.602158245126 0.000519528349729
nummult44 -0.140882422417 0.000519130227634
goschol 0.0374517048309 0.0005185160115
lastmonmeth1 0.00617156482349 0.000509678883923
marend03_i 1.34548982786 0.000508595805745
oldwantr -0.0365084263058 0.000507506178707
anybc12 -0.114135095935 0.000501802382412
currmeth1 0.00406916354485 0.000501356820751
lifprtnr 0.00582578317498 0.000500311619035
wantresp -0.0362156324931 0.000499185401079
nummult43 -0.143642856323 0.000497812794674
cmfstuse -2.52860907475e-05 0.000497141284702
monsx1213 -0.026075386317 0.000491166065205
methhist441 0.00540465340309 0.000483784581809
cmstrtmc -0.0166566435579 0.000481651499383
othinc 0.0447633839059 0.000476856638206
rscreenhisp 0.0236813934593 0.000476426760997
methhist391 0.00598269910708 0.000475424139854
mar1bir1_i -0.206941901924 0.000471636146133
monsx1214 -0.025096035457 0.000467295146568
monsx1190 -0.0235251248882 0.000466048008855
monsx1187 -0.0233283870774 0.000462472638631
wantrp03 -0.0314376055697 0.000458928308223
oldwr03 -0.0314376055697 0.000458928308223
monsx1195 -0.0235400400181 0.000456432334255
nummult41 -0.138473304772 0.00044894496209
lossnum -0.0469863390691 0.000448712595947
monsx1203 -0.0234074797521 0.00043804949964
mar1bir1 -9.22664882413e-05 0.00043270255561
agepreg_i -0.610367233969 0.000428924881649
everovrs 0.0642190135396 0.00042021295691
b1premar 0.0773451007755 0.000412972736861
ovacyst 0.0259690731375 0.000412023845708
ovarect 0.261413819672 0.000409292734128
staytog 0.0401816079196 0.000407474075739
cmprgend -0.000102803269508 0.00040699954457
cmbabdob -0.000102803269508 0.00040699954457
iud -0.034112581235 0.000406173325389
numfmhh 0.0267943693582 0.000405882492032
hieduc_r 0.0168050187211 0.000405201190318
hieduc 0.0168050187211 0.000405201190318
ssi 0.0347732648287 0.000399390213411
family -0.0368468605981 0.000398979896282
selfinc -0.0235258877095 0.000398147426198
lsexdate 0.00114478546551 0.000395014210969
rmarout09_i -0.529028615875 0.000392326510952
marout09_i -0.529028615875 0.000392326510952
rmarout01_i -0.174540775085 0.000386935941114
datend03_i -0.436600718643 0.000385680197075
datcon03_i -0.436600718643 0.000385680197075
b1premar_i -0.184254610018 0.000383280010303
cmintfin -0.000102960817613 0.000382248844893
monsx1212 -0.0222960841126 0.000371282529694
diafragm -0.0315469361251 0.000366672050915
currprtt 0.0794692593035 0.000365028785171
maternlv_i -0.879580561507 0.000362008047941
marcon06_i -0.224458368081 0.000360169080526
sxok18 -0.0402929708643 0.000359115560696
sameman 0.0179625740129 0.000357556898975
cmlstsxx 5.46670227705e-05 0.00035529733821
nummult18 0.124652672544 0.000352842169166
hlptrans -0.0367758872313 0.000349169494977
monsx1199 -0.0207775655927 0.00034689842961
rftptx 0.0515210657067 0.000345219833203
monsx1210 -0.0211176215066 0.000342686741235
monsx1224 -0.0223309157502 0.000342593109557
monsx1198 -0.0202129982759 0.000339731878451
okcohab 0.0383663700307 0.000337673287842
prevcohb 0.0433652537083 0.000333010527647
monsx1205 -0.0206429203777 0.000332407094491
educmom 0.00351012744015 0.00032925524408
rstrstat -0.0655436980709 0.000329171938461
hpwnold -0.0192885496303 0.00032760957382
mardis04_i 0.892441230285 0.000323054838255
fmethod1_i -0.328120409501 0.000318364486803
race_r 0.0593579038 0.000314402188104
race 0.0593579038 0.000314402188104
agecon_i -0.511863821888 0.0003097707929

















In [179]:



    


model = smf.ols('prglngth ~ birthord + C(race) + C(paydu) + + C(diabetes)', data=df)
results = model.fit()
regression.SummarizeResults(results)



    


















    






Intercept   38.7   (0)
C(race)[T.2]   0.0984   (0.0417)
C(race)[T.3]   0.00545   (0.944)
C(paydu)[T.2]   -0.0763   (0.0678)
C(paydu)[T.8]   -1.39   (0.028)
C(paydu)[T.9]   -1.92   (0.0236)
C(diabetes)[T.5]   0.273   (0.000327)
C(diabetes)[T.8]   1.01   (0.341)
C(diabetes)[T.9]   0.317   (0.71)
birthord   -0.0535   (0.00596)
R^2 0.005232
Std(ys) 1.898
Std(res) 1.893

Exercise 11.2





In [210]:



    


def GoMiningBabySex(df):
    df.dropna(subset=['babysex'])
    df['boy'] = (df.babysex==1).astype(int)
    
    t = []
    
    for name in df.columns:
        try:
            if df[name].var() < 1e-7:
                continue
            
            formula = "boy ~ " + str(name)
            model = smf.logit(formula, data=df)
            nobs = len(model.endog)
            if nobs < len(df) / 2:
                continue
                
            results = model.fit()
        except:
            continue
        
        t.append((results.prsquared, name))
    return t

bs = GoMiningBabySex(df)



    


















    






Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692962
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692850
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692903
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692766
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692579
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.686617
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693026
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692903
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693033
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692915
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693104
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693112
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692594
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692778
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692843
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692628
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692825
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693029
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693036
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692678
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692612
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693047
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693079
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692923
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692921
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692910
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692908
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692547
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692656
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692590
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692808
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692293
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692935
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692942
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692945
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692875
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692846
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692876
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692575
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692820
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692591
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692880
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692655
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692922
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692895
         Iterations 5
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692784
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.692643
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692840
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692815
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.686323
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692934
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692920
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692892
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692875
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692846
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692840
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693041
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692888
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692963
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692893
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692921
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692927
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692469
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692919
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693022
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692950
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693041
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693025
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693022
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692856
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693075
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692887
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692322
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692848
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692881
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692876
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692961
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692780
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693103
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693111
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693074
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693125
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693106
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693030
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693099
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693108
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693056
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693127
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693128
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692801
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693079
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692966
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692915
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692836
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693039
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692824
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692950
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693102
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692943
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692727
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692868
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693074
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692956
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693064
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692909
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692967
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692845
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692844
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692783
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692799
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692856
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693129
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693137
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692900
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692842
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692911
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692941
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692905
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692821
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692963
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692972
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693032
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693052
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692734
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692647
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693033
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692843
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692899
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692726
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693055
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692744
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692857
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692948
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692923
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692938
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692947
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692933
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692832
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693070
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693086
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693123
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693067
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692852
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693020
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693028
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693020
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693022
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693020
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692967
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692819
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692682
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692676
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692886
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692867
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692886
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692944
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692920
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692852
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692846
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692966
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692940
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692913
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692932
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692848
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692721
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692577
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692676
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692781
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693023
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692807
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693027
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692754
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693027
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692723
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693079
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692678
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693108
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692915
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693126
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692830
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693077
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693060
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693051
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692915
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692881
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692681
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693046
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692946
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692934
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692629
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692782
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692888
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692739
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692624
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692795
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692927
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692893
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692805
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692910
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692944
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692954
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692924
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692871
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692852
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692575
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692884
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693099
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692881
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693029
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693047
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692956
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692962
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693023
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692580
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692950
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692807
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693099
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693040
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692922
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692835
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692874
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692875
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692921
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692952
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692846
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692616
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692907
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692863
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692721
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692953
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692876
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692888
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692803
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692732
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692921
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692563
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692808
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692877
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692774
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692716
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693034
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692883
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692863
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692850
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692740
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693030
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692876
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692861
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692534
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692876
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692580
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692931
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692682
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692823
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692889
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692799
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692881
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692943
         Iterations 17
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692961
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692863
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692899
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692878
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692864
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692918
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692696
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692863
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692894
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692828
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692972
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692863
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692928
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692934
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692944
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692934
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692970
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692946
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692934
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692942
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693061
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693125
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692804
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692817
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692885
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692844
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693083
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692636
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692714
         Iterations 33
Optimization terminated successfully.
         Current function value: 0.692915
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692967
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692729
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692776
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692708
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692727
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692803
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692808
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692958
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692867
         Iterations 33
Optimization terminated successfully.
         Current function value: 0.692865
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692660
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692791
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692860
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692860
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692756
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692874
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692834
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692784
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692699
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692784
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692768
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692954
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692920
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692956
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692932
         Iterations 4
Warning: Maximum number of iterations has been exceeded.









    






/Users/davidgoldberg/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py:466: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)
/Users/davidgoldberg/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py:466: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)










    






         Current function value: 0.692967
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692266
         Iterations 5
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692700
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692917
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692807
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692838
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693033
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693017
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692902
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692770
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692867
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692867
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692910
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692758
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693108
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692931
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692830
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692893
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692944
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692923
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692923
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692936
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692735
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692903
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692883
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692892
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692854
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692961
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692854
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692961
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692910
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692966
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Warning: Maximum number of iterations has been exceeded.









    






/Users/davidgoldberg/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py:466: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)
/Users/davidgoldberg/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py:466: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)










    






         Current function value: 0.692943
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692832
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692795
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692693
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692460
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692822
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692677
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692945
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692815
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693019
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 3


















In [211]:



    


regression.MiningReport(bs)



    


















    






totalwgt_lb 0.00967200535184
birthwgt_lb 0.0092489816712 BD-3 BIRTHWEIGHT IN POUNDS - 1ST BABY FROM THIS PREGNANCY
constat3 0.0010867056923 3RD PRIORITY CODE FOR CURRENT CONTRACEPTIVE STATUS
lbw1 0.00104859533196 LOW BIRTHWEIGHT - BABY 1
nplaced 0.00100691837365 # OF R'S BIO CHILDREN SHE PLACED FOR ADOPTION (BASED ON BPA)
infever 0.000806910464076 EVER USED INFERTILITY SERVICES OF ANY KIND
frsteatd 0.0007600842059 AGE (IN MOS) WHEN 1ST SUPPLEMENTED - 1ST FROM THIS PREG
splstwk1 0.000731746186575 IF-1 H/P DOING WHAT LAST WEEK (EMPLOYMENT STATUS) 1ST MENTION
outcom02 0.000698741289162 OUTCOME OF PREGNANCY - 2ND
fmarout5 0.000681996122597 FORMAL MARITAL STATUS AT PREGNANCY OUTCOME
nummult34 0.000656746961037 NUMBER OF METHODS REPORTED IN (OCT 2001)
coh1dur 0.000654992932551 DURATION (IN MONTHS) OF R'S FIRST COHABITATION
brnout_r 0.000641068785277 IB-8 R BORN OUTSIDE OF US
brnout 0.000641068785277 IB-8 R BORN OUTSIDE OF US
bpa_bdscheck1 0.000635132392883 WHETHER 1ST LIVEBORN BABY FROM THIS PREGNANCY WAS BPA OR BDS
rmarout6 0.000620359718917 INFORMAL MARITAL STATUS AT PREGNANCY OUTCOME - 6 CATEGORIES
agepreg_i 0.000617875893733 AGEPREG IMPUTATION FLAG
usefstp 0.000616134686552 EF-3 USE METHOD AT FIRST SEX WITH 1ST PARTNER IN PAST 12 MONTHS?
nummult41 0.000583717969493 NUMBER OF METHODS REPORTED IN (MAY 2002)
educmom 0.000582193107397 MOTHER'S (OR MOTHER-FIGURE'S) EDUCATION
abort12 0.000563624754693 FA-3B RECEIVED ABORTION LAST 12 MONTHS
numfirsm1 0.000556625653879 TOTAL NUMBER OF RESPONSES IN EB-1 FIRSMETH - PRESCRIPTION METHODS
lbw1_i 0.000542950837496 LBW1 IMPUTATION FLAG
p1yhsage 0.000528237261171 CI-6 PARTNER'S AGE AT 1ST SEX-1ST REPORTED PARTNER IN LAST 12 MOS
agecon_i 0.000525964435053 AGECON IMPUTATION FLAG
pmarpreg 0.000525037533603 WHETHER PREGNANCY ENDED BEFORE R'S 1ST MARRIAGE (PREMARITALLY)
mar1con1_i 0.000518178906611 MAR1CON1 IMPUTATION FLAG
nummult35 0.000514953291495 NUMBER OF METHODS REPORTED IN (NOV 2001)
nummult39 0.000510947362069 NUMBER OF METHODS REPORTED IN (MAR 2002)
nummult17 0.000504586705056 NUMBER OF METHODS REPORTED IN (MAY 2000)

















In [221]:



    


model = smf.logit('boy ~ fmarout5==5 + totalwgt_lb + nplaced + infever==1', data=df)
results = model.fit()
regression.SummarizeResults(results)



    


















    






Optimization terminated successfully.
         Current function value: 0.683971
         Iterations 6
Intercept   -1.5   (4.32e-29)
fmarout5 == 5[T.True]   0.199   (1.11e-05)
infever == 1[T.True]   0.26   (7.28e-05)
totalwgt_lb   0.193   (1.55e-28)
nplaced   -1.78   (0.00491)
R^2 0.01307

Exercise 11.3

Use a Poisson Regression to predict how many children a woman has born if she is 35 years old, black, college graduate, with household income greater than 75000





In [222]:



    


def makeJoin():
    live, firsts, others = first.MakeFrames()
    live = live[live.prglngth>30]
    resp = chap01soln.ReadFemResp()
    resp.index = resp.caseid
    join = live.join(resp, on='caseid', rsuffix='_r')
    return join

join = makeJoin()



    











In [259]:



    


join = join.dropna(subset=['agepreg'])
join.numbabes.replace([97], np.nan, inplace=True)
join['is35'] = ((join.agepreg >= 35) * (join.agepreg < 36)).astype(int)
formula = 'numbabes ~ C(race) + age_r + C(havedeg) + C(totinc)'
model = smf.poisson(formula, data=join)
results = model.fit()
results.summary()



    


















    






Optimization terminated successfully.
         Current function value: 1.586266
         Iterations 7










    
Out[259]:








Poisson Regression Results


  Dep. Variable:     numbabes       No. Observations:      4239  




  Model:              Poisson       Df Residuals:          4218  




  Method:               MLE         Df Model:                20  




  Date:          Wed, 22 Jul 2015   Pseudo R-squ.:       0.01613 




  Time:              12:43:50       Log-Likelihood:      -6724.2 




  converged:           True         LL-Null:             -6834.4 




                                    LLR p-value:        9.888e-36








                       coef     std err      z      P>|z| [95.0% Conf. Int.] 




  Intercept             0.1345     0.100     1.345  0.179    -0.062     0.331




  C(race)[T.2]         -0.0509     0.024    -2.082  0.037    -0.099    -0.003




  C(race)[T.3]         -0.0722     0.041    -1.747  0.081    -0.153     0.009




  C(havedeg)[T.5.0]     0.0497     0.021     2.334  0.020     0.008     0.091




  C(totinc)[T.2]        0.0624     0.102     0.610  0.542    -0.138     0.263




  C(totinc)[T.3]        0.2575     0.096     2.671  0.008     0.069     0.447




  C(totinc)[T.4]        0.3055     0.090     3.389  0.001     0.129     0.482




  C(totinc)[T.5]        0.1083     0.096     1.122  0.262    -0.081     0.297




  C(totinc)[T.6]        0.1836     0.086     2.131  0.033     0.015     0.352




  C(totinc)[T.7]        0.0953     0.087     1.098  0.272    -0.075     0.266




  C(totinc)[T.8]        0.0441     0.087     0.508  0.612    -0.126     0.215




  C(totinc)[T.9]        0.0146     0.088     0.166  0.868    -0.157     0.186




  C(totinc)[T.10]       0.0197     0.088     0.223  0.824    -0.154     0.193




  C(totinc)[T.11]      -0.0378     0.086    -0.440  0.660    -0.206     0.131




  C(totinc)[T.12]      -0.0291     0.085    -0.341  0.733    -0.196     0.138




  C(totinc)[T.13]      -0.0168     0.085    -0.197  0.844    -0.184     0.150




  C(totinc)[T.14]      -0.0394     0.082    -0.481  0.630    -0.200     0.121




  C(totinc)[T.97]       0.1448     0.288     0.502  0.616    -0.420     0.710




  C(totinc)[T.98]      -0.0501     0.099    -0.507  0.612    -0.244     0.144




  C(totinc)[T.99]       0.0711     0.131     0.545  0.586    -0.185     0.327




  age_r                 0.0208     0.002    12.038  0.000     0.017     0.024




















In [260]:



    


columns = ['race', 'age_r', 'havedeg', 'totinc']
new = pandas.DataFrame([[1, 35, 1, 14]], columns=columns)
y = results.predict(new)
y



    


















    
Out[260]:








array([ 2.27943811])

Exercise 11.4

use multinomial logistic regression to determine the categorical marital status of a 25 year old white high school graduate whose annual income is about $45,000





In [273]:



    


resp = chap01soln.ReadFemResp()

formula = 'rmarital ~ age_r + C(race) + C(educat) + C(totincr)'
model = smf.mnlogit(formula, data=resp)
results = model.fit()
results.summary()



    


















    






Warning: Maximum number of iterations has been exceeded.
         Current function value: 1.054219
         Iterations: 35










    






/Users/davidgoldberg/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py:466: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)










    
Out[273]:








MNLogit Regression Results


  Dep. Variable:     rmarital       No. Observations:      7643 




  Model:              MNLogit       Df Residuals:          7508 




  Method:               MLE         Df Model:               130 




  Date:          Wed, 22 Jul 2015   Pseudo R-squ.:       0.1906 




  Time:              13:06:13       Log-Likelihood:      -8057.4




  converged:           False        LL-Null:             -9954.7




                                    LLR p-value:          0.000 








     rmarital=2       coef     std err      z      P>|z| [95.0% Conf. Int.] 




  Intercept            2.5873     0.319     8.102  0.000     1.961     3.213




  C(race)[T.2]        -0.6489     0.112    -5.790  0.000    -0.869    -0.429




  C(race)[T.3]        -0.7345     0.173    -4.248  0.000    -1.073    -0.396




  C(educat)[T.10]      0.3770     0.197     1.915  0.055    -0.009     0.763




  C(educat)[T.11]     -0.2196     0.210    -1.044  0.296    -0.632     0.193




  C(educat)[T.12]     -0.1215     0.138    -0.881  0.378    -0.392     0.149




  C(educat)[T.13]     -0.2167     0.172    -1.262  0.207    -0.553     0.120




  C(educat)[T.14]     -0.0890     0.168    -0.529  0.596    -0.419     0.241




  C(educat)[T.15]     -0.3148     0.214    -1.469  0.142    -0.735     0.105




  C(educat)[T.16]     -0.7445     0.195    -3.812  0.000    -1.127    -0.362




  C(educat)[T.17]     -0.5642     0.273    -2.068  0.039    -1.099    -0.030




  C(educat)[T.18]     -1.1754     0.409    -2.872  0.004    -1.977    -0.373




  C(educat)[T.19]      0.1120     0.288     0.389  0.697    -0.452     0.676




  C(totincr)[T.2]      0.5144     0.331     1.553  0.121    -0.135     1.164




  C(totincr)[T.3]      0.1893     0.341     0.555  0.579    -0.480     0.858




  C(totincr)[T.4]     -0.1125     0.308    -0.365  0.715    -0.716     0.491




  C(totincr)[T.5]     -0.1790     0.287    -0.624  0.533    -0.742     0.383




  C(totincr)[T.6]     -0.3620     0.281    -1.290  0.197    -0.912     0.188




  C(totincr)[T.7]     -0.3301     0.279    -1.184  0.236    -0.876     0.216




  C(totincr)[T.8]     -0.3530     0.276    -1.279  0.201    -0.894     0.188




  C(totincr)[T.9]     -0.5145     0.280    -1.837  0.066    -1.064     0.034




  C(totincr)[T.10]    -0.4745     0.290    -1.635  0.102    -1.043     0.094




  C(totincr)[T.11]    -0.8088     0.282    -2.873  0.004    -1.361    -0.257




  C(totincr)[T.12]    -1.1122     0.288    -3.860  0.000    -1.677    -0.547




  C(totincr)[T.13]    -1.1297     0.294    -3.838  0.000    -1.707    -0.553




  C(totincr)[T.14]    -1.3146     0.279    -4.711  0.000    -1.862    -0.768




  age_r               -0.0864     0.006   -14.865  0.000    -0.098    -0.075




     rmarital=3       coef     std err      z      P>|z| [95.0% Conf. Int.] 




  Intercept           -7.4010     1.299    -5.697  0.000    -9.947    -4.855




  C(race)[T.2]        -0.1220     0.366    -0.333  0.739    -0.840     0.596




  C(race)[T.3]         0.4449     0.514     0.865  0.387    -0.563     1.453




  C(educat)[T.10]      0.3587     0.699     0.513  0.608    -1.011     1.729




  C(educat)[T.11]      1.1767     0.514     2.288  0.022     0.169     2.185




  C(educat)[T.12]      0.2003     0.464     0.431  0.666    -0.710     1.110




  C(educat)[T.13]      0.6569     0.528     1.245  0.213    -0.378     1.691




  C(educat)[T.14]      0.3438     0.563     0.610  0.542    -0.760     1.448




  C(educat)[T.15]    -37.3705  8.59e+07 -4.35e-07  1.000 -1.68e+08  1.68e+08




  C(educat)[T.16]     -1.2998     1.086    -1.197  0.231    -3.429     0.829




  C(educat)[T.17]    -16.4896  3599.796    -0.005  0.996 -7071.961  7038.982




  C(educat)[T.18]    -28.3278   1.4e+06 -2.03e-05  1.000 -2.74e+06  2.74e+06




  C(educat)[T.19]    -17.0263  5973.424    -0.003  0.998 -1.17e+04  1.17e+04




  C(totincr)[T.2]      0.6312     0.900     0.701  0.483    -1.133     2.395




  C(totincr)[T.3]      0.7179     0.906     0.792  0.428    -1.059     2.494




  C(totincr)[T.4]      1.0778     0.801     1.345  0.179    -0.493     2.648




  C(totincr)[T.5]     -0.1090     0.862    -0.126  0.899    -1.798     1.580




  C(totincr)[T.6]     -0.9137     0.936    -0.976  0.329    -2.749     0.921




  C(totincr)[T.7]     -0.1814     0.845    -0.215  0.830    -1.837     1.474




  C(totincr)[T.8]     -0.6882     0.892    -0.772  0.440    -2.436     1.060




  C(totincr)[T.9]     -0.8010     0.895    -0.895  0.371    -2.556     0.954




  C(totincr)[T.10]    -1.9791     1.247    -1.587  0.112    -4.423     0.464




  C(totincr)[T.11]    -2.4027     1.246    -1.928  0.054    -4.846     0.040




  C(totincr)[T.12]    -1.7117     1.030    -1.662  0.097    -3.731     0.307




  C(totincr)[T.13]    -2.2688     1.253    -1.811  0.070    -4.724     0.187




  C(totincr)[T.14]    -2.8383     1.256    -2.260  0.024    -5.300    -0.377




  age_r                0.1138     0.026     4.411  0.000     0.063     0.164




     rmarital=4       coef     std err      z      P>|z| [95.0% Conf. Int.] 




  Intercept           -3.7385     0.439    -8.525  0.000    -4.598    -2.879




  C(race)[T.2]        -0.2686     0.122    -2.207  0.027    -0.507    -0.030




  C(race)[T.3]        -0.6078     0.214    -2.839  0.005    -1.027    -0.188




  C(educat)[T.10]      1.0340     0.277     3.736  0.000     0.491     1.576




  C(educat)[T.11]      0.8267     0.269     3.071  0.002     0.299     1.354




  C(educat)[T.12]      0.6323     0.202     3.129  0.002     0.236     1.028




  C(educat)[T.13]      1.1848     0.219     5.411  0.000     0.756     1.614




  C(educat)[T.14]      1.0894     0.220     4.953  0.000     0.658     1.520




  C(educat)[T.15]      1.1691     0.251     4.654  0.000     0.677     1.662




  C(educat)[T.16]      0.8574     0.232     3.696  0.000     0.403     1.312




  C(educat)[T.17]      0.8909     0.300     2.966  0.003     0.302     1.480




  C(educat)[T.18]      1.0343     0.314     3.297  0.001     0.419     1.649




  C(educat)[T.19]      1.5258     0.300     5.090  0.000     0.938     2.113




  C(totincr)[T.2]      0.5443     0.388     1.401  0.161    -0.217     1.305




  C(totincr)[T.3]      0.5518     0.388     1.423  0.155    -0.208     1.312




  C(totincr)[T.4]      0.4444     0.348     1.276  0.202    -0.238     1.127




  C(totincr)[T.5]     -0.0201     0.347    -0.058  0.954    -0.701     0.661




  C(totincr)[T.6]     -0.0391     0.330    -0.118  0.906    -0.686     0.608




  C(totincr)[T.7]     -0.2231     0.334    -0.668  0.504    -0.878     0.432




  C(totincr)[T.8]     -0.1620     0.328    -0.494  0.621    -0.805     0.481




  C(totincr)[T.9]     -0.5187     0.332    -1.562  0.118    -1.170     0.132




  C(totincr)[T.10]    -0.6662     0.346    -1.923  0.054    -1.345     0.013




  C(totincr)[T.11]    -0.9324     0.335    -2.786  0.005    -1.588    -0.277




  C(totincr)[T.12]    -1.2501     0.342    -3.653  0.000    -1.921    -0.579




  C(totincr)[T.13]    -1.8745     0.371    -5.054  0.000    -2.602    -1.148




  C(totincr)[T.14]    -2.0887     0.344    -6.074  0.000    -2.763    -1.415




  age_r                0.0611     0.007     8.274  0.000     0.047     0.076




     rmarital=5       coef     std err      z      P>|z| [95.0% Conf. Int.] 




  Intercept           -0.4618     0.434    -1.065  0.287    -1.312     0.388




  C(race)[T.2]        -0.7824     0.152    -5.160  0.000    -1.080    -0.485




  C(race)[T.3]        -0.4941     0.234    -2.109  0.035    -0.953    -0.035




  C(educat)[T.10]      0.5865     0.287     2.043  0.041     0.024     1.149




  C(educat)[T.11]     -0.1031     0.318    -0.325  0.746    -0.726     0.520




  C(educat)[T.12]      0.1001     0.208     0.482  0.630    -0.307     0.507




  C(educat)[T.13]      0.5279     0.237     2.229  0.026     0.064     0.992




  C(educat)[T.14]      0.1296     0.259     0.501  0.616    -0.377     0.637




  C(educat)[T.15]     -0.3197     0.358    -0.893  0.372    -1.022     0.382




  C(educat)[T.16]     -0.2415     0.295    -0.818  0.413    -0.820     0.337




  C(educat)[T.17]     -0.5236     0.491    -1.066  0.287    -1.487     0.439




  C(educat)[T.18]     -1.1554     0.739    -1.563  0.118    -2.605     0.294




  C(educat)[T.19]     -0.1020     0.550    -0.186  0.853    -1.180     0.976




  C(totincr)[T.2]      0.2858     0.362     0.790  0.430    -0.423     0.995




  C(totincr)[T.3]      0.2362     0.363     0.650  0.516    -0.476     0.948




  C(totincr)[T.4]     -0.2794     0.339    -0.825  0.409    -0.943     0.384




  C(totincr)[T.5]     -0.8008     0.339    -2.363  0.018    -1.465    -0.137




  C(totincr)[T.6]     -1.0848     0.332    -3.270  0.001    -1.735    -0.435




  C(totincr)[T.7]     -1.3169     0.345    -3.820  0.000    -1.993    -0.641




  C(totincr)[T.8]     -1.4121     0.347    -4.069  0.000    -2.092    -0.732




  C(totincr)[T.9]     -1.7248     0.364    -4.740  0.000    -2.438    -1.012




  C(totincr)[T.10]    -1.6028     0.377    -4.254  0.000    -2.341    -0.864




  C(totincr)[T.11]    -2.1400     0.380    -5.628  0.000    -2.885    -1.395




  C(totincr)[T.12]    -2.0765     0.370    -5.614  0.000    -2.801    -1.352




  C(totincr)[T.13]    -2.4733     0.420    -5.895  0.000    -3.296    -1.651




  C(totincr)[T.14]    -2.8213     0.394    -7.154  0.000    -3.594    -2.048




  age_r                0.0018     0.009     0.200  0.841    -0.016     0.019




     rmarital=6       coef     std err      z      P>|z| [95.0% Conf. Int.] 




  Intercept            6.8391     0.249    27.424  0.000     6.350     7.328




  C(race)[T.2]        -1.3694     0.083   -16.504  0.000    -1.532    -1.207




  C(race)[T.3]        -1.4115     0.126   -11.238  0.000    -1.658    -1.165




  C(educat)[T.10]      0.6048     0.162     3.741  0.000     0.288     0.922




  C(educat)[T.11]      0.5489     0.154     3.571  0.000     0.248     0.850




  C(educat)[T.12]      0.1009     0.113     0.897  0.370    -0.120     0.321




  C(educat)[T.13]      0.0674     0.134     0.503  0.615    -0.195     0.330




  C(educat)[T.14]      0.2297     0.133     1.727  0.084    -0.031     0.490




  C(educat)[T.15]      0.2108     0.156     1.348  0.178    -0.096     0.517




  C(educat)[T.16]      0.2923     0.131     2.233  0.026     0.036     0.549




  C(educat)[T.17]      0.4071     0.175     2.330  0.020     0.065     0.750




  C(educat)[T.18]      0.5499     0.197     2.799  0.005     0.165     0.935




  C(educat)[T.19]      0.8683     0.205     4.238  0.000     0.467     1.270




  C(totincr)[T.2]      0.2612     0.263     0.994  0.320    -0.254     0.776




  C(totincr)[T.3]     -0.1218     0.264    -0.461  0.645    -0.640     0.397




  C(totincr)[T.4]     -0.1371     0.230    -0.595  0.552    -0.589     0.315




  C(totincr)[T.5]     -0.8255     0.222    -3.714  0.000    -1.261    -0.390




  C(totincr)[T.6]     -0.9032     0.214    -4.218  0.000    -1.323    -0.484




  C(totincr)[T.7]     -0.8304     0.212    -3.921  0.000    -1.246    -0.415




  C(totincr)[T.8]     -1.0041     0.211    -4.748  0.000    -1.419    -0.590




  C(totincr)[T.9]     -1.0310     0.211    -4.884  0.000    -1.445    -0.617




  C(totincr)[T.10]    -1.0496     0.220    -4.777  0.000    -1.480    -0.619




  C(totincr)[T.11]    -1.3029     0.210    -6.211  0.000    -1.714    -0.892




  C(totincr)[T.12]    -1.4874     0.210    -7.082  0.000    -1.899    -1.076




  C(totincr)[T.13]    -1.3953     0.212    -6.596  0.000    -1.810    -0.981




  C(totincr)[T.14]    -1.4509     0.201    -7.226  0.000    -1.844    -1.057




  age_r               -0.1758     0.005   -36.869  0.000    -0.185    -0.166




















In [275]:



    


columns = ['age_r','race','educat','totincr']
new = pandas.DataFrame([[25, 2, 12, 11]], columns=columns)
y = results.predict(new)
y



    


















    
Out[275]:








array([[ 0.43494745,  0.13738324,  0.00044677,  0.02698496,  0.0170361 ,
         0.38320148]])

43% are currently married 14% are not married but currently living with opp sex partner An extremely small amount are widowed 2.6% are divorced 1.7% are separated due to marital discord 38% have never been maried.





In [281]:



    


columns = ['age_r','race','educat','totincr']
new = pandas.DataFrame([[43, 2, 16, 14]], columns=columns)
y = results.predict(new)
y



    


















    
Out[281]:








array([[ 0.86989957,  0.01875586,  0.0009998 ,  0.06387563,  0.01264673,
         0.0338224 ]])

















In [ ]:

Content source: goodwordalchemy/thinkstats_notes_and_exercises

Similar notebooks:

notebook.community | gallery | about