Introduction to Robust Estimation and Hypothesis Testing Second Edition Rand R. Wilcox Department of Psychology University of Southern California AMSTERDAM • BOSTON (cid:127) HEIDELBERG (cid:127) LONDON NEWYORK (cid:127) OXFORD (cid:127) PARIS (cid:127) SANDIEGO SANFRANCISCO (cid:127) SINGAPORE (cid:127) SYDNEY (cid:127) TOKYO AcquisitionEditor BarbaraA.Holland AssociateAcquisitionEditor TomSinger SeniorProjectManager AngelaG.Dooley MarketingManager PhilipPritchard CoverDesign RichardHannus CoverImage Superstock Composition CEPHA CoverPrinter PhoenixColor,Inc. InteriorPrinter TheMaple-VailBookManufacturingGroup,Inc. ElsevierAcademicPress 30CorporateDrive,Suite400,Burlington,MA01803,USA 525BStreet,Suite1900,SanDiego,CA92101-4495,USA 84Theobald’sRoad,LondonWC1X8RR,UK Thisbookisprintedonacid-freepaper. Copyright©2005,ElsevierInc.Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyany means,electronicormechanical,includingphotocopy,recording,oranyinformation storageandretrievalsystem,withoutpermissioninwritingfromthepublisher. PermissionsmaybesoughtdirectlyfromElsevier’sScience&TechnologyRights DepartmentinOxford,UK:phone:(+44)1865843830,fax:(+44)1865853333, e-mail:[email protected] viatheElsevierhomepage(http://elsevier.com),byselecting“CustomerSupport” andthen“ObtainingPermissions.” LibraryofCongressCataloging-in-PublicationData Applicationsubmitted BritishLibraryCataloguinginPublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN:0-12-751542-9 ForallinformationonallElsevierAcademicPressPublications visitourWebsiteatwww.books.elsevier.com PrintedintheUnitedStatesofAmerica 04 05 06 07 08 09 9 8 7 6 5 4 3 2 1 Contents Preface................................................................. xvii Chapter1 Introduction .............................................. 1 1.1 ProblemswithAssumingNormality................................ 2 1.2 Transformations.................................................... 6 1.3 TheInfluenceCurve................................................ 7 1.4 TheCentralLimitTheorem......................................... 8 1.5 IstheANOVAFRobust?........................................... 9 1.6 Regression.......................................................... 10 1.7 MoreRemarks...................................................... 10 1.8 UsingtheComputer:RandS-PLUS................................. 11 1.9 SomeData-ManagmentIssues...................................... 13 1.9.1 EliminatingMissingValues................................ 17 Chapter2 AFoundationforRobustMethods........................ 19 2.1 BasicToolsforJudgingRobustness................................. 20 2.1.1 QualitativeRobustness.................................... 21 2.1.2 InfinitesimalRobustness................................... 23 2.1.3 QuantitativeRobustness................................... 25 2.2 SomeMeasuresofLocationandTheirInfluenceFunction........... 26 2.2.1 Quantiles.................................................. 26 2.2.2 TheWinsorizedMean..................................... 27 iii iv Contents 2.2.3 TheTrimmedMean....................................... 29 2.2.4 M-MeasuresofLocation................................... 30 2.2.5 R-MeasuresofLocation.................................... 33 2.3 MeasuresofScale................................................... 33 2.3.1 MeanDeviationfromtheMean............................ 34 2.3.2 MeanDeviationfromtheMedian.......................... 35 2.3.3 MedianAbsoluteDeviation................................ 35 2.3.4 Theq-QuantileRange...................................... 35 2.3.5 TheWinsorizedVariance.................................. 36 2.4 Scale-EquivariantM-MeasuresofLocation.......................... 36 2.5 WinsorizedExpectedValues........................................ 38 Chapter3 EstimatingMeasuresofLocationandScale .............. 43 3.1 ABootstrapEstimateofaStandardError........................... 44 3.1.1 RandS-PLUSFunctionbootse............................ 46 3.2 DensityEstimators................................................. 47 3.2.1 NormalKernel............................................ 47 3.2.2 Rosenblatt’sShiftedHistogram............................ 48 3.2.3 TheExpectedFrequencyCurve............................ 48 3.2.4 AnAdaptiveKernelEstimator............................. 49 3.2.5 RandS-PLUSFunctionsskerd,kerden,kdplot,rdplot, akerd,andsplot.......................................... 50 3.3 TheSampleTrimmedMean......................................... 56 3.3.1 RandS-PLUSFunctiontmean............................. 59 3.3.2 EstimatingtheStandardErroroftheTrimmedMean....... 59 3.3.3 RandS-PLUSFunctionswin,winvar,andtrimse.......... 64 3.3.4 EstimatingtheStandardErroroftheSampleMedian,M... 64 3.3.5 RandS-PLUSFunctionmsmedse........................... 65 3.4 TheFinite-SampleBreakdownPoint................................ 65 3.5 EstimatingQuantiles............................................... 66 3.5.1 EstimatingtheStandardErroroftheSampleQuantile...... 67 3.5.2 RandS-PLUSFunctionqse................................ 69 3.5.3 TheMaritz–JarrettEstimateoftheStandardErrorofxˆq..... 69 Contents v 3.5.4 RandS-PLUSFunctionmjse.............................. 70 3.5.5 TheHarrell–DavisEstimator............................... 71 3.5.6 RandS-PLUSFunctionhd................................. 72 ˆ 3.5.7 ABootstrapEstimateoftheStandardErrorofθ ........... 72 q 3.5.8 RandS-PLUSFunctionhdseb............................. 72 3.6 AnM-EstimatorofLocation........................................ 73 3.6.1 ComputinganM-EstimatorofLocation.................... 79 3.6.2 RandS-PLUSFunctionmest.............................. 81 3.6.3 EstimatingtheStandardErroroftheM-Estimator.......... 81 3.6.4 RandS-PLUSFunctionmestse............................ 84 3.6.5 ABootstrapEstimateoftheStandardErrorofµˆm.......... 84 3.6.6 RandS-PLUSFunctionmestseb........................... 85 3.7 One-StepM-Estimator.............................................. 85 3.7.1 RandS-PLUSFunctiononestep........................... 86 3.8 W-Estimators....................................................... 87 3.9 TheHodges–LehmannEstimator................................... 88 3.10 SkippedEstimators................................................. 88 3.10.1 RandS-PLUSFunctionsmomandbmean.................... 89 3.11 SomeComparisonsoftheLocationEstimators...................... 90 3.12 MoreMeasuresofScale............................................. 92 3.12.1 TheBiweightMidvariance................................. 93 3.12.2 RandS-PLUSFunctionbivar............................. 96 3.12.3 ThePercentageBendMidvariance......................... 96 3.12.4 RandS-PLUSFunctionpbvar............................. 98 3.12.5 TheInterquartileRange.................................... 98 3.12.6 RandS-PLUSFunctionidealf............................ 99 3.13 SomeOutlierDetectionMethods.................................... 99 3.13.1 RulesBasedonMeansandVariances...................... 99 3.13.2 AMethodBasedontheInterquartileRange................ 100 3.13.3 Carling’sModification..................................... 100 3.13.4 AMAD-MedianRule...................................... 101 3.13.5 RandS-PLUSFunctionsoutboxandout................... 101 3.14 Exercises........................................................... 102 vi Contents Chapter4 ConfidenceIntervalsintheOne-SampleCase ........... 105 4.1 ProblemsWhenWorkingwithMeans............................... 105 4.2 Theg-and-hDistribution........................................... 110 4.3 InferencesAbouttheTrimmedMean............................... 113 4.3.1 RandS-PLUSFunctiontrimci............................ 117 4.4 BasicBootstrapMethods............................................ 117 4.4.1 ThePercentileBootstrapMethod.......................... 118 4.4.2 RandS-PLUSFunctiononesampb.......................... 119 4.4.3 Bootstrap-tMethod........................................ 119 4.4.4 BootstrapMethodsWhenUsingaTrimmedMean......... 121 4.4.5 Singh’sModification....................................... 125 4.4.6 RandS-PLUSFunctionstrimpbandtrimcibt............. 126 4.5 InferencesAboutM-Estimators..................................... 127 4.5.1 RandS-PLUSFunctionsmestciandmomci................ 129 4.6 ConfidenceIntervalsforQuantiles.................................. 130 4.6.1 AlteranativeMethodfortheMedian....................... 132 4.6.2 RandS-PLUSFunctionsqmjci,hdci,sint, qci,andqint.............................................. 133 4.7 ConcludingRemarks............................................... 134 4.8 Exercises........................................................... 135 Chapter5 ComparingTwoGroups.................................. 137 5.1 TheShiftFunction.................................................. 139 5.1.1 TheKolmogorov–SmirnovTest............................ 142 5.1.2 RandS-PLUSFunctionsks,kssig,kswsig, andkstiesig.............................................. 145 5.1.3 TheSBandandWBandfortheShiftFunction............. 147 5.1.4 RandS-PLUSFunctionssbandandwband................. 148 5.1.5 ConfidenceBandfortheDecilesOnly...................... 151 5.1.6 RandS-PLUSFunctionshifthd........................... 153 5.1.7 RandS-PLUSFunctionsg2plotandsplotg2.............. 155 5.2 Student’stTest..................................................... 155 Contents vii 5.3 TheYuen–WelchTest............................................... 159 5.3.1 RandS-PLUSFunctionyuen.............................. 161 5.3.2 ABootstrap-tMethodforComparingTrimmedMeans..... 162 5.3.3 S-PLUSFunctionyuenbt................................... 165 5.4 InferencesBasedonaPercentileBootstrapMethod.................. 167 5.4.1 ComparingM-Estimators.................................. 168 5.4.2 ComparingTrimmedMeans............................... 169 5.4.3 RandS-PLUSFunctionstrimpb2,pb2gen,andm2ci....... 169 5.5 ComparingMeasuresofScale....................................... 170 5.5.1 ComparingVariances...................................... 170 5.5.2 RandS-PLUSFunctioncomvar2........................... 171 5.5.3 ComparingBiweightMidvariances........................ 171 5.5.4 RandS-PLUSFunctionb2ci.............................. 171 5.6 PermutationTests.................................................. 172 5.6.1 RandS-PLUSFunctionpermg............................. 173 5.7 SomeHeteroscedastic,Rank-BasedMethods........................ 173 5.7.1 RandS-PLUSFunctionmee................................ 174 5.7.2 HandlingTiedValues..................................... 176 5.7.3 RandS-PLUSFunctionscidandbmp...................... 180 5.8 ComparingTwoIndependentBinomials............................ 181 5.8.1 Storer–KimMethod....................................... 182 5.8.2 Beal’sMethod............................................. 183 5.8.3 RandS-PLUSFunctionstwobinomandtwobici........... 184 5.9 ComparingDependentGroups..................................... 184 5.9.1 ComparingDeciles........................................ 185 5.9.2 RandS-PLUSFunctionshiftdhd.......................... 186 5.9.3 ComparingTrimmedMeans............................... 188 5.9.4 RandS-PLUSFunctionyuend............................. 190 5.9.5 ABootstrap-tMethodforMarginalTrimmedMeans....... 191 5.9.6 RandS-PLUSFunctionydbt.............................. 192 5.9.7 PercentileBootstrap:ComparingM-EstimatorsandOther MeasuresofLocationandScale............................ 192 5.9.8 RandS-PLUSFunctionbootdpci.......................... 194 5.9.9 ComparingVariances...................................... 195 viii Contents 5.9.10 TheSignTestandInferencesAboutthe BinomialDistribution...................................... 196 5.9.11 RandS-PLUSFunctionbinomci........................... 198 5.10 Exercises........................................................... 199 Chapter6 SomeMultivariateMethods.............................. 203 6.1 GeneralizedVariance............................................... 203 6.2 Depth.............................................................. 204 6.2.1 MahalanobisDepth........................................ 204 6.2.2 HalfspaceDepth........................................... 205 6.2.3 ComputingHalfspaceDepth.............................. 207 6.2.4 RandS-PLUSFunctionsdepth2,depth,fdepth,fdepthv2, unidepth,depth2.for,depth3.for,fdepth.for, fdepthv2.for,andufdepth.for........................... 210 6.2.5 ProjectionDepth........................................... 211 6.2.6 RandS-PLUSFunctionspdisandpdis.for............... 212 6.2.7 OtherMeasuresofDepth.................................. 213 6.2.8 RandS-PLUSFunctionzdepth............................ 214 6.3 SomeAffine-EquivariantEstimators................................ 214 6.3.1 Minimum-VolumeEllipsoidEstimator..................... 215 6.3.2 TheMinimum-CovarianceDeterminantEstimator......... 216 6.3.3 S-EstimatorsandConstrainedM-Estimators............... 216 6.3.4 Donoho–GaskoGeneralizationofaTrimmedMean........ 218 6.3.5 RandS-PLUSFunctionsdmeananddmean.for............. 219 6.3.6 TheStahel–DonohoW-Estimator.......................... 220 6.4 MultivariateOutlierDetectionMethods............................. 221 6.4.1 ARelplot.................................................. 222 6.4.2 RandS-PLUSFunctionrelplot........................... 223 6.4.3 TheMVEMethod......................................... 226 6.4.4 TheMCDMethod......................................... 226 6.4.5 RandS-PLUSFunctionscov.mveandcov.mcd............. 226 6.4.6 RandS-PLUSFunctionsoutmveandout................... 227 6.4.7 TheMGVMethod......................................... 228 6.4.8 RandS-PLUSFunctionoutmgv............................ 231 Contents ix 6.4.9 AProjectionMethod....................................... 231 6.4.10 RandS-PLUSFunctionsoutproandoutpro.for.......... 233 6.4.11 CommentsonChoosingaMethod......................... 234 6.5 ASkippedEstimatorofLocationandScatter........................ 236 6.5.1 RandS-PLUSFunctionssmean,smean.for,wmcd,wmve, mgvmean,andspat......................................... 238 6.6 ConfidenceRegionandInferenceBasedontheOPEstimatorof Location............................................................ 240 6.6.1 RandS-PLUSFunctionssmeancrandsmeancr.for........ 241 6.6.2 InferencesBasedontheMGVEstimator................... 243 6.6.3 RandS-PLUSFunctionsmgvcr............................ 244 6.7 Two-SampleCase.................................................. 244 6.7.1 RandS-PLUSFunctionssmean2andsmean2.for.......... 245 6.8 MultivariateDensityEstimators.................................... 245 6.9 ATwo-Sample,Projection–TypeExtensionofthe Wilcoxon–Mann–WhitneyTest..................................... 247 6.9.1 RandS-PLUSFunctionsmulwmwandmulwmw.for.......... 249 6.10 ARelativeDepthAnalogoftheWilcoxon–Mann–WhitneyTest..... 250 6.10.1 RandS-PLUSFunctionsmwmwandmwmw.for............... 252 6.11 ComparisonsBasedonDepth....................................... 253 6.11.1 RandS-PLUSFunctionslsqs3,lsqs3.for,anddepthg2... 256 6.12 ComparingDependentGroupsBasedonAllPairwiseDifferences... 259 6.12.1 RandS-PLUSFunctiondfried............................ 261 6.13 Exercises........................................................... 262 Chapter7 One-WayandHigherDesignsfor IndependentGroups...................................... 265 7.1 TrimmedMeansandaOne-WayDesign............................ 266 7.1.1 AWelch-TypeAdjustedDegreesofFreedomProcedure... 266 7.1.2 RandS-PLUSFunctiont1way............................. 268 7.1.3 AGeneralizationofBox’sMethod......................... 270 7.1.4 RandS-PLUSFunctionbox1way........................... 270 7.1.5 ComparingMedians....................................... 271