Methods of Multivariate Analysis Second Edition ALVINC.RENCHER BrighamYoungUniversity AJOHNWILEY&SONS,INC.PUBLICATION Methods of Multivariate Analysis Second Edition Methods of Multivariate Analysis Second Edition ALVINC.RENCHER BrighamYoungUniversity AJOHNWILEY&SONS,INC.PUBLICATION ∞ Thisbookisprintedonacid-freepaper. Copyright c 2002byJohnWiley&Sons,Inc.Allrightsreserved. (cid:1) PublishedsimultaneouslyinCanada. Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedinanyform orbyanymeans,electronic,mechanical,photocopying,recording,scanningorotherwise,exceptas permittedunderSections107or108ofthe1976UnitedStatesCopyrightAct,withouteithertheprior writtenpermissionofthePublisher,orauthorizationthroughpaymentoftheappropriateper-copyfeeto theCopyrightClearanceCenter,222RosewoodDrive,Danvers,MA01923,(978)750-8400,fax(978) 750-4744.RequeststothePublisherforpermissionshouldbeaddressedtothePermissionsDepartment, JohnWiley&Sons,Inc.,605ThirdAvenue,NewYork,NY10158-0012,(212)850-6011,fax(212) 850-6008.E-Mail:[email protected]. Fororderingandcustomerservice,call1-800-CALL-WILEY. LibraryofCongressCataloging-in-PublicationData Rencher,AlvinC.,1934– Methodsofmultivariateanalysis/AlvinC.Rencher.—2nded. p. cm.—(Wileyseriesinprobabilityandmathematicalstatistics) “AWiley-Intersciencepublication.” Includesbibliographicalreferencesandindex. ISBN0-471-41889-7(cloth) 1. Multivariateanalysis. I. Title. II. Series. QA278.R452001 519.535—dc21 2001046735 ′ PrintedintheUnitedStatesofAmerica 10 9 8 7 6 5 4 3 2 1 Contents 1. Introduction 1 1.1 WhyMultivariateAnalysis?, 1 1.2 Prerequisites, 3 1.3 Objectives, 3 1.4 BasicTypesofDataandAnalysis, 3 2. MatrixAlgebra 5 2.1 Introduction, 5 2.2 NotationandBasicDefinitions, 5 2.2.1 Matrices,Vectors,andScalars, 5 2.2.2 EqualityofVectorsandMatrices, 7 2.2.3 TransposeandSymmetricMatrices, 7 2.2.4 SpecialMatrices, 8 2.3 Operations, 9 2.3.1 SummationandProductNotation, 9 2.3.2 AdditionofMatricesandVectors, 10 2.3.3 MultiplicationofMatricesandVectors, 11 2.4 PartitionedMatrices, 20 2.5 Rank, 22 2.6 Inverse, 23 2.7 PositiveDefiniteMatrices, 25 2.8 Determinants, 26 2.9 Trace, 30 2.10 OrthogonalVectorsandMatrices, 31 2.11 EigenvaluesandEigenvectors, 32 2.11.1 Definition, 32 2.11.2 I AandI A, 33 + − 2.11.3 tr(A)and A, 34 | | 2.11.4 PositiveDefiniteandSemidefiniteMatrices, 34 2.11.5 TheProductAB, 35 2.11.6 SymmetricMatrix, 35 v vi CONTENTS 2.11.7 SpectralDecomposition, 35 2.11.8 SquareRootMatrix, 36 2.11.9 SquareMatricesandInverseMatrices, 36 2.11.10 SingularValueDecomposition, 36 3. CharacterizingandDisplayingMultivariateData 43 3.1 MeanandVarianceofaUnivariateRandomVariable, 43 3.2 CovarianceandCorrelationofBivariateRandomVariables, 45 3.2.1 Covariance, 45 3.2.2 Correlation, 49 3.3 ScatterPlotsofBivariateSamples, 50 3.4 GraphicalDisplaysforMultivariateSamples, 52 3.5 MeanVectors, 53 3.6 CovarianceMatrices, 57 3.7 CorrelationMatrices, 60 3.8 MeanVectorsandCovarianceMatricesforSubsetsof Variables, 62 3.8.1 TwoSubsets, 62 3.8.2 ThreeorMoreSubsets, 64 3.9 LinearCombinationsofVariables, 66 3.9.1 SampleProperties, 66 3.9.2 PopulationProperties, 72 3.10 MeasuresofOverallVariability, 73 3.11 EstimationofMissingValues, 74 3.12 DistancebetweenVectors, 76 4. TheMultivariateNormalDistribution 82 4.1 MultivariateNormalDensityFunction, 82 4.1.1 UnivariateNormalDensity, 82 4.1.2 MultivariateNormalDensity, 83 4.1.3 GeneralizedPopulationVariance, 83 4.1.4 DiversityofApplicationsoftheMultivariateNormal, 85 4.2 PropertiesofMultivariateNormalRandomVariables, 85 4.3 EstimationintheMultivariateNormal, 90 4.3.1 MaximumLikelihoodEstimation, 90 4.3.2 DistributionofyandS, 91 4.4 AssessingMultivariateNormality, 92 4.4.1 InvestigatingUnivariateNormality, 92 4.4.2 InvestigatingMultivariateNormality, 96 CONTENTS vii 4.5 Outliers, 99 4.5.1 OutliersinUnivariateSamples, 100 4.5.2 OutliersinMultivariateSamples, 101 5. TestsonOneorTwoMeanVectors 112 5.1 MultivariateversusUnivariateTests, 112 5.2 Testson(cid:1)with(cid:2)Known, 113 5.2.1 ReviewofUnivariateTestfor H µ µ 0 0 : = withσ Known, 113 5.2.2 MultivariateTestfor H (cid:1) (cid:1) with(cid:2)Known, 114 0 0 : = 5.3 Testson(cid:1)When(cid:2)IsUnknown, 117 5.3.1 ReviewofUnivariatet-Testfor H µ µ withσ 0 0 : = Unknown, 117 5.3.2 Hotelling’sT2-Testfor H (cid:1) (cid:1) with(cid:2)Unknown, 117 0 0 : = 5.4 ComparingTwoMeanVectors, 121 5.4.1 ReviewofUnivariateTwo-Samplet-Test, 121 5.4.2 MultivariateTwo-SampleT2-Test, 122 5.4.3 LikelihoodRatioTests, 126 5.5 TestsonIndividualVariablesConditionalonRejectionof H by 0 theT2-Test, 126 5.6 ComputationofT2, 130 5.6.1 ObtainingT2fromaMANOVAProgram, 130 5.6.2 ObtainingT2fromMultipleRegression, 130 5.7 PairedObservationsTest, 132 5.7.1 UnivariateCase, 132 5.7.2 MultivariateCase, 134 5.8 TestforAdditionalInformation, 136 5.9 ProfileAnalysis, 139 5.9.1 One-SampleProfileAnalysis, 139 5.9.2 Two-SampleProfileAnalysis, 141 6. MultivariateAnalysisofVariance 156 6.1 One-WayModels, 156 6.1.1 UnivariateOne-WayAnalysisofVariance(ANOVA), 156 6.1.2 MultivariateOne-WayAnalysisofVarianceModel (MANOVA), 158 6.1.3 Wilks’TestStatistic, 161 6.1.4 Roy’sTest, 164 6.1.5 PillaiandLawley–HotellingTests, 166