Smoothing Spline ANOVA Models Chong Gu DepartmentofStatistics PurdueUniversity June 21, 2012 ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 1/45 Outline 1 Introduction Cubic Spline and Penalized Likelihood Functional ANOVA Decomposition R Package gss 2 Estimation and Inference Splines as Bayes Estimates Efficient Approximation Cross-Validation Bayesian Confidence Intervals Kullback-Leibler Projection 3 Regression Models Non-Gaussian Regression Regression with Correlated Data 4 Density and Hazard Estimation Density and Conditional Density Hazard and Relative Risk ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45 Outline 1 Introduction Cubic Spline and Penalized Likelihood Functional ANOVA Decomposition R Package gss 2 Estimation and Inference Splines as Bayes Estimates Efficient Approximation Cross-Validation Bayesian Confidence Intervals Kullback-Leibler Projection 3 Regression Models Non-Gaussian Regression Regression with Correlated Data 4 Density and Hazard Estimation Density and Conditional Density Hazard and Relative Risk ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45 Outline 1 Introduction Cubic Spline and Penalized Likelihood Functional ANOVA Decomposition R Package gss 2 Estimation and Inference Splines as Bayes Estimates Efficient Approximation Cross-Validation Bayesian Confidence Intervals Kullback-Leibler Projection 3 Regression Models Non-Gaussian Regression Regression with Correlated Data 4 Density and Hazard Estimation Density and Conditional Density Hazard and Relative Risk ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45 Outline 1 Introduction Cubic Spline and Penalized Likelihood Functional ANOVA Decomposition R Package gss 2 Estimation and Inference Splines as Bayes Estimates Efficient Approximation Cross-Validation Bayesian Confidence Intervals Kullback-Leibler Projection 3 Regression Models Non-Gaussian Regression Regression with Correlated Data 4 Density and Hazard Estimation Density and Conditional Density Hazard and Relative Risk ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45 Outline 1 Introduction Cubic Spline and Penalized Likelihood Functional ANOVA Decomposition R Package gss 2 Estimation and Inference Splines as Bayes Estimates Efficient Approximation Cross-Validation Bayesian Confidence Intervals Kullback-Leibler Projection 3 Regression Models Non-Gaussian Regression Regression with Correlated Data 4 Density and Hazard Estimation Density and Conditional Density Hazard and Relative Risk ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 3/45 Cubic Smoothing Spline ◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1] i i i i ∈ and ǫ N(0,σ2), one is to estimate η(x). i ∼ ◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx. ◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1]. 1 2 q i 1 q ··· ∪ ◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one + ↑ interpolates data, at λ = , η(x) = β +β x. 0 1 ∞ ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45 Cubic Smoothing Spline ◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1] i i i i ∈ and ǫ N(0,σ2), one is to estimate η(x). i ∼ ◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx. ◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1]. 1 2 q i 1 q ··· ∪ ◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one + ↑ interpolates data, at λ = , η(x) = β +β x. 0 1 ∞ ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45 Cubic Smoothing Spline ◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1] i i i i ∈ and ǫ N(0,σ2), one is to estimate η(x). i ∼ ◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx. ◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1]. 1 2 q i 1 q ··· ∪ ◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one + ↑ interpolates data, at λ = , η(x) = β +β x. 0 1 ∞ ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45 Cubic Smoothing Spline ◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1] i i i i ∈ and ǫ N(0,σ2), one is to estimate η(x). i ∼ ◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx. ◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1]. 1 2 q i 1 q ··· ∪ ◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one + ↑ interpolates data, at λ = , η(x) = β +β x. 0 1 ∞ ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45