LS,LAD組合損失的高維統計性質分析

張凌潔,蘇美紅,張海

(西北大學數學系,陜西西安 710127)

LS,LAD組合損失的高維統計性質分析

張凌潔,蘇美紅,張海

(西北大學數學系,陜西西安 710127)

線性模型;高維;穩健估計;罰穩健估計;LS+LAD的凸組合

DO I：10.3969/j.issn.1008-5513.2013.05.014

1 引言

研究與應用的階段,將穩健統計數據擴展到其他評估和測試的問題,同時建立穩健估計的漸近理論,并講述有關穩健性的相關知識;文獻[6]對文獻[4]提出的案例(a)lim sup p＜∞做了分析;文獻[7]給出M估計中每個估計量的漸近有效性;文獻[8]介紹“一步法”的Huber(M)估計線性模型;文獻[9-11]給出?β的一致正態漸近分布;文獻[12]提出多參數線性模型M估計的漸近性和一致性;文獻[13-15]在一般損失函數下給出高維穩健估計和高維罰穩健估計求解‖?β‖(‖?β-β0‖)的方程組,并對如何適當選擇損失函數的問題做以分析.

經典地,通常研究p固定或p/n→0(觀測數n→∞比預測數p→∞的速度快)的情況,對于噪聲服從正態分布,最小二乘LS是優的,而關于損失函數ρ是雙指數分布,最小絕對偏差LAD是優的.

估計未知參數β時,當ε的分布是已知的(如正態的,均勻的,Weibull的等),通常采用最大似然估計法來估計未知參數.若ε的分布是未知的,通常采用LS﹑M inMax(MM)和LAD等作估計.如果誤差是正態分布,LS和最大似然估計是相同的,但是在響應變量和解釋變量中, LS卻受離群值的影響.對響應變量LAD是穩健的,但LAD對于解缺乏唯一性.近幾年提出組合的方法是為了處理不確定性模型選擇的問題,該方法不僅節省了計算時間,提高了估計精度,而且在不確定性模型選擇時,也給出了較好的估計量.比如組合的方法可以改善回歸的性能問題[16];用于穩固和收縮系數估計的組合方法能提高預測[17];用回歸函數的參數和非參數的組合回歸估計時,組合估計量優于核估計量[18].

為了減弱LS受離群值的影響和LAD對解缺乏唯一性,用LS+LAD的凸組合形式[19-20],即其中0≤δ≤1.顯然,當δ=0時,模型為LAD估計,當δ=1時,模型為LS估計.適當選擇δ是為了得到未知參數的最小漸近方差.組合模型允許組合一些已有模型來估算誤差,對已有模型的估計進行改善,使其具有更多的性質:使不確定性模型的選擇有了依據,節省了計算時間﹑提高了預測精度和估計的收斂率.特別,組合模型解決了解缺乏唯一性的問題.

然而,損失函數是LS+LAD凸組合形式的高維性質還不清楚.本文主要是在高維背景(觀測數n和預測數p均趨于無窮大,即下,對LS+LAD的高維穩健性質(p＜n)和高維罰穩健性質(p?n)作以分析,性質分析中主要運用了prox函數和Stein′s identity[14],得到了穩健估計和罰穩健估計的顯示表達,結果顯示這種凸組合損失函數模型集成了LS和LAD損失的優點,同時消弱了它們的不足,具有優良的高維統計性質.

2 LS+LAD高維穩健性質分析(p＜n)

3 LS+LAD高維罰穩健性質分析(p?n)

4 結論

在高維穩健回歸中,LS估計和LAD估計已有相對完善的理論結果,但是它們還存在一定的問題.LS在響應變量和解釋變量中受離群值的影響;LAD在解釋變量中受離群值的影響,同時還對解缺乏唯一性.

本文主要針對損失函數為LS+LAD的凸組合形式,研究了高維背景(觀測數n和預測數p均趨于無窮大,即

?運用了prox函數和Stein′s identity,得到了凸組合損失下高維穩健估計‖β‖和高維罰穩健回歸估計‖β?-β0‖的顯示表達,結果表明這種凸組合損失函數模型集成了LS和LAD損失的優點,同時消弱了它們的不足,具有優良的高維統計性質.

[1]陳希孺,趙林城.線性模型中的M方法[M].上海:上海科學出版社,1996.

[2]Huber P J.Robust estimation of a location parameter[J].Ann.Statist.,1964,35,73-101.

[3]Huber P J.The 1972Wald lecture.Robust statistics:A review[J].Ann.Statist.,1972,43:1041-1067.

[4]Huber P J.Robust regression:asym p totics,con jectures and M onte Carlo[J].Ann.Statist.,1973,1:799-821.

[5]Huber P J,Ronchetti E M.Robust Statistics[M].2nd ed.Hoboken,NJ:John W iley and Sons Inc,2009.

[6]Relles D.Robust Regression by M odifed Least Squares[C].New Haven:Ph.D.Thesis,Yale University,1968.

[7]Yohai V J.Robust Estimation in the Linear M odel[C].New Haven:Ph.D.Thesis,Yale University,1974.

[8]Bickel P J.One-step Huber estimates in the linearmodel[J].J.Amer.Statist.Assoc.,1975,70:428-434.

[9]Portnoy S.Asym p totic behavior of M-estim ators of p regression param eters w hen p2/n is large[J].Ann. Statist.,1984,12:1298-1309.

[10]Portnoy S.Asym p totic behavior of M estimators of p regression parameters when p2/n is large[J].Ann. Statist.,1985,13:1403-1417.

[11]Le Cam L.On the assum ptions used to prove asym ptotic norm ality of m aximum likelihood estim ates[J]. Ann.Statistics,1970,41:802-828.

[12]M ammen E.Asym ptotics w ith increasing dimension for robust regression w ith app lications to the bootstrap[J].Ann.Statist.,1989,17:382-400.

[13]Nou redd ine E l Karoui,Derek Bean,Peter B ickel,et al.On robust regression w ith high-d im ensional p red ictors[J].Proc.Natl.Acad.Sci.USA,2013,110(36):14557-14562.

[14]Derek Bean,Peter Bickel,Noureddine El Karoui,et al.Penalized Robust Regression in High-Dimension[C]. Berkeley:Technical Reports of Departm en t of Statistics University of California,2011.

[15]Derek Bean,Peter Bickel,Noureddine El Karoui,et al.Op timal Ob jective Function in High-Dimensional regression[C].Berkeley:Technical Reports of Department of Statistics University of California,2011.

[16]Sculley D.Combined regression and ranking[J].Washington,DC,USA,K.D.D.,2010,10:25-28.

[17]W ei X iaoqiao.regression-based forecast com bination m ethods[J].Rom anian Jou rnal for Econom ic Forecasting,2009(4):5-18.

[18]Fan Yanqin.Asym p totic Normality of a combined regression estimator[J].Journal ofM u ltivariate Analysis, 1999,71:191-240.

[19]Enrique Castillo,Carm en Castillo,A li S Had i,et al.Combined regression m od les[J].Com pu t.Stat., 2009,24:37-66.

[20]Rosasco E,De V ito A,Caponnetto M,et al.Are loss functions all the same?[J].Neural.Com put., 2003,16(5):1063-1076.

[21]Akaike H.In formation Theory and an Extension of the Mmaximum Likelihood Princip le[D].Budapest: Akadem iai K iado,1973.

[22]Schwarz G.Estimating the dimension of amodel[J].Ann Stat.,1978,6:461-464.

[23]Xu Zongben,Zhang Hai,W ang Yao,et al.L1/2Regu larization[J].Sci.China In f.Sci.,2010,53:1159-1169.

[24]Zhang Hai,Liang Yong,Gou HaiLiang,et al.The essential ability of sparse reconstruction of dif erent com p ressive sensing strategies[J].Sci.China In f.Sci.,2012,55:2582-2589.

[25]Xu Zongben,Chang X iangyu,Xu Fengm in,et al.L1/2Regu larization:a th reshold ing rep resentation theory and a fast solver[J].Neu ral Networks and Learning System s.2012,23(7):1013-1027.

[26]Tibshirani R.Regression shrinkage and selection via the Lasso[J].Journal of the Royal Statistical Society Series,1996,58(1):267-288.

[27]M allow s C L.Som e comm ents on Cp[J].Technom etrics,1973,15:661-675.

[28]Peter Buhlmann Sara van de Geer.Statistics for High-Dimensional Data[M].New York:Springer,2010.

[29]Fan JQ,LiR Z.Variab le selection via nonconcave penalized likelihood and its oracle p roperties[J].Journal of the American Statistical Association,2001,31:1348-1360.

The statistical analysis of the com bined loss of
LS,LAD in h igh-d im ension

Zhang Lingjie,Su Meihong,Zhang Hai

(Department of Mathematics,Northwest University,X i′an 710127,China)

This article studies a convex combination of the Least Squares(LS)and Least Absolute Deviation(LAD).By studying the robust statistical properties of high-dimensional and penalized robust statistical p roperties of high d im ension when the number of observations n and the num ber of p rediction p tends to inf nitythe exp ressions of robust estim ation and penalized robust estim ation are obtained.The

result reveals that the loss function model of convex combination combines the advantages of the LSand LAD, at the same time,it relatively weakens their shortcom ings,thus it has excellent high dimensional statistical p roperties.

linearmodel,high dimension,robust estimation,penalized robust estimation, convex combination of LS+LAD

O23

A

1008-5513(2013)05-0536-08

2013-05-16.

國家自然科學基金(11171272).

張凌潔(1986-),碩士生,研究方向:機器學習.

2010 MSC:94A 15