An Introductitoon ComputationLaela rninTgh eory CopyrigMhatteedr ial An Introductitoon Computational LeaTrhneionrgy MichaeKle aJr.n s UmeshV .V azirani TheM IT Press CambridgMea,s sachusetts LondonE,n gland CopyrigMhatteedr ial @1994M assachuseItntsst itoufTt eec hnology Allr ighrtess ervNeod p.a rotf t hibso okm ayb er eproduced ina ny forbmy a nye lectroornm iecc hanimceaaln s (incplhuodtioncgo pying, recordoirni gn,f ormation asntdro ertargiee wviatlhu)ot p ermission in writifnrgo mt hep ublisher. Thibso okw asty pesetb yt hea uthoarnsd w as printaenddb ounidn theU niteSdt atoefsA merica. LibraorfyC ongreCsast aloging-in-PDuabtlai cation KearnsM, ichaJe.l An introducttoic oonm putatiolneaalrn itnhge or/y M ichaelJ . KearnUsm,e shV .V azira.ni. p. cm. Includbeisb Hographriecfaelr enacnedis n dex. ISDN0 -262-11193-4 1.M achinlee arnin2g.A. r tificiinatle llig3e.nAc leg.o rithms. 4.N eura.nle tworkIs..V aziraUlm1eis, hV irkuma.rI.I T.i tle. Q325.5.K14949 4 006.3-dc20 94-16588 CIP 109876 Copyrighted Material Contents Preface xi 1 The ProbablAyp proximateCloyr recLte arninMgo del 1 1.1A Rectangle Learning Game 1 1.2A General Model 6 121 Definitoifot nh eP ACM odel 7 . . 1.2.R2e presentSaitziaeon ndI nstaDnicmee nsion 12 1.3L earniBnogo leCaonn junctions 16 1.4I ntractaobfiL leiatryn i3n-Tge rmD NF Formulae 18 1.5U singC N3F Formultaeo A v oiIdn tractability 22 - 1.6E xercises 26 17. BibliogrNaoptheisc 28 2 Occam' 8R azor 31 2.1O ccamL earnianngdS uccinctness 33 CopyrigMhatteedr ial Contents vi 2.2I mprovtihnegS amplSei zfeo Lre arngi Cnonjunctions3 7 2.3L earngi CnonjunctiwointsFh e wR elevaVnatr iables 38 2.4L earngi DnecisLiiosnt s 42 2.5E xercises 44 2.6B ibliogrNaoptheisc 46 3 TheV apnik-ChervonDeinmkeinss ion 49 3.1W henC anI nfiniCtlases esB eL earnweidt ah F iniStaem ple4?9 3.2T heV apnik-Chervonenkis Dimension 50 3.3E xamploefts h eV C Dimension 51 3.4A PolynomBioalu nod nl 11c(8)1 54 3.5A PolynomBioauln odn t heS ampleS izfeo PrA CL earni5n7g 3.5.T1h eI mportaonfcf e- Nets 57 3.5.A2 Smallf -Neftr oRamn domS ampling 59 3.6S amplSei zLeo weBro unds 62 3.7A n ApplicattoiN oenu rNale tworks 64 3.8E xercises 67 3.9B ibliogrNaopthiesc 70 4 Weaka ndS trong Learning 13 4.1A RelaxDeedfi nitoifoL ne arning? 73 4.2 Boostgit nheC onfidence 76 CopyrigMhatteedr ial Contents vii 4.3Bo ostign theA ccuarcy 78 4.3.1 79 A ModesAtcc uracByo ostiPnrgo cedure 4.3.2 81 ErroArn alysfiotsrh eM odesPtr ocedure 4.3.A 3Re cusriveA ccuraBcoyo tsingA lgorithm 85 4.3.4 88 Boundintgh eD eptohf t heR ecurns io 4.3.5 AnalyosfiF si elrtinEgffi ciency 89 4.3.6 FinishUipn g 96 101 4.4E xercises 4.5B iliobgraphic Notes 102 5 Learningt hiePn r esencoefN oise 103 104 5.1T heC lassificNaotiisoMeno del 5.2A n AlgoritfhoLmre arniCnogn junctfiroonSmst a tistic1s 06 5.3 108 TheS tatisQtuiecryaL le arniMnogd el 5.4 SimulatSitnagt isQtuiecrailie nts h eP resenocfNe o ise 111 5.4.1 A NicDee cmoposiitono fP x. 112 5.4.2 Solving foarn E stimaotfPe x. 114 5.4.3 115 GuessianngdV erifytihneNg o isRea te 5.4.D4e scrpitoin of timhuela tioSnA logrithm 117 5.5 119 Exerciess 5.6 BibliogrNaoptheisc 121 CopyrigMhatteedr ial Contents viii 6 InherUennptr edictability 123 6.1 RepresentDaetpieonnd eanntdI ndependent Har1d2n3e ss 6.2 TheD iscrCeutbee R ootP roblem 124 6.2.1T heD ifficulotfyD isrcetCeu beR oots 126 6.2.2D iscrCeutbee R ootass a Learning Problem 128 6.3 SmalBlo oleCainrc uitsA reI nherenUtnplryed ictable 131 6.4 ReducitnhgeD eptohf In herentlyU npredcitableC ircui1t3s3 6.4.1E xpadningt heI nput 135 6.5 A GenerMaelt hoadn dI tAsp plicattioNo enu arlN etwor1ks3 9 6.6 Exercises 140 6.7 Biblogiraphic Notes 141 7 ReducibiliinPt Ay C Learning 143 7.1R educing DNFt oM onotoDnNeF 144 7.2A GeneralM ethdo foRre ducibility 147 194 7.3R educiBnogo leFaonr multaoFe i niAtueto mata 7.4E xercises 153 7.5B ibliogrNaoptheisc 154 8 LearningF initAeu tomatbay Experimentation 155 8.1A ctiavned P assivLee raning 155 8.2 ExacLte arniUnsgi nQgu ersi e 158 CopyrigMhatteedr ial Contents ix 8.3E xacLte arnionfFg in ite Automata 160 8.3.1Ac cessS tringsa ndD istniguishingS trings 160 8.3.2A n Efficientyl ComputbaleS tatPea rtition 162 8.3.3T heT entatiev Hypotheits is 164 8.3.4U snig Cao unterexample 166 8.3.5T heA lgoriftohLrme arning Finite Automata 169 8.3.6R unninTgim e Analyiss 171 8.4L earniwnigt hoau Rte set 174 8.4.1U sinag H oming SequteoLn ecaer n 176 8.4.2B uildnig Ha omingS equenUcsei nOgv ersiGzeend- eralizCeldas sificaTrteieosn 178 8.4.3T heN o-ResAelgto rithm 181 8.4.4M akinSgu rLe(1 B uilGdesn eralCizleasds ification Trees 182 8.5E xericses 185 8.6B ibliogrNaoptheisc 186 9 AppendiSxo:m eT oolfso Prr obabilAinsatliycs is 189 9.1T heU nioBno und 189 9.2M arkovI'nse quality 189 9.3C hernoffB ounds 190 CopyrigMhatteedr ial x Contents Bibliography 193 Index 205 CopyrigMhatteedr ial Preface Int heF allt ermo f1 990w,e j ointtalugyh ta graduatsee minianrc om putatiolneala rntihnego riynt hec omputsecri endceep artmoefnt th e UniversoiftCy a lifoartnB ierake leTyhem atertihaalit sp resenhteerde . has itosr igiinnts h at coursbeo,t hi nc ontneta ndex positRiaotnh.e r thana ttemtpotg ivea ne xhaustoivveer vioeftw h irasp idleyx panding andc hangianrgeo afr esearwcehh a,v ter iteodca reufllys elefcutn damen tatlo pitchsa dte monstriamtpeo rtparnitn citphlatem sa yb ea plpicable ina widesre ttitnhga nt heo neex am inde hereI.n t het echnisceacl tionwse,h avet reidt oe mphasiiznet uitwihoenn evpeors siwbhliels,et ill providpirnegc iasreg uments. Theb ooiks i nteenddf orre searcahnedsr tsu deinnta sr tifiicnitaell li gencen,e uraneltw orks,t heorectoimcpault secri enancdes tatisatnidc s, anyoneel sien tereisntm eadt hematmiocdaell osfl e arinng Iti sa ppro . priatef oru seas thec entrtaelxi tn a specialsiemziendca ourr s,eo ras a supplemteenxtitan la broader courtshea pte hrapsa lssot uditehse viewpoitnatkseb ny a rtificinitaellgle ince andn eurnaelt worWkhsi.l e Chapt1el ra yasc ommofno undatfioaorln tl h esu bsequmeantte rtiahle, latchearp terasr ee ssentisaelllfy- conatnadim naeydb er easde lectively andi nan y ordeErx.e rciasreeprs ov ided att hee ndo fe acchh apter. Someb riceofm menotsnt hee xpectbeackdgr onud oft her eadaerre approprhiearteFe.a miliawritihts yo mbeas itco oolfst hfeo rmaanla lysis ofa lgoritish mecnses sar,yas isf amiliawriittohyn ly them oste lemen tarnyo tioonfsc omplextihetoryy, s ucash NP-completeFnoerts hse. CopyrigMhatteedr ial

An Introduction to Computational Learning Theory PDF

215 Pages

1994

24.051 MB

English

by Michael J. Kearns, Umesh Vazirani

#

Checking for file health...

Preview An Introduction to Computational Learning Theory

An Introductitoon ComputationLaela rninTgh eory CopyrigMhatteedr ial An Introductitoon Computational LeaTrhneionrgy MichaeKle aJr.n s UmeshV .V azirani TheM IT Press CambridgMea,s sachusetts LondonE,n gland CopyrigMhatteedr ial @1994M assachuseItntsst itoufTt eec hnology Allr ighrtess ervNeod p.a rotf t hibso okm ayb er eproduced ina ny forbmy a nye lectroornm iecc hanimceaaln s (incplhuodtioncgo pying, recordoirni gn,f ormation asntdro ertargiee wviatlhu)ot p ermission in writifnrgo mt hep ublisher. Thibso okw asty pesetb yt hea uthoarnsd w as printaenddb ounidn theU niteSdt atoefsA merica. LibraorfyC ongreCsast aloging-in-PDuabtlai cation KearnsM, ichaJe.l An introducttoic oonm putatiolneaalrn itnhge or/y M ichaelJ . KearnUsm,e shV .V azira.ni. p. cm. Includbeisb Hographriecfaelr enacnedis n dex. ISDN0 -262-11193-4 1.M achinlee arnin2g.A. r tificiinatle llig3e.nAc leg.o rithms. 4.N eura.nle tworkIs..V aziraUlm1eis, hV irkuma.rI.I T.i tle. Q325.5.K14949 4 006.3-dc20 94-16588 CIP 109876 Copyrighted Material Contents Preface xi 1 The ProbablAyp proximateCloyr recLte arninMgo del 1 1.1A Rectangle Learning Game 1 1.2A General Model 6 121 Definitoifot nh eP ACM odel 7 . . 1.2.R2e presentSaitziaeon ndI nstaDnicmee nsion 12 1.3L earniBnogo leCaonn junctions 16 1.4I ntractaobfiL leiatryn i3n-Tge rmD NF Formulae 18 1.5U singC N3F Formultaeo A v oiIdn tractability 22 - 1.6E xercises 26 17. BibliogrNaoptheisc 28 2 Occam' 8R azor 31 2.1O ccamL earnianngdS uccinctness 33 CopyrigMhatteedr ial Contents vi 2.2I mprovtihnegS amplSei zfeo Lre arngi Cnonjunctions3 7 2.3L earngi CnonjunctiwointsFh e wR elevaVnatr iables 38 2.4L earngi DnecisLiiosnt s 42 2.5E xercises 44 2.6B ibliogrNaoptheisc 46 3 TheV apnik-ChervonDeinmkeinss ion 49 3.1W henC anI nfiniCtlases esB eL earnweidt ah F iniStaem ple4?9 3.2T heV apnik-Chervonenkis Dimension 50 3.3E xamploefts h eV C Dimension 51 3.4A PolynomBioalu nod nl 11c(8)1 54 3.5A PolynomBioauln odn t heS ampleS izfeo PrA CL earni5n7g 3.5.T1h eI mportaonfcf e- Nets 57 3.5.A2 Smallf -Neftr oRamn domS ampling 59 3.6S amplSei zLeo weBro unds 62 3.7A n ApplicattoiN oenu rNale tworks 64 3.8E xercises 67 3.9B ibliogrNaopthiesc 70 4 Weaka ndS trong Learning 13 4.1A RelaxDeedfi nitoifoL ne arning? 73 4.2 Boostgit nheC onfidence 76 CopyrigMhatteedr ial Contents vii 4.3Bo ostign theA ccuarcy 78 4.3.1 79 A ModesAtcc uracByo ostiPnrgo cedure 4.3.2 81 ErroArn alysfiotsrh eM odesPtr ocedure 4.3.A 3Re cusriveA ccuraBcoyo tsingA lgorithm 85 4.3.4 88 Boundintgh eD eptohf t heR ecurns io 4.3.5 AnalyosfiF si elrtinEgffi ciency 89 4.3.6 FinishUipn g 96 101 4.4E xercises 4.5B iliobgraphic Notes 102 5 Learningt hiePn r esencoefN oise 103 104 5.1T heC lassificNaotiisoMeno del 5.2A n AlgoritfhoLmre arniCnogn junctfiroonSmst a tistic1s 06 5.3 108 TheS tatisQtuiecryaL le arniMnogd el 5.4 SimulatSitnagt isQtuiecrailie nts h eP resenocfNe o ise 111 5.4.1 A NicDee cmoposiitono fP x. 112 5.4.2 Solving foarn E stimaotfPe x. 114 5.4.3 115 GuessianngdV erifytihneNg o isRea te 5.4.D4e scrpitoin of timhuela tioSnA logrithm 117 5.5 119 Exerciess 5.6 BibliogrNaoptheisc 121 CopyrigMhatteedr ial Contents viii 6 InherUennptr edictability 123 6.1 RepresentDaetpieonnd eanntdI ndependent Har1d2n3e ss 6.2 TheD iscrCeutbee R ootP roblem 124 6.2.1T heD ifficulotfyD isrcetCeu beR oots 126 6.2.2D iscrCeutbee R ootass a Learning Problem 128 6.3 SmalBlo oleCainrc uitsA reI nherenUtnplryed ictable 131 6.4 ReducitnhgeD eptohf In herentlyU npredcitableC ircui1t3s3 6.4.1E xpadningt heI nput 135 6.5 A GenerMaelt hoadn dI tAsp plicattioNo enu arlN etwor1ks3 9 6.6 Exercises 140 6.7 Biblogiraphic Notes 141 7 ReducibiliinPt Ay C Learning 143 7.1R educing DNFt oM onotoDnNeF 144 7.2A GeneralM ethdo foRre ducibility 147 194 7.3R educiBnogo leFaonr multaoFe i niAtueto mata 7.4E xercises 153 7.5B ibliogrNaoptheisc 154 8 LearningF initAeu tomatbay Experimentation 155 8.1A ctiavned P assivLee raning 155 8.2 ExacLte arniUnsgi nQgu ersi e 158 CopyrigMhatteedr ial Contents ix 8.3E xacLte arnionfFg in ite Automata 160 8.3.1Ac cessS tringsa ndD istniguishingS trings 160 8.3.2A n Efficientyl ComputbaleS tatPea rtition 162 8.3.3T heT entatiev Hypotheits is 164 8.3.4U snig Cao unterexample 166 8.3.5T heA lgoriftohLrme arning Finite Automata 169 8.3.6R unninTgim e Analyiss 171 8.4L earniwnigt hoau Rte set 174 8.4.1U sinag H oming SequteoLn ecaer n 176 8.4.2B uildnig Ha omingS equenUcsei nOgv ersiGzeend- eralizCeldas sificaTrteieosn 178 8.4.3T heN o-ResAelgto rithm 181 8.4.4M akinSgu rLe(1 B uilGdesn eralCizleasds ification Trees 182 8.5E xericses 185 8.6B ibliogrNaoptheisc 186 9 AppendiSxo:m eT oolfso Prr obabilAinsatliycs is 189 9.1T heU nioBno und 189 9.2M arkovI'nse quality 189 9.3C hernoffB ounds 190 CopyrigMhatteedr ial x Contents Bibliography 193 Index 205 CopyrigMhatteedr ial Preface Int heF allt ermo f1 990w,e j ointtalugyh ta graduatsee minianrc om putatiolneala rntihnego riynt hec omputsecri endceep artmoefnt th e UniversoiftCy a lifoartnB ierake leTyhem atertihaalit sp resenhteerde . has itosr igiinnts h at coursbeo,t hi nc ontneta ndex positRiaotnh.e r thana ttemtpotg ivea ne xhaustoivveer vioeftw h irasp idleyx panding andc hangianrgeo afr esearwcehh a,v ter iteodca reufllys elefcutn damen tatlo pitchsa dte monstriamtpeo rtparnitn citphlatem sa yb ea plpicable ina widesre ttitnhga nt heo neex am inde hereI.n t het echnisceacl tionwse,h avet reidt oe mphasiiznet uitwihoenn evpeors siwbhliels,et ill providpirnegc iasreg uments. Theb ooiks i nteenddf orre searcahnedsr tsu deinnta sr tifiicnitaell li gencen,e uraneltw orks,t heorectoimcpault secri enancdes tatisatnidc s, anyoneel sien tereisntm eadt hematmiocdaell osfl e arinng Iti sa ppro . priatef oru seas thec entrtaelxi tn a specialsiemziendca ourr s,eo ras a supplemteenxtitan la broader courtshea pte hrapsa lssot uditehse viewpoitnatkseb ny a rtificinitaellgle ince andn eurnaelt worWkhsi.l e Chapt1el ra yasc ommofno undatfioaorln tl h esu bsequmeantte rtiahle, latchearp terasr ee ssentisaelllfy- conatnadim naeydb er easde lectively andi nan y ordeErx.e rciasreeprs ov ided att hee ndo fe acchh apter. Someb riceofm menotsnt hee xpectbeackdgr onud oft her eadaerre approprhiearteFe.a miliawritihts yo mbeas itco oolfst hfeo rmaanla lysis ofa lgoritish mecnses sar,yas isf amiliawriittohyn ly them oste lemen tarnyo tioonfsc omplextihetoryy, s ucash NP-completeFnoerts hse. CopyrigMhatteedr ial

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.