An Introduction to Time Series Modeling

CONTENTS 

Preface 7 

Abbreviations 9 

Notational conventions 11 

CHAPTER 1 Introduction 13 

CHAPTER 2 Stochastic vectors 21 

2.1  Introduction 21 

2.2  Stochastic vectors 23 

2.2.1  Properties and peculiarities 23 

2.2.2  Conditional expectations 26 

2.2.3  Normal distributed vectors 29 

2.2.4  Linear projections of Normal distributed vectors 31 

2.3 Exercises 35 

CHAPTER 3 Stochastic processes 37 

3.1  Introduction 37 

3.2  Properties and peculiarities 39 

3.2.1  Estimating the mean and the covariance sequence 43 

3.2.2  Vector representation 48 

3.3  The power spectral density 53 

3.4  Filtering of a stochastic process 57 

3.5 The basic linear processes 62 

3.5.1  The moving average process 62 

3.5.2  The autoregressive process 68 

3.5.3  The Levinson-Durbin algorithm⋆ 74 

3.5.4  The ARMA process 78 

3.6  Estimating the power spectral density 82 

3.7  Exercises 89 

CHAPTER 4 Identification and modeling 95 

4.1  Introduction 95 

4.2  Finding an appropriate model structure 97 

4.2.1  The partial autocorrelation function 98 

4.2.2  The inverse autocorrelation function⋆ 102 

4.2.3  The extended sample autocorrelation function⋆ 104 

4.3 Data with trends and seasons 110 

4.3.1  Deterministic trend 111 

4.3.2  Stochastic trend 112 

4.3.3  Constant trend 117 

4.3.4  Seasonaltrend 119 

4.4  Using a transformation to stabilize the variance 123 

4.5  Transfer function models 131 

4.6  Intervention analysis⋆ 146 

4.7  Outliers and robust estimation⋆ 150 

4.8  Exercises 157 

CHAPTER 5 Estimation and testing 159 

5.1  Introduction 159 

5.2  Estimating the unknown parameters 160 

5.2.1  Least squares estimation 162 

5.2.2  Weighted least squares 170 

5.2.3  Prediction error method 172 

5.2.4  Maximum likelihood estimation 175 

5.2.5  The Cramér-Rao lower bound⋆ 179 

5.3 Estimating the model order 186 

5.3.1 Information theoretic models 188 

5.4 Residualanalysis 194 

5.4.1  Testing the estimated ACF and PACF 195 

5.4.2  Testing the cumulative periodogram 197 

5.4.3  Testing for sign changes 198 

5.4.4  Testing if the residual is Normal distributed 199 

5.5  Two modeling examples 200 

5.6  Testing for periodicities⋆ 207 

5.7  Exercises 217 

CHAPTER 6 Prediction of stochastic processes 223 

6.1  Introduction 223 

6.2  Optimal linear prediction 224 

6.3  Prediction of ARMA processes 227 

6.4  Prediction of ARMAX processes 240 

6.5  Prediction of Box-Jenkins processes 245 

6.6  Exercises 247 

CHAPTER 7 Multivariate processes 249 

7.1  Introduction 249 

7.2  Common multivariate processes 251 

7.3  The multivariate Yule-Walker equations 256 

7.4  Identification and estimation 259 

7.5  Maximum likelihood estimation 265 

7.5.1  The case of a known covariance matrix 265 

7.5.2  The case of an unknown covariance matrix 266 

7.6  Multivariate residual analysis 268 

7.7  Robust covariance matrix estimation⋆ 274 

7.8  Exercises 279 

CHAPTER 8 Tracking dynamic systems 281 

8.1  Introduction 281 

8.2  Recursive least squares 282 

8.3  Recursive PEM⋆ 285 

8.4  The linear state space representation 289 

B.5  The Rayleigh distribution 

B.6  The Rice distribution 322 

B.7  The Poisson distribution 322 

B.8  The Student’s t-distribution 323 

B.9  The binomial distribution 323 

B.10  The Wishart distribution 324 

APPENDIX A Some useful formulae 315 

A.1  Matrix inversion lemmas 315 

A.2  Euler’s formula and trigonometric relations 316 

A.3  Kronecker products and the vec operator 317 

A.4  Cauchy-Schwarz inequality 318 

APPENDIX B Probability distributions 319 

B.1  The Normal distributed vectors 319 

B.2  The χ2-distribution 320 

B.3  The Cauchy distribution 320 

B.4  The F-distribution 321 

B.5  The Rayleigh distribution 

B.6  The Rice distribution 322 

B.7  The Poisson distribution 322 

B.8  The Student’s t-distribution 323 

B.9  The binomial distribution 323 

B.10  The Wishart distribution 324 

APPENDIX C Matlab functions 325 

APPENDIX D Exercise solutions 327 

Bibliography 377
 Index 387