Most of the methods that we have discussed so far assume that each sample in a data set is entirely independent in space and time. That is to say that we assume that the prior state of a variable does not influence its current or future state. A data set of this type is said to be serially uncorrelated. Perhaps the simplest example of this is a coin toss experiment. If we are using a fair coin, then each flip has an equally likely chance of being a head or tail. Moreover, the state of the last toss (heads or tails) has no impact on the next. In the real world however, many processes of interest are in fact serially (or temporally) correlated, and thus exhibit systematic behavior in time or space or both. Consider for example, the energy of an earthquake that propagates away from the eppicenter, or fuel prices that may exhibit systematic seasonal changes (high heating costs in winter - low heating costs in summer in northern climates).
The study of processes of this type requires an entirely different branch of statistics that is referred to as Time Series Analysis (TSA). TSA is a tremendously broad topic of study that is employed by fields ranging from business and finance to the social sciences, engineering, and the life and biological sciences. TSA methods can be bivariate (measurements of one variable as a function of time) or multivariate (multiple variables as a function of time). It is also important to point out that in general TSA methods can be applied to any series of data that is correlated. The independent variable need not be time.
A time dependent process can be either finite or periodic. A finite process is one that has a beginning and end, such as an earthquake, explosion, or an animal call. A periodic process is one that repeats over and over. We will touch on some basic methods applied to periodic (cyclic) processes in the remainder of the class. Time series analysis of finite processes is a topic beyond the scope of this class.
Perhaps the most fundamental time series question that we can ask is: "How interdependent are the successive samples in the record?" To use some fancy time series jargon, you could also say "How persistent is the process that generates the record?"
To answer questions of this type, we can use a variation of correlation analysis that is referred to as "serial correlation analysis" or "autocorrelation". Let's assume that we have a time dependent process g(t) that is sampled at t=1, 2, 3, 4 ... n. The serial correlation or autocorrelation function of a time series is determined by finding the linear correlation coefficient of the time series compared against successively lagged copies of itself. Because this analysis is based on the correlation coefficient, its values will range from -1.0 to 1.0. If we compare the time series against itself, (the zero lag case), the correlation will be exactly equal to 1.0. To find the correlation coefficient for the first lag of the time series, we offset the time series by one time step (or lag) and find the linear correlation between g(t) for t=2,n and g(t-1) for t=1, (n-1). We can continue to offset the time series by an additional time steps to determine the serial correlation for greater lags. Notice that we could form both positive or negative lags depending on the way that we shift the records relative to each other. Keep in mind that each time we increase the lag, the number of points used to calculate the correlation coefficient decreases by two. Because the reliability of the correlation coefficient depends on the sample size, it is not recommended to calculate the autocorrelation function past lags greater than 1/4 or 1/3 of the total record length, N.
If we plot the autocorrelation function for the time series as a function of lag, we can learn quite a bit about it. First, the number of lags that it takes for the autocorrelation function to reach zero correlation is referred to as the decorrelation time scale. If a process is very random, this time scale will be very short. If the process is persistent, this time will be longer. We can use this information to determine the degree of independence of successive samples in the data set. If, for example, a monthly time series has a decorrelation time scale of 3, then there is overlap in the information contained in any three successive samples. This has a profound influence on how we estimate the degrees of freedom of the record. Rather than basing the degrees of freedom (df) on N, the record length, it is more appropriate in this case to determine the effective degrees of freedom (df*) based on N/3. How will this affect the significance of any statistics we calculate for the time series? Second, if the process is periodic, the serial correlation function will cycle from positive to negative and back again. This gives is a measure of the period or length of the cycle in the data.
A related method is called autoregression. Can you think of how autoregression might be similar to autocorrelation analysis and how it might differ? What type of information might we obtain from an autoregression analysis?
Once we have determined that a record may exhibit periodicity by calculating its autocorrelation function, we will want to better characterize that periodicity. To do this, we will need a way to express the time series as the interaction of a number of different processes. That is to say that we want to find some basis set that can be used to represent the original time series. There are a number of possibilities. Perhaps the simplest way to do this is to represent the original time series in terms as a combination of sinusoids. Specifically a combination of sine and cosine waves. This is the basis of Fourier Analysis, which can be used to take a complex time series and decompose it into small number of fundamental sinusoidal components.
Some properties of a wave that will come in handy -
The amplitude of the wave is measured from peak to trough. The period (T) of the wave is time between two wave peaks. The frequency (F) of the wave is the rate at which successive peaks pass a fixed point in space, or the number of cycles per unit time. It is also useful to remember that the period of a a wave is the inverse of its frequency (T = 1/f).
What do we gain by representing a time series as a series of sinusoids? We could calculate the variance of the record as we have done previously, but that is a rather bulk statistic. What happens if the variability in the record arises from two or more independent processes that have very different time scales (frequencies) associated with them. The variance of the record tells us nothing about the relative importance of these processes that operate at different frequencies. Fourier analysis provides us with a way to isolate the variance in the record as a function of sinusoids of differing frequencies. It then is possible to determine the variance with the record as a function of frequency. The results are presented as a power spectrum, a plot of spectral density (variance) as a function of frequency.
Some critical properties of the Fourier Transform model:
Record length, N: This is the total number of samples in the record or time series.
Sampling interval, dt: This is the time step between successive samples in the record. The time series methods that we discuss require a constant sampling interval. If the original sample interval in not constant, then the record must be interpolated. It is best to keep the number of interpolated samples close to the record length N. Interpolation adds noise to the time series; interpolating at much greater than the original record length does not provide new information.
Nyquist frequency, fn: determines the highest frequency of variations that we can resolve in our time series. This is dependent on the sampling interval and is given by fn = 1/(2dt). What this means is that the smallest period wave (highest frequency) that we can resolve has a period that is exactly twice the sample interval (i.e. one sample for the peak of the wave and one for the trough).
The character of a process determines the shape of its power spectra.
If a process is random, then its autocorrelation function is non-cyclic and the magnitude of the lagged correlation rapidly decreases to zero correlation. Such a process has roughly equal amounts of spectral density (variance) at all fourier frequencies. In analogy to the visible part of the electromagnetic spectrum, such a process is said to have a "white" spectrum.
If instead, a process is periodic and persistent, then its autocorrelation function is cyclic and the magnitude of the lagged correlation slowly decreases to zero correlation. Such a process has greater spectral density (variance) at lower frequencies (longer period). In analogy to the visible part of the electromagnetic spectrum, such a process is said to have a "Red" spectrum. Most geophysical processes exhibit Red spectra. Some good examples are the spectra of climate related processes.
If a process is periodic and limited to a single pure frequency or a small range of frequencies, then the power spectra will exhibit discrete spectral peaks. Discrete peaks may be superimposed on either a white or a red background.