DSC/e Lecture Series - October 5: Visualization of big time series data

It is becoming increasingly common for organizations to collect very large amounts of data over time. Data visualization is essential for exploring and understanding structures and patterns, and to identify unusual observations. However, the sheer quantity of data available challenges current time series visualization methods. One approach to this problem is to compute a vector of features on each time series, measuring characteristics of the series. For example, the features may include lag correlation, strength of seasonality, spectral entropy, etc. Then we use a principal component decomposition on the features, and plot the first few principal components. This enables us to explore a lower dimensional space and discover interesting structure and unusual observations.

I will demonstrate the ideas by identifying unusual behavior on web mail servers, finding characteristics of time series that are difficult to forecast, and generating new time series unlike any seen before.