Predictive Algorithm Engine

Analyzing the Past to Predict the Future
Using predictive analytics to forecast key resource utilization patterns has the potential to change the IT world’s approach to problem prevention and capacity planning.    The benefit to  IT departments is that it allows them to leverage historical availability and performance data to anticipate future events.   It is the “crystal ball” of problem prevention, allowing IT departments to be truly proactive instead of reactive.

The Science of Predictive Analytics
Predictive analytics uses historical measurements to predict future trends, probabilities and behavior patterns.   The two basic elements of predictive analytics are predictors and predictor groups.   A predictor is a metric that can be compared against itself historically or against a pre-selected group of related metrics to predict future behavior patterns.  Groups of selected metrics are combined and analyzed as a single entity, called prediction groups.   When subjected to mathematical analysis, the prediction group can be used to forecast future events with a high level of reliability. In predictive modeling, data is collected, a statistical model using mathematical constructs is created, predictions are generated and the output is analyzed and revised as additional data becomes available.  This iterative process is required to continuously improve the algorithms used to generate the predictions. 

Preventing IT Firefighting using Historical Trending and Predictive Analysis
Remote DBA Expert’s monitoring architecture collects and stores key resource utilization indicators daily in a historical database repository.  Historical tracking allows the team to track past usage spikes to facilitate problem analysis activities.  It also allows Remote DBA Experts Proactive Monitoring and Response Center team (PMRC) to forecast when performance will be good and provide a possible reason when performance degrades.   The customer’s knowledge of usage spikes allows them to better understand why performance degrades during a particular time period and prevents them from trying to solve the same performance problem on a regular basis.

Remote DBA Experts Predictive Algorithm Engine
Early last year, Remote DBA  Experts initiated a project that leverages historical data to  predict when key resources will become over-utilized.   It became readily apparent that existing technologies could not be utilized to capture the variety of patterns generated by key resource indicators.   For example, although the chart above can be effectively used to review historical and current utilizations, the patterns are useless for forecast analysis.  Mathematical algorithms are required to filter the raw data points to allow a clearer picture of future behavior.   Remote DBA Experts coined the term “Predictive Algorithm Engine” to name the new product architecture that would be designed to forecast future resource utilizations.

Led by a Carnegie Mellon Mathematician, Remote DBA Experts’ PMRC team is currently implementing advanced predictive algorithms to forecast availability and performance problems which will allow RDBAE to take proactive action to prevent their occurrence. Utilizing historical data as input, the mathematical constructs and algorithms analyze the data to project the utilization of key availability and performance indicators weeks and months in advance.

The example below shows a very basic prediction.   The prediction is from a production environment that was generated in mid-year of 2010.  The historical data was supplied as input to the Predictive Algorithm Engine to generate a projected failure date and time at the end of December which was months (and months) in the future.

Custom Metric Collection and Analysis
Remote DBA Experts is also able to work with customers to create a custom projection/patterning for additional monitoring criteria to show both projected growth and possible usage patterns that may be affecting them.       Representatives from the Remote DBA Experts’ PMRC team meet with customers to analyze their needs and determine if the existing algorithms deployed in the Predictive Algorithm Engine can be used to accommodate the new custom projections. If new algorithms are required, PMRC mathematicians will analyze the metrics and customer requirements to determine the scope of the work required.  The analysis will determine how much historical data is required, the algorithms utilized and the quality of the projection.