On a daily basis, our business metrics may either increase or decrease, and occasionally, we may observe significant spikes or drops. When a change in metric occurs, we evaluate whether it warrants a response or if it is simply an anomaly that we can ignore.

Six years ago, a friend posed a challenging question to me – how to determine which changes in metrics require immediate attention? I had no answer then, and Universal Analytics was of no help since it lacked an anomaly detection tool. 

However, I refused to give up on this problem and spent countless hours researching and studying. Today, I possess a reliable solution that can accurately identify when a change in metrics has occurred and when to take prompt action. 

The solution has been game-changing for my clients, and I guarantee it will be just as beneficial for your business. 

Alright, let’s start and get right into it. I’ll begin with the first fundamental solution.

Compare to Previous Times Span

Comparing the present time span with previous ones is a simple yet essential solution that greatly benefits small and medium-sized businesses. This approach does not require any additional budget allocation or expertise in statistics. Doing so can provide valuable insights and help businesses make informed decisions.

It is also essential to mention that you can compare the current time period with the same time span from this year, a year, or two years ago. If your business experiences seasonality, there is no way of comparing your high season with the lowest season.

Note: There are profound ways to remove seasonality from metrics, but it’s not a part of this essay. Do you want to learn them? Reply in the comments.

Most analytics solutions (Google Analytics 4, Google Data Studio, Mixpanel, Amplitude) offer this functionality by default. Nowadays, finding a tool that doesn’t offer it is rather difficult. 

Boost Your Business with Data-Driven Marketing Solutions

Analytics Implementation

Level up your analytics to track every funnel step with precision and drive better results

Data Analysis

Uncover actionable insights and optimize every step of your business journey

CRO

Unleash the power of CRO and run experiments to boost conversions and revenues.

Contact Us

Over 90 satisfied clients & counting

Moving Average or Rolling Average

The second solution you can use to determine if there is a trend in the metric direction that will clearly tell you if you need to take any action is the moving average or rolling average.

According to Wikipedia, the moving average is a calculation to analyze data points by creating a series of averages of different selections of the full data set.

This approach is commonly used with time series data to smooth out short-term fluctuations and highlight longer-term trends or cycles. 

In order to calculate it, just add up a set of recent data points and divide the sum by the number of time periods. 

The rolling average is often confused with the moving average. While the rolling average is the average of all values available, the moving average is the average of the last set number of values.

In the table below, I demonstrate how to calculate these two metrics in Google Sheets or Microsoft Excel.

One of the advantages of using a moving average is that it helps to identify the exact time when a change started to occur. This feature enables you to revert to the original solution mentioned in this article, compare two different time periods, and identify the root cause that led to the change in direction.

Some analytics tools offer moving or rolling averages features, while others don’t. For instance, Mixpanel, Tableau, and Google Looker Studio offer it, while Google Analytics 4 doesn’t have it. 

Normal Distribution: Anomaly Detection

The third solution uses statistics to detect anomalies, a proven and practical approach. By leveraging normal distribution, it’s possible to find subtle irregularities that may otherwise go undetected. This method is a powerful tool for identifying patterns and outliers that can provide valuable insights and drive impactful decisions.

In time series and other statistical analyses, the normal distribution is the most commonly assumed type of distribution. The standard normal distribution has two parameters – the mean and the standard deviation.

Calculating the mean and the standard deviation for the metric period allows us to determine whether the spike or drop in the metric was an anomaly. 

To determine the anomaly, you should do the following:

• Calculate the mean or average of the metric values for the observed period

• Calculate the standard deviation of the metric values for the observed period

• Calculate a 95% confidence interval (boundaries) to determine outliers (anomalies) by the following formulas:
  • First boundary: Mean – 2 * standard deviation
  • Second boundary: Mean + 2 * standard deviation

After doing the steps above, you will get two additional chart lines that will allow you to see clearly if the metric value was an anomaly or not. 

Some analytics tools offer anomaly detection features, while others don’t. For instance, Google Analytics 4 and Tableau offer it, while Mixpanel doesn’t have it.

Other options

Although there are other more advanced techniques to detect anomalies, they require the knowledge and experience of using statistics.

A few of them are STL decomposition, auto-regressive integrated moving average (ARIMA), classification and regression trees (CART), detection using forecasting, and others. 

Most of these solutions can be done in Python or R and couldn’t be achieved in Google Sheets or Microsoft Excel. Therefore, this article doesn’t cover them.

Boost Your Business with Data-Driven Marketing Solutions

Analytics Implementation

Level up your analytics to track every funnel step with precision and drive better results

Data Analysis

Uncover actionable insights and optimize every step of your business journey

CRO

Unleash the power of CRO and run experiments to boost conversions and revenues.

Contact Us

Over 90 satisfied clients & counting

Takeaway

Managing a business involves tracking various metrics that fluctuate on a daily basis. It can be challenging to decide when to act upon these fluctuations and when to focus on other business tasks. 

This article provides a solution for SMBs to identify outliers or anomalies in their time-series data within 30 minutes and take appropriate actions. 

If you know of any other ways that SMBs can detect anomalies in their data that are not covered in this article, please feel free to share your thoughts in the comments section below.