How to Calculate the Standard Error of Mean in Python

To calculate the standard error of the mean (SEM) in Python, use scipy library’s sem() function.

For instance, let’s calculate the SEM for a group of numbers:

from scipy.stats import sem

# Create a dataset
data = [19, 2, 12, 3, 100, 2, 3, 2, 111, 82, 4]

# Calculate the standard error of mean
s = sem(data)

print(s)

Output:

13.172598656753378

If you do not have scipy installed, run:

pip install scipy

That was the quick answer. But make sure to read along to learn about the standard error and how to implement the function yourself.

What Is the Standard Error of Mean (SEM)

The standard error of the mean (SEM) is an estimate of the standard deviation.

The SEM is used to measure how close sample means are likely to be to the true population mean. This gives a good indication as to where a given sample actually lies in relation to its corresponding population.

The standard error of the mean follows the following formula:

Where σ is the standard deviation and n is the number of samples.

How to Implement Standard Error of Mean Function in Python

To write a function that calculates the standard error of the mean in Python, you first need to implement a function that calculates the standard deviation of the data.

What Is Standard Deviation

Standard deviation is a measure of how far numbers lie from the average.

For example, if we look at a group of men we find that most of them are between 5’8” and 6’2” tall. Those who lie outside this range make up only a small percentage of the group. The standard deviation identifies the percentage by which the numbers tend to vary from the average.

The standard deviation follows the formula:

Where:

$\sigma$ = sample standard deviation
= the size of the population
= each value from the population
$\mu$ = the sample mean (average)

How to Calculate Standard Deviation in Python

Assuming you do not use a built-in standard deviation function, you need to implement the above formula as a Python function to calculate the standard deviation.

Here is the implementation of standard deviation in Python:

from math import sqrt

def stddev(data):
    N = len(data)
    mu = float(sum(data) / len(data))
    s = [(x_i - mu) ** 2 for x_i in data]

    return sqrt(float(sum(s) / (N - 1)))

The Standard Error of Mean in Python

Now that you have set up a function to calculate the standard deviation, you can write the function that calculates the standard error of the mean.

Here is the code:

def sem(data):
    return stddev(data) / sqrt(len(data))

Now you can use this function.

For example:

data = [19, 2, 12, 3, 100, 2, 3, 2, 111, 82, 4]
sem_data = sem(data)

print(sem_data)

Output:

13.172598656753378

To verify that this really is the SEM, use a built-in SEM function to double-check. Let’s use the one you already saw in the introduction:

from scipy.stats import sem

# Create a dataset
data = [19, 2, 12, 3, 100, 2, 3, 2, 111, 82, 4]

# Calculate the standard error of mean
s = sem(data)

print(s)

As a result, you get the same output as the custom implementation yielded.

13.172598656753378

This completes our example of building the functionality for calculating the standard error of the mean in Python.

Here is the full code used in this example for your convenience:

from math import sqrt

def stddev(data):
    N = len(data)
    mu = float(sum(data) / len(data))
    s = [(x_i - mu) ** 2 for x_i in data]
    return sqrt(float(sum(s) / (N - 1)))

def sem(data):
    return stddev(data) / sqrt(len(data))

data = [19, 2, 12, 3, 100, 2, 3, 2, 111, 82, 4]
sem_data = sem(data)

print(sem_data)

This is the hard way to obtain the standard error of the mean in Python.

Usually, when you have a common problem, you should rely on using existing functionality as much as possible.

Let’s next take a look at the two ways to find the standard error of mean in Python using built-in functionality.

How to Use Existing Functionality to Calculate the Standard Error of Mean in Python

Standard Error of Mean Using Scipy

You have seen this approach already twice in this guide.

The scipy module comes in with a built-in sem() function. This directly calculates the standard mean of error for a given dataset.

For instance:

from scipy.stats import sem

# Create a dataset
data = [19, 2, 12, 3, 100, 2, 3, 2, 111, 82, 4]

# Calculate the standard error of mean
s = sem(data)

print(s)

Output:

13.172598656753378

Standard Error of Mean Using Numpy

You can also use NumPy module to calculate the standard error of the mean in Python.

However, there is no dedicated sem() function in numpy. But there is a function called std() that calculates the standard deviation.

So, to calculate the SEM with NumPy, calculate the standard deviation and divide it by the square root of the data size.

For example:

import numpy as np

data = [19, 2, 12, 3, 100, 2, 3, 2, 111, 82, 4]
sem_data = np.std(data, ddof=1) / np.sqrt(np.size(data))

print(sem_data)

Output:

13.172598656753378

Conclusion

Today you learned how to calculate the standard error of the mean in Python.

To recap, the standard error of the mean is an estimate of the standard deviation of all samples that could be drawn from a particular population.

To calculate the SEM in Python, you can use scipy‘s sem() function.

Another way to calculate SEM in Python is by using the NumPy module. But there is no direct sem() function there. Thus you need to use the standard deviation and the equation of SEM.

The laborious approach to find the SEM is to implement the sem() function yourself. To do this, you need to implement the functionality to calculate the standard deviation first. Then the rest is simple.

Thanks for reading. Happy coding!