Calculate Streaming Mean and Variance

Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Locked
User avatar
drpratten
 
Posts: 3
Joined: Tue May 31, 2022 6:37 am

Calculate Streaming Mean and Variance

Post by drpratten »

A recent project required identifying the mean and variance of a stream of inputs.

Here is the Python function that I used to calculate the mean and variance of the stream. I couldn't find any implementations so I wrote my own using exponential smoothing.

Code: Select all

def streaming_average_and_variance(new_value, existing_aggregate, exponential_decay_over_n_events=1000):
    (old_mean, old_var) = existing_aggregate
    new_mean = ((old_mean * (exponential_decay_over_n_events - 1)) + new_value) / exponential_decay_over_n_events
    new_var = ((old_var * (exponential_decay_over_n_events - 1)) + pow(new_value - new_mean, 2)) / exponential_decay_over_n_events
    return new_mean, new_var
Here is how to call it for each value in

Code: Select all

stream_data

Code: Select all

streaming_statistics = list()
streaming_aggregate = (0, 0)
for value in stream_data:
    streaming_aggregate = streaming_average_and_variance(value, streaming_aggregate, 1000)
    streaming_statistics.append(streaming_aggregate)
All the best,

David

User avatar
jwcooper
 
Posts: 1004
Joined: Tue May 01, 2012 9:08 pm

Re: Calculate Streaming Mean and Variance

Post by jwcooper »

Thank you for providing example code. Others will likely find this helpful in the future.

Locked
Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.

Return to “Internet of Things: Adafruit IO and Wippersnapper”