adix/mvstat

Search:
Group by:
Source   Edit  

Summary stats built in running/online fashion (as std/stats) BUT over (maybe) MOVING data windows (via pop) and (sometimes) 50X faster & a million X more accurate. Speed up comes from SIMD auto-vectorization in whole openArray[] calls (in --passC:-ffast-math|-Ofast backend modes) aided by "shift" idea at en.wikipedia.org/wiki/Algorithms_for_calculating_variance (both simpler & faster than Welford). Both var & non-var overloads are provided to allow caching 1.0/n which may be identical-but-expensive (eg. reporting at each cycle of a 1 pop per push (so fixed n) window over data).

Note: this all costs more in both time & space than exponentially weighted equivalents but has precise rather than infinite memory which can be nice. I.e., it can perfectly "forget" a large spike when it leaves a window.

Types

BasicStats[F] = object
  n*: int                    ## sample size
  min*, max*, mean*, sdev*: F ## the usual suspects.
Result type for parallel-mergable stats Source   Edit   
MovingStat[F; C] = object
  options*: set[Option]
  n*, n4Inv: int             ## amount of pushed data
  min*, max*: F
  lgHisto*: LgHisto[C]
Statistical data accumulator Source   Edit   
Option = enum
  OrderStats
Source   Edit  

Procs

proc `$`[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): string
A string representation of a MovingStat[F,C]. Source   Edit   
func `+`[F](a, b: BasicStats[F]): BasicStats[F]
Operator notation for add. Source   Edit   
func `+=`[F](a: var BasicStats[F]; b: BasicStats[F])
Operator notation for add. Source   Edit   
func add[F](a: var BasicStats[F]; b: BasicStats[F])
Combines two BasicStats. Useful to combine partial result sets. Source   Edit   
func basicStats[F: SomeFloat](xs: openArray[F]): BasicStats[F] {.
    codegenDecl: "__attribute__((optimize(\"Ofast\", \"fast-math\"))) $# $#$#".}
Some summary stats in 1-pass using vector-CPU instructions, if possible. Source   Edit   
func cdf[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]; x: float): float {.
    inline.}
Estimated moving CDF(x) Source   Edit   
func clear[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]) {.inline.}
Reset s to same as var s: MovingStat[F,C]. Source   Edit   
func init[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]; a = 1e-16;
                                        b = 1e+20; n = 8300;
                                        options: set[Option] = {}) {.inline.}
Initialized a MovingStat[F,C] with a [-b,b] log-spaced histogram. Source   Edit   
func initMovingStat[F: SomeFloat; C: SomeInteger](a = 1e-16; b = 1e+20;
    n = 8300; options: set[Option] = {}): MovingStat[F, C] {.inline.}
Return initialized MovingStat[F,C] with a [-b,b] log-spaced histogram. Source   Edit   
func kurtosis[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.inline.}
Moving excess-relative to Gaussian kurtosis (population) over data window. Source   Edit   
func kurtosis[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float {.
    inline.}
Moving excess-relative to Gaussian kurtosis (population) over data window. Source   Edit   
func kurtosis[T: SomeNumber](xs: openArray[T]; accum = 32): float
excess kurtosis (population). accum != 32 => 64-bit accumulation. Source   Edit   
func kurtosisS[F: SomeFloat; C: SomeInteger](s`gensym35: MovingStat[F, C]): float
Source   Edit   
func kurtosisS[F: SomeFloat; C: SomeInteger](
    s`gensym35: var MovingStat[F, C]): float
Source   Edit   
func kurtosisS[T: SomeNumber](xs`gensym35: openArray[T]; accum`gensym35 = 32): float
Source   Edit   
func mean[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.inline.}
Moving mean/average over data window. Source   Edit   
func mean[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float {.inline.}
Moving mean/average over data window. Source   Edit   
func mean[T: SomeNumber](xs: openArray[T]): float {.
    codegenDecl: "__attribute__((optimize(\"Ofast\", \"fast-math\"))) $# $#$#".}
Source   Edit   
func nInv[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): F {.inline.}
A reciprocal caching 1.0/s.n; cannot update reciprocal. Source   Edit   
func nInv[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): F {.inline.}
A reciprocal caching 1.0/s.n. Source   Edit   
func pop[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]; x: SomeNumber)
Pops (aka removes the influence) of a value x from running sums. Source   Edit   
func push[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]; x: SomeNumber)
Pushes a value x into running sums. Source   Edit   
func quantile[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]; q: float): float {.
    inline.}
Estimated moving quantile q; E.g. q=0.5 is the moving median. Source   Edit   
func range[F: SomeFloat](xs: openArray[F]): (F, F) {.
    codegenDecl: "__attribute__((optimize(\"Ofast\", \"fast-math\"))) $# $#$#".}
Data range in 1-pass using vector-CPU instructions, if possible. E.g., let (mn,mx) = x.range. Source   Edit   
func skewness[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.inline.}
Moving skewness (population) over data window. Source   Edit   
func skewness[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float {.
    inline.}
Moving skewness (population) over data window. Source   Edit   
func skewness[T: SomeNumber](xs: openArray[T]; accum = 32): float
skewness (population). accum != 32 => 64-bit accumulation. Source   Edit   
func skewnessS[F: SomeFloat; C: SomeInteger](s`gensym28: MovingStat[F, C]): float
Source   Edit   
func skewnessS[F: SomeFloat; C: SomeInteger](
    s`gensym28: var MovingStat[F, C]): float
Source   Edit   
func skewnessS[T: SomeNumber](xs`gensym28: openArray[T]; accum`gensym28 = 32): float
Source   Edit   
func standardDeviation[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.
    inline.}
Moving standard deviation (population) over data window. Source   Edit   
func standardDeviation[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float {.
    inline.}
Moving standard deviation (population) over data window. Source   Edit   
func standardDeviation[T: SomeNumber](xs: openArray[T]; accum = 32): float
standard deviation (population). accum != 32 => 64-bit accumulation. Source   Edit   
func standardDeviationS[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float
Source   Edit   
func standardDeviationS[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float
Source   Edit   
func standardDeviationS[T: SomeNumber](xs: openArray[T]; accum = 32): float
Source   Edit   
func stderror[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.inline.}
Moving standard error (standard deviation of the mean) over data window. Source   Edit   
func stderror[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float {.
    inline.}
Moving standard error (standard deviation of the mean) over data window. Source   Edit   
func stderror[T: SomeNumber](xs: openArray[T]; accum = 32): float
standard error (std dev of the mean). accum != 32 => 64-bit accumulation. Source   Edit   
func sum[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.inline.}
Moving sum/total over data window. Source   Edit   
func variance[F: SomeFloat; C: SomeInteger](s: MovingStat[F, C]): float {.inline.}
Moving variance (population) over data window. Source   Edit   
func variance[F: SomeFloat; C: SomeInteger](s: var MovingStat[F, C]): float {.
    inline.}
Moving variance (population) over data window. Source   Edit   
func variance[T: SomeNumber](xs: openArray[T]; accum = 32): float {.
    codegenDecl: "__attribute__((optimize(\"Ofast\", \"fast-math\"))) $# $#$#".}
variance (population). accum != 32 => 64-bit accumulation. Source   Edit   
func varianceS[F: SomeFloat; C: SomeInteger](s`gensym21: MovingStat[F, C]): float
Source   Edit   
func varianceS[F: SomeFloat; C: SomeInteger](
    s`gensym21: var MovingStat[F, C]): float
Source   Edit   
func varianceS[T: SomeNumber](xs`gensym21: openArray[T]; accum`gensym21 = 32): float
Source   Edit  

Made with Nim. Generated: 2025-06-22 12:33:37 UTC