lar::util::StatCollector< T, W > Class Template Reference

Collects statistics on a single quantity (weighted) More...

#include "StatCollector.h"

Inheritance diagram for lar::util::StatCollector< T, W >:

Public Types
using	This_t = StatCollector< T, W >
	this type More...
using	Data_t = T
	type of the data More...
using	Weight_t = typename Base_t::Weight_t
	type of the weight More...

Public Member Functions
void	clear ()
	Clears all the statistics. More...
Add elements
void	add (Data_t value, Weight_t weight=Weight_t(1.0))
	Adds one entry with specified value and weight. More...
template<typename Iter >
void	add_unweighted (Iter begin, Iter end)
	Adds entries from a sequence with weight 1. More...
template<typename Iter , typename Pred >
void	add_unweighted (Iter begin, Iter end, Pred extractor)
	Adds entries from a sequence with weight 1. More...
template<typename Cont , typename Pred >
void	add_unweighted (Cont cont, Pred extractor)
	Adds all entries from a container, with weight 1. More...
template<typename Cont >
void	add_unweighted (Cont cont)
	Adds all entries from a container, with weight 1. More...
template<typename VIter , typename WIter , typename VPred , typename WPred = identity>
void	add_weighted (VIter begin_value, VIter end_value, WIter begin_weight, VPred value_extractor, WPred weight_extractor=WPred())
	Adds entries from a sequence with individually specified weights. More...
template<typename Iter >
void	add_weighted (Iter begin, Iter end)
	Adds entries from a sequence with individually specified weights. More...
template<typename Cont >
void	add_weighted (Cont cont)
	Adds entries from a sequence with individually specified weights. More...
Statistic retrieval
int	N () const
	Returns the number of entries added. More...
Weight_t	Weights () const
	Returns the sum of the weights. More...
Weight_t	Sum () const
	Returns the weighted sum of the values. More...
Weight_t	SumSq () const
	Returns the weighted sum of the square of the values. More...
Weight_t	Average () const
	Returns the value average. More...
Weight_t	Variance () const
	Returns the square of the RMS of the values. More...
Weight_t	RMS () const
	Returns the root mean square. More...
Weight_t	AverageWeight () const
	Returns the arithmetic average of the weights. More...

Protected Types
using	Variable_t = details::DataTracker2< Data_t, Weight_t >

Protected Member Functions
void	add (Weight_t weight)
	Adds the specified weight to the statistics. More...

Static Protected Member Functions
template<typename V >
static constexpr V	sqr (V const &v)
	Returns the square of the specified value. More...

Protected Attributes
Variable_t	x
	accumulator for variable x More...
int	n = 0
	number of added entries More...
Weight_t	w = Weight_t(0)
	total weight More...

Private Types
using	Base_t = details::WeightTracker< W >

Detailed Description

template<typename T, typename W = T>
class lar::util::StatCollector< T, W >

Collects statistics on a single quantity (weighted)

---

Template Parameters

T	type of the quantity
W	type of the weight (as T by default)

This is a convenience class, as easy to use as:

lar::util::StatCollector<double> stat;
stat.add(3.0, 2.0);
stat.add(4.0, 2.0);
stat.add(5.0, 1.0);
std::cout << "Statistics from " << stat.N() << " entries: "
  << stat.Average() << std::endl;

or also

std::vector<std::pair<double, double>> values({
  { 3.0, 2.0 },
  { 4.0, 2.0 },
  { 5.0, 1.0 },
  });
lar::util::StatCollector<double> stat;
stat.add_weighted(values.begin(), values.end());
std::cout << "Statistics from " << stat.N() << " entries: "
  << stat.Average() << std::endl;

that should both print: "Statistics from 3 entries: 3.8".

Other functions are available allowing addition of weighted and unweighted data from collections. For additional examples, see the unit test StatCollector_test.cc .

StatCollector::Variance() is known to be very sensitive to rounding errors, since it uses the formula E[x^2] - E^2[x]. If the variance is effectively small the formula can become negative. In case of negative value produced by the variance formula, the Variance() method will round up to zero. If you know roughly the average of the items you are add()ing, you can reduce the rounding error by subtracting it from the input value; you'll have to shift the average back, while the variance will not be affected; also the sums will be shifted. Example:

// fill the values, shifted
for (auto element: elements)
  sc.add(element - elements[0], (*weight)++);

auto sum_weights = sc.Weights();
auto sum_values = sc.Sum() - elements[0] * sc.Weights();
auto sum_values2 = sc.SumSq() + sqr(elements[0]) * sc.Weights()
  + 2. * elements[0] * sc.Sum();
auto average = sc.Average() + elements[0];
auto variance = sc.Variance();

A small variance implies values of similar magnitude, and therefore subtracting any single one of them (in the example, the first one) is more effective than it seems. As a rule of thumb, if you are collecting statistics for elements with N significant bits, a 2N significant bits is recommended for statistics collection. That means double for float, and long double for double (although usually long double is only marginally more precise than double, typically 25 or 50% more).

Definition at line 243 of file StatCollector.h.

Member Typedef Documentation

template<typename T, typename W = T>

using lar::util::StatCollector< T, W >::Base_t = details::WeightTracker<W>

private

Definition at line 244 of file StatCollector.h.

template<typename T, typename W = T>

using lar::util::StatCollector< T, W >::Data_t = T

type of the data

Definition at line 248 of file StatCollector.h.

template<typename T, typename W = T>

using lar::util::StatCollector< T, W >::This_t = StatCollector<T, W>

this type

Definition at line 247 of file StatCollector.h.

template<typename T, typename W = T>

using lar::util::StatCollector< T, W >::Variable_t = details::DataTracker2<Data_t, Weight_t>

protected

Definition at line 437 of file StatCollector.h.

template<typename T, typename W = T>

using lar::util::StatCollector< T, W >::Weight_t = typename Base_t::Weight_t

type of the weight

Definition at line 249 of file StatCollector.h.

Member Function Documentation

template<typename W>

void lar::util::details::WeightTracker< W >::add ( Weight_t  weight )

inlineinherited

Adds the specified weight to the statistics.

Definition at line 53 of file StatCollector.h.

Referenced by lar::util::details::FitDataCollector< T, D >::add().

{ ++n; w += weight; }

template<typename T , typename W >

void lar::util::StatCollector< T, W >::add	(	Data_t	value,
		Weight_t	weight = Weight_t(1.0)
	)

Adds one entry with specified value and weight.

Definition at line 855 of file StatCollector.h.

References lar::util::StatCollector< T, W >::add_unweighted(), and x.

Referenced by lar::util::details::WeightTracker< W >::AverageWeight(), cluster::HoughBaseAlg::FastTransform(), hit::HitCheater::FindHitsOnChannel(), cluster::ClusterParamsAlg::GetAverages(), and evd::GraphCluster::produce().

{
  Base_t::add(weight);
  x.add(value, weight);
} // StatCollector<T, W>::add()

template<typename T, typename W = T>

template<typename Iter >

void lar::util::StatCollector< T, W >::add_unweighted ( Iter  begin, Iter  end  )

inline

Adds entries from a sequence with weight 1.

Template Parameters

Iter	forward iterator to the elements to be added

Parameters

begin	iterator pointing to the first element to be added
end	iterator pointing after the last element to be added

The value pointed by the iterator must be convertible to the Data_t type.

Definition at line 269 of file StatCollector.h.

References evd::details::begin(), and evd::details::end().

Referenced by lar::util::StatCollector< T, W >::add().

        {  add_unweighted(begin, end, identity()); }

template<typename T , typename W >

template<typename Iter , typename Pred >

void lar::util::StatCollector< T, W >::add_unweighted	(	Iter	begin,
		Iter	end,
		Pred	extractor
	)

Adds entries from a sequence with weight 1.

Template Parameters

Iter	forward iterator to the elements to be added
Pred	a predicate to extract the element from iterator value

Parameters

begin	iterator pointing to the first element to be added
end	iterator pointing after the last element to be added
extractor	the predicate extracting the value to be inserted

The predicate is required to react to a call like with:

Data_t Pred::operator() (typename Iter::value_type);

Definition at line 865 of file StatCollector.h.

{
  std::for_each
    (begin, end, [this, extractor](auto item) { this->add(extractor(item)); });
} // StatCollector<T, W>::add_unweighted(Iter, Iter, Pred)

template<typename T, typename W = T>

template<typename Cont , typename Pred >

void lar::util::StatCollector< T, W >::add_unweighted ( Cont  cont, Pred  extractor  )

inline

Adds all entries from a container, with weight 1.

Template Parameters

Cont	type of container of the elements to be added
Pred	a predicate to extract the element from iterator value

Parameters

cont	container of the elements to be added
extractor	the predicate extracting the value to be inserted

The predicate is required to react to a call like with:

Data_t Pred::operator() (typename Cont::value_type);

The container must support the range-based for loop syntax, that is is must have std::begin<Cont>() and std::end<Cont>() defined.

Definition at line 303 of file StatCollector.h.

References evd::details::begin(), and evd::details::end().

        { add_unweighted(std::begin(cont), std::end(cont), extractor); }

template<typename T, typename W = T>

template<typename Cont >

void lar::util::StatCollector< T, W >::add_unweighted ( Cont  cont )

inline

Adds all entries from a container, with weight 1.

Template Parameters

Cont	type of container of the elements to be added

Parameters

cont	container of the elements to be added

The container must support the range-based for loop syntax, that is is must have std::begin<Cont>() and std::end<Cont>() defined. The value in the container must be convertible to the Data_t type.

Definition at line 316 of file StatCollector.h.

References evd::details::begin(), and evd::details::end().

        { add_unweighted(std::begin(cont), std::end(cont)); }

template<typename T , typename W >

template<typename VIter , typename WIter , typename VPred , typename WPred >

void lar::util::StatCollector< T, W >::add_weighted	(	VIter	begin_value,
		VIter	end_value,
		WIter	begin_weight,
		VPred	value_extractor,
		WPred	weight_extractor = WPred()
	)

Adds entries from a sequence with individually specified weights.

Template Parameters

VIter	forward iterator to the elements to be added
WIter	forward iterator to the weights
VPred	a predicate to extract the element from iterator value
WPred	a predicate to extract the weight from iterator value

Parameters

begin_value	iterator pointing to the first element to be added
end_value	iterator pointing after the last element to be added
begin_weight	iterator pointing to the weight of first element
value_extractor	predicate extracting the value to be inserted
weight_extractor	predicate extracting the value to be inserted

Each value is added with the weight pointed by the matching element in the list pointed by begin_weight: the element *(begin_value) will have weight *(begin_weight), the next element *(begin_value + 1) will have weight *(begin_weight + 1), etc.

The predicates are required to react to a call like with:

Data_t VPred::operator() (typename VIter::value_type);
Weight_t WPred::operator() (typename WIter::value_type);

Definition at line 874 of file StatCollector.h.

  {
  while (begin_value != end_value) {
    add(value_extractor(*begin_value), weight_extractor(*begin_weight));
    ++begin_value;
    ++begin_weight;
  } // while
} // StatCollector<T, W>::add_weighted(VIter, VIter, WIter, VPred, WPred)

template<typename T , typename W >

template<typename Iter >

void lar::util::StatCollector< T, W >::add_weighted	(	Iter	begin,
		Iter	end
	)

Adds entries from a sequence with individually specified weights.

Template Parameters

Iter	forward iterator to (value, weight) pairs to be added

Parameters

begin	iterator pointing to the first element to be added
end	iterator pointing after the last element to be added

The value pointed by the iterator must be a pair with first element convertible to the Data_t and the second element convertible to Weight_t. For more complicate structures, use the version with two predicates (using the weight iterator the same as the value iterator).

Definition at line 890 of file StatCollector.h.

                                                                  {
  
  std::for_each(begin, end, [this](auto p) { this->add(p.first, p.second); });
  
} // StatCollector<T, W>::add_weighted(Iter, Iter)

template<typename T, typename W = T>

template<typename Cont >

void lar::util::StatCollector< T, W >::add_weighted ( Cont  cont )

inline

Adds entries from a sequence with individually specified weights.

Template Parameters

Cont	type of container of (value, weight) pairs to be added

Parameters

cont	container of (value, weight) pairs to be added

The values in the container must be pairs with first element convertible to the Data_t and the second element convertible to Weight_t.

Definition at line 380 of file StatCollector.h.

References evd::details::begin(), clear(), and evd::details::end().

        { add_weighted(std::begin(cont), std::end(cont)); }

template<typename T , typename W >

lar::util::StatCollector< T, W >::Weight_t lar::util::StatCollector< T, W >::Average

(

)

const

Returns the value average.

Returns

the value average

Exceptions

std::range_error

if the total weight is 0 (usually: no data)

Definition at line 907 of file StatCollector.h.

Referenced by cluster::ClusterParamsAlg::GetAverages().

{
  if (Weights() == Weight_t(0))
    throw std::range_error("StatCollector<>::Average(): divide by 0");
  return Sum() / Weights();
} // StatCollector<T, W>::Average()

template<typename T, typename W = T>

Weight_t lar::util::StatCollector< T, W >::AverageWeight ( ) const

inline

Returns the arithmetic average of the weights.

Returns

the weight average

Exceptions

std::range_error

if no entry was added

Definition at line 431 of file StatCollector.h.

{ return Base_t::AverageWeight(); }

template<typename T , typename W >

void lar::util::StatCollector< T, W >::clear ( )

inline

Clears all the statistics.

Definition at line 899 of file StatCollector.h.

References clear(), and x.

Referenced by hit::HitCheater::FindHitsOnChannel().

                                              {
  Base_t::clear();
  x.clear();
} // StatCollector<T, W>::clear()

template<typename T, typename W = T>

int lar::util::StatCollector< T, W >::N ( ) const

inline

Returns the number of entries added.

Definition at line 392 of file StatCollector.h.

{ return Base_t::N(); }

template<typename T , typename W >

lar::util::StatCollector< T, W >::Weight_t lar::util::StatCollector< T, W >::RMS

(

)

const

Returns the root mean square.

Returns

the RMS of the values

Exceptions

std::range_error	if the total weight is 0 (see Variance())
std::range_error	if Variance() is negative (due to rounding errors)

Definition at line 927 of file StatCollector.h.

References lar::util::StatCollector2D< T, W >::add().

Referenced by cluster::HoughBaseAlg::FastTransform(), hit::HitCheater::FindHitsOnChannel(), cluster::ClusterParamsAlg::GetAverages(), and evd::GraphCluster::produce().

{
  const Weight_t rms2 = Variance();
  if (rms2 < Weight_t(0)) return 0.;
//    throw std::range_error("StatCollector<>::RMS(): negative RMS^2");
  return std::sqrt(rms2);
} // StatCollector<T, W>::RMS()

template<typename W>

template<typename V >

static constexpr V lar::util::details::WeightTracker< W >::sqr ( V const &  v )

inlinestaticinherited

Returns the square of the specified value.

Definition at line 74 of file StatCollector.h.

{ return v*v; }

template<typename T, typename W = T>

Weight_t lar::util::StatCollector< T, W >::Sum ( ) const

inline

Returns the weighted sum of the values.

Definition at line 398 of file StatCollector.h.

References x.

Referenced by cluster::HoughBaseAlg::FastTransform(), cluster::ClusterParamsAlg::GetAverages(), and evd::GraphCluster::produce().

{ return x.Sum(); }

template<typename T, typename W = T>

Weight_t lar::util::StatCollector< T, W >::SumSq ( ) const

inline

Returns the weighted sum of the square of the values.

Definition at line 401 of file StatCollector.h.

References x.

{ return x.SumSq(); }

template<typename T , typename W >

lar::util::StatCollector< T, W >::Weight_t lar::util::StatCollector< T, W >::Variance

(

)

const

Returns the square of the RMS of the values.

Returns

the square of the RMS of the values

Exceptions

std::range_error

if the total weight is 0 (usually: no data)

Definition at line 917 of file StatCollector.h.

References max, and geo::sqr().

{
  if (Weights() == Weight_t(0))
    throw std::range_error("StatCollector<>::Variance(): divide by 0");
  return std::max(Weight_t(0), (SumSq() - sqr(Sum()) / Weights()) / Weights());
} // StatCollector<T, W>::Variance()

template<typename T, typename W = T>

Weight_t lar::util::StatCollector< T, W >::Weights ( ) const

inline

Returns the sum of the weights.

Definition at line 395 of file StatCollector.h.

{ return Base_t::Weights(); }

Member Data Documentation

template<typename W>

int lar::util::details::WeightTracker< W >::n = 0

protectedinherited

number of added entries

Definition at line 77 of file StatCollector.h.

template<typename W>

Weight_t lar::util::details::WeightTracker< W >::w = Weight_t(0)

protectedinherited

total weight

Definition at line 78 of file StatCollector.h.

template<typename T, typename W = T>

Variable_t lar::util::StatCollector< T, W >::x

protected

accumulator for variable x

Definition at line 439 of file StatCollector.h.

---

The documentation for this class was generated from the following file:

• lardata/v06_47_03/source/lardata/Utilities/StatCollector.h