NekoCWD/libcamera
NekoCWD/libcamera
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 7 Wiki Activity Actions

libipa: histogram: Fix quantile() calculation for fractional results

Browse source 
The calculation of the frac variable is based solely on integers and
therefore results in the fractional part being either 0 or 1.

In the original code from RaspberryPi this is mitigated by casting the
nominator to a double. This works for most cases, but fails when q is
very small because of the quantization introduced by item being an
integer.

Fix both issues by doing the full calculation in double and remove the
should_fail tag.

Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This commit is contained in:
Stefan Klug 2025-04-01 14:36:11 +02:00
parent 781e2f4d0c
commit 8936e81e3f
2 changed files with 3 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

3
src/ipa/libipa/histogram.cpp
View file

@ -130,7 +130,8 @@ double Histogram::quantile(double q, uint32_t first, uint32_t last) const
if (cumulative_[first + 1] == cumulative_[first]) if (cumulative_[first + 1] == cumulative_[first])
frac = 0; frac = 0;
else else
frac = (item - cumulative_[first]) / (cumulative_[first + 1] - cumulative_[first]); frac = (q * total() - cumulative_[first])
/ (cumulative_[first + 1] - cumulative_[first]);
return first + frac; return first + frac;
} }

2
test/ipa/libipa/meson.build
View file

@ -2,7 +2,7 @@
libipa_test = [ libipa_test = [
{'name': 'fixedpoint', 'sources': ['fixedpoint.cpp']}, {'name': 'fixedpoint', 'sources': ['fixedpoint.cpp']},
{'name': 'histogram', 'sources': ['histogram.cpp'], 'should_fail': true}, {'name': 'histogram', 'sources': ['histogram.cpp']},
{'name': 'interpolator', 'sources': ['interpolator.cpp']}, {'name': 'interpolator', 'sources': ['interpolator.cpp']},
] ]