For the implementation of a manual colour temperature setting, it is
necessary to read predefined colour gains per colour temperature from
the tuning file. Implement this in a backwards compatible way. If no
gains are contained in the tuning file, loading just continues as
before.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The RGB to YCbCr conversion matrix mentioned in a comment, coming from
the hardware documentation, does not match any of the canonical matrices
specified by any standard. While researching where the values came from,
it became apparent they are likely Bt.601 limited range coefficients
rounded to 6 bits of decimal precision. Record this in comments.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Replace the manual vector and matrix calculations with usage of the
Vector and Matrix classes. This simplifies the code and improves
readability.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Replace the manual storage of gains in the IPA active state and frame
context with usage of the RGB class. This simplifies the code thanks to
usage of the arithmetic functions provided by the RGB class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
The rec601LuminanceFromRGB() and estimateCCT() functions take RGB
triplets as three variables. Replace them with instances of the RGB
class and adapt the users accordingly. Only variables passed directly to
these functions are converted to RGB instances, further conversion of
IPA modules to the RGB class will be performed separately.
While at it, fix a typo in the documentation of the estimateCCT()
function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Use the centralised libipa helpers instead of open-coding common
functions.
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Sometimes the ISP produces statistics only with a subset of statistic
types being valid. It doesn't happen normally, but was observed in the
wild. Check for the RKISP1_CIF_ISP_STAT_AWB bit to prevent using invalid
or outdated data. As it doesn't happen regularly add an error message to
get notified when it happens.
For simpler code structure, the ColourTemperature metadata entry gets
written unconditionally and overwritten later if needed.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The LSP autoformatter doesn't like some of the current formatting, let's
make it happier. Note that not all of its suggestions were accepted
because readability is preferred and adjusting .clang-format may not be
easy or possible.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The includes that are not used can be removed.
Signed-off-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Use the new ISP parameters abstraction class RkISP1Params to access the
ISP parameters in the IPA algorithms. The class replaces the pointer to
the rkisp1_params_cfg structure passed to the algorithms' prepare()
function, and is used to access individual parameters blocks.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
When the colour temperature estimation gets skipped, the metadata isn't
populated. Fix that by filling the metadata early in the function.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The frame context is used to store data used for processing that frame.
It is later used to either act as input for other algorithms or to fill
the metadata. For the colour temperature this is not needed, as the
meatadata shall not contain the value that was active when the image was
processed, but the value that was calculated based on the statistics for
that image. This is no functional change.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
When the color gains are set manually it is possible to specify a
gain that wrapped the hardware limits. It would also be possible to
further tune the floating point limits, but that is an error prone
approach. So the limits are imposed on the integers, just before writing
to the hardware. This noticeably reduces some oscillations in the awb
regulation.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Source files in libcamera start by a comment block header, which
includes the file name and a one-line description of the file contents.
While the latter is useful to get a quick overview of the file contents
at a glance, the former is mostly a source of inconvenience. The name in
the comments can easily get out of sync with the file name when files
are renamed, and copy & paste during development have often lead to
incorrect names being used to start with.
Readers of the source code are expected to know which file they're
looking it. Drop the file name from the header comment block.
The change was generated with the following script:
----------------------------------------
dirs="include/libcamera src test utils"
declare -rA patterns=(
['c']=' \* '
['cpp']=' \* '
['h']=' \* '
['py']='# '
['sh']='# '
)
for ext in ${!patterns[@]} ; do
files=$(for dir in $dirs ; do find $dir -name "*.${ext}" ; done)
pattern=${patterns[${ext}]}
for file in $files ; do
name=$(basename ${file})
sed -i "s/^\(${pattern}\)${name} - /\1/" "$file"
done
done
----------------------------------------
This misses several files that are out of sync with the comment block
header. Those will be addressed separately and manually.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reorder functions in the base ipa::Algorithm and its derived classes to
match the calling order: queueRequest(), prepare() and process(). This
makes the code flow easier to read. No functional change intended.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Fill the frame metadata in the AGC and AWB algorithm's prepare()
function. Additional metadata for other algorithms will be addressed
later.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Extend the Algorithm::process() function with a metadata control list,
to be filled by individual algorithms with frame metadata. Update the
rkisp1 and ipu3 IPA modules accordingly, and drop the dead code in the
IPARkISP1::prepareMetadata() function while at it.
This only creates the infrastructure, filling metadata in individual
algorithms will be handled separately.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The red and blue gains are computed by dividing the green mean by the
red and blue means respectively. An offset of 1 is added to the dividers
to avoid divisions by zero. This introduces a bias in the gain values.
Fix it by clamping the divisors to a minimum of 1.0 instead of adding an
offset.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
When the RGB means are too small, gains and color temperature can't be
meaningfully calculated. Freeze the AWB in that case, using the
previously calculated values.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The gain values are currently clamped to the range [0.0, 3.996] used by
the hardware. A zero value makes little sense, as it would completely
remove the contribution of the corresponding color channel from the AWB
accumulators, but worse, would lead to divisions by zero when
calculating the raw means in subsequent iterations. Prevent this by
setting the minimum gain value to 1/256.
While at it, clamp the gain values before filtering them, to improve the
stability of the control loop.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Due to hardware rounding errors in the YCbCr means, the calculated RGB
means may be negative. This would lead to negative gains, messing up
calculation. Prevent this by clamping the means to positive values.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Extend the debug message in Awb::process() to log the means and color
temperature in addition to the gains. This is useful for debugging the
algorithm behaviour. While at it, set the showpoint flag to print a
fixed number of digits after the decimal point, making logs more
readable.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
RkISP actually supports two modes for color means, RGB and YCbCr. The
variables where the means are stored are identically named regardless of
the color means mode that's been selected.
Since the gains are computed in RGB mode, a conversion needs to be done
when the mode is YCbCr, which is unnecessary when RGB mode is selected.
This adds support for RGB means mode too, by checking at runtime which
mode is selected at a given time. The default is still set to YCbCr mode
for now.
Cc: Quentin Schulz <foss+libcamera@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The AWB statistics are computed after the ISP applies the colour gains.
This means that the red, green and blue means do not match the data
coming directly from the sensor, but are multiplied by the colour gains
that were used for the frame on which the statistics have been computed.
The AWB algorithm needs to take this into account when calculating the
colour gains for the next frame. Do so by dividing the means by the
gains that were applied to the frame, retrieved from the frame context.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Rework the algorithm's usage of the active state and frame context to
store data in the right place.
The active state stores two distinct categories of information:
- The consolidated value of all algorithm controls. Requests passed to
the queueRequest() function store values for controls that the
application wants to modify for that particular frame, and the
queueRequest() function updates the active state with those values.
The active state thus contains a consolidated view of the value of all
controls handled by the algorithm.
- The value of parameters computed by the algorithm when running in auto
mode. Algorithms running in auto mode compute new parameters every
time statistics buffers are received (either synchronously, or
possibly in a background thread). The latest computed value of those
parameters is stored in the active state in the process() function.
The frame context also stores two categories of information:
- The value of the controls to be applied to the frame. These values are
typically set in the queueRequest() function, from the consolidated
control values stored in the active state. The frame context thus
stores values for all controls related to the algorithm, not limited
to the controls specified in the corresponding request, but
consolidated from all requests that have been queued so far.
For controls that can be specified manually or computed by the
algorithm depending on the operation mode (such as the colour gains),
the control value will be stored in the frame context in
queueRequest() only when operating in manual mode. When operating in
auto mode, the values are computed by the algorithm and stored in the
frame context in prepare(), just before being stored in the ISP
parameters buffer.
The queueRequest() function can also store ancillary data in the frame
context, such as flags to indicate if (and what) control values have
changed compared to the previous request.
- Status information computed by the algorithm for a frame. For
instance, the colour temperature estimated by the algorithm from ISP
statistics calculated on a frame is stored in the frame context for
that frame in the process() function.
The active state and frame context thus both contain identical members
for most control values, but store values that have a different meaning.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Now that the Algorithm::prepare() function takes a frame number, we can
use it to replace the IPAActiveState::frameCount member.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
The RkISP1 IPA module creates a single instance of its IPAFrameContext
structure, effectively using it more as an active state than a per-frame
context. To prepare for the introduction of a real per-frame context,
move all the members of the IPAFrameContext structure to a new
IPAActiveState structure. The IPAFrameContext becomes effectively
unused at runtime, and will be populated back with per-frame data after
converting the RkISP1 IPA module to using a frame context queue.
The IPAActiveState structure will slowly morph into a different entity
as individual algorithm get later ported to the frame context API.
While at it, fix a typo in the documentation of the
Agc::computeExposure() function that incorrectly refers to the frame
context instead of the global context.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
IPA modules have access to incoming Request's controls list and need to
store them in the frame context at queueRequest() time. Pass the frame
context to the Algorithm::queueRequest() function.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Frame contexts will become the core component of IPA modules, always
available to functions of the algorithms. To indicate and prepare for
this, turn the frame context pointer passed to Algorithm::process() into
a reference.
The RkISP1 IPA module doesn't use frame contexts yet, so pass a dummy
context for now.
While at it, drop an unneeded [[maybe_unused]] from Agc::process() and
add a missing parameter documentation for the frameContext argument to
Awb::process().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Add an enable variable in the awb struct in IPASessionConfiguration
which indicates if the AWB hardware module is enabled. This will allow
other algorithms to retrieve this information.
Signed-off-by: Florian Sylvestre <fsylvestre@baylibre.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Add support for manually controlling the color gains on the rkisp1 IPA.
To that end, add and plumb the AwbEnable and ColourGains controls. As
per-frame controls aren't supported yet in the rkisp1 IPA, simply apply
and perform checks on the controls immediately.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To prepare for dynamic instantiation of algorithms from the tuning file,
register the algorithms with the Module class.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Currently we have a single structure of IPAFrameContext but
subsequently, we shall have a ring buffer (or similar) container
to keep IPAFrameContext structures for each frame.
It would be a hassle to query out the frame context required for
process() (since they will reside in a ring buffer) by the IPA
for each process. Hence, prepare the process() libipa template to
accept a particular IPAFrameContext early on.
As for this patch, we shall pass in the pointer as nullptr, so
that the changes compile and keep working as-is.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The RkISP1 ISP calculates a mean value for Y, Cr and Cb at each frame.
There is a RGB mode which could theoretically give us the values for R,
G and B directly, but it seems to be failing right now.
Convert those values into R, G and B and estimate the gain to apply in a
grey world.
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Tested-by: Peter Griffin <peter.griffin@linaro.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
