Commit graph

50 commits

Author SHA1 Message Date
Jean-Michel Hautbois
a0734f72d4 ipa: ipu3: awb: Add support for color temperature
The AWB estimates the color temperature, but it is not used at all. It
can be useful for debug purpose at least, but also for lux estimation
later, to be able to know the temperature estimated for a given frame.

Add a new member to the IPAFrameContext::awb for this purpose, and
update the value in AWB.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 11:00:05 +01:00
Jean-Michel Hautbois
891ec3f872 ipa: ipu3: agc: Store exposure in units of time
The minimum and maximum exposure are stored in lines. Replace it by
values in time to simplify the calculations.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 11:00:05 +01:00
Jean-Michel Hautbois
93af8ea616 ipa: ipu3: agc: Update previous exposure value
Previously, the exposure value was calculated based on the estimated
shutter time and gain applied. Now that we have the real values for the
current frame, use those before estimating the next one and rename the
variable accordingly.

As the exposure value is updated in the beginning of the computation,
there is no need to initialize effectiveExposureValue anymore in the
configure call, and it can be a local variable and not a class variable
anymore.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 11:00:05 +01:00
Jean-Michel Hautbois
f8f07f9468 ipa: ipu3: agc: Improve gain calculation
When an image is partially saturated, its brightness is not increasing
linearly when the shutter time or gain increases. It is a big issue with
a backlight as the algorithm is fading to darkness right now.

Introduce a function to estimate the brightness of the frame, based on
the current exposure/gain and loop on it several times to estimate it
again and approach the non linear function.

Inspired-by: 7de5506c30 ("libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images")
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-11-15 11:00:05 +01:00
Jean-Michel Hautbois
3d81620b20 ipa: ipu3: agc: Refactor ev gain calculation and testing
When we compute the new gain, we use the iqMean_ and estimate an
exposure value gain to apply. Return early when the gain is less than
1%.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 11:00:05 +01:00
Jean-Michel Hautbois
4fdf1e4f5e ipa: ipu3: agc: Compute the gain for each frame
Now that we have the real exposure applied at each frame, remove the
early return based on a frame counter and compute the gain for each
frame.

Introduce a number of startup frames during which the filter speed is
1.0, meaning we apply instantly the exposure value calculated and not a
slower filtered one. This is used to have a faster convergence, and
those frames may be dropped in a future development to hide the
convergance process from the viewer.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 11:00:05 +01:00
Jean-Michel Hautbois
5614c95794 ipa: ipu3: agc: Limit the number of saturated cells
When the histogram is calculated, we check if a cell is saturated or not
before cumulating its green value. This is wrong, and it can lead to an
empty histogram in case of a fully saturated frame.

Use a constant to limit the amount of pixels within a cell before
considering it saturated. If at the end of the loop we still have an
empty histogram, then make it a fully saturated one.

Bug: https://bugs.libcamera.org/show_bug.cgi?id=84
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 10:49:13 +01:00
Jean-Michel Hautbois
5233581e51 ipa: ipu3: Use sensor controls to update frameContext
The pipeline handler populates the new sensorControls ControlList, to
have the effective exposure and gain values for the current frame. This
is done when a statistics buffer is received.

Make those values the frameContext::sensor values for the frame when the
EventStatReady event is received.

AGC also needs to use frameContext.sensor as its input values and
frameContext.agc as its output values. Modify computeExposure by passing
it the frameContext instead of individual exposure and gain values.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 10:49:13 +01:00
Jean-Michel Hautbois
4d4694cd9f ipa: ipu3: agc: Clamp shutter speed
In case the maximum exposure received from the sensor is very high, we
can have a very high shutter speed with a small analogue gain, and it
may result in very slow framerate. We are not really supporting it for
the moment, so clamp the shutter speed to an arbitrary value of 60ms.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Tested-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:10:34 +02:00
Kieran Bingham
c46c889e20 ipa: ipu3: algorithms: awb: Privatize internal structures
The struct RGB and struct AwbStatus are used only by the internal
implementation of the AWB algorithm module.

Move them into the private class declaration.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:33 +02:00
Kieran Bingham
f225154018 ipa: ipu3: Fix the IPU3 AWB doxygen references
The AWB AwbStatus structure is contained within the Awb class.
Fix the Doxygen reference so that it can be found.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:29 +02:00
Jean-Michel Hautbois
a1f0df8654 ipa: ipu3: tonemapping: Add the documentation for ToneMapping
The tone mapping algorithm is currently undocumented.

Provide an introduction and overview to the implementation as the class
definition and document how the algorithm operates in the process and
prepare methods.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:03:37 +02:00
Jean-Michel Hautbois
ec5ceceaf7 ipa: ipu3: tonemapping: Implement configure call
Let the algorithm perform its initial configuration. Implement
configure() to set a default gamma value and let process do the updates
needed.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:43 +02:00
Jean-Michel Hautbois
acbad32b2e ipa: ipu3: tonemapping: Generate the LUT only on gamma change
The tone mapping algorithm calculates the gamma curve for every frame,
regardless of whether the gamma value has changed or not. This issue is
exasperated as we currently hardcode the gamma to a single value.

Optimise the implementation to only recalculate the look up table when
the gamma setting is changed, and store the gamma setting of the LUT
curve as part of the IPA context.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:41 +02:00
Jean-Michel Hautbois
acf6b42ab4 ipa: ipu3: agc: Document AGC mean-based algorithm
The AGC class was not documented while developing. Extend that to
reference the origins of the implementation, and improve the
descriptions on how the algorithm operates internally.

While at it, rename the functions which have bad names.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:36 +02:00
Jean-Michel Hautbois
faeb07e897 ipa: ipu3: awb: Reword accumulator documentation
Now that we moved the diagram into the AWB class documentation, reword the
accumulator documentation to make it clear it is not meant to be used
only in AWB.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:34 +02:00
Jean-Michel Hautbois
7635f71a0a ipa: ipu3: awb: Add AWB class documentation
The AWB algorithm is based on the Grey world algorithm and uses the
statistics generated by the ImgU for that. Explain how it uses those,
and reference the original algorithm at the same time.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-26 14:02:24 +02:00
Jean-Michel Hautbois
43d098ce5f ipa: ipu3: Use sensor limits for analogue gain
Instead of using constants for the analogue gains limits, use the
minimum and maximum from the configured sensor.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22 09:29:25 +02:00
Jean-Michel Hautbois
58486847f0 ipa: ipu3: agc: Remove unused variables
We currently control the exposure value by the shutter speed and the
analogue gain. We can't use the digital gain to have more than the
maximum exposure value calculated because we are not controlling it.

Remove unused code associated with this digital gain.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22 09:29:25 +02:00
Jean-Michel Hautbois
cbf2c9a612 ipa: ipu3: agc: Refactor condition on exposure correction
Simplify the reading by removing one level of indentation to return
early when the change is small between two calls.

Reword the LOG() message when we are correctly exposed, and move the
lastFrame_ variable to update it even if the change is small.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 09:29:25 +02:00
Jean-Michel Hautbois
cd0b3402ea ipa: ipu3: agc: Introduce previous exposure value
We need to calculate the gain on the previous exposure value calculated.
Now that we initialise the exposure and gain values in configure(), we
know the initial exposure value, and we can set it before any loop is
running.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:31:32 +02:00
Jean-Michel Hautbois
fd5a82ea88 ipa: ipu3: agc: Rename gains properly
We have mixed terms between gain, analogue gain and the exposure value
gain.

Make it clear when we are using the analogue gain from the sensor, and
when we are using the calculated gain to be applied to the exposure
value to reach the target.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:31:32 +02:00
Jean-Michel Hautbois
17dbae2325 ipa: ipu3: agc: Simplify division of exposure/gain
Until now, the algorithm makes complex assumptions when dividing the
exposure and analogue gain values. Instead, use a simpler clamping of
the shutter speed first, and then of the analogue gain, based on the
limits configured.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22 07:31:28 +02:00
Jean-Michel Hautbois
f1cf01d13b ipa: ipu3: agc: Use filtered exposure values
We are filtering the exposure value to limit the gain to apply, but we
are not using the result.

Fix it.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:30:18 +02:00
Jean-Michel Hautbois
a58a784e53 ipa: ipu3: agc: Change analogue gain limits
The gains are currently set as a uint32_t while the analogue gain is
passed as a double. We also have a default maximum analogue gain of 15
which is quite high for a number of sensors.

Use a maximum value of 8 which should really be configured by the IPA
and not fixed as it is now. While at it make it a double.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:29:53 +02:00
Jean-Michel Hautbois
355503233b ipa: ipu3: agc: Change exposure limits
We are using arbitrary constants for the exposure limit in a number of
lines.

Instead of using static constants for those, use the limits of the
sensor passed in IPASessionConfiguration and cache those.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:29:53 +02:00
Jean-Michel Hautbois
02686a052a ipa: ipu3: agc: Rename exposure values properly
The exposure value is filtered in filterExposure() using the
currentExposure_ and setting a prevExposure_ variable. This is misnamed
as it is not the previous exposure, but a filtered value.

Rename it accordingly.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:29:53 +02:00
Jean-Michel Hautbois
fac6734a4f ipa: ipu3: awb: Change minimal green threshold value
When zones are used for the grey world algorithm, they are only
considered if their average green value is at least 32/255 to exclude
zones that are too dark and don't provide relevant colour information
(on the opposite side of the spectrum, saturated regions are excluded by
the ImgU statistics engine).

The algorithm requires a minimal number of zones that meet this criteria
in order to run. Now that we correct the black level, the 32/255 minimal
value is a bit high and prevents the algorithm for running in low-light
conditions. Lower the value to 16/255 to fix it.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:29:53 +02:00
Jean-Michel Hautbois
9469fededd ipa: ipu3: awb: Use saturation under 90%
The AWB grey world algorithm tries to find a grey value and it can't do
it on over-exposed images. To exclude those, the saturation ratio is
used for each cell, and the cell is included only if this ratio is 0.

Now that we have changed the threshold, more cells may be considered as
partially saturated and excluded, preventing the algorithm from running
efficiently.

Change that behaviour, and consider 90% as a good enough ratio.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22 07:29:53 +02:00
Jean-Michel Hautbois
7c9c1a2a92 ipa: ipu3: set frameContext before controls
The AGC frame context needs to be initialised correctly for the first
iteration. Until now, the IPA uses the minimum exposure and gain values
and caches those in local variables.

In order to give the sensor limits to AGC, create a new structure in
IPASessionConfiguration. Store the exposure in time (and not line
duration) and the analogue gain after CameraSensorHelper conversion.

Set the gain and exposure appropriately to the current values known to
the IPA and remove the setting of exposure and gain in IPAIPU3 as those
are now fully controlled by IPU3Agc.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-22 07:25:48 +02:00
Jean-Michel Hautbois
59687683a1 ipa: ipu3: awb: Set a threshold for the green saturation
We can have a saturation ratio per cell, giving the percentage of pixels
over a threshold within a cell where 100% is set to 0xff.

The parameter structure 'ipu3_uapi_awb_config_s' contains four fields to
set the threshold, one per channel.
The blue field is also used to configure the ImgU and make it calculate
the saturation ratio or not.

Set a green value saturated when it is more than 230 (90% of the maximum
value 255, coded as 8191). As this is the only channel used for AGC,
there is no need to apply it to the other ones.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-22 07:25:42 +02:00
Laurent Pinchart
3b07397f0e ipa: ipu3: agc: Remove "using" directive from agc.h header
"using" directives are harmful in headers, as they propagate the
namespace short-circuit to all files that include the header, directly
or indirectly. Drop the directive from agc.h, and use utils::Duration
explicitly. While at it, shorten the namespace qualifier from
libcamera::utils:: to utils:: in agc.cpp for Duration.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-10-19 13:14:54 +03:00
Laurent Pinchart
5ae92dae58 ipa: ipu3: awb: Don't pass member variable to member function
The Awb::generateZones() member function fills the zones vector passed
as an argument, which is actually a member variable. Use it directly in
the function.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-10-19 13:14:53 +03:00
Jean-Michel Hautbois
09a221eb47 ipa: ipu3: Replace ipa::ipu3::algorithms::Ipu3AwbCell
The intel-ipu3.h public interface from the kernel does not define how to
parse the statistics for a cell. This had to be identified by a process
of reverse engineering, and later identifying the structures from [0]
leading to our custom definition of struct Ipu3AwbCell.

[0]
https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h

To improve the kernel interface, a proposal has been made to the
linux-kernel [1] to incorporate the memory layout for each cell into the
intel-ipu3 header directly.

[1]
https://lore.kernel.org/linux-media/20211005202019.253353-1-jeanmichel.hautbois@ideasonboard.com/

Update our local copy of the intel-ipu3.h to match the proposal and
change the AGC and AWB algorithms to reference that structure directly,
allowing us to remove the deprecated custom Ipu3AwbCell definition.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 17:58:17 +02:00
Jean-Michel Hautbois
e34ebe008f ipa: ipu3: agc: Rewrite and simplify the brightness loop
Now that we know how the AWB statistics are formatted, use a simplified
loop in processBrightness() to parse the green values and get the
histogram.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 17:58:17 +02:00
Jean-Michel Hautbois
8437893900 ipa: ipu3: awb: Introduce Black Level Correction
The pixels output by the camera normally include a black level, because
sensors do not always report a signal level of '0' for black. Pixels at
or below this level should be considered black and to achieve that, we
need to substract an offset to all the pixels. This can be taken into
account by reading the lowest value of a special region on sensors which
is not exposed to light. This provides a substracting factor to be
able to adjust the expected black levels in the resulting images.

For a camera outputting 10-bit pixel values (in the range 0 to 1023) a
typical black level might be 64. It is a fixed value, obtained by
capturing a raw frame with minimum exposure and gain fixed to 1.0 while
covering the sensor (the darker the better). We consider it good enough
as a very first approximation, until we measure it during a tuning
process and include it in a configuration file

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 17:58:17 +02:00
Jean-Michel Hautbois
380b08754f ipa: ipu3: awb: Use the line stride for the stats
The statistics buffer 'ipu3_uapi_awb_raw_buffer' stores the ImgU
calculation results in a buffer aligned horizontally to a multiple of 4
cells. The AWB loop should take care of it to add the proper offset
between lines and avoid any staircase effect.

It is no longer required to pass the grid configuration context to the
private functions called from process() which simplifies the code flow.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:59:40 +02:00
Jean-Michel Hautbois
03132d0ff9 ipa: ipu3: awb: Correct the gain multipliers
The gains have a precision u3.13, range [0, 8[ which means that a gain
multiplier value of 1.0 is represented as a multiplication by 8192 in
the ImgU. Correct the gains as this was misunderstood in the first
place.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:59:40 +02:00
Jean-Michel Hautbois
3a240e56a8 ipa: ipu3: awb: Correct the relevant zones proportion
The algorithm uses the statistics of a cell only if there is not too
much saturated pixels in it. The grey world algorithm works fine when
there are a limited number of outliers.
Consider a valid zone to be at least 80% of unsaturated cells in it.
This value could very well be configurable, and make the algorithm more
or less tolerant.

While at it, implement it in a configure() call as it will not change
during execution, and cache the cellsPerZone values estimated with
std::round as we are using cmath.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:59:40 +02:00
Jean-Michel Hautbois
e7860ce808 ipa: ipu3: awb: Make the naming consistent
The variables mix the terms cell, region and zone. It can confuse the
reader, and make the algorithm more difficult to follow. Rename the
local variables to be consistent with their definitions:
- Cells are defined in Pixels
- Zones are defined in Cells
There is no "region" as such, so replace it with the correct term.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:27:22 +02:00
Jean-Michel Hautbois
a90bc0e7b2 ipa: ipu3: Change Accumulator structure layout
The pixel component sums for the Accumulator are inconsistent with other
similar structures such as the IPAFrameContext::awb::gains. Group the
red, green, and blue sums together in a struct and store them as
uint64_t to reduce potential architectural differences.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:27:22 +02:00
Jean-Michel Hautbois
25ae4ad89f ipa: ipu3: Rename IspStatsRegion to Accumulator
The IspStatsRegion structure was introduced as an attempt to prepare for
a generic AWB algorithm structure. The structure name by itself is not
explicit and it is too optimistic to try and make a generic one for now.

Its role is to accumulate the pixels in a given zone. Rename it to
accumulator, and remove the uncounted field at the same time. It is
always possible to know how many pixels are not relevant for the
algorithm by calculating total-counted. The uncounted field was only
declared and not used. Amend the documentation accordingly.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:27:22 +02:00
Jean-Michel Hautbois
22ccbe2e5c ipa: ipu3: Move the AWB stats structures
The structure Ipu3AwbCell describes the AWB stats layout on the kernel
side. We will need it to be used by the AGC algorithm to be introduced
later, so let's make it visible from ipa::ipu3::algorithms and not only
for the AWB class.

The IspStatsRegion will be needed by AGC too, so let's move it in the
same namespace too.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-06 15:27:21 +02:00
Jean-Michel Hautbois
359ade5027 ipa: ipu3: awb: remove local storage of the grid
The IPASessionConfiguration now has the grid configuration stored. Use
it it at prepare() and process() calls in AWB and pass it as a reference
to the private functions when needed.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-23 11:04:08 +02:00
Jean-Michel Hautbois
b3befe2ef0 ipa: ipu3: agc: remove local storage of the grid
The IPASessionConfiguration now has the grid configuration stored. Use
it at process() call in AGC and pass it as a reference to the private
functions when needed.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-08-23 11:04:08 +02:00
Jean-Michel Hautbois
93802f600c ipa: ipu3: Move IPU3 agc into algorithms
Now that the interface is properly used by the AGC class, move it into
ipa::ipu3::algorithms and let the loops do the calls.
As we need to exchange the exposure_ and gain_ by passing them through the
FrameContext, use the calculated values in setControls() function to
ease the reading.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20 12:11:29 +02:00
Jean-Michel Hautbois
b145ae4242 ipa: ipu3: Move IPU3 awb into algorithms
Now that the interface is properly used by the AWB class, move it into
ipa::ipu3::algorithms and let the loops do the calls.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20 12:11:29 +02:00
Jean-Michel Hautbois
fcf56b9716 ipa: ipu3: Introduce a modular tone mapping algorithm
Introduce a new algorithm to manage the tone mapping handling of the
IPU3.

The initial algorithm is chosen to configure the gamma contrast curve
which moves the implementation out of AWB for simplicity. As it is
initialised with a default gamma value of 1.1, there is no need to use
the default table at initialisation anymore.

This demonstrates the way to use process() call when the EventStatReady
comes in. The function calculates the LUT in the context of a frame, and
when prepare() is called, the parameters are filled with the updated
values.

AGC is modified to take the new process interface into account.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20 12:11:28 +02:00
Jean-Michel Hautbois
b3a2882b36 ipa: ipu3: Add the functions to the Algorithm class
Introduce three functions in the Algorithm class to manage algorithms:
- configure which is called when IPA is configured only
- prepare called on EventFillParams event at each frame when the request
is queued
- process called on EventStatReady event at each frame completion when
the statistics have been generated.

The existing AGC implementation already has a function named process(),
though it has different arguments. Adding the new virtual process()
interface causes a compiler warning due to the AGC implementation
overloading a virtual function, even though the overload can be resolved
correctly.

Temporarily disable the warning in this commit to maintain bisection
until the AGC is converted to the new interface.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20 12:11:28 +02:00
Jean-Michel Hautbois
a47a6906a2 ipa: move libipa::Algorithm to ipa/ipu3/algorithms
The abstract Algorithm class was originally placed in libipa as an
attempt define a generic algorithm container. This was a little
optimistic and pushed a bit far too early.

Move the Algorithm class into the IPU3 which is the only user of the
class, as we adapt it to support modular algorithm components for the
IPU3.

Not documenting the namespace may cause issues with Doxygen in libipa.
The file libipa.cpp is thus created as an empty file for now, but we
can leverage it in the future to add more global libipa documentation,
and possibly code too.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-08-20 12:11:11 +02:00