Commit graph

2783 commits

Author SHA1 Message Date
Jean-Michel Hautbois
5233581e51 ipa: ipu3: Use sensor controls to update frameContext
The pipeline handler populates the new sensorControls ControlList, to
have the effective exposure and gain values for the current frame. This
is done when a statistics buffer is received.

Make those values the frameContext::sensor values for the frame when the
EventStatReady event is received.

AGC also needs to use frameContext.sensor as its input values and
frameContext.agc as its output values. Modify computeExposure by passing
it the frameContext instead of individual exposure and gain values.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 10:49:13 +01:00
Han-Lin Chen
726d6087c0 ipu3: ipa: Report effective sensor controls with statistics to IPA
The Intel close sourced IPA requires the effective controls applied to
the sensor when the statistics are generated. Report effective sensor controls
with the statistics to IPA.

Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>
[Jean-Michel: Reword s/stastistics/statistics and move reset after IPA
start]
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 10:49:13 +01:00
Han-Lin Chen
f12efa6730 ipa: ipu3: Extend ipu3 ipa interface for sensor controls
IPU3Event and IPU3Action use single ControlList for both libcamera and
V4L2 controls, and it's content could be either one based on the
context.  Extend IPU3Event and IPU3Action for sensor V4L2 controls, and
preserve the original one for only libcamera Controls to make the
content of an event more specific.

Signed-off-by: Han-Lin Chen <hanlinchen@chromium.org>
[Jean-Michel: remove lensControls from the original patch]
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-15 10:49:13 +01:00
Kieran Bingham
6b288f8f4f libcamera: pipeline: ipu3: Fix minCrop indentation
When updating the controls the calculation for minCrop incorrectly
indents the parameters to scaledBy().

Fix it.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
2021-11-11 12:12:17 +00:00
Umang Jain
1b30992b62 android: Camera3RequestDescriptor: Provide a constructor for StreamBuffer
Provide a constructor for StreamBuffer and use that while populating
Camera3RequestDescriptor::buffers_ vector. Also provide the default
move-constructor (required as StreamBuffer is stored in a vector in
Camera3RequestDescriptor) and destructor for the StreamBuffer struct.

Also declare a default move assignment operator and disable the
copy constructor and move operator explicitly with
LIBCAMERA_DISABLE_COPY().

While at it, initialize pointers members in the StreamBuffer struct
to nullptr, with StreamBuffer::status set to Status::Success by default.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-11-08 17:34:45 +05:30
Umang Jain
701266653c android: mm: Null check for CameraBufferManager
cros::CameraBufferManager can be nullptr if there is an error in
its creation. Place a null-check guard to check it.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-11-08 16:50:27 +05:30
Jean-Michel Hautbois
a5b323e05d ipa: ipu3: List the events in their order of calling
The IPU3 IPA has three events which are handled from the pipeline
handler.

The events are received in the sequence, EventProcessControls,
EventFillParams, and finally EventStatReady, while the code lists these
in a different order.

Update the flow of IPAIPU3::processEvent() to match the expected
sequence of events, to help support the reader in interpreting the flow
of events through the IPA.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
2021-11-06 16:37:08 +01:00
Kieran Bingham
4b8c50bda2 libcamera: framebuffer: Fix isContiguous log message
The isContiguous debug message is inverted.
Correct the logic.

Reported-by: Roman Stratiienko <roman.o.stratiienko@globallogic.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-04 16:17:21 +00:00
Naushir Patuck
06008c6e81 pipeline: raspberrypi: Fix rounding issue in findBestFormat()
The aspect ratio calculation divides two integer values then casts to a double.
This might reduce precision when scoring for aspect rato differences.

Fix this by casting the integer to a double before the division.

Reported-by: Coverity CID=361652
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 13:03:50 +00:00
Arducam info
5004d8a969 libcamera: ipa: raspberrypi: Add support for imx519 sensor
The necessary tuning file and CamHelper is added for the imx519 sensor.

The imx519 is a 16MP rolling shutter sensor. To enable
it, please add

dtoverlay=imx519

to the /boot/config.txt file and reboot the Pi.

Signed-off-by: Lee Jackson <info@arducam.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:15:03 +00:00
Naushir Patuck
460da2b580 pipeline: raspberrypi: Account for a missing Unicam embedded data node
The unicam driver no longer registers an embedded data node if the sensor does
not provide this stream. Account for this in the pipeline handler match routine
by not assuming it is always present.

Add a warning if Unicam and the CamHelper do not agree on the presense of sensor
embedded data, and disable its usage in these cases.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Naushir Patuck
ca67bf09f1 pipeline: raspberrypi: Remove "unpacked" format penalty in mode selection
With the recent change to switch to programming the sensor device directly,
the notion of packed vs unpacked modes are not relevent, since that is a
Unicam format construct. Remove any scoring based on packed/unpacked modes.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Naushir Patuck
83a5128161 pipeline: raspberrypi: Convert the pipeline handler to use media controller
Switch the pipeline handler to use the new Unicam media controller based driver.
With this change, we directly talk to the sensor device driver to set controls
and set/get formats in the pipeline handler.

This change requires the accompanying Raspberry Pi linux kernel change at
https://github.com/raspberrypi/linux/pull/4645. If this kernel change is not
present, the pipeline handler will fail to run with an error message informing
the user to update the kernel build.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Naushir Patuck
5acc21fd04 pipeline: raspberrypi: Apply sensor flips at the start of configure()
Sensor flips might change the Bayer order of the requested format. The existing
code would set a sensor format along with the appropriate Unicam and ISP input
formats, but reset the latter two on start() once the flips had been requested.

We can now set the sensor flips just before we set the sensor mode in
configure(), thereby not needing the second pair of format sets in start().

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Naushir Patuck
91bd175c8f libcamera: bayer_format: Add PixelFormat conversion helpers to BayerFormat class
Add BayerFormat::toPixelFormat() and BayerFormat::fromPixelFormat() helper
functions to convert between BayerFormat and PixelFormat types.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
[Kieran: Minor checkstyle fix]
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Naushir Patuck
16ab13eaa9 libcamera: bayer_format: Add unpacked mono 10-bit format to the conversion table
Add BayerFormat conversions for formats::R10 (10-bit unpacked) format.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Naushir Patuck
c308ae532a libcamera: bayer_format: Rework BayerFormat conversion table
Rename the bayerToV4l2 conversion table to bayerToFormat. Update the table to
hold both the PixelFormat and V4L2PixelFormat conversions for a given
BayerFormat. This will allow converting between BayerFormat and PixelFormat
types in a subsequent change.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:45 +00:00
Laurent Pinchart
364ae3b78d libcamera: bayer_format: Turn BayerFormat::Packing into scoped enum
The unscoped enum BayerFormat::Packing leads to usage of the ambiguous
BayerFormat::None enumerator. Turn the enumeration into a scoped enum to
force usage of BayerFormat::Packing::None, and drop the now redundant
"Packed" suffix for the CSI2 and IPU3 packing.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:44 +00:00
Naushir Patuck
9e3470ad3e libcamera: formats: Add R10_CSI2P format
This new formats corresponds to the V4L2 V4L2_PIX_FMT_Y10P format, and is a
CSI2-packed version of the DRM_FORMAT_R10 format.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:44 +00:00
Laurent Pinchart
264a673d28 libcamera: formats: Add R10 and R12 formats
These new formats corresponds to the V4L2 V4L2_PIX_FMT_Y10 and
V4L2_PIX_FMT_Y12 formats, and are the little-endian version of the
DRM_FORMAT_R10 and DRM_FORMAT_R12 formats.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-11-02 12:08:44 +00:00
Kieran Bingham
76bd9f3d80 libcamera: pixel_format: remove duplicated return documentation
The PixelFormat::toString() has two \return statements in its
doxygen documentation.

Remove the redundant one.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-27 14:31:49 +01:00
Kieran Bingham
0a1aaa8d55 libcamera: v4l2_videodevice: provide hasMediaController()
The V4L2Capability has helpers to interogate the capabilities
of a device.

V4L2VideoDevice::enumPixelformats accesses the raw capabilites to check
if the device is supported by a MediaController device.

Provide a helper, and update the usage.

Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-27 14:31:49 +01:00
Jean-Michel Hautbois
4d4694cd9f ipa: ipu3: agc: Clamp shutter speed
In case the maximum exposure received from the sensor is very high, we
can have a very high shutter speed with a small analogue gain, and it
may result in very slow framerate. We are not really supporting it for
the moment, so clamp the shutter speed to an arbitrary value of 60ms.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Tested-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:10:34 +02:00
Kieran Bingham
40eb230999 ipa: ipu3: ipa_context: Fix doxygen warnings
The IPAFrameContext uses unnamed structures to group items. Doxygen
doesn't seem to support this properly, documentation isn't properly
generated and warnings are output during compilation. Suppress the
warning with a workaround that still results in incorrect generated
documentation until Doxygen gets fixed.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
[JMH: Fix doxygen variable usage]
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:08:37 +02:00
Jean-Michel Hautbois
f79f7331cd ipa: ipu3: Implement an empty stop() function
While the stop() function does not currently perform any action, it forms
part of the IPA interface and is a public function in the class.

Promote it to a full (but basic) function implementation and begin the
documentation accordingly so that there is an appropriate stub to
perform stop operations if they come up.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26 14:08:37 +02:00
Jean-Michel Hautbois
9b5f9e6487 ipa: ipu3: Fix badly documented context variables
Doxygen warns us because the structures are referenced as \struct while
they should be \var. Fix it.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:08:37 +02:00
Kieran Bingham
5faa0b1a2a ipa: ipu3: Isolate ipa_context documentation
The IPU3 IPA core is growing with additional documentation. The
ipa_context documentation is stored here, but it pushes the IPU3
documentation and implementation further from the head of the file.

Furthermore, the ipa_context documentation is outside of the ipa::ipu3
namespace and isn't identified correctly by Doxygen.

Move the ipa_context to its own compilation object even though there
isn't any code, but to maintain consistency with our documentation
model.

Correctly re-introduce the documentation into the libcamera::ipa::ipu3
namespace during the move.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:37 +02:00
Kieran Bingham
c46c889e20 ipa: ipu3: algorithms: awb: Privatize internal structures
The struct RGB and struct AwbStatus are used only by the internal
implementation of the AWB algorithm module.

Move them into the private class declaration.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:33 +02:00
Kieran Bingham
f225154018 ipa: ipu3: Fix the IPU3 AWB doxygen references
The AWB AwbStatus structure is contained within the Awb class.
Fix the Doxygen reference so that it can be found.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:29 +02:00
Kieran Bingham
6d89ca6b65 ipa: ipu3: ipa_context: Fix file reference
The ipa_context.h entry incorrectly referenced its file name.

Fix it.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:26 +02:00
Kieran Bingham
372698ece3 Documentation: IPU3 IPA Design guide
The IPU3 IPA implements the basic 3A using the ImgU ISP.

Provide an overview document to describe its operations, and provide a
block diagram to help visualise how the components are put together to
assist any new developers exploring the code.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
2021-10-26 14:08:17 +02:00
Jean-Michel Hautbois
a1f0df8654 ipa: ipu3: tonemapping: Add the documentation for ToneMapping
The tone mapping algorithm is currently undocumented.

Provide an introduction and overview to the implementation as the class
definition and document how the algorithm operates in the process and
prepare methods.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:03:37 +02:00
Jean-Michel Hautbois
ec5ceceaf7 ipa: ipu3: tonemapping: Implement configure call
Let the algorithm perform its initial configuration. Implement
configure() to set a default gamma value and let process do the updates
needed.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:43 +02:00
Jean-Michel Hautbois
acbad32b2e ipa: ipu3: tonemapping: Generate the LUT only on gamma change
The tone mapping algorithm calculates the gamma curve for every frame,
regardless of whether the gamma value has changed or not. This issue is
exasperated as we currently hardcode the gamma to a single value.

Optimise the implementation to only recalculate the look up table when
the gamma setting is changed, and store the gamma setting of the LUT
curve as part of the IPA context.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:41 +02:00
Jean-Michel Hautbois
acf6b42ab4 ipa: ipu3: agc: Document AGC mean-based algorithm
The AGC class was not documented while developing. Extend that to
reference the origins of the implementation, and improve the
descriptions on how the algorithm operates internally.

While at it, rename the functions which have bad names.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:36 +02:00
Jean-Michel Hautbois
faeb07e897 ipa: ipu3: awb: Reword accumulator documentation
Now that we moved the diagram into the AWB class documentation, reword the
accumulator documentation to make it clear it is not meant to be used
only in AWB.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 14:02:34 +02:00
Jean-Michel Hautbois
7635f71a0a ipa: ipu3: awb: Add AWB class documentation
The AWB algorithm is based on the Grey world algorithm and uses the
statistics generated by the ImgU for that. Explain how it uses those,
and reference the original algorithm at the same time.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-26 14:02:24 +02:00
Jean-Michel Hautbois
db5f8d3271 ipa: ipu3: Explicitly use the statistics parameter
The stats pointer is marked as [[maybe_unused]]. This is a leftover from
a previous commit which was here to keep the compatibility while
transitioning to the new iterative algorithms.

Remove this attribute to make it explicit that stats are really used to
feed the algorithms.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26 14:02:23 +02:00
Jean-Michel Hautbois
08df88e5b4 ipa: ipu3: Document the IPAIPU3 class
Clarify the roles and interactions between the pipeline handler events
and the algorithm calls by documenting all the remaining functions of
the class.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26 14:02:22 +02:00
Jean-Michel Hautbois
da0d5caa7f ipa: ipu3: Document IPAIPU3::configure
Further extend the documentation for the IPAIPU3::configure operation.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26 14:02:21 +02:00
Jean-Michel Hautbois
346baa856b ipa: ipu3: Document IPAIPU3 class interface
The IPU3 IPA is maturing to a modular and extensible system capable of
handling specific algorithms for the processing blocks on the ImgU.

Provide a top-level class documentation to provide an overview of the
IPA, detailing what events are used and what algorithms are currently
supported, as well as the limitations currently imposed.

Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
2021-10-26 14:02:19 +02:00
Umang Jain
b1cefe38f3 android: post_processor: Make post processing async
Introduce a dedicated worker class derived from libcamera::Thread.
The worker class maintains a queue for post-processing requests
and waits for a post-processing request to become available.
It will process them as per FIFO before de-queuing it from the
queue.

The entire post-processing handling iteration is locked under
streamsProcessMutex_ which helps us to queue all the post-processing
request at once, before any of the post-processing completion slot
(streamProcessingComplete()) is allowed to run for post-processing
requests completing in parallel. This helps us to manage both
synchronous and asynchronous errors encountered during the entire
post processing operation. Since a post-processing operation can
even complete after CameraDevice::requestComplete() has returned,
we need to check and complete the descriptor from
streamProcessingComplete() running in the PostProcessorWorker's
thread.

This patch also implements a flush() for the PostProcessorWorker
class which is responsible to purge post-processing requests
queued up while a camera is stopping/flushing. It is hooked with
CameraStream::flush(), which isn't used currently but will be
used when we handle flush/stop scenarios in greater detail
subsequently (in a different patchset).

The libcamera request completion handler CameraDevice::requestComplete()
assumes that the request that has just completed is at the front of the
queue. Now that the post-processor runs asynchronously, this isn't true
anymore, a request being post-processed will stay in the queue and a new
libcamera request may complete. Remove that assumption, and use the
request cookie to obtain the Camera3RequestDescriptor.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
2021-10-26 16:11:17 +05:30
Umang Jain
05862a7e35 android: post_processor: Drop return value for process()
PostProcessor::process() is invoked by CameraStream class
in case any post-processing is required for the camera stream.
The failure or success is checked via the value returned by
CameraStream::process().

Now that the post-processor notifies about the post-processing
completion operation, we can drop the return value of
PostProcessor::process(). The status of post-processing is passed
to CameraDevice::streamProcessingComplete() by the
PostProcessor::processComplete signal's slot.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-10-26 16:11:17 +05:30
Umang Jain
6b706e1c3e android: Track and notify post processing of streams
Notify that the post processing for a request has been completed,
via a signal. The signal is emitted with a context pointer along with
status of the buffer. The function CameraDevice::streamProcessingComplete()
will finally set the status on the request descriptor and complete the
descriptor if all the streams requiring post processing are completed.
If buffer status obtained is in error state, notify the status to the
framework and set the overall error status on the descriptor via
setBufferStatus().

We need to track the number of streams requiring post-processing
per Camera3RequestDescriptor (i.e. per capture request). Introduce
a std::map to track the post-processing of streams. The nodes
are dropped from the map when a particular stream post processing
is completed (or on error paths). A std::map is selected for tracking
post-processing requests, since we will move post-processing to be
asynchronous in subsequent commits. A vector or queue will not be
suitable as the sequential order of post-processing completion
of various requests won't be guaranteed then.

A streamsProcessMutex_ has been introduced here as well, which will be
applicable to guard access to descriptor's pendingStreamsToProcess_ when
post-processing is moved to be asynchronous in subsequent commits.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-10-26 16:11:17 +05:30
Umang Jain
79cdb1f19d android: post_processor: Consolidate contextual information
Save and provide the context for post-processor of a camera stream
via Camera3RequestDescriptor::StreamBuffer. We extend the structure
to include source and destination buffers for the post processor, along
with CameraStream::Type::Internal buffer pointer (if any). In addition
to that, a back pointer to Camera3RequestDescriptor is convenient to
get access to overall descriptor (status, metadata settings etc.).

Also, migrate CameraStream::process() and PostProcessor::process()
signature to use Camera3RequestDescriptor::StreamBuffer only. This
will be helpful when we move to async post-processing in subsequent
commits.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-10-26 16:11:17 +05:30
Umang Jain
64bcbd0e2c android: camera_device: Refactor descriptor status and sendCaptureResults()
Currently, we use Camera3RequestDescriptor::Status to determine:
- When the descriptor has been completely processed by HAL
- Whether any errors were encountered, during its processing

Both of these are essential to know whether the descriptor is eligible
to call process_capture_results() through sendCaptureResults().
When a status(Success/Error) is set on the descriptor, it is ready to
be sent back via sendCaptureResults(). However, this might lead to
undesired results especially when sendCaptureResults() runs in a
different thread (for e.g. stream's post-processor async completion
slot).

This patch decouples the descriptor status (Success/Error) from the
descriptor's completion status (pending or complete). The advantage
of this is we can set the completion status when the descriptor has
been processed fully by the layer and we can set the error status on
the descriptor wherever an error is encountered, throughout the
lifetime of the descriptor in the HAL layer.

While at it, introduce a wrapper completeDescriptor() around
sendCaptureResults(). completeDescriptor() as the name suggests will
mark the descriptor as complete, so it is ready to be sent back.
The locking mechanism is moved from sendCaptureResults() to this wrapper
since the intention is to use completeDescriptor() in place of existing
sendCaptureResults() calls.

Also make sure the sequence of abortRequest() call happens in the same
order at all places i.e. after its added to the descriptors_ queue. Fix
one of the abortRequest() call accordingly.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
2021-10-26 16:11:17 +05:30
Umang Jain
ed9eb080e9 android: post_processor_jpeg: Replace encoder_ nullptr check
Instead of simply returning if encoder_ is nullptr, fail hard
via an assertion. It is quite unlikely that encoder_ could only
be null as a result of a fatal bug in the code, so be loud about
the failure.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 16:11:17 +05:30
Umang Jain
d559905c5f android: camera_stream: Replace post-processor nullptr check
Instead of checking postProcessor for nullptr, replace this
check with an assertion that checks if the camera stream's
type is not Type::Direct. Since it makes no sense to call
CameraStream::process() on a Type::Direct camera stream.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-26 16:11:17 +05:30
Kieran Bingham
88a90ba2a7 libcamera: request: Use external CameraControlValidator
Each Request is currently creating its own CameraControlValidator
using the Camera instance at construction.

Now that the Camera exposes its own CameraControlValidator on its
private interface, use that one on all Requests.

Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-25 09:56:50 +01:00
Kieran Bingham
42f5a75001 libcamera: camera: Create a CameraControlValidator
Create a Camera-specific CameraControlValidator for the Camera instance.
This will allow requests to use a single validator instance without
having to construct their own.

Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
2021-10-25 09:56:00 +01:00