NekoCWD/libcamera
NekoCWD/libcamera
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 7 Wiki Activity Actions

controls: Introduce AEGC-related controls

Browse source 
Introduce the AeState, ExposureTimeMode and AnalogueGainMode controls
to model the AEGC algorithm block.

The three controls allow applications to select the exposure time and
analogue gain computation calculation mode (auto vs manual)
independently from one another, while the AeState control reports the
global state for the AEGC algorithm.

The new controls are meant to replace the existing AeEnable and AeLocked
controls, which are momentarily kept not to break compilation of
platforms making use of them.

Bug: https://bugs.libcamera.org/show_bug.cgi?id=42
Bug: https://bugs.libcamera.org/show_bug.cgi?id=43
Bug: https://bugs.libcamera.org/show_bug.cgi?id=47
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
This commit is contained in:
Paul Elder 2025-01-13 15:59:35 -06:00 • committed by Laurent Pinchart
parent b01918978c
commit 3becdbcbe8
2 changed files with 245 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

265
src/libcamera/control_ids_core.yaml
View file

@ -28,6 +28,71 @@ controls:
\sa AeEnable
- AeState:
type: int32_t
direction: out
description: |
Report the AEGC algorithm state.
The AEGC algorithm computes the exposure time and the analogue gain
to be applied to the image sensor.
The AEGC algorithm behaviour is controlled by the ExposureTimeMode and
AnalogueGainMode controls, which allow applications to decide how
the exposure time and gain are computed, in Auto or Manual mode,
independently from one another.
The AeState control reports the AEGC algorithm state through a single
value and describes it as a single computation block which computes
both the exposure time and the analogue gain values.
When both the exposure time and analogue gain values are configured to
be in Manual mode, the AEGC algorithm is quiescent and does not actively
compute any value and the AeState control will report AeStateIdle.
When at least the exposure time or analogue gain are configured to be
computed by the AEGC algorithm, the AeState control will report if the
algorithm has converged to stable values for all of the controls set
to be computed in Auto mode.
\sa AnalogueGainMode
\sa ExposureTimeMode
enum:
- name: AeStateIdle
value: 0
description: |
The AEGC algorithm is inactive.
This state is returned when both AnalogueGainMode and
ExposureTimeMode are set to Manual and the algorithm is not
actively computing any value.
- name: AeStateSearching
value: 1
description: |
The AEGC algorithm is actively computing new values, for either the
exposure time or the analogue gain, but has not converged to a
stable result yet.
This state is returned if at least one of AnalogueGainMode or
ExposureTimeMode is auto and the algorithm hasn't converged yet.
The AEGC algorithm converges once stable values are computed for
all of the controls set to be computed in Auto mode. Once the
algorithm converges the state is moved to AeStateConverged.
- name: AeStateConverged
value: 2
description: |
The AEGC algorithm has converged.
This state is returned if at least one of AnalogueGainMode or
ExposureTimeMode is Auto, and the AEGC algorithm has converged to a
stable value.
If the measurements move too far away from the convergence point
then the AEGC algorithm might start adjusting again, in which case
the state is moved to AeStateSearching.
# AeMeteringMode needs further attention:
# - Auto-generate max enum value.
# - Better handling of custom types.
@ -109,6 +174,13 @@ controls:
The exposure modes specify how the desired total exposure is divided
between the exposure time and the sensor's analogue gain. They are
platform specific, and not all exposure modes may be supported.
When one of AnalogueGainMode or ExposureTimeMode is set to Manual,
the fixed values will override any choices made by AeExposureMode.
\sa AnalogueGainMode
\sa ExposureTimeMode
enum:
- name: ExposureNormal
value: 0
@ -130,13 +202,15 @@ controls:
Specify an Exposure Value (EV) parameter.
The EV parameter will only be applied if the AE algorithm is currently
enabled.
enabled, that is, at least one of AnalogueGainMode and ExposureTimeMode
are in Auto mode.
By convention EV adjusts the exposure as log2. For example
EV = [-2, -1, -0.5, 0, 0.5, 1, 2] results in an exposure adjustment
of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x].
\sa AeEnable
\sa AnalogueGainMode
\sa ExposureTimeMode
- ExposureTime:
type: int32_t
@ -146,17 +220,108 @@ controls:
This value is specified in micro-seconds.
Setting this value means that it is now fixed and the AE algorithm may
not change it. Setting it back to zero returns it to the control of the
AE algorithm.
This control will only take effect if ExposureTimeMode is Manual. If
this control is set when ExposureTimeMode is Auto, the value will be
ignored and will not be retained.
\sa AnalogueGain AeEnable
When reported in metadata, this control indicates what exposure time
was used for the current frame, regardless of ExposureTimeMode.
ExposureTimeMode will indicate the source of the exposure time value,
whether it came from the AE algorithm or not.
\todo Document the interactions between AeEnable and setting a fixed
value for this control. Consider interactions with other AE features,
such as aperture and aperture/shutter priority mode, and decide if
control of which features should be automatically adjusted shouldn't
better be handled through a separate AE mode control.
\sa AnalogueGain
\sa ExposureTimeMode
- ExposureTimeMode:
type: int32_t
direction: inout
description: |
Controls the source of the exposure time that is applied to the image
sensor.
When set to Auto, the AE algorithm computes the exposure time and
configures the image sensor accordingly. When set to Manual, the value
of the ExposureTime control is used.
When transitioning from Auto to Manual mode and no ExposureTime control
is provided by the application, the last value computed by the AE
algorithm when the mode was Auto will be used. If the ExposureTimeMode
was never set to Auto (either because the camera started in Manual mode,
or Auto is not supported by the camera), the camera should use a
best-effort default value.
If ExposureTimeModeManual is supported, the ExposureTime control must
also be supported.
Cameras that support manual control of the sensor shall support manual
mode for both ExposureTimeMode and AnalogueGainMode, and shall expose
the ExposureTime and AnalogueGain controls. If the camera also has an
AEGC implementation, both ExposureTimeMode and AnalogueGainMode shall
support both manual and auto mode. If auto mode is available, it shall
be the default mode. These rules do not apply to black box cameras
such as UVC cameras, where the available gain and exposure modes are
completely dependent on what the device exposes.
\par Flickerless exposure mode transitions
Applications that wish to transition from ExposureTimeModeAuto to direct
control of the exposure time without causing extra flicker can do so by
selecting an ExposureTime value as close as possible to the last value
computed by the auto exposure algorithm in order to avoid any visible
flickering.
To select the correct value to use as ExposureTime value, applications
should accommodate the natural delay in applying controls caused by the
capture pipeline frame depth.
When switching to manual exposure mode, applications should not
immediately specify an ExposureTime value in the same request where
ExposureTimeMode is set to Manual. They should instead wait for the
first Request where ExposureTimeMode is reported as
ExposureTimeModeManual in the Request metadata, and use the reported
ExposureTime to populate the control value in the next Request to be
queued to the Camera.
The implementation of the auto-exposure algorithm should equally try to
minimize flickering and when transitioning from manual exposure mode to
auto exposure use the last value provided by the application as starting
point.
1. Start with ExposureTimeMode set to Auto
2. Set ExposureTimeMode to Manual
3. Wait for the first completed request that has ExposureTimeMode
set to Manual
4. Copy the value reported in ExposureTime into a new request, and
submit it
5. Proceed to run manual exposure time as desired
\sa ExposureTime
enum:
- name: ExposureTimeModeAuto
value: 0
description: |
The exposure time will be calculated automatically and set by the
AE algorithm.
If ExposureTime is set while this mode is active, it will be
ignored, and its value will not be retained.
When transitioning from Manual to Auto mode, the AEGC should start
its adjustments based on the last set manual ExposureTime value.
- name: ExposureTimeModeManual
value: 1
description: |
The exposure time will not be updated by the AE algorithm.
When transitioning from Auto to Manual mode, the last computed
exposure value is used until a new value is specified through the
ExposureTime control. If an ExposureTime value is specified in the
same request where the ExposureTimeMode is changed from Auto to
Manual, the provided ExposureTime is applied immediately.
- AnalogueGain:
type: float
@ -167,17 +332,77 @@ controls:
The value of the control specifies the gain multiplier applied to all
colour channels. This value cannot be lower than 1.0.
Setting this value means that it is now fixed and the AE algorithm may
not change it. Setting it back to zero returns it to the control of the
AE algorithm.
This control will only take effect if AnalogueGainMode is Manual. If
this control is set when AnalogueGainMode is Auto, the value will be
ignored and will not be retained.
\sa ExposureTime AeEnable
When reported in metadata, this control indicates what analogue gain
was used for the current request, regardless of AnalogueGainMode.
AnalogueGainMode will indicate the source of the analogue gain value,
whether it came from the AEGC algorithm or not.
\todo Document the interactions between AeEnable and setting a fixed
value for this control. Consider interactions with other AE features,
such as aperture and aperture/shutter priority mode, and decide if
control of which features should be automatically adjusted shouldn't
better be handled through a separate AE mode control.
\sa ExposureTime
\sa AnalogueGainMode
- AnalogueGainMode:
type: int32_t
direction: inout
description: |
Controls the source of the analogue gain that is applied to the image
sensor.
When set to Auto, the AEGC algorithm computes the analogue gain and
configures the image sensor accordingly. When set to Manual, the value
of the AnalogueGain control is used.
When transitioning from Auto to Manual mode and no AnalogueGain control
is provided by the application, the last value computed by the AEGC
algorithm when the mode was Auto will be used. If the AnalogueGainMode
was never set to Auto (either because the camera started in Manual mode,
or Auto is not supported by the camera), the camera should use a
best-effort default value.
If AnalogueGainModeManual is supported, the AnalogueGain control must
also be supported.
For cameras where we have control over the ISP, both ExposureTimeMode
and AnalogueGainMode are expected to support manual mode, and both
controls (as well as ExposureTimeMode and AnalogueGain) are expected to
be present. If the camera also has an AEGC implementation, both
ExposureTimeMode and AnalogueGainMode shall support both manual and
auto mode. If auto mode is available, it shall be the default mode.
These rules do not apply to black box cameras such as UVC cameras,
where the available gain and exposure modes are completely dependent on
what the hardware exposes.
The same procedure described for performing flickerless transitions in
the ExposureTimeMode control documentation can be applied to analogue
gain.
\sa ExposureTimeMode
\sa AnalogueGain
enum:
- name: AnalogueGainModeAuto
value: 0
description: |
The analogue gain will be calculated automatically and set by the
AEGC algorithm.
If AnalogueGain is set while this mode is active, it will be
ignored, and it will also not be retained.
When transitioning from Manual to Auto mode, the AEGC should start
its adjustments based on the last set manual AnalogueGain value.
- name: AnalogueGainModeManual
value: 1
description: |
The analogue gain will not be updated by the AEGC algorithm.
When transitioning from Auto to Manual mode, the last computed
gain value is used until a new value is specified through the
AnalogueGain control. If an AnalogueGain value is specified in the
same request where the AnalogueGainMode is changed from Auto to
Manual, the provided AnalogueGain is applied immediately.
- AeFlickerMode:
type: int32_t

30
src/libcamera/control_ids_draft.yaml
View file

@ -80,36 +80,6 @@ controls:
High quality aberration correction which might reduce the frame
rate.
- AeState:
type: int32_t
direction: out
description: |
Control to report the current AE algorithm state. Currently identical to
ANDROID_CONTROL_AE_STATE.
Current state of the AE algorithm.
enum:
- name: AeStateInactive
value: 0
description: The AE algorithm is inactive.
- name: AeStateSearching
value: 1
description: The AE algorithm has not converged yet.
- name: AeStateConverged
value: 2
description: The AE algorithm has converged.
- name: AeStateLocked
value: 3
description: The AE algorithm is locked.
- name: AeStateFlashRequired
value: 4
description: The AE algorithm would need a flash for good results
- name: AeStatePrecapture
value: 5
description: |
The AE algorithm has started a pre-capture metering session.
\sa AePrecaptureTrigger
- AwbState:
type: int32_t
direction: out