The ControlEnum and Control helper classes defined in gen-controls.py
are useful for other generator scripts. Move them to a separate file to
make it possible to share them.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Jinja2 templates help separate the logic related to the template from
the generation of the data. The python code becomes much clearer as a
result.
As an added bonus, we can use a single template file for both controls
and properties.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The libcamera.h header is a top-level library header that contains every
other libcamera header. It is currently generated by listing the files
in include/libcamera/ and dropping the .in suffix from template files.
This assumes a 1:1 mapping between generated header file names and the
name of their templates.
Drop that assumption and base the libcamera.h generation on the
libcamera public headers listed in meson.build. This makes the
libcamera.h header generation more future-proof.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Python scripts run as part of the build process need to take a few
actions specific to the environment in which they operate. One of those
is disabling the Python bytecode cache, to avoid writing .pyc files to
the source tree. This is done manually in the IPC generate.py and
parser.py scripts.
The current implementation is not ideal because it hardcodes in the
scripts information related to the environment in which they operate. As
those scripts are part of libcamera this is more of a theoretical issue
than a practical one. A second issue is that future Python scripts will
need to duplicate similar mechanisms, resulting in a higher maintenance
burden.
Address the issue with a different approach, by creating a meson
environment for the Python scripts, and passing it to the
custom_target() functions. The environment only disables the bytecode
cache for now.
The diffstat shows an increase in code size. This is expected to be
offset by usage of the environment for more Python scripts, as well as
support of more variables in the environment.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
We have multiple code generation scripts in utils/, mixed with other
miscellaneous utilities, as well as a larger code base based on mojom in
utils/ipc/. To make code sharing easier between the generator scripts,
without creating a mess in the utils/ directory, move all the code
generation code to utils/codegen/.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Change the first parameter of Tuner.add() to accept either a list of
modules or a single module. This allows more compact code and is in sync
with Tuner.set_output_order().
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The instances of the static modules need to be passed to
tuner.set_output_order() instead of the class. This is the reason for
the intermediate variables. To keep the code tidy, apply the same
pattern to the other modules.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
When debugging issues with the Commit class, a __repr__ method proved to
be useful to quickly print all the parsed information about a commit. To
avoid reimplementing the method over and over again in the future, add
it to the class, even if it is not actually used.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
When running checkstyle.py in a pre-commit hook, there is either no
commit message at all (when committing staged changes as a new commit),
or the commit message comes from a previous commit being amended. In
either case, there's no new commit message yet, and thus nothing to
validate for the title and trailers checkers. The trailers checker, in
particular, will always flag an error, making all commits fail.
To fix that, just skip the two checkers during pre-commit.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
When parsing commit messages, the Commit class tries to optimize the
process by invoking git-show once only, extracting both the commit
author, title and modified files in one go. As a result, the derived
Amendment class needs to implement the commit parsing separately, as the
modified files need to be extracted differently for amendments (and the
commit ID also needs to be retrieved differently). Furthermore, because
of the list of named files, extracting the trailers needs to invoke
git-show separately.
Improve the situation by reworking the commit message parsing in three
steps. In the first step, use git-show to extract the commit ID, author,
title and body. In the second step, invoke git-interpret-trailers to
extract the trailers from the body that was previously extracted. The
third and final step extracts the list of modified files, using
different methods for regular commits and amendments.
This allows sharing code for the first two steps between the Commit and
Amendment classes, making the code simpler. The Commit class still
invokes git three times, while the Amendment class runs it three times
instead of four, improving performance slightly.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Trailers are extracted from commits using the '(trailers:*)' formatting
specifier. git ignores dividers when doing so, as if the --no-divider
options was passed to git-interpret-trailers. This prevents trailers
from being located when the patch contains a local changelog.
There is unfortuantely no 'no-no-divider' option to the trailers format
specifier. Fix the issue by extracting trailers with
git-interpret-trailers, that gives better control of the parsing.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The TrailersChecker enforces the presence of a Signed-off-by tag in the
trailer, but doesn't verify that the tag matches the commit's author.
Add that verification, as required by the libcamera contribution
process.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the Commit class with an author property, retrieved from the
commit. It will be used to extend checkers.
While at it, drop the unneeded .strip() call when retrieving the title
for amendment commits. The call got carried over from code that
initially needed it to strip the new line character, but that need
disappeard with usage of .splitlines().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
All commits to libcamera must include a Signed-off-by line, and that
rule is enforced through git hooks and CI. This however doesn't prevent
patches from being submitted without an SoB tag, as noticed multiple
times in the past. Extend the checkstyle.py trailer checker to issue a
warning when the SoB line is missing to try and improve the situation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The version string of libcamera is presently appended with a date/time
argument in the case of a dirty tree to show further detail of when it
was built. This string is generated with the iso-8601 parameter to
'date' and produces a date string in the form:
"2024-06-17T22:15:30+01:00"
Strict posix shells or implementations of 'date' which may be optimised
for size such as the one provided by busybox or Alpine Linux may not
support the '--iso-8601' flag.
To support builds on those platforms, use a direct POSIX-compliant
date-format string instead matching as closely as possible to the
iso-8601 definition.
An exact match is not possible with those restrictions so 'date
%Y-%m-%dT%H:%M:%S%Z' is used which produces 2024-06-17T22:15:30BST.
The use of the human readable timezone identifier provides a friendlier
output for the string than a timezone offset at the expense of being
less sortable in the exceptionally low risk and unlikely event of two
custom builds being compared at the same time of different timezones.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Zach DeCook <zachdecook@librem.one>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
With the addition of gamma out correction the relative luminance target
was set too low. As brightness is a bit subjective it is difficult to
come up with the "correct" value. With 0.5 the patch 22 on the macbeth
chart (neutral grey, 18% reflectance) ended up a bit below 50% grey,
which seems reasonable.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
In cases where the calibration image contains super dark areas, or when
an invalid blacklevel was supplied, the grid might get close to zero or
negative. This would have bad effects on the 1/grid later. So clamp the
values to a small positive number.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The current LSC algorithm for the rkisp1 just forwards the LSC tables to
the hardware, so absolute factors are needed and not ratios compared to
green. Therefore every channel needs to be calculated independently.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Add awb, blc, cproc, filter, and gamma to the tuning file. These don't
need any configuration.
At the moment there are no inter-module dependencies in the tuning
process. We can therefore safely sort them alphabetically. As soon as
the first dependency gets introduced (most likely lsc -> ccm) we will
see how to solve that.
The output order controls the order of processing in the IPA. It is now
also in alphabetical order which happens to be no change for the modules
that existed previously. For the others, there is no need for a specific
order.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add a static module class, that can be used to add static data to the
tuning file. This is propably not the best solution, but allows us to
progress without writing lots of dummy classes for static cases.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
There are valid cases where a module returns None. E.g. no images were
provided for LSC calibration. We should however define proper semantics
there. Continue with a warning for now.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Make it clear that no lsc calibration was done by returning None instead
of a incomplete configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Implement a minimal ccm calibration module. For now it doesn't take the
results from lsc into account and supports rkisp1 only.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Remove the need for the Cam object, as we don't want to port
it from Raspberry Pi.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
In the Image class the variable holding the color temperature is named
color instead of col which was used by the raspberry pi scripts. Rename
accordingly.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Based on the input images, the lsc values could exceed the range allowed
by the rkisp1. As we are now clipping the values, we can simplify the
value mapping.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Print a info on every image that gets processed and a warning on every
image that gets ignored.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add the missing pieces and store the result inside the image object.
This solution is not very nice, and should be refactored soon. For that
we need a concept to collect temperature and/or image specific results
in a central place. For now it serves the purpose.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
At the moment this just reads the yaml file and returns it verbatim.
This needs to evolve further in the near future.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In the tuning datasets, the files had names like
'imx335_1600l_3000k_1.dng'. That failed on the old filename parsing
function. As there is no need to dictate the order of the tags, split
the big regex into chunks and parse them one by one. This also makes
the code easier to digest.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The old function uses PIL to save the image, which is not in the
requirements file. As we are already requiring opencv, use that to save
images instead of an additional dependency
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In ctt_ccm.py the logging functionality of the Cam object was used. As
we don't want to port over that class, it needs to be replaced anyways.
While at it, also replace the eprint function as it doesn't add any
value over the logging framework and misses the ability for easy log
formatting.
For nice output formatting add the coloredlogs library.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Fix imports to match new structure in the files copied from Raspberry
Pi. Add missing imports in macbeth.py. Add missing dependencies to
requirements.txt.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add a requirements file to ease the installation and use of the tuning
scripts. Document that in the readme. No debian packages are provided as
rawpy is not packaged as deb. So pip has to be used anyways.
Also add pyyaml which was missing in the dependencies.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Copy visualize_macbeth_chart from raspberry pi. It is copied verbatim
and does not work in this state.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Paul Elder <paul.elder@ideasonboard.com>
Copy ctt_{awb,ccm,colors,ransac} from the raspberrypi tuning scripts as
basis for the libcamera implementation. color.py was renamed to
ctt_colors.py to better express the origin.
The files were taken from commit 66479605ba ("utils: raspberrypi: ctt:
Improve the Macbeth Chart search reliability").
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Paul Elder <paul.elder@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Opencv fails to load the image. The added license destroys the magic
number. Fix, by moving the licence below the magic number.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul ELder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Port commit 66479605ba ("utils: raspberrypi: ctt: Improve the Macbeth
Chart search reliability") into libtuning.
Previously the code would brighten up images in case the Macbeth Chart
is slightly dark, and also zoom in on sections of it to look for
charts occupying less of the field of view. But it would not do both
together.
This change makes the search for smaller charts also repeat that
search for the brightened up images that it made earlier, thereby
increasing the chances of success for non-optimal tuning images.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The Image class incorrectly parses data in GBRG bayer formats as the
indices to the channels are set incorrectly - fix it.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Add the skeletal AGC module to the rkisp1 tuning script. For now it just
spits out hardcoded values.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add a skeletal AGC module just so that we can have some AGC tuning
values that we can use to test during development of AGC in the IPAs. As
rkisp1 is the main target, we only add support for rkisp1 for now.
The parameters are mostly copied from the hardcoded values in ctt,
except for the metering modes.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
A max_gain parameter is added to the config file which we pass to the
lens shading calibration. This clamps the maximum luminance gain that
gets written into the tuning files so as to prevent overflows.
It is particularly useful for lenses that cut off the light completely
from the sensor corners, and allows usable tables to be generated for
them.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
This change adds functionality to the convert_tuning.py script to
convert between vc4 and pisp target tuning files. The conversion is
done on a best effort basis, and should provide functional tuning files.
However, a full tuning for the target platform is always preferred.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The various boilerplate parts of the tuning file are extended to
include the necessary extra bits for HDR, specifically:
* rpi.denoise has different configurations for HDR modes
* rpi.agc now has extra channels for HDR
* rpi.hdr parameters are added.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Changed how users select which platform to tune for. Now users
specify a command line argument, '-t', to specify which target
platform.
Signed-off-by: Ben Benson <ben.benson@raspberrypi.com>
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Added the ability to tune the chromatic aberration correction
within the ctt. There are options for cac_only or to tune as part
of a larger tuning process. CTT will now recognise any files that
begin with "cac" as being chromatic aberration tuning files.
Signed-off-by: Ben Benson <ben.benson@raspberrypi.com>
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The old ctt.py and alsc_only.py scripts are removed.
Instead of ctt.py use ctt_vc4.py or ctt_pisp.py, depending on your
target platform.
Instead of alsc_only.py use alsc_vc4.py or alsc_pisp.py, again
according to your platform.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Naushir Patuck <naush@raspberrypi.com>
Tested-by: Naushir Patuck <naush@raspberrypi.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
linux/udmabuf.h will be used by upcoming DmaBufAllocator changes and it
is not available on some older kernels.
Add it to the headers to sync.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The issue checkers display the line number and line content of each
offending line, but don't show the location of the issue within a line.
Improve checkstyle by adding a marker that points to the exact location.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
