The CommitChecker, StyleChecker and Formatter classes duplicate code.
Create a new CheckerBase class to factor out common code.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The tracepoints.h file is generated from the tracepoints.h.in template
by the gen-tp-header.py script. The template contains a {{year}}
placeholder for the copyright year, which the script fills with the
current year. This breaks reproducible builds with at least the openSUSE
debugsource package.
As the gen-tp-header.py script doesn't add any copyrightable contents to
the tracepoints.h file, fix this by replacing the {{year}} placeholder
with the year of the last copyright-worthy change to tracepoints.h.in.
Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The Co-developed-by tag must be followed by a corresponding SoB line.
Enforce this rule in the git pre-push hook.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
While building the core libcamera library for ABI compatibility checks,
the pycamera python bindings may get built if the required dependencies
are found.
The ABI compatibility checks do not validate or verify the Python API -
so remove this from the intermediate build steps.
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The issues detected and fixed by autopep8 are the same as the ones
detected by pycodestyle. As the formatter runs unconditionally we can
remove the checker.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reporting style issues on python files is great, automatically fixing
them is even better. Add a call to autopep8 for python files. This fixes
the same issues as the ones reported by pycodestyle.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The ipa_interface.h file includes a number of headers that are not
directly used. Remove them, and add them to the source files that
include ipa_interface.h as required.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Depending on the types used in the IPA interface, generated headers may
use the std::string class. Include <string> when needed.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
When given a relative path to the kernel git tree,
update-kernel-headers.sh fails to execute the headers_install.sh script
from the kernel sources. Fix it by turning the kernel directory into an
absolute path.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Replace manual extraction of data from YAML with the Control helper
class. This centralizes YAML parsing and avoids manual mistakes.
In order to import the controls module, add the utils/codegen/ directory
to the PYTHONPATH through the Python build environment.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The ControlEnum and Control helper classes defined in gen-controls.py
are useful for other generator scripts. Move them to a separate file to
make it possible to share them.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Jinja2 templates help separate the logic related to the template from
the generation of the data. The python code becomes much clearer as a
result.
As an added bonus, we can use a single template file for both controls
and properties.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The libcamera.h header is a top-level library header that contains every
other libcamera header. It is currently generated by listing the files
in include/libcamera/ and dropping the .in suffix from template files.
This assumes a 1:1 mapping between generated header file names and the
name of their templates.
Drop that assumption and base the libcamera.h generation on the
libcamera public headers listed in meson.build. This makes the
libcamera.h header generation more future-proof.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Python scripts run as part of the build process need to take a few
actions specific to the environment in which they operate. One of those
is disabling the Python bytecode cache, to avoid writing .pyc files to
the source tree. This is done manually in the IPC generate.py and
parser.py scripts.
The current implementation is not ideal because it hardcodes in the
scripts information related to the environment in which they operate. As
those scripts are part of libcamera this is more of a theoretical issue
than a practical one. A second issue is that future Python scripts will
need to duplicate similar mechanisms, resulting in a higher maintenance
burden.
Address the issue with a different approach, by creating a meson
environment for the Python scripts, and passing it to the
custom_target() functions. The environment only disables the bytecode
cache for now.
The diffstat shows an increase in code size. This is expected to be
offset by usage of the environment for more Python scripts, as well as
support of more variables in the environment.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
We have multiple code generation scripts in utils/, mixed with other
miscellaneous utilities, as well as a larger code base based on mojom in
utils/ipc/. To make code sharing easier between the generator scripts,
without creating a mess in the utils/ directory, move all the code
generation code to utils/codegen/.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Change the first parameter of Tuner.add() to accept either a list of
modules or a single module. This allows more compact code and is in sync
with Tuner.set_output_order().
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The instances of the static modules need to be passed to
tuner.set_output_order() instead of the class. This is the reason for
the intermediate variables. To keep the code tidy, apply the same
pattern to the other modules.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
When debugging issues with the Commit class, a __repr__ method proved to
be useful to quickly print all the parsed information about a commit. To
avoid reimplementing the method over and over again in the future, add
it to the class, even if it is not actually used.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
When running checkstyle.py in a pre-commit hook, there is either no
commit message at all (when committing staged changes as a new commit),
or the commit message comes from a previous commit being amended. In
either case, there's no new commit message yet, and thus nothing to
validate for the title and trailers checkers. The trailers checker, in
particular, will always flag an error, making all commits fail.
To fix that, just skip the two checkers during pre-commit.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
When parsing commit messages, the Commit class tries to optimize the
process by invoking git-show once only, extracting both the commit
author, title and modified files in one go. As a result, the derived
Amendment class needs to implement the commit parsing separately, as the
modified files need to be extracted differently for amendments (and the
commit ID also needs to be retrieved differently). Furthermore, because
of the list of named files, extracting the trailers needs to invoke
git-show separately.
Improve the situation by reworking the commit message parsing in three
steps. In the first step, use git-show to extract the commit ID, author,
title and body. In the second step, invoke git-interpret-trailers to
extract the trailers from the body that was previously extracted. The
third and final step extracts the list of modified files, using
different methods for regular commits and amendments.
This allows sharing code for the first two steps between the Commit and
Amendment classes, making the code simpler. The Commit class still
invokes git three times, while the Amendment class runs it three times
instead of four, improving performance slightly.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Trailers are extracted from commits using the '(trailers:*)' formatting
specifier. git ignores dividers when doing so, as if the --no-divider
options was passed to git-interpret-trailers. This prevents trailers
from being located when the patch contains a local changelog.
There is unfortuantely no 'no-no-divider' option to the trailers format
specifier. Fix the issue by extracting trailers with
git-interpret-trailers, that gives better control of the parsing.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The TrailersChecker enforces the presence of a Signed-off-by tag in the
trailer, but doesn't verify that the tag matches the commit's author.
Add that verification, as required by the libcamera contribution
process.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Extend the Commit class with an author property, retrieved from the
commit. It will be used to extend checkers.
While at it, drop the unneeded .strip() call when retrieving the title
for amendment commits. The call got carried over from code that
initially needed it to strip the new line character, but that need
disappeard with usage of .splitlines().
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
All commits to libcamera must include a Signed-off-by line, and that
rule is enforced through git hooks and CI. This however doesn't prevent
patches from being submitted without an SoB tag, as noticed multiple
times in the past. Extend the checkstyle.py trailer checker to issue a
warning when the SoB line is missing to try and improve the situation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Milan Zamazal <mzamazal@redhat.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The version string of libcamera is presently appended with a date/time
argument in the case of a dirty tree to show further detail of when it
was built. This string is generated with the iso-8601 parameter to
'date' and produces a date string in the form:
"2024-06-17T22:15:30+01:00"
Strict posix shells or implementations of 'date' which may be optimised
for size such as the one provided by busybox or Alpine Linux may not
support the '--iso-8601' flag.
To support builds on those platforms, use a direct POSIX-compliant
date-format string instead matching as closely as possible to the
iso-8601 definition.
An exact match is not possible with those restrictions so 'date
%Y-%m-%dT%H:%M:%S%Z' is used which produces 2024-06-17T22:15:30BST.
The use of the human readable timezone identifier provides a friendlier
output for the string than a timezone offset at the expense of being
less sortable in the exceptionally low risk and unlikely event of two
custom builds being compared at the same time of different timezones.
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Zach DeCook <zachdecook@librem.one>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
With the addition of gamma out correction the relative luminance target
was set too low. As brightness is a bit subjective it is difficult to
come up with the "correct" value. With 0.5 the patch 22 on the macbeth
chart (neutral grey, 18% reflectance) ended up a bit below 50% grey,
which seems reasonable.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
In cases where the calibration image contains super dark areas, or when
an invalid blacklevel was supplied, the grid might get close to zero or
negative. This would have bad effects on the 1/grid later. So clamp the
values to a small positive number.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
The current LSC algorithm for the rkisp1 just forwards the LSC tables to
the hardware, so absolute factors are needed and not ratios compared to
green. Therefore every channel needs to be calculated independently.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Add awb, blc, cproc, filter, and gamma to the tuning file. These don't
need any configuration.
At the moment there are no inter-module dependencies in the tuning
process. We can therefore safely sort them alphabetically. As soon as
the first dependency gets introduced (most likely lsc -> ccm) we will
see how to solve that.
The output order controls the order of processing in the IPA. It is now
also in alphabetical order which happens to be no change for the modules
that existed previously. For the others, there is no need for a specific
order.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add a static module class, that can be used to add static data to the
tuning file. This is propably not the best solution, but allows us to
progress without writing lots of dummy classes for static cases.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
There are valid cases where a module returns None. E.g. no images were
provided for LSC calibration. We should however define proper semantics
there. Continue with a warning for now.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Make it clear that no lsc calibration was done by returning None instead
of a incomplete configuration.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Implement a minimal ccm calibration module. For now it doesn't take the
results from lsc into account and supports rkisp1 only.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Remove the need for the Cam object, as we don't want to port
it from Raspberry Pi.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
In the Image class the variable holding the color temperature is named
color instead of col which was used by the raspberry pi scripts. Rename
accordingly.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Based on the input images, the lsc values could exceed the range allowed
by the rkisp1. As we are now clipping the values, we can simplify the
value mapping.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Print a info on every image that gets processed and a warning on every
image that gets ignored.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add the missing pieces and store the result inside the image object.
This solution is not very nice, and should be refactored soon. For that
we need a concept to collect temperature and/or image specific results
in a central place. For now it serves the purpose.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
At the moment this just reads the yaml file and returns it verbatim.
This needs to evolve further in the near future.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In the tuning datasets, the files had names like
'imx335_1600l_3000k_1.dng'. That failed on the old filename parsing
function. As there is no need to dictate the order of the tags, split
the big regex into chunks and parse them one by one. This also makes
the code easier to digest.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The old function uses PIL to save the image, which is not in the
requirements file. As we are already requiring opencv, use that to save
images instead of an additional dependency
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
In ctt_ccm.py the logging functionality of the Cam object was used. As
we don't want to port over that class, it needs to be replaced anyways.
While at it, also replace the eprint function as it doesn't add any
value over the logging framework and misses the ability for easy log
formatting.
For nice output formatting add the coloredlogs library.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Fix imports to match new structure in the files copied from Raspberry
Pi. Add missing imports in macbeth.py. Add missing dependencies to
requirements.txt.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Add a requirements file to ease the installation and use of the tuning
scripts. Document that in the readme. No debian packages are provided as
rawpy is not packaged as deb. So pip has to be used anyways.
Also add pyyaml which was missing in the dependencies.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Copy visualize_macbeth_chart from raspberry pi. It is copied verbatim
and does not work in this state.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Paul Elder <paul.elder@ideasonboard.com>
Copy ctt_{awb,ccm,colors,ransac} from the raspberrypi tuning scripts as
basis for the libcamera implementation. color.py was renamed to
ctt_colors.py to better express the origin.
The files were taken from commit 66479605ba ("utils: raspberrypi: ctt:
Improve the Macbeth Chart search reliability").
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Acked-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Acked-by: Paul Elder <paul.elder@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Opencv fails to load the image. The added license destroys the magic
number. Fix, by moving the licence below the magic number.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul ELder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Port commit 66479605ba ("utils: raspberrypi: ctt: Improve the Macbeth
Chart search reliability") into libtuning.
Previously the code would brighten up images in case the Macbeth Chart
is slightly dark, and also zoom in on sections of it to look for
charts occupying less of the field of view. But it would not do both
together.
This change makes the search for smaller charts also repeat that
search for the brightened up images that it made earlier, thereby
increasing the chances of success for non-optimal tuning images.
Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The Image class incorrectly parses data in GBRG bayer formats as the
indices to the channels are set incorrectly - fix it.
Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
