Refactor uart (#13585)

* serial - refactor serial resource handling

- port related resources (tx/rx/inverted,dma)
  - are stored sequentially, with space in place of ports that are not
  enabled (RESOURCE_UART_COUNT + RESOURCE_LPUART_COUNT +
     RESOURCE_SOFTSERIAL_COUNT)
  - RESOURCE_UART_OFFSET, RESOURCE_LPUART_OFFSET, RESOURCE_SOFTSERIAL_OFFSET
  - resource entries are pointing into this array (UART, LPUART, SOFTSERIAL)
    - both pins and DMA
    - inverter is supproted only for UART + LPUART (makes sense only
     for UART, to be removed)
  - softSerialPinConfig is removed, it is no longer necessary
- serialResourceIndex(identifier) is used universally to get correct
   index into resource array
- unified handling of resources where possible
- serialOwnerTxRx() + serialOwnerIndex() are used for displaying resources
   correctly to user.
- serialType(identifier) implemented
  - serialOwnerTxRx / serialOwnerIndex are trivial with it
  - large switch() statemens are greatly simplified

* serial - merge code duplicated in all UART implementations

- drivers/serial_uart_hw.c contains merged serialUART code.
  Code did not match exactly. Obvious cases are fixed, more complicated
  use #ifs
- pin inversion unified
- uartOpen is refactored a bit

* serial - refactor uartDevice

- use 'compressed' enum from uartDeviceIdx_e. Only enabled ports are
   in this enum.
- uartDeviceIdx directly indexes uartDevice (no search necessary, no
    wasted space)
- use `serialPortIdentifier_e identifier;` for uartHardware
- for DMA remap, define only entries for enabled ports (uartDeviceIdx_e
   does not exist disabled port)

* serial - apply changes to inverter

New implementation is trivial

* serial - HAL - rxIrq, txIrq replaces by irqn

There is only one IRQ for serial

* serial - serial_post.h

Generated code to normalize target configuration.
jinja2 template is used to generate header file. Port handling is
unified a lot.

SERIAL_<type><n>_USED 0/1 - always defined, value depends on target configuration
SERIAL_<type>_MASK        - bitmask of used ports or given type. <port>1 is BIT(0)
SERIAL_<type>_COUNT       - number of enabled ports of given type
SERIAL_<type>_MAX         - <index of highest used port> + 1, 0 when no port is enabled

* targets - remove automatically calculated valued from configs

serial_post.h generated it

* serial - remove definitions replaced by serial_post.h

* serial - change LPUART to UART_LP1 in places

LPUART is mostly handled as another UART port, this change reflects it

* serial - use ARRAYLEN / ARRAYEND in some places

replaces constant that may not match actual array size

* serial - adapt softserial to changes

* serial - whitespace, comments

* softserial - fix serialTimerConfigureTimebase

* serial - suspicious code

* serial - unittests

* serial - cleanup

* serial - simpler port names

Is this useful ?

* serial - no port count for SITL necessary

* serial - fix unittests

include serial_post.h, some ports are defined, so normalization will
work fine

* timers - remove obsolete defines from SITL and unittests

* serial - cosmetic improvements

comments, whitespace, minor refactoring

* serial - fix serialInit bug

serialPortToDisable was preventing further tests

* serial - fix possible NULL pointer dereference

* serial - move serialResourceIndex to drivers

* serial - refactor pp/od and pulldown/pullup/nopull

Centralize serial options handling, decouple it from MCU type

Move some code into new drivers/serial_impl.c

* serial - make port.identifier valid early in port initialization

* serial - fix unittest

Code is moved around a bit to make unittests implementation easier

* serial - bug - fix off-by-one error

uart1/softserial1/lpuart was not working properly because of this

* serial - whipespace + formating + style + comments

No functional changes

* utils - add popcount functions

Wrap __builtin_popcount for easier use. Refactor existing code using
it. Update code using BITCOUNT macro in dynamic context (BITCOUNT is
for compile-time use only)

* serial - inverter - simplify code

* pinio - prevent array access overflow

* serial - refactor SERIAL_BIDIR_*

SERIAL_BIDIR_OD / SERIAL_BIDIR_PP
SERIAL_PULL_DEFAULT / SERIAL_PULL_NONE / SERIAL_PULL_PD

* serial - simplify code

minor refactoring
- cleaner AVOID_UARTx_FOR_PWM_PPM (unused anyway)
- serialFindPortConfiguration* - remove code duplication
- use serialType in smartaudio

* serial - add port names

- const string is assiociated with each compiled-in port (easy to pass around)
- findSerialPortByName

* cli - improve serialpassthrough

- support port options (in current mode argument)
- support port names (`serialpassthrough uart1`)
- improve error handling; check more parse errors

* cli - resources - minor refactor

- prevent SERIAL_TX_NONSENSE when parsing resource type
- fix possible NULL pointer dereference
- store resource tag only after checking all conditions
- slighty better error reporting
- `resource serial_tx 1` will print resource assignment

* serial - remane pulldown to SERIAL_PULL_SMARTAUDIO

Make sure nobody uses it by
mistake. SERIAL_PULL_SMARTAUDIO_SUPER_DANGEROUS_USE_AT_YOUR_OWN_RISK_THIS_WILL_BREAK_EVERYTHING_AND_KILL_YOUR_PET_FISH
would be better choice, but try shorter version first.

* smartaudio - minor refactor/improvement

- Fix softserial on AT32
- make it easier to handle SA on per-cpu basis

* softserial - minor refactoring

no functional changes

* UART - move AF handling before MCU dependent code

* UART - adapt APM32 / APM32F4 code

- Modeled after F4 where possible
- come code moved from APM32 to APM32F4, possbily not necessary, but
   it improves similarity with STM32F4

* UART - APM32 - remove per-pin AF

* UART - APM32 - bugfix - fix pinswap #if conditions

* UART - apply some improvemnts from APM32 to STM32

* UART - add todo for F4 serial speed

* UART - fix typo

* UART - refactor support for USE_SMARTAUDIO_NOPULLDOWN

* UART - typos, comments, minor improvements

* UART - move code into enableRxIrq

TODO: split into mcu-specific filer (but in sepatate PR)

* UART - add UART_TRAIT_xx

makes #if test easier and more consistent
more traits shall follow

* UART - fix variable name

unused branch, would trigger compilation error otherwise

* UART - use tables instead of switch

* UART - smartaudio

minor style change + better comments

* UART - unify mspPorts iteration

* UART - fix spelling

* UART - another typo

* UART - fix serialResourceIndex

offset must be added, not subtracted
offset can be negative

* UART - fix typo

Bad day ...

* UART - use const table

GCC does optimize it better.
Should not cause functional change

* UART - use OwnerIndex for inverter

- 1 based
- only UART has inversion

* UART - refactor serial_resource a bit

Single table + helper function. New table is easier to read

* UART - serial_impl header is necessary

* UART - style-only changes

typos
unify whitespace
comment improvement
add some const modifiers
use cfg in uartConfigureDma
minor improvemnt of for loops

* UART - remove LPUART inverter

LPUART always supports internal inversion, code was incomplete and
unused

* UART - update jinja template, regenerate files

* UART - enable UART module RCC clk before configuring DMA

Does not change anything, UART RCCs are enabled unconditionally

src/utils/gen-serial-j2.py

@@ -0,0 +1,94 @@
+from jinja2 import Environment, FileSystemLoader
+import jinja2
+import os
+from datetime import datetime
+import pprint
+
+# generate normalization preprocessor file for serial ports
+
+# configuration for template generation
+serials = {
+    "UART": {"ids": [i + 1 for i in range(10)],
+             "inverter": True,
+             },
+    "LPUART": {"ids": [1],
+               "depends": {"UART"},
+#               "inverter": True,   # TODO: old code compatibility only, disabled
+               },
+    "SOFTSERIAL": {"ids": [i + 1 for i in range(2)],
+                   "use_enables_all": True,
+                   "force_continuous": True,
+                   },
+    "VCP": {"singleton": True,
+            "no_pins": True}
+}
+
+def flatten_config(data):
+    flattened_list = []
+    for key, value in data.items():
+        flattened_dict = {'typ': key}
+        # Update this new dictionary with the inner dictionary's items
+        flattened_dict.update(value)
+        flattened_list.append(flattened_dict)
+    return flattened_list
+
+def pprint_filter(value, indent=4):
+    return pprint.pformat(value, indent=indent)
+
+def rdepends_filter(cfg, typ):
+    return list(set([ c['typ'] for c in cfg if typ in c['depends'] ]))
+
+def main():
+    # Setup Jinja2 environment
+    current_dir = os.path.dirname(os.path.abspath(__file__))
+    # Set the template directory relative to the current script
+    template_dir = os.path.join(current_dir, 'templates')
+    env = Environment(loader=FileSystemLoader(template_dir),
+                      autoescape=True,
+                      undefined=jinja2.StrictUndefined,  # Throws an error on undefined variables
+                      trim_blocks = True,
+                      )
+    env.filters['pprint'] = pprint_filter
+    env.filters['zip'] = zip
+    env.filters['rdepends'] = rdepends_filter
+    template = env.get_template('serial_post.h')
+
+    # Read license file
+    license_path = 'DEFAULT_LICENSE.md'
+    with open(license_path, 'r') as file:
+        t_license = file.read()
+
+    context = {}
+    context['license'] = t_license
+    context['date_generated'] = datetime.now().strftime('%Y-%m-%d')
+    context['user_config'] = serials
+    config = flatten_config(serials)
+    context['config'] = config
+
+    for cfg in config:
+        singleton = cfg.setdefault('singleton', False)
+        no_pins = cfg.setdefault('no_pins', False)
+        inverter = cfg.setdefault('inverter', False)
+        cfg.setdefault("use_enables_all", False)
+        cfg.setdefault("force_continuous", False)
+        cfg.setdefault("depends", {})
+        typ = cfg['typ']
+        if singleton:
+            cfg['ports']= [ f"{typ}" ]
+        else:
+            cfg['ports'] = [ f"{typ}{i}" for i in cfg["ids"] ]
+
+    # Render the template with the license content
+    output = template.render(**context)
+
+    if False:
+        # Output or save to a file
+        output_path = './src/main/target/serial_post.h'
+        with open(output_path, 'w') as file:
+            file.write(output)
+        print(f"Generated file saved to {output_path}")
+    else:
+        print(output)
+
+if __name__ == '__main__':
+    main()