Report footprint changes in CI

[nashif]

A nice feature we used to have back in the days is the ability to report footprint changes, RAM and ROM for each pull request. We would measure the baseline data and then measure with the PR commits applied and report if there are any changes.

We are planning to introduce the same things utilising the size_report script we already have which generates a report for both RAM and ROM reports of any Zephyr binaries. A diff tool can then be used to extract the data and report differences, so for example if we generate RAM reports for Zephyr v2.5.0 and then another report for current main branch we can generate the diff this way:

# ~/fpdiff.py ram_v2_5_0.json ram_2_6_0_rc3.json
root -> +1328
:/_kernel -> -4
tests/benchmarks/footprints/src/workq.c/workq -> +16
arch/arm/core/aarch32/cortex_m/mpu/nxp_mpu.c/static_regions_num (-1) disappeared.
:/k_sys_work_q (+192) is new.
drivers/clock_control/clock_control_mcux_mcg.c/__devstate_dts_ord_35 (+4) is new.
drivers/clock_control/clock_control_mcux_sim.c/__devstate_dts_ord_8 (+4) is new.
drivers/timer/sys_clock_init.c/__devstate_sys_init_sys_clock_driver_init0 (+4) is new.
drivers/pinmux/pinmux_mcux.c/__devstate_dts_ord_17 (+4) is new.
drivers/pinmux/pinmux_mcux.c/__devstate_dts_ord_20 (+4) is new.
drivers/pinmux/pinmux_mcux.c/__devstate_dts_ord_24 (+4) is new.
drivers/pinmux/pinmux_mcux.c/__devstate_dts_ord_27 (+4) is new.
drivers/pinmux/pinmux_mcux.c/__devstate_dts_ord_43 (+4) is new.
drivers/gpio/gpio_mcux.c/__devstate_dts_ord_18 (+4) is new.
drivers/gpio/gpio_mcux.c/__devstate_dts_ord_21 (+4) is new.
drivers/gpio/gpio_mcux.c/__devstate_dts_ord_25 (+4) is new.
drivers/gpio/gpio_mcux.c/__devstate_dts_ord_28 (+4) is new.
drivers/gpio/gpio_mcux.c/__devstate_dts_ord_44 (+4) is new.
drivers/serial/uart_mcux.c/__devstate_dts_ord_61 (+4) is new.
drivers/serial/uart_mcux.c/__devstate_dts_ord_70 (+4) is new.
arch/arm/core/aarch32/mpu/nxp_mpu.c/static_regions_num (+1) is new.
kernel/work.c/pending_cancels (+8) is new.
kernel/system_work_q.c/sys_work_q_stack (+1056) is new.

The script is actually very simple and basic, can be enhanced to visualise the output and make it more user-friendly:

#!/usr/bin/env python3

from anytree.importer import DictImporter
from anytree import RenderTree, Node, RenderTree, AsciiStyle, PreOrderIter
from anytree.search  import find
import sys
from colorama import Fore, Back, Style
import json

importer = DictImporter()

with open(sys.argv[1], "r") as f:
    data = json.load(f)
    root1 = importer.import_(data['symbols'])

with open(sys.argv[2], "r") as f:
    data = json.load(f)
    root2 = importer.import_(data['symbols'])

for node in PreOrderIter(root1):
    n = find(root2, lambda node2: node2.identifier == node.identifier)
    if n:
        if n.size != node.size:
            diff = n.size - node.size
            if not n.children or not n.parent:
                if diff < 0:
                    print(f"{n.identifier} -> {Fore.GREEN}{diff}{Fore.RESET}")
                else:
                    print(f"{n.identifier} -> {Fore.RED}+{diff}{Fore.RESET}")

    else:
        if not node.children:
            print(f"{node.identifier} ({Fore.GREEN}-{node.size}{Fore.RESET}) disappeared.")

for node in PreOrderIter(root2):
    n = find(root1, lambda node2: node2.identifier == node.identifier)
    if not n:
        if not node.children:
            print(f"{node.identifier} ({Fore.RED}+{node.size}{Fore.RESET}) is new.")

Provide and feedback or suggestions here please.

[nashif]

@robwoolley FYI

[nashif]

the above json files were produced using
west build -b frdm_k64f tests/benchmarks/footprints and then ran the size_report script directly, because the format has changed between the 2 releases, so for example:

size_report -e build/zephyr/zephyr.elf  -z ${PWD} -o build/  --json ram1.json ram

[nashif]

...and there is a bug in the generated JSON file with duplicate identifiers that needs to be fix..