Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync csa branch with main #265

Merged
merged 10 commits into from
Jan 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/tools/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Source files for these tools are located at `scripts/tools`.

../scripts/tools/memory/README
../scripts/tools/spake2p/README
../scripts/tools/ELF_SIZE_TOOLING

```

Expand Down
2 changes: 1 addition & 1 deletion integrations/docker/images/base/chip-build/version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
103 : Changed sysroot for crosscompile image (new ubuntu version and includes NFC libs)
104 : [Telink] Update Docker image (Zephyr update)
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ RUN set -x \
&& : # last line

# Setup Zephyr
ARG ZEPHYR_REVISION=c05c461b1119782cc839cf436fa04ec5e1fb2c8c
ARG ZEPHYR_REVISION=52c23bb5bfa7b08fb2499fda8c34cbd3418e0c1d
WORKDIR /opt/telink/zephyrproject
RUN set -x \
&& python3 -m pip install --break-system-packages -U --no-cache-dir west \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ COPY --from=telink /opt/telink/zephyr-sdk-0.16.1 /opt/telink/zephyr-sdk-0.16.1

COPY --from=tizen /opt/tizen-sdk /opt/tizen-sdk

COPY --from=crosscompile /opt/ubuntu-22.04.1-aarch64-sysroot /opt/ubuntu-22.04.1-aarch64-sysroot
COPY --from=crosscompile /opt/ubuntu-24.04-aarch64-sysroot /opt/ubuntu-24.04-aarch64-sysroot

COPY --from=ameba /opt/ameba /opt/ameba

Expand Down Expand Up @@ -124,7 +124,7 @@ ENV NRF5_TOOLS_ROOT=/opt/NordicSemiconductor/nRF5_tools
ENV OPENOCD_PATH=/opt/openocd/
ENV QEMU_ESP32=/opt/espressif/qemu/qemu-system-xtensa
ENV QEMU_ESP32_DIR=/opt/espressif/qemu
ENV SYSROOT_AARCH64=/opt/ubuntu-22.04.1-aarch64-sysroot
ENV SYSROOT_AARCH64=/opt/ubuntu-24.04-aarch64-sysroot
ENV TELINK_ZEPHYR_BASE=/opt/telink/zephyrproject/zephyr
ENV TELINK_ZEPHYR_SDK_DIR=/opt/telink/zephyr-sdk-0.16.1
ENV TI_SYSCONFIG_ROOT=/opt/ti/sysconfig_1.18.1
Expand Down
63 changes: 63 additions & 0 deletions scripts/tools/ELF_SIZE_TOOLING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# ELF binary size information

## Individual size information

`file_size_from_nm.py` is able to build an interactive tree map of
methods/namespaces sizes within an elf binary.

Use it to determine how much space specific parts of the code take. For example:

```
./scripts/tools/file_size_from_nm.py \
--zoom '::chip::app' \
./out/qpg-qpg6105-light/chip-qpg6105-lighting-example.out
```

could result in a graph like:

![image](./FileSizeOutputExample.png)

## Determine difference between two binaries

`binary_elf_size_diff` provides the ability to compare two elf files. Usually
you can build the master branch of a binary and save it somewhere like
`./out/master.elf` and then re-build with changes and compare.

Example runs:

```
> ~/devel/chip-scripts/bindiff.py \
./out/qpg-qpg6105-light/chip-qpg6105-lighting-example.out \
./out/qpg-master.out

Type Size Function
------- ------ -----------------------------------------------------------------------------------------------------------------------
CHANGED -128 chip::app::CodegenDataModelProvider::WriteAttribute(chip::app::DataModel::WriteAttributeRequest const&, chip::app::A...
CHANGED -76 chip::app::InteractionModelEngine::CheckCommandExistence(chip::app::ConcreteCommandPath const&, chip::app::DataModel...
CHANGED -74 chip::app::reporting::Engine::CheckAccessDeniedEventPaths(chip::TLV::TLVWriter&, bool&, chip::app::ReadHandler*)
REMOVED -58 chip::app::DataModel::EndpointFinder::EndpointFinder(chip::app::DataModel::ProviderMetadataTree*)
REMOVED -44 chip::app::DataModel::EndpointFinder::Find(unsigned short)
CHANGED 18 chip::app::WriteHandler::WriteClusterData(chip::Access::SubjectDescriptor const&, chip::app::ConcreteDataAttributePa...
ADDED 104 chip::app::DataModel::ValidateClusterPath(chip::app::DataModel::ProviderMetadataTree*, chip::app::ConcreteClusterPat...
ADDED 224 chip::app::WriteHandler::CheckWriteAllowed(chip::Access::SubjectDescriptor const&, chip::app::ConcreteDataAttributeP...
TOTAL -34


```

```
> ~/devel/chip-scripts/bindiff.py \
--output csv --skip-total \
./out/qpg-qpg6105-light/chip-qpg6105-lighting-example.out ./out/qpg-master.out

Type,Size,Function
CHANGED,-128,"chip::app::CodegenDataModelProvider::WriteAttribute(chip::app::DataModel::WriteAttributeRequest const&, chip::app::AttributeValueDecoder&)"
CHANGED,-76,"chip::app::InteractionModelEngine::CheckCommandExistence(chip::app::ConcreteCommandPath const&, chip::app::DataModel::AcceptedCommandEntry&)"
CHANGED,-74,"chip::app::reporting::Engine::CheckAccessDeniedEventPaths(chip::TLV::TLVWriter&, bool&, chip::app::ReadHandler*)"
REMOVED,-58,chip::app::DataModel::EndpointFinder::EndpointFinder(chip::app::DataModel::ProviderMetadataTree*)
REMOVED,-44,chip::app::DataModel::EndpointFinder::Find(unsigned short)
CHANGED,18,"chip::app::WriteHandler::WriteClusterData(chip::Access::SubjectDescriptor const&, chip::app::ConcreteDataAttributePath const&, chip::TLV::TLVReader&)"
ADDED,104,"chip::app::DataModel::ValidateClusterPath(chip::app::DataModel::ProviderMetadataTree*, chip::app::ConcreteClusterPath const&, chip::Protocols::InteractionModel::Status)"
ADDED,224,"chip::app::WriteHandler::CheckWriteAllowed(chip::Access::SubjectDescriptor const&, chip::app::ConcreteDataAttributePath const&)"

```
Binary file added scripts/tools/FileSizeOutputExample.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
218 changes: 218 additions & 0 deletions scripts/tools/binary_elf_size_diff.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
#!/usr/bin/env -S python3 -B
#
# Copyright (c) 2025 Project CHIP Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# Processes 2 ELF files via `nm` and outputs the
# diferences in size. Example calls:
#
# scripts/tools/bindiff.py \
# ./out/updated_binary.elf \
# ./out/master_build.elf
#
# scripts/tools/bindiff.py \
# --output csv \
# --no-demangle \
# ./out/updated_binary.elf \
# ./out/master_build.elf
#
#
# Requires:
# - click
# - coloredlogs
# - cxxfilt
# - tabulate

import csv
import logging
import os
import subprocess
import sys
from dataclasses import dataclass
from enum import Enum, auto
from pathlib import Path

import click
import coloredlogs
import cxxfilt
import tabulate


@dataclass
class Symbol:
symbol_type: str
name: str
size: int


# Supported log levels, mapping string values required for argument
# parsing into logging constants
__LOG_LEVELS__ = {
"debug": logging.DEBUG,
"info": logging.INFO,
"warn": logging.WARN,
"fatal": logging.FATAL,
}


class OutputType(Enum):
TABLE = (auto(),)
CSV = (auto(),)


__OUTPUT_TYPES__ = {
"table": OutputType.TABLE,
"csv": OutputType.CSV,
}


def get_sizes(p: Path, no_demangle: bool):
output = subprocess.check_output(
["nm", "--print-size", "--size-sort", "--radix=d", p.as_posix()]
).decode("utf8")

result = {}

for line in output.split("\n"):
if not line.strip():
continue

_, size, t, name = line.split(" ")
size = int(size, 10)

if not no_demangle:
name = cxxfilt.demangle(name)

result[name] = Symbol(symbol_type=t, name=name, size=size)

return result


def default_cols():
try:
# if terminal output, try to fit
return os.get_terminal_size().columns - 29
except Exception:
return 120


@click.command()
@click.option(
"--log-level",
default="INFO",
show_default=True,
type=click.Choice(list(__LOG_LEVELS__.keys()), case_sensitive=False),
help="Determines the verbosity of script output.",
)
@click.option(
"--output",
default="TABLE",
show_default=True,
type=click.Choice(list(__OUTPUT_TYPES__.keys()), case_sensitive=False),
help="Determines the type of the output (use CSV for easier parsing).",
)
@click.option(
"--skip-total",
default=False,
is_flag=True,
help="Skip the output of a TOTAL line (i.e. a sum of all size deltas)"
)
@click.option(
"--no-demangle",
default=False,
is_flag=True,
help="Skip CXX demangling. Note that this will not deduplicate inline method instantiations."
)
@click.option(
"--style",
default="simple",
show_default=True,
help="tablefmt style for table output (e.g.: simple, plain, grid, fancy_grid, pipe, orgtbl, jira, presto, pretty, psql, rst)",
)
@click.option(
"--name-truncate",
default=default_cols(),
show_default=True,
type=int,
help="Truncate function name to this length (for table output only). use <= 10 to disable",
)
@click.argument("f1", type=Path)
@click.argument("f2", type=Path)
def main(
log_level,
output,
skip_total,
no_demangle,
style: str,
name_truncate: int,
f1: Path,
f2: Path,
):
log_fmt = "%(asctime)s %(levelname)-7s %(message)s"
coloredlogs.install(level=__LOG_LEVELS__[log_level], fmt=log_fmt)

r1 = get_sizes(f1, no_demangle)
r2 = get_sizes(f2, no_demangle)

output_type = __OUTPUT_TYPES__[output]

# at this point every key has a size information
# We are interested in sizes that are DIFFERENT (add/remove or changed)
delta = []
total = 0
for k in set(r1.keys()) | set(r2.keys()):
if k in r1 and k in r2 and r1[k].size == r2[k].size:
continue

# At this point the value is in v1 or v2
s1 = r1[k].size if k in r1 else 0
s2 = r2[k].size if k in r2 else 0
name = r1[k].name if k in r1 else r2[k].name

if k in r1 and k in r2:
change = "CHANGED"
elif k in r1:
change = "ADDED"
else:
change = "REMOVED"

if (
output_type == OutputType.TABLE
and name_truncate > 10
and len(name) > name_truncate
):
name = name[: name_truncate - 4] + "..."

delta.append([change, s1 - s2, name])
total += s1 - s2

delta.sort(key=lambda x: x[1])
if not skip_total:
delta.append(["TOTAL", total, ""])

HEADER = ["Type", "Size", "Function"]

if output_type == OutputType.TABLE:
print(tabulate.tabulate(delta, headers=HEADER, tablefmt=style))
elif output_type == OutputType.CSV:
writer = csv.writer(sys.stdout)
writer.writerow(HEADER)
writer.writerows(delta)
else:
raise Exception("Unknown output type: %r" % output)


if __name__ == "__main__":
main(auto_envvar_prefix="CHIP")
Loading
Loading