llext: explicitly define loader storage types #81016

pillo79 · 2024-11-06T17:26:29Z

This pull request improves LLEXT heap usage by avoiding unnecessary allocations. It extends the current CONFIG_LLEXT_STORAGE_WRITABLE solution with a mechanism that covers all use cases, and uses this new information to optimize the memory usage when actually possible.

Current situation

LLEXT always allocates a copy of a region on the heap unless these three specific conditions are met:

CONFIG_LLEXT_STORAGE_WRITABLE is set in the build, AND
the current loader implements llext_peek, AND
the section is not BSS, so there is data in the file.

Limitations of the current situation

The above check does not take into account additional constraints imposed by, for example, hardware memory proections. Enabling CONFIG_LLLEXT_STORAGE_WRITABLE with an MPU/MMU-enabled build is definitely unsupported and most probably will not work due to alignment issues.
When the ELF is stored in accessible read only memory, no optimization is performed; even sections that only contain read only data and do not need to have relocations applied, are copied verbatim in the heap and used from there.
The fact that a loader implements the llext_peek function does not mean the pointers returned by it can be stored and used after loading. There may be temporary buffers which provide llext_peek but have a short lifetime.
Having a global Kconfig option does not neatly adapt to some use cases. For example, the same project might have extensions in cold storage (file system), in read-only memory, and copied into RAM. Each llext_load call should use the correct set of optimizations depending on the used loader.

Proposed changes

A new enum llext_storage_type is introduced, which is meant to describe the lifetime and access patterns LLEXT can expect from a loader's data. The following options are available:
- LLEXT_STORAGE_TEMPORARY: ELF data is only available during llext_load(); even if the loader supports directly accessing the memory via llext_peek(), the buffer contents will be discarded afterwards.
  LLEXT will allocate copies of all required data into its heap.
- LLEXT_STORAGE_PERSISTENT: ELF data is stored in a read-only buffer that is guaranteed to be always accessible for as long as the extension is loaded.
  LLEXT may directly reuse constant data from the buffer, but may still allocate copies if relocations need to be applied.
- LLEXT_STORAGE_WRITABLE: ELF data is stored in a writable memory buffer that is guaranteed to be always accessible for as long as the extension is loaded.
  LLEXT may freely modify and reuse data in the buffer; so, after the extension is unloaded, the contents should be re-initialized before a subsequent llext_load() call.
Storage-specific initializers for llext_buf_loader have been added: LLEXT_TEMPORARY_BUF_LOADER, LLEXT_PERSISTENT_BUF_LOADER, LLEXT_WRITABLE_BUF_LOADER. This makes the intent explicit in the code.
The fs_loader is classified as TEMPORARY, which is also the safe default for any custom loader that does not define the new .storage field.
To keep full compatibility with existing code, the storage type of LLEXT_BUF_LOADER is determined by the CONFIG_LLEXT_STORAGE_WRITABLE Kconfig option between LLEXT_STORAGE_PERSISTENT and LLEXT_STORAGE_WRITABLE. New use should be discouraged, because it does not state the desired buffer storage type in the code.
LLEXT now identifies the sections / regions that require relocations early in the loading process. When it's time to choose how to map them, this information is used to enable read-only access even on LLEXT_STORAGE_PERSISTENT buffers, to those sections whose data is actually known not to change. This can greatly improve memory usage, as seen below.

Results

The following table compares heap memory usage, after applying the PR, on targets that support the readonly test suite:

lyakh · 2024-11-07T13:40:50Z

include/zephyr/llext/loader.h

+	/**
+	 * @brief Storage type of the underlying data accessed by this loader.
+	 */
+	enum llext_storage_type storage;


It seems to me this should rather go to struct llext_load_param

As discussed in Discord, IMO the way the data is handled must be strictly tied to the loader.

Consider the case of fs_loader, for example. It is implicitly TEMPORARY - there is no data that can be read long-term from its storage - even though it might even support peek in the future (say, the file is available in some cache, perhaps!).

If you move this field into llext_load_param, people will try to use fs_loader with WRITABLE. It makes no sense and it should fail, but how do you test for this specific case inside llext_load? And how do you support other app-specific loader implementations?

@pillo79 wouldn't it be possible for the user to select a wrong macro with your approach too? Also we can define several macros for the parameters to extend LLEXT_LOAD_PARAM_DEFAULT

@pillo79 wouldn't it be possible for the user to select a wrong macro with your approach too?

Well, the default initializer for fs_loader sets the storage to TEMPORARY, no way around it unless you actually fill the loader struct by yourself. And the different default initializers for BUF_LOADER make any mistake pretty easy to spot.

Also we can define several macros for the parameters to extend LLEXT_LOAD_PARAM_DEFAULT

The whole point of this PR is to unambiguously state the storage type of the data exposed by each loader. Moving this to an optional struct defeats the purpose IMO (and my question above would still apply).

The whole point of this PR is to unambiguously state the storage type of the data exposed by each loader. Moving this to an optional struct defeats the purpose IMO (and my question above would still apply).

@pillo79 ok, I rest my case :-) The point - as you say - seems to be, that loader parameters are optional and we really want an explicit selection of a loader type to be compulsory. Looks a bit confusing IMHO to have "parameters" in two separate locations, but if others are fine with this - I can live with this too.

I believe this is the right spot for this. It's an attribute of the loader type. Imagine a network loader, usb loader, uart loader, etc. We want to know as part of the loader API this attribute.

Making this const perhaps helps signal that its a non-mutable attribute?

Like struct device we could also split this struct up into its immutable parts (this enum and the function pointers) and the mutable parts (hdrs/sects/sect_map) which may further help differentiate the intent as well as how they get placed into an ELF image themselves (ROM vs RAM)

Making this const perhaps helps signal that its a non-mutable attribute?

Makes sense, although it should apply to all function pointers as well. But that last part will wait for the more comprehensive splitting of const/non-const fields which I think is also useful.

github-actions · 2025-01-11T00:33:32Z

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

github-actions · 2025-03-13T00:35:23Z

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

Adding this information allows easy cross-reference of the effects of memory optimization changes between subsequent commits. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>

This patch introduces an 'enum llext_storage_type' that defines the storage type of the ELF data that will be loaded: - LLEXT_STORAGE_TEMPORARY: ELF data is only available during llext_load(); even when the loader supports directly accessing the memory via llext_peek(), the buffer contents will be discarded afterwards. LLEXT will allocate copies of all required data into its heap. - LLEXT_STORAGE_PERSISTENT: ELF data is stored in a *read-only* buffer that is guaranteed to be always accessible for as long as the extension is loaded. LLEXT may directly reuse constant data from the buffer, but may still allocate copies if relocations need to be applied. - LLEXT_STORAGE_WRITABLE: ELF data is stored in a *writable* memory buffer that is guaranteed to be always accessible for as long as the extension is loaded. LLEXT may freely modify and reuse data in the buffer; so, after the extension is unloaded, the contents should be re-initialized before a subsequent llext_load() call. To keep full compatibility with existing code, the storage type of LLEXT_BUF_LOADER is determined by the CONFIG_LLEXT_STORAGE_WRITABLE Kconfig option between LLEXT_STORAGE_PERSISTENT and LLEXT_STORAGE_WRITABLE. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>

This patch adds three new macros to initialize an llext_buf_loader structure with a specific storage type. The existing LLEXT_BUF_LOADER macro is discouraged, since its storage type is not explicitly defined in the code but relies on the CONFIG_LLEXT_STORAGE_WRITABLE option. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>

This patch moves the alignment checks for MPU and MMU regions to the beginning of the llext_copy_region() function. This is done to ensure that the correct region alignment and size are used even in the case where the region is reused from the ELF file buffer, avoiding MMU/MPU configuration issues. This also relaxes the same checks for regions that are accessed by the kernel only (e.g. symbol and string tables), which do not need special MMU/MPU treatment. This exposed an inconsistency in the MMU code, which was setting the permission on the correct regions, but later restoring the default permissions on every region, including the now-unaligned ones. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>

When loading an extension, read-only sections that do not have relocations and whose data is accessible from the ELF buffer can be directly mapped as-is in the extension memory. This avoids the need to allocate and copy unmodified data from the ELF buffer to the LLEXT heap. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>

Add a note on the new loader types available in LLEXT.. Signed-off-by: Luca Burelli <l.burelli@arduino.cc> Signed-off-by:

pillo79 · 2025-03-20T11:10:28Z

Rebased and updated to current LLEXT state.

I want to proceed with this since it unlocks the possibility of having RODATA being left in Flash - a pretty sizable difference when large constants are defined in the code. And with further PLT work, TEXT could also be left there... 🤩

teburd

This seems like a great change, and surprisingly small for the result.

No blockers from me, but maybe some ideas for improvements if you like any of them.

teburd · 2025-03-20T13:35:16Z

include/zephyr/llext/buf_loader.h

+ * @param _buf Buffer containing the ELF binary
+ * @param _buf_len Buffer length in bytes
+ */
+#define LLEXT_BUF_LOADER(_buf, _buf_len) \


I'm in favor of deprecating this given the variants are now available at construction time

I would love to, but it's a little tricky as it is used everywhere in the test suite (and TBH it really works great there!).
I will submit a separate PR if I manage to make the change look nice 🙂

teburd · 2025-03-20T13:40:43Z

include/zephyr/llext/loader.h

+	/**
+	 * @brief Storage type of the underlying data accessed by this loader.
+	 */
+	enum llext_storage_type storage;


I believe this is the right spot for this. It's an attribute of the loader type. Imagine a network loader, usb loader, uart loader, etc. We want to know as part of the loader API this attribute.

Making this const perhaps helps signal that its a non-mutable attribute?

Like struct device we could also split this struct up into its immutable parts (this enum and the function pointers) and the mutable parts (hdrs/sects/sect_map) which may further help differentiate the intent as well as how they get placed into an ELF image themselves (ROM vs RAM)

teburd · 2025-03-20T13:47:19Z

subsys/llext/llext_link.c

 		 */
+		if (ldr->storage != LLEXT_STORAGE_WRITABLE) {


It'd be great to have a nice table somewhere of what the effects of linkages (relocatable vs dynamic) and -fPIC/-fPIE do to the ELF in terms of where relocations are needed.

Like you noted in the PR, it'd be great to support a XIP setup but if this requires rewriting the elf sections into a new on-flash image that's kind of painful. I know at EOSS one idea that came up was having something that would likely be useful for SOF as well which is once again seperating linking and loading, so you could have like a usb mass storage type thing taking images, linking and storing on flash for XIP usage for example.

isn't that also platform- or even toolchain-specific? It would be great to have fully documented what sections are created for each of those build types but it certainly is well out of scope of this PR

Indeed I'd love to read up some kind of documentation on that as well 😁
Rules and behavior of linkers seem to be really scattered folk lore, "the wise guys did that a long time ago" kind of stuff.

It is! I follow the wild linker development as it’s neat, and have used mold, gnu ld, and Linux’s ELF loading code as reference but the lack of information in one spot is a sore point.

teburd · 2025-03-20T13:50:36Z

subsys/llext/llext_load.c

+			enum llext_mem target_region = ldr->sect_map[shdr->sh_info].mem_idx;
+
+			if (target_region != LLEXT_MEM_COUNT) {
+				ldr->sects[target_region].sh_flags |= SHF_LLEXT_HAS_RELOCS;


sh_flags today is mostly there to store the ELF flags. Would it be too disasterous to do something like

ldr->sect_has_relocs[target_region] = true;

instead here? you could even instead set the relocation section if you like...

ldr->sect_to_rel_sect[target_region] = i;

which might be useful, using UINT32_MAX or something to store "no relocation section"

I chose some unused ELF bits to avoid another allocation, but I like the idea and will try to refactor that as well 🙂

lyakh · 2025-03-21T07:28:12Z

subsys/llext/llext_link.c

 		 */
+		if (ldr->storage != LLEXT_STORAGE_WRITABLE) {


isn't that also platform- or even toolchain-specific? It would be great to have fully documented what sections are created for each of those build types but it certainly is well out of scope of this PR

lyakh · 2025-03-21T07:58:20Z

doc/services/llext/load.rst

-:c:struct:`llext_buf_loader`.
+needed by the :c:func:`llext_load` function. Several loaders are already provided:
+
+ * An implementation over a buffer containing an ELF in addressable memory in


some typo here, did you mean "if?"

Reused the text without looking, thanks for spotting! 👍

lyakh · 2025-03-21T07:59:11Z

doc/services/llext/load.rst

+   memory is available as :c:struct:`llext_buf_loader`. To use this kind of
+   loader, it is helpful to use one of the :c:macro:`LLEXT_TEMPORARY_BUF_LOADER`,
+   :c:macro:`LLEXT_PERSISTENT_BUF_LOADER`, or :c:macro:`LLEXT_WRITABLE_BUF_LOADER`
+   macros to tell LLEXT the appropriate type of memory buffer.


should we mention the wrappers directly?

I think so. This is introducing LLEXT, so the first thing you need to know is how to start tinkering with it... IMHO they are even more useful than the struct reference 🙂

pillo79

Thanks for the review comments, will do another round with the suggested improvements 👍

pillo79 · 2025-03-20T15:32:41Z

include/zephyr/llext/buf_loader.h

+ * @param _buf Buffer containing the ELF binary
+ * @param _buf_len Buffer length in bytes
+ */
+#define LLEXT_BUF_LOADER(_buf, _buf_len) \


I would love to, but it's a little tricky as it is used everywhere in the test suite (and TBH it really works great there!).
I will submit a separate PR if I manage to make the change look nice 🙂

pillo79 · 2025-03-21T08:28:01Z

doc/services/llext/load.rst

-:c:struct:`llext_buf_loader`.
+needed by the :c:func:`llext_load` function. Several loaders are already provided:
+
+ * An implementation over a buffer containing an ELF in addressable memory in


Reused the text without looking, thanks for spotting! 👍

pillo79 · 2025-03-21T08:32:58Z

doc/services/llext/load.rst

+   memory is available as :c:struct:`llext_buf_loader`. To use this kind of
+   loader, it is helpful to use one of the :c:macro:`LLEXT_TEMPORARY_BUF_LOADER`,
+   :c:macro:`LLEXT_PERSISTENT_BUF_LOADER`, or :c:macro:`LLEXT_WRITABLE_BUF_LOADER`
+   macros to tell LLEXT the appropriate type of memory buffer.


I think so. This is introducing LLEXT, so the first thing you need to know is how to start tinkering with it... IMHO they are even more useful than the struct reference 🙂

pillo79 · 2025-03-21T08:37:27Z

include/zephyr/llext/loader.h

+	/**
+	 * @brief Storage type of the underlying data accessed by this loader.
+	 */
+	enum llext_storage_type storage;


Making this const perhaps helps signal that its a non-mutable attribute?

Makes sense, although it should apply to all function pointers as well. But that last part will wait for the more comprehensive splitting of const/non-const fields which I think is also useful.

pillo79 · 2025-03-21T08:44:43Z

subsys/llext/llext_link.c

 		 */
+		if (ldr->storage != LLEXT_STORAGE_WRITABLE) {


Indeed I'd love to read up some kind of documentation on that as well 😁
Rules and behavior of linkers seem to be really scattered folk lore, "the wise guys did that a long time ago" kind of stuff.

pillo79 · 2025-03-21T08:48:48Z

subsys/llext/llext_load.c

+			enum llext_mem target_region = ldr->sect_map[shdr->sh_info].mem_idx;
+
+			if (target_region != LLEXT_MEM_COUNT) {
+				ldr->sects[target_region].sh_flags |= SHF_LLEXT_HAS_RELOCS;


I chose some unused ELF bits to avoid another allocation, but I like the idea and will try to refactor that as well 🙂

lyakh reviewed Nov 7, 2024

View reviewed changes

github-actions bot added the Stale label Jan 11, 2025

pillo79 removed the Stale label Jan 11, 2025

github-actions bot added the Stale label Mar 13, 2025

pillo79 added 3 commits March 19, 2025 11:37

llext: (cosmetic) add allocated size to debug message

9249e32

Adding this information allows easy cross-reference of the effects of memory optimization changes between subsequent commits. Signed-off-by: Luca Burelli <l.burelli@arduino.cc>

pillo79 force-pushed the pr-llext-storage branch from 111f46e to 597a47b Compare March 19, 2025 10:39

pillo79 removed the Stale label Mar 19, 2025

pillo79 force-pushed the pr-llext-storage branch from 597a47b to af528cd Compare March 19, 2025 11:06

pillo79 mentioned this pull request Mar 19, 2025

[DNM] llext: test loader storage types thesofproject/sof#9909

Draft

pillo79 added 2 commits March 19, 2025 14:13

pillo79 force-pushed the pr-llext-storage branch from af528cd to f4f67eb Compare March 19, 2025 13:20

doc: llext: add note on the loader types

e55e2d2

Add a note on the new loader types available in LLEXT.. Signed-off-by: Luca Burelli <l.burelli@arduino.cc> Signed-off-by:

pillo79 marked this pull request as ready for review March 19, 2025 17:27

zephyrbot added the area: llext Linkable Loadable Extensions label Mar 19, 2025

zephyrbot requested a review from teburd March 19, 2025 17:28

zephyrbot assigned teburd Mar 19, 2025

pillo79 requested a review from lyakh March 20, 2025 11:10

teburd approved these changes Mar 20, 2025

View reviewed changes

lyakh approved these changes Mar 21, 2025

View reviewed changes

pillo79 commented Mar 21, 2025

View reviewed changes

nashif merged commit 766122f into zephyrproject-rtos:main Mar 21, 2025
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

llext: explicitly define loader storage types #81016

llext: explicitly define loader storage types #81016

pillo79 commented Nov 6, 2024 •

edited

Loading

lyakh Nov 7, 2024

pillo79 Nov 7, 2024

lyakh Nov 8, 2024

pillo79 Nov 8, 2024

lyakh Nov 11, 2024

teburd Mar 20, 2025

pillo79 Mar 21, 2025

github-actions bot commented Jan 11, 2025

github-actions bot commented Mar 13, 2025

pillo79 commented Mar 20, 2025

teburd left a comment

teburd Mar 20, 2025

pillo79 Mar 20, 2025

teburd Mar 20, 2025

teburd Mar 20, 2025

lyakh Mar 21, 2025

pillo79 Mar 21, 2025

teburd Mar 21, 2025 •

edited

Loading

teburd Mar 20, 2025

pillo79 Mar 21, 2025

lyakh Mar 21, 2025

lyakh Mar 21, 2025

pillo79 Mar 21, 2025

lyakh Mar 21, 2025

pillo79 Mar 21, 2025

pillo79 left a comment

pillo79 Mar 20, 2025

pillo79 Mar 21, 2025

pillo79 Mar 21, 2025

pillo79 Mar 21, 2025

pillo79 Mar 21, 2025

pillo79 Mar 21, 2025

llext: explicitly define loader storage types #81016

llext: explicitly define loader storage types #81016

Conversation

pillo79 commented Nov 6, 2024 • edited Loading

Current situation

Limitations of the current situation

Proposed changes

Results

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jan 11, 2025

github-actions bot commented Mar 13, 2025

pillo79 commented Mar 20, 2025

teburd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

teburd Mar 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pillo79 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pillo79 commented Nov 6, 2024 •

edited

Loading

teburd Mar 21, 2025 •

edited

Loading