Using LuaJIT

[espdev]

Hello,

It would be nice to have the benefits of using JIT for transforms written in Lua. mlua crate supports LuaJIT as a feature. Have you considered enabling this feature for Vector build? Is there any reason why LuaJIT is not being used in Vector now? At a minimum, it would be nice to be able to customize Lua version when building Vector. Currently it is hardcoded as lua54.

Thanks.

[jszwedko]

Hi @espdev !

I believe that would be a compile time option rather than something we could do at runtime; i.e. we have to pick one to distribute Vector with. You can change that value and build Vector locally to use LuaJIT though. I'm not terribly familiar with the Lua ecosystem, but one thing that jumps out about LuaJIT is that it appears to only guarantee compatibility with Lua 5.1 programs which seems limiting.

[espdev]

@jszwedko

A compile time option is a good idea, that would be enough. I understand that LuaJIT is late with support for a modern version of Lua. However, it is often the performance that is needed, and not all the modern features of the language. For example, I just need to have a statefull transform, and Lua version 5.1 or 5.2 will probably be enough in my case.

[jszwedko]

Gotcha, yeah, that is good context. I'd suggest building Vector and swapping in LuaJIT and seeing if it improves performance for you (you can swap the mlua feature flags in Cargo.toml). It'd be nice if we could support that as a runtime option, but it doesn't seem like it.

[khvzak]

I would suggest trying Luau (supported by mlua) instead. It's faster than Lua 5.4, has optional static types, plus many modern features like string interpolation and sandboxing.
Also seems Roblox actively working on adding AoT/JIT to Luau which would make it a good alternative to LuaJIT.

[splitice]

I would actually second luajit over any more esoteric lua implementations. It's modern enough for most people and significantly faster (although it might be worth checking that with tincy scripts)

Openresty uses it and its pretty damn awesome

[splitice]

FYI To anyone who thinks lua has no place. We use luajit in small area to fill a niche that I doubt vrl will ever fill. State shared between events.

We use it to tag netconsole (linux kenel) messages with hostnames (which we supply to the same source via a different channel).

This is not the kind of thing I would ever expect to see VRL handle.