Fix issue with specialization using arithmetic expressions (#6084) #6946

Job	Run time
build (linux, debug, gcc, x86_64)	13m 53s
build (linux, release, gcc, x86_64)	19m 59s
build (linux, release, gcc, wasm)	18m 36s
build (macos, debug, clang, aarch64)	9m 10s
build (macos, release, clang, aarch64)	19m 31s
build (windows, debug, cl, x86_64)	17m 1s
build (windows, release, cl, x86_64)	18m 17s
build (linux, release, gcc, aarch64, smoke, false, self-hosted, Linux, ARM64, true)	23m 45s
	2h 20m 12s

Provide feedback