Feedback on Removing the Isolate & Latency Questions

[Cornfritter]

Happy to have found out about this plugin! I've been looking for more precise audio playback in Flutter for a project which is basically an audio sequencer. It has an audio event loop in an isolate which triggers sounds.

First, I've tried the branch without the built-in isolate, and it's great for this project. I can run SoLoud in the same isolate as the event loop, eliminating latency from sending messages between the event loop and playback isolates. I hope this is incorporated into the main branch.

I'm running into other latency issues though, and hope someone might have some advice.

1. The elapsed time of await play seems to spike 10x every once in a while (when playing a series of short sounds), from on the order of 0.1ms to 1ms.

2. Measuring latency with only the elapsed time of await play doesn't seem sufficient. I hear more fluctuations than are represented by that. So I think there must be something in SoLoud taking a variable amount of time after the handle is returned.

3. When initially playing a series of sounds, the elapsed time for the first sound is significantly higher, around 4ms, than the rest (excluding the above issues). This can be mitigated by around half by playing the loaded sounds at 0 volume before playing the sequence. It's still quite high though.

Any suggestions or info about this would be much appreciated! And/or where to start looking in the source code for causes of these things; I assume SoLoud has been well optimized over the years though... I also realize my environment might be a factor, running in an iPhone simulator.

Thanks a lot!

[alnitak]

Hi @Cornfritter, thanks for your feedback!! Hope the PR without the isolate will land soon.

As I understand your wish is to implement an audio sequencer. Here a low latency is important, but we are using a language with a Garbage Collector. This means that we cannot prevent/predict when GC will do its job. This job is always made on the main isolate and can produce the latency problem. It could take 20 ms when it needs to generate collection and up to (IIRC) 100 or more ms when it compacts memory.

Saying that, the C code does not have a GC and maybe it's possible to optimize it for a sequencer use case. On the Dart side IMHO there is not much we can do, but maybe, I never tried, the use of microtasks (Future.microtask() or scheduleMicrotask()).

There is also the "audio buffer" length variable when we encounter latency problem. This is discussed here.

If you want to look at the code, it could be somehow simple: in the src/bindings.cpp are exposed the functions called with dart:ffi from Flutter. It is used to manage src/player.cpp which is a layer on top of SoLoud cpp library.

I'll appreciate any of your findings!

[Cornfritter]

@alnitak, thank you! That's very helpful info. I was thinking about buffer size. I'll report back anything which might be useful.

[Cornfritter]

Reducing the buffer size resolves most of the fluctuations I was hearing! And I agree with the discussion in that other thread that a feature to configure buffer size in Dart would be really helpful.

Inspired by this, I'll probably implement the audio loop in C with FFI to improve things further.

Thanks again!

[alnitak]

Let me know your progress, even by email if you want.

[Cornfritter]

Hi again- I've made a lot of progress, mainly on the sequencer which now runs in a pthread in C called with Dart FFI from an isolate. I haven't made any adjustments to Soloud besides the buffer size, but I'm becoming more familiar with the code.

I think the most helpful thing would be if I could call the Soloud play function from C. Everything else isn't as time-critical, and I'd rather not start from scratch with the Soloud library.

So, do you think this is possible? I guess it'd mean passing a reference to the Soloud instance (and maybe AudioSources) from Dart to C. Any other ideas?

[Cornfritter]

To clarify, I'm talking about calling Soloud in C from C. Now, I have a callback from C to Dart where I call Soloud.

[alnitak]

If I understand well, you want to call SoLoud C lib from C. This happens in player.cpp where the SoLoud::Soloud soloud; is defined in the header.
The bindings.cpp has the duty to expose player.cpp to the Dart side.

So, do you think this is possible? I guess it'd mean passing a reference to the Soloud instance (and maybe AudioSources) from Dart to C. Any other ideas?

The SoLoud lib works with SoLoud::AudioSource (ie to play that audio source) or with SoLoud::handle (ie the unique handle returned by the play() for that audio source).
In player.cpp is stored a vector of active sounds: std::vector<std::shared_ptr<ActiveSound>> sounds;. This means that from Dart you can pass sound hashes or handles to find the correct SoLoud::AudioSource in that vector.

So from Dart side you only need to pass the AudioSource.SoundHandle.id if needed or the Dart AudioSource.SoundHash.hash to find the owner of that hash in the ActiveSound vector.

The Dart SoLoud.activeSounds reflect what in C is std::vector<std::shared_ptr<ActiveSound>> sounds;.

Hope this helps!

[Cornfritter]

Thanks so much for the info, and sorry for the delayed reply! Just getting back to this. I realize this is probably more of a Dart FFI question (obviously I'm new to it).

I'm still not clear if it's possible to access the same instance of Soloud from Dart and C? I probably didn't explain this well before. What I'm looking to do is to init and other things from Dart, and to play from C. I see where a pointer to player is declared in bindings.cpp, but I don't see where to get a hold of it on the Dart side (to pass it back to C, if that's how to do it). Again, any info much appreciated!!