Does MLT cache avformat producers with same resource file path?

[remorses]

If I create an MLT graph (using an xml file) with 2 avformat producers that point to the same resource video file, does MLT treat them as 2 different files?

Does the avformat producer have to open the file 2 times to get the video metadata?

[ddennedy]

Yes, producers are separate; it does not cache in the manner that you asked about.

[remorses]

Do you think it's possible to add a cache using the producer resource file path as key? Would you like a PR for this?

Or maybe I can sponsor this work? This feature would greatly improve performance for my use case.

[ddennedy]

I’m not interested in doing the work or maintaining it in the framework or avformat producer.

You can already reference a producer multiple times in a composition with a caveat. Mixing an avformat producer with itself such as in a transition like cross fade may produce corrupt audio because there is a buffer of decoded audio samples in the object’s state. But it’s been a long time since I tried it and looked into it. It could be now there is something to flush that buffer between calls to get audio from non-sequential frames.

So, your script or application can maintain a map of file name to producer to reuse the objects. Or, are you using the melt command line tool?

[remorses]

I am currently using the xml producer to create the MLT graph. What I really care about is the initialization speed, the time it takes for ffmpeg to analyze the video file and start consuming frames.

I think this feature would also help improve Shotcut performance and other editors that depend on MLT.

For example, when dragging the same audio file in the Shotcut editor it produces 2 different chains. Adding a global cache would help making opening Shotcut projects that reference the same file open and render much faster.

Generated mlt file example

<?xml version="1.0" standalone="no"?>
<mlt LC_NUMERIC="C" version="7.30.0" title="Shotcut version 25.01.25" producer="main_bin">
  <profile description="automatic" width="1710" height="1280" progressive="1" sample_aspect_num="1" sample_aspect_den="1" display_aspect_num="171" display_aspect_den="128" frame_rate_num="60" frame_rate_den="1" colorspace="709"/>
  <playlist id="main_bin">
    <property name="xml_retain">1</property>
  </playlist>
  <producer id="black" in="00:00:00.000" out="00:01:34.533">
    <property name="length">00:01:34.550</property>
    <property name="eof">pause</property>
    <property name="resource">0</property>
    <property name="aspect_ratio">1</property>
    <property name="mlt_service">color</property>
    <property name="mlt_image_format">rgba</property>
    <property name="set.test_audio">0</property>
  </producer>
  <playlist id="background">
    <entry producer="black" in="00:00:00.000" out="00:01:34.533"/>
  </playlist>
  <chain id="chain0" out="00:01:18.617">
    <property name="length">00:01:18.633</property>
    <property name="eof">pause</property>
    <property name="resource">/Volumes/1tb sabrent/downloaded youtube videos/Dune - The Planet Arrakis - (HDR - 4K - 5.1) [ng_DIUTuBZE]_no_vocals.mp3</property>
    <property name="mlt_service">avformat-novalidate</property>
    <property name="meta.media.nb_streams">1</property>
    <property name="meta.media.0.stream.type">audio</property>
    <property name="meta.media.0.codec.sample_fmt">fltp</property>
    <property name="meta.media.0.codec.sample_rate">44100</property>
    <property name="meta.media.0.codec.channels">2</property>
    <property name="meta.media.0.codec.layout">stereo</property>
    <property name="meta.media.0.codec.name">mp3float</property>
    <property name="meta.media.0.codec.long_name">MP3 (MPEG audio layer 3)</property>
    <property name="meta.media.0.codec.bit_rate">320000</property>
    <property name="seekable">1</property>
    <property name="audio_index">0</property>
    <property name="video_index">-1</property>
    <property name="creation_time">2025-03-13T16:12:09</property>
    <property name="astream">0</property>
    <property name="shotcut:skipConvert">1</property>
    <property name="shotcut:hash">563f241b0436a5dcb5f26cb36efd3cf9</property>
    <property name="shotcut:caption">Dune - The Planet Arrakis - (HDR - 4K - 5.1) [ng_DIUTuBZE]_no_vocals.mp3</property>
    <property name="xml">was here</property>
  </chain>
  <chain id="chain1" out="00:01:18.617">
    <property name="length">00:01:18.633</property>
    <property name="eof">pause</property>
    <property name="resource">/Volumes/1tb sabrent/downloaded youtube videos/Dune - The Planet Arrakis - (HDR - 4K - 5.1) [ng_DIUTuBZE]_no_vocals.mp3</property>
    <property name="mlt_service">avformat-novalidate</property>
    <property name="meta.media.nb_streams">1</property>
    <property name="meta.media.0.stream.type">audio</property>
    <property name="meta.media.0.codec.sample_fmt">fltp</property>
    <property name="meta.media.0.codec.sample_rate">44100</property>
    <property name="meta.media.0.codec.channels">2</property>
    <property name="meta.media.0.codec.layout">stereo</property>
    <property name="meta.media.0.codec.name">mp3float</property>
    <property name="meta.media.0.codec.long_name">MP3 (MPEG audio layer 3)</property>
    <property name="meta.media.0.codec.bit_rate">320000</property>
    <property name="seekable">1</property>
    <property name="audio_index">0</property>
    <property name="video_index">-1</property>
    <property name="creation_time">2025-03-13T16:12:09</property>
    <property name="astream">0</property>
    <property name="shotcut:skipConvert">1</property>
    <property name="shotcut:hash">563f241b0436a5dcb5f26cb36efd3cf9</property>
    <property name="shotcut:caption">Dune - The Planet Arrakis - (HDR - 4K - 5.1) [ng_DIUTuBZE]_no_vocals.mp3</property>
  </chain>
  <playlist id="playlist0">
    <property name="shotcut:video">1</property>
    <property name="shotcut:name">V1</property>
    <entry producer="chain0" in="00:00:00.000" out="00:00:15.900"/>
    <entry producer="chain1" in="00:00:00.000" out="00:01:18.617"/>
  </playlist>
  <tractor id="tractor0" title="Shotcut version 25.01.25" in="00:00:00.000" out="00:01:34.533">
    <property name="shotcut">1</property>
    <property name="shotcut:projectAudioChannels">2</property>
    <property name="shotcut:projectFolder">0</property>
    <track producer="background"/>
    <track producer="playlist0"/>
    <transition id="transition0">
      <property name="a_track">0</property>
      <property name="b_track">1</property>
      <property name="mlt_service">mix</property>
      <property name="always_active">1</property>
      <property name="sum">1</property>
    </transition>
    <transition id="transition1">
      <property name="a_track">0</property>
      <property name="b_track">1</property>
      <property name="version">0.1</property>
      <property name="mlt_service">frei0r.cairoblend</property>
      <property name="threads">0</property>
      <property name="disable">1</property>
    </transition>
  </tractor>
</mlt>

[ddennedy]

You can do what I described in your own XML.

[ddennedy]

Regarding performance most files people are using can degrade performance because seeking is expensive due to temporal compression. If multiple references to the same producer are used around the same time each frame requested by the consumer may come out of order and cause seeking.

So, if someone chooses to contribute this (at a high level outside the avformat producer) it needs to be smart about when to do it or not, which means the cache needs to accommodate multiple objects and a way to choose an appropriate one.

[bmatherly]

What I really care about is the initialization speed, the time it takes for ffmpeg to analyze the video file and start consuming frames.

the avformat-novalidate producer might help with that:

https://mltframework.org/plugins/ProducerAvformat-novalidate/

Also, I don't think the optimization you are proposing will really help. Depending on the container, making an arbitrary seek into an open producer probably takes about as much resources as opening the file and seeking. I don't think it would save a lot.

Maybe you would want an optimization that opens the file multiple times, but each producer somehow pre-seeks to the first frame (possibly asynchronously in a different thread long before the first frame is needed).

[remorses]

Thank you for the info, tried using avformat-novalidate but does not seem to make a big difference.

I tried changing the avformat producer to add a cache and reloading an xml file now takes half a second instead of 3, pretty cool! I basically re-implemented melt with a --watch option that reloads the file when the file changes

I am very bad at C programming so now it does not do any cleanup and just leaks memory

[ddennedy]

That is interesting but the cache should return a miss when the producer object has any properties that override whatever was cached. In other words, the cache key needs to include a number of other properties in addition to resource. And maybe those values can be hashed. It could also do something like factor in whether a video track has B-frames and skip cache to avoid the seek penalties we described above.