Description

It seems that the "cueTime" for an ID3 cue (in-band metadata) is out of sync when the live HLS stream is audio only. If the live HLS stream includes video, the "cueTime" appears to be synced correctly.

See lib/mp4/transmuxer.js, lines 861-862:

Regarding the screenshot above, "keepOriginalTimestamps" is true and the value for "cueTime" seems to be set relative to the start time of the stream itself. However, in the past, I think the "cueTime" was set relative to the player's start time, which ensured that the cues would activate at the correct time.

See lib/utils/clock.js, lines 25-27 & 41-47:

Regarding the second screenshot, it would seem desirable to set "keepOriginalTimestamps" to false, to ensure the ID3 cues are synchronized relative to the player's current time. However, doing so seems to have various consequences. For example, setting this to false within "segment-loader.js (@videojs/http-streaming/src/segment-loader.js, line 684) breaks playback altogether. I also tried setting this to false at line 862 of "mux.js/lib/mp4/transmuxer.js", but this only fixed the cue synchronization for my audio stream in Firefox (the cues didn't activate in Chrome).

All that said, I don't know what the issue is exactly. We're currently using VideoJS version 7.7.6 in production and it doesn't have this problem. However, we want to upgrade to a newer version of VideoJS because of this recent issue: videojs/video.js#7905

For my testing above, I am using VJS version 7.20.2. I've also tested versions 7.10.0, 7.18.1 and 7.19.2 and found similar issues with cue synchronization and audio only streams.

Finally, I observed an interesting difference between versions 7.19.2 and 7.20.2, regarding a live stream with video and audio. In version 7.19.2, the "cueTime" is set relative to the player's current time (as if keepOriginalTimestamps is false), and the cues activate at the correct time (expected behavior since the stream includes video). However, in version 7.20.2, the "cueTime" is set relative to the stream's start time (as if keepOriginalTimestamps is true), which is strange because the cues still seem to activate at the correct time. In both versions, I believe keepOriginalTimestamps is set to true (which is why I said "as if" in the parenthesis above).

Somewhat Related Issues
videojs/http-streaming#638
#293
videojs/video.js#7902
videojs/http-streaming#130

Steps to Reproduce
1.) Start a live audio only HLS stream.
2.) Inject ID3 cues into the stream.
3.) Inspect the ID3 cues when they come in. In my case, I added logging statements to "videojs/src/tracks/text-tracks.js" and mux.js/lib/mp4/transmuxer.js.
4.) Take note of when the cues actually get pushed into activeCues and the "cuechange" event is triggered.

Example Logging Statement (audio only stream)

In this screenshot, the "cueTime" is waaaaay out in the future compared to the player's current time. The stream's latency is about 20-30 seconds, so the "cueTime" should be around 8.122203 + 20-30 seconds. One thing to note, you may have a hard time noticing this issue if your player's start time is close to the stream's start time. I recommend waiting a minute or two before starting the player on your test page.

Errors
None

What version of Video.js are you using?
7.20.2

Video.js plugins used.
None

What browser(s) including version(s) does this occur with?
Chrome Version 105.0.5195.102 (64-bit)
Firefox Version 104.0.1 (64-bit)

What OS(es) and version(s) does this occur with?
Windows 10

Please let me know if I'm missing anything, thanks!