Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Priority to a normal audio, skip visual_impaired audio track unless was only audio (HLS ts segment with video_audio tracks) #415

Open
Murmur opened this issue Jul 1, 2022 · 1 comment

Comments

@Murmur
Copy link

Murmur commented Jul 1, 2022

We have HLS source where ts segments contain one VIDEO, multiple AUDIO tracks in a segment file. Current mux.ts library takes any first audio PID even if it was an additional type=visual_impaired track.

We cannot control the ordering of audio or remuxing to separate m3u8 manifest tracks. I made changes to parsePmt function to select first normal_audio and fallback to any first audio (current behaviour) if normal audio was not found.

I think this would be usefull be a default in muxjs library? This simple rule works fine even if audio PIDs or language codes changed while playing a hls ts stream, transition from one programme to next one may have different audio component tags, but no problem.

      parsePmt = function parsePmt(payload, pmt) {
        var sectionLength, tableEnd, programInfoLength, offset; // PMTs can be sent ahead of the time when they should actually
        // take effect. We don't believe this should ever be the case
        // for HLS but we'll ignore "forward" PMT declarations if we see
        // them. Future PMT declarations have the current_next_indicator
        // set to zero.

        if (!(payload[5] & 0x01)) {
          return;
        } // overwrite any existing program map table

        var firstAudio=null, firstAudioNormal=null;
        self.programMapTable = {
          video: null,
          audio: null,
          'timed-metadata': {}
        }; // the mapping table ends at the end of the current section

        sectionLength = (payload[1] & 0x0f) << 8 | payload[2];
        tableEnd = 3 + sectionLength - 4; // to determine where the table is, we have to figure out how
        // long the program info descriptors are

        programInfoLength = (payload[10] & 0x0f) << 8 | payload[11]; // advance the offset to the first entry in the mapping table
        offset = 12 + programInfoLength;

        while (offset < tableEnd) {
          // 27=VIDEO_h264, 15=AUDIO_adtsaac, 6=SUBTITLE_dvb
          var streamType = payload[offset];
          var pid = (payload[offset + 1] & 0x1F) << 8 | payload[offset + 2]; // only map a single elementary_pid for audio and video stream types
          // TODO: should this be done for metadata too? for now maintain behavior of multiple metadata streams

          // parse audio components, skip visualimpaired audio and take any first normal audio
          var infoLen = ((payload[offset + 3] & 0x0F) << 8 | payload[offset + 4]);
          var descLang="";
          var descType=0xFF; //0x00=normal_audio, 0x03=visual_impaired_audio
          if(payload[offset+5]==0x0A || payload[offset+5]==0x59) { // 0x0A=langDesc, 0x59=subDesc
            var descLen = payload[offset+6];
            if(descLen>=4) {
              descLang= String.fromCharCode(payload[offset+7]) 
                + String.fromCharCode(payload[offset+8])
                + String.fromCharCode(payload[offset+9]);
              descType=payload[offset+10]
            }
          }
         // console.log(`type=${streamType}, pid=${pid}` +"(0x"+bin.toHexString(pid)+"), infoLen="+infoLen 
         //    + ", lang="+descLang + ", type="+ descType);

          if (streamType === streamTypes.H264_STREAM_TYPE && self.programMapTable.video === null) {
            self.programMapTable.video = pid;
          //} else if (streamType === streamTypes.ADTS_STREAM_TYPE && self.programMapTable.audio === null) {
            //self.programMapTable.audio = pid;			
          } else if (streamType === streamTypes.ADTS_STREAM_TYPE) {
            if(firstAudio===null) firstAudio=pid; // any first audio even visual_impaired_audio
            if(firstAudioNormal===null && descType==0x00) firstAudioNormal=pid; // first normal_audio
          } else if (streamType === streamTypes.METADATA_STREAM_TYPE) {
            // map pid to stream type for metadata streams
            self.programMapTable['timed-metadata'][pid] = streamType;
          } // move to the next table entry
          // skip past the elementary stream descriptors, if present
          
          offset += infoLen+5; //offset += ((payload[offset + 3] & 0x0F) << 8 | payload[offset + 4]) + 5;
        } // record the map on the packet as well

        self.programMapTable.audio = firstAudioNormal!=null ? firstAudioNormal : firstAudio;
        pmt.programMapTable = self.programMapTable;
      };
@Murmur
Copy link
Author

Murmur commented Jul 1, 2022

Or how do videojs|muxjs works any ideas how to inject an application callback to parsePmt function? Something like track = appCallback.selectAudioTrack(tracks[ list of streamType, descLang, descType, pid ]) let application decide a priority logic?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant