Dual Audio Example in HLS Streaming with FFmpeg
HTTP Live Streaming is an adaptive bitrate streaming protocol. It's the first released ABR (Adaptive bitrate streaming) streaming protocol, mostly used in the Video on Demand (VoD) services and OTT platforms to be more specific.
HLS (HTTP Live Streaming) is a streaming protocol developed by Apple that is used to deliver video and audio content over the internet. The dual audio feature in HLS allows a single video stream to be delivered with multiple audio tracks, allowing viewers to select the audio track that they want to listen to.
For example, a dual audio HLS stream might contain separate audio tracks for different languages, allowing viewers to switch between languages as they watch the video. Alternatively, a dual audio HLS stream might contain different audio tracks for different audio formats or qualities, allowing viewers to choose the audio track that best suits their needs.
To use the dual audio feature in HLS, the video stream must be encoded with multiple audio tracks and the HLS playlist must include references to these tracks. Viewers can then use a player that supports dual audio (such as the native HLS player on Apple devices) to switch between audio tracks as they watch the video.
Watching this video you will understand how dual audio works in Netflix or similar ott platforms cause the core idea is pretty much the same for all protocols of adaptive bitrate streaming.
Basically, audio streams are segmented separately and then referenced to the video, also a default is chosen to select automatically. Watch the full video to understand clearly.
Here is an example of an FFmpeg command that can be used to create a dual audio HLS master playlist file with 3 different bitrates (360p, 480p, 720p) from an input video named input.m4v and two audio tracks named english.mp3 and french.mp3:
ffmpeg -i input.m4v -i english.mp3 -i french.mp3 -map 0:v -map 1:a -map 2:a -c:v libx264 -c:a aac -b:v 360k -b:a 128k -f hls -hls_time 10 -hls_playlist_type vod -hls_segment_filename "360p_%03d.ts" 360p.m3u8 -c:v libx264 -c:a aac -b:v 480k -b:a 128k -f hls -hls_time 10 -hls_playlist_type vod -hls_segment_filename "480p_%03d.ts" 480p.m3u8 -c:v libx264 -c:a aac -b:v 720k -b:a 128k -f hls -hls_time 10 -hls_playlist_type vod -hls_segment_filename "720p_%03d.ts" 720p.m3u8
This command uses FFmpeg to encode the video and audio streams and create HLS segments for each bitrate. The -i options specify the input files, and the -map options specify which streams should be included in the output. The -c:v and -c:a options specify the codecs to be used for the video and audio streams, respectively. The -b:v and -b:a options specify the bitrates for the video and audio streams, respectively
MPEG DASH Explained: • MPEG-DASH Streaming Explained for Beg...
HLS vs Dash comparison: • MPEG DASH vs HLS Streaming
Downloading HLS videos: • Playlist
Adding dual audio in HTTP live streaming is not that straightforward so far. I made this quick tutorial for beginners with practical examples showing how to add dual or multiple audios for a video with the help of FFmpeg. #hls #vod #ott #streaming #livestreaming #ffmpeg #dualaudio #vod
