projects / HLS-Stream-Creator.git / commitdiff
summary | shortlog | log | commit | commitdiff | tree
side by side | plain
Allowed segment filename prefix to be specified. See HLS-9
Allowed segment filename prefix to be specified. See HLS-9

file:a/HLS-Stream-Creator.sh -> file:b/HLS-Stream-Creator.sh 
--- a/HLS-Stream-Creator.sh
+++ b/HLS-Stream-Creator.sh
@@ -20,7 +20,7 @@
 #   list of conditions and the following disclaimer in the documentation and/or
 #   other materials provided with the distribution.
 # 
-#   Neither the name of the {organization} nor the names of its
+#   Neither the name of Ben Tasker nor the names of his
 #   contributors may be used to endorse or promote products derived from
 #   this software without specific prior written permission.
 # 
@@ -38,9 +38,33 @@
 ######################################################################################
 
 # Basic config
+OUTPUT_DIRECTORY=${OUTPUT_DIRECTORY:-'./output'}
 
 # Change this if you want to specify a path to use a specific version of FFMPeg
-FFMPEG='ffmpeg'
+FFMPEG=${FFMPEG:-'ffmpeg'}
+
+# Number of threads which will be used for transcoding. With newer FFMPEGs and x264
+# encoders "0" means "optimal". This is normally the number of CPU cores.
+NUMTHREADS=${NUMTHREADS:-"0"}
+
+# Video codec for the output video. Will be used as an value for the -vcodec argument
+VIDEO_CODEC=${VIDEO_CODEC:-"libx264"}
+
+# Video codec for the output video. Will be used as an value for the -acodec argument
+AUDIO_CODEC=${AUDIO_CODEC:-"libfdk_aac"}
+
+# Additional flags for ffmpeg
+FFMPEG_FLAGS=${FFMPEG_FLAGS:-""}
+
+# If the input is a live stream (i.e. linear video) this should be 1
+LIVE_STREAM=${LIVE_STREAM:-0}
+
+# Video bitrates to use in output (comma seperated list if you want to create an adaptive stream.)
+# leave null to use the input bitrate
+OP_BITRATES=${OP_BITRATES:-''}
+
+# Determines whether the processing for adaptive streams should run sequentially or not
+NO_FORK=${NO_FORK:-0}
 
 # Lets put our functions here
 
@@ -54,12 +78,25 @@
 HTTP Live Stream Creator
 Version 1
 
-Copyright (C) 2013 B Tasker
+Copyright (C) 2013 B Tasker, D Atanasov
 Released under BSD 3 Clause License
 See LICENSE
 
 
-Usage: HLS-Stream-Creator.sh inputfile segmentlength(seconds)
+Usage: HLS-Stream-Creator.sh -[lf] [-c segmentcount] -i [inputfile] -s [segmentlength(seconds)] -o [outputdir] -b [bitrates]  [-p filename]
+
+	-i	Input file
+	-s	Segment length (seconds)
+	-o	Output directory (default: ./output)
+	-l	Input is a live stream
+	-c	Number of segments to include in playlist (live streams only) - 0 is no limit
+	-b	Output video Bitrates (comma seperated list for adaptive streams)
+	-f	Foreground encoding only (don't fork the encoding processes into the background - adaptive non-live streams only)
+	-p	Playlist filename
+	-t	Segment filename prefix
+
+Deprecated Legacy usage:
+	HLS-Stream-Creator.sh inputfile segmentlength(seconds) [outputdir='./output']
 
 EOM
 
@@ -68,59 +105,116 @@
 }
 
 
-## Create the Initial M3U8 file including the requisite headers
-#
-# Usage: create_m3u8 streamname segmentlength
-#
-function create_m3u8(){
-
-# We'll add some more headers in a later version, basic support is all we need for now
-# The draft says we need CRLF so we'll use SED to ensure that happens
-cat << EOM | sed 's/$/\r/g' > output/$1.m3u8
-#EXTM3U
-#EXT-X-TARGETDURATION:$2
-#EXT-X-MEDIA-SEQUENCE:0
-#EXT-X-VERSION:3
+function createStream(){
+# For VoD and single bitrate streams the variables we need will exist in Global scope.
+# for live adaptive streams though, that won't be the case, so we need to take them as arguments
+# Some are global though, so we'll leave those as is
+
+playlist_name="$1"
+output_name="$2"
+bitrate="$3"
+
+$FFMPEG -i "$INPUTFILE" \
+    -loglevel error -y \
+    -vcodec "$VIDEO_CODEC" \
+    -acodec "$AUDIO_CODEC" \
+    -threads "$NUMTHREADS" \
+    -map 0 \
+    -flags \
+    -global_header \
+    -f segment \
+    -segment_list "$playlist_name" \
+    -segment_time "$SEGLENGTH" \
+    -segment_format mpeg_ts \
+    $bitrate \
+    $FFMPEG_ADDITIONAL \
+    $FFMPEG_FLAGS \
+    "$OUTPUT_DIRECTORY/$output_name"
+}
+
+
+function createVariantPlaylist(){
+playlist_name="$1"
+echo "#EXTM3U" > "$playlist_name"
+}
+
+
+function appendVariantPlaylistentry(){
+playlist_name=$1
+playlist_path=$2
+playlist_bw=$(( $3 * 1000 )) # bits not bytes :)
+
+cat << EOM >> "$playlist_name"
+#EXT-X-STREAM-INF:BANDWIDTH=$playlist_bw
+$playlist_path
 EOM
-}
-
-
-## Append a movie segment to the M3U8
-#
-# Usage: append_segment streamname SegmentLength(Seconds) SegmentFilename
-#
-function append_segment(){
-
-cat << EOM | sed 's/$/\r/g' >> output/$1.m3u8
-#EXTINF:$2
-$3
-EOM
-}
-
-
-## Close the M3U8 file
-#
-# Found that ffplay skips the first few segments if this isn't included.
-#
-# Usage: close_m3u8 streamname
-#
-function close_m3u8(){
-cat << EOM | sed 's/$/\r/g' >> output/$1.m3u8
-#EXT-X-ENDLIST
-EOM
-}
-
-
-# The fun begins! Think of this as function main
-
+
+}
+
+
+function awaitCompletion(){
+# Monitor the encoding pids for their completion status
+while [ ${#PIDS[@]} -ne 0 ]; do
+    # Calculate the length of the array
+    pid_length=$((${#PIDS[@]} - 1))
+
+    # Check each PID in the array
+    for i in `seq 0 $pid_length`
+    do
+	  # Test whether the pid is still active
+	  if ! kill -0 ${PIDS[$i]} 2> /dev/null
+	  then
+		echo "Encoding for bitrate ${BITRATE_PROCESSES[$i]}k completed"
+		unset BITRATE_PROCESSES[$i]
+		unset PIDS[$i]
+	  fi
+    done
+    PIDS=("${PIDS[@]}") # remove any nulls
+    sleep 1
+done
+}
+
+
+# This is used internally, if the user wants to specify their own flags they should be
+# setting FFMPEG_FLAGS
+FFMPEG_ADDITIONAL=''
+LIVE_SEGMENT_COUNT=0
 
 # Get the input data
 
-# Basic Usage is going to be
-# cmd.sh inputfile segmentlength 
-
-INPUTFILE=$1
-SEGLENGTH=$2
+# This exists to maintain b/c
+LEGACY_ARGS=1
+
+# If even one argument is supplied, switch off legacy argument style
+while getopts "i:o:s:c:b:p:t:lf" flag
+do
+	LEGACY_ARGS=0
+        case "$flag" in
+                i) INPUTFILE="$OPTARG";;
+                o) OUTPUT_DIRECTORY="$OPTARG";;
+                s) SEGLENGTH="$OPTARG";;
+		l) LIVE_STREAM=1;;
+		c) LIVE_SEGMENT_COUNT="$OPTARG";;
+		b) OP_BITRATES="$OPTARG";;
+		f) NO_FORK=1;;
+		p) PLAYLIST_PREFIX="$OPTARG";;
+		t) SEGMENT_PREFIX="$OPTARG";;
+        esac
+done
+
+
+if [ "$LEGACY_ARGS" == "1" ]
+then
+  # Old Basic Usage is 
+  # cmd.sh inputfile segmentlength 
+
+  INPUTFILE=${INPUTFILE:-$1}
+  SEGLENGTH=${SEGLENGTH:-$2}
+  if ! [ -z "$3" ]
+  then
+    OUTPUT_DIRECTORY=$3
+  fi
+fi
 
 
 # Check we've got the arguments we need
@@ -129,10 +223,8 @@
   print_usage
 fi
 
-
-
 # FFMpeg is a pre-requisite, so let check for it
-if hash ffmpeg 2> /dev/null
+if hash $FFMPEG 2> /dev/null
 then
   # FFMpeg exists
   echo "ffmpeg command found.... continuing"
@@ -142,92 +234,94 @@
   exit 1
 fi
 
-
-# Now we want to make sure out input file actually exists
-if ! [ -f "$INPUTFILE" ]
-then
-  echo "Error: You gave me an incorrect filename. Please re-run specifying something that actually exists!"
-  exit 1
-fi
-
-
-
-# OK, so from here, what we want to do is to split the file into appropriately sized chunks,
-# re-encoding each to H.264 with MP3 audio, all to go into an MPEG2TS container
-#
-# The protocol appears to support MP4 as well though, so we may well look at that later.
-#
-# Essentially we want to create the chunks by running
-#
-# ffmpeg -i "$INPUTFILE" -vcodec libx264 -acodec mp3 -ss "START_POINT" -t "$SEGLENGTH" -f mpegts output/"$INPUTFILE"_"$N".ts
-
-# First we need the duration of the video
-DURATION=$($FFMPEG -i "$INPUTFILE" 2>&1 | grep Duration | cut -f 4 -d ' ')
-
-# Now we need to break out the duration into a time we can use
-DUR_H=$(echo "$DURATION" | cut -d ':' -f 1)
-DUR_M=$(echo "$DURATION" | cut -d ':' -f 2)
-DUR_X=$(echo "$DURATION" | cut -d ':' -f 3 | cut -d '.' -f 1)
-
-# Calculate the duration in seconds
-let "DURATION_S = ( DUR_H * 60 + DUR_M ) * 60 + DUR_X"
-
-
-# Check we've not got empty media
-if [ "$DURATION_S" == "0" ]
-then
-  echo "You've given me an empty media file!"
-  exit 1
-fi
-
-
-# Now we've got our Duration, we need to work out how many segments to create
-N='1'
-START_POS='0'
-let 'N_FILES = DURATION_S / SEGLENGTH + 1'
-
-# For now, INPUTFILENAME is going to == INPUTFILE
-# Later, we'll change so that INPUTFILE could be an absolute path, whilst INPUTFILENAME will just be the filename
-INPUTFILENAME=$INPUTFILE
-
-
-# Create the M3U8 file
-create_m3u8 "$INPUTFILENAME" "$SEGLENGTH"
-
-# Finally, lets build the output filename format
-OUT_NAME=$INPUTFILENAME"_%03d.ts"
-
-
-
-# Processing Starts
-
-while [ "$START_POS" -lt "$DURATION_S" ]
-do
-
-  OUTPUT=$( printf "$OUT_NAME" "$N" )
-  echo "Creating $OUTPUT ($N/$N_FILES)..."
-  $FFMPEG -i "$INPUTFILE" -loglevel quiet -vcodec libx264 -acodec mp3 -ss "$START_POS" -t "$SEGLENGTH" -f mpegts output/"$OUTPUT"
-
-  let "N = N + 1"
-  let "START_POS = START_POS + SEGLENGTH"
-
-  # If we're on the last segment, the duration may be less than the seglenth, so we need to reflect this in the m3u8
-  if ! [ "$START_POS" -lt "$DURATION_S" ]
-  then
-    SEG_DURATION=$($FFMPEG -i output/"$OUTPUT" 2>&1 | grep Duration | cut -f 4 -d ' ')
-    # Now we need to break out the duration into a time we can use
-    DUR_H=$(echo "$SEG_DURATION" | cut -d ':' -f 1)
-    DUR_M=$(echo "$SEG_DURATION" | cut -d ':' -f 2)
-    DUR_X=$(echo "$SEG_DURATION" | cut -d ':' -f 3 | cut -d '.' -f 1)
-
-    # Calculate the duration in seconds
-    let "SEGLENGTH = ( DUR_H * 60 + DUR_M ) * 60 + DUR_X"
-  fi
-
-  # Append the file reference to the M3U8
-  append_segment "$INPUTFILENAME" "$SEGLENGTH" "$OUTPUT"
-
-done
-
-# Add the close tag (ffplay gives some weird behaviour without this!)
-close_m3u8 "$INPUTFILENAME"
+# Check output directory exists otherwise create it
+if [ ! -w $OUTPUT_DIRECTORY ]
+then
+  echo "Creating $OUTPUT_DIRECTORY"
+  mkdir -p $OUTPUT_DIRECTORY
+fi
+
+if [ "$LIVE_STREAM" == "1" ]
+then
+    FFMPEG_ADDITIONAL+="-segment_list_flags +live"
+
+    if [ "$LIVE_SEGMENT_COUNT" -gt 0 ]
+    then
+	FFMPEG_ADDITIONAL+=" -segment_list_size $LIVE_SEGMENT_COUNT"
+    fi
+fi
+
+
+# Pulls file name from INPUTFILE which may be an absolute or relative path.
+INPUTFILENAME=${INPUTFILE##*/}
+
+# If a prefix hasn't been specified, use the input filename
+PLAYLIST_PREFIX=${PLAYLIST_PREFIX:-$INPUTFILENAME}
+SEGMENT_PREFIX=${SEGMENT_PREFIX:-$PLAYLIST_PREFIX}
+
+# Set the bitrate
+if [ ! "$OP_BITRATES" == "" ]
+then
+      # Make the bitrate list easier to parse
+      OP_BITRATES=${OP_BITRATES//,/$'\n'}
+
+      # Create an array to house the pids for backgrounded tasks
+      declare -a PIDS
+      declare -a BITRATE_PROCESSES
+
+      # Get the variant playlist created
+      createVariantPlaylist "$OUTPUT_DIRECTORY/${PLAYLIST_PREFIX}_master.m3u8"
+      for br in $OP_BITRATES
+      do
+	    appendVariantPlaylistentry "$OUTPUT_DIRECTORY/${PLAYLIST_PREFIX}_master.m3u8" "${PLAYLIST_PREFIX}_${br}.m3u8" "$br"
+      done
+
+      # Now for the longer running bit, transcode the video
+      for br in $OP_BITRATES
+      do
+	      BITRATE="-b:v ${br}k -bufsize ${br}k"
+	      # Finally, lets build the output filename format
+	      OUT_NAME="${SEGMENT_PREFIX}_${br}_%05d.ts"
+	      PLAYLIST_NAME="$OUTPUT_DIRECTORY/${PLAYLIST_PREFIX}_${br}.m3u8"
+
+	      echo "Generating HLS segments for bitrate ${br}k - this may take some time"
+
+	      if [ "$NO_FORK" == "0" ] || [ "$LIVE_STREAM" == "1" ]
+	      then
+		      # Processing Starts
+		      createStream "$PLAYLIST_NAME" "$OUT_NAME" "$BITRATE" &
+		      PID=$!
+		      PIDS=(${PIDS[@]} $PID)
+		      BITRATE_PROCESSES=(${BITRATE_PROCESSES[@]} $br)
+	      else
+		      createStream "$PLAYLIST_NAME" "$OUT_NAME" "$BITRATE"
+	      fi
+	      # Will deal with exit statuses shortly.
+	      #|| exit 1
+
+      done
+
+      if [ "$NO_FORK" == "0" ] || [ "$LIVE_STREAM" == "1" ]
+      then
+	    # Monitor the background tasks for completion
+	    echo "All transcoding processes started, awaiting completion"
+	    awaitCompletion
+      fi
+
+else
+
+  # No bitrate specified
+
+  # Finally, lets build the output filename format
+  OUT_NAME="${SEGMENT_PREFIX}_%05d.ts"
+  PLAYLIST_NAME="$OUTPUT_DIRECTORY/${PLAYLIST_PREFIX}.m3u8"
+
+  echo "Generating HLS segments - this may take some time"
+
+  # Processing Starts
+
+  createStream "$PLAYLIST_NAME" "$OUT_NAME" "$BITRATE"
+
+
+fi
+
file:a/README.md -> file:b/README.md 
--- a/README.md
+++ b/README.md
@@ -16,38 +16,76 @@
 Usage is incredibly simple
 
 ```
-./HLS-Stream-Creator.sh inputfile segmentlength(seconds)
+./HLS-Stream-Creator.sh -[lf] [-c segmentcount] -i [inputfile] -s [segmentlength(seconds)] -o [outputdir] -b [bitrates]
+
+
+Deprecated Legacy usage:
+	HLS-Stream-Creator.sh inputfile segmentlength(seconds) [outputdir='./output']
+
 ```
 
 So to split a video file called *example.avi* into segments of 10 seconds, we'd run
 
 ```
-./HLS-Stream-Creator.sh example.avi 10
+./HLS-Stream-Creator.sh -i example.avi -s 10
 ```
 
+**Arguments**
+
+```
+    Mandatory Arguments:
+
+	-i [file]	Input file
+	-s [s]  	Segment length (seconds)
+
+    Optional Arguments:
+
+	-o [directory]	Output directory (default: ./output)
+	-c [count]	Number of segments to include in playlist (live streams only) - 0 is no limit
+	-b [bitrates]	Output video Bitrates in kb/s (comma seperated list for adaptive streams)
+	-l		Input is a live stream
+	-f		Foreground encoding only (adaptive non-live streams only)
+```
+
+
+Adaptive Streams
+------------------
+
+As of [HLS-6](http://projects.bentasker.co.uk/jira_projects/browse/HLS-6.html) the script can now generate adaptive streams with a top-level variant playlist for both VoD and Linear input streams.
+
+In order to create seperate bitrate streams, pass a comma seperated list in with the *-b* option
+
+```
+./HLS-Stream-Creator.sh -i example.avi -s 10 -b 28,64,128,256
+```
+
+By default, transcoding for each bitrate will be forked into the background - if you wish to process the bitrates sequentially, pass the *-f* option
+
+```
+./HLS-Stream-Creator.sh -i example.avi -s 10 -b 28,64,128,256 -f
+```
+
+In either case, in accordance with the HLS spec, the audio bitrate will remain unchanged
 
 
 Output
 -------
 
-As of version 1, the HLS resources will be output to the directory *output*. These will consist of video segments encoded in H.264 with MP3 audio (should be AAC really, but I'd compiled *ffmpeg* without) and an m3u8 file in the format
+As of version 1, the HLS resources will be output to the directory *output*. These will consist of video segments encoded in H.264 with AAC audio and an m3u8 file in the format
 
 >\#EXTM3U  
 >\#EXT-X-MEDIA-SEQUENCE:0  
 >\#EXT-X-VERSION:3  
 >\#EXT-X-TARGETDURATION:10  
 >\#EXTINF:10, no desc  
->example_001.ts  
+>example_00001.ts  
 >\#EXTINF:10, no desc  
->example_002.ts  
+>example_00002.ts  
 >\#EXTINF:10, no desc  
->example_003.ts  
+>example_00003.ts  
 >\#EXTINF:5, no desc  
->example_004.ts  
+>example_00004.ts  
 >\#EXT-X-ENDLIST
-
-
-
 
 
 
@@ -61,6 +99,23 @@
 ```
 
 
+Additional Environment Variables
+-------------------------------
+
+There are few environment variables which can control the ffmpeg behaviour.
+
+* `VIDEO_CODEC` - The encoder which will be used by ffmpeg for video streams. Examples: _libx264_, _nvenc_
+* `AUDIO_CODEC` - Encoder for the audio streams. Examples: _aac_, _libfdk_acc_, _mp3_, _libfaac_
+* `NUMTHREADS` - A number which will be passed to the `-threads` argument of ffmpeg. Newer ffmpegs with modern libx264 encoders will use the optimal number of threads by default.
+* `FFMPEG_FLAGS` - Additional flags for ffmpeg. They will be passed without any modification.
+
+Example usage:
+
+```
+export VIDEO_CODEC="nvenc"
+export FFMPEG_FLAGS="-pix_fmt yuv420p -profile:v"
+./HLS-Stream-Creator.sh example.avi 10
+```
 
 License
 --------
@@ -68,4 +123,8 @@
 HLS-Stream-Creator is licensed under the [BSD 3 Clause License](http://opensource.org/licenses/BSD-3-Clause) and is Copyright (C) 2013 [Ben Tasker](http://www.bentasker.co.uk)
 
 
+Issue Tracking
+----------------
 
+Although the Github issue tracker can be used, the bulk of project management (such as it is) happens in JIRA. See [projects.bentasker.co.uk](http://projects.bentasker.co.uk/jira_projects/browse/HLS.html) for a HTML mirror of the tracking.
+
Simple Bash Script to take a media file, segment it and create an M3U8 playlist for serving using HLS https://github.com/bentasker/HLS-Stream-Creator