Master Uploading HLS Files to S3 Bucket in Real-Time with Ant Media Server

In this blog post, we’ll discover how to elevate your live streaming experience by uploading HLS files to S3 bucket in real-time using the Ant Media Server.

Ant Media Server supports uploading the HLS files to all the S3 compatible buckets like AWS S3, OVH object storage, Digital Ocean Spaces, Wasabi storage, and Google Cloud Storage, etc. In Earlier versions of Ant Media Server, the recorded HLS files were uploaded to the S3 buckets after the stream ended.

But starting from the Ant Media Server version 2.6.2, you can now seamlessly upload HLS files to the S3 buckets in real-time while your stream is live, hence delivering an instant and captivating experience to your viewers.

Prerequisites

Before diving into the process of uploading HLS files to S3 buckets, it’s important to ensure you have the essentials in place. So before we begin, it is important to have your S3 bucket ready.

You can create your bucket on any S3-compatible cloud of your liking. If you’re not sure how to create an S3 bucket, please refer to our detailed document on how to create S3 buckets in different cloud platforms.

I’m going to use the AWS S3 bucket for this blog post. So now that the S3 bucket is ready and we have the required credentials, let’s go ahead and save the details.

Step 1: Saving the bucket credentials

The bucket details can be easily saved on the Ant Media Server dashboard by going to the Application > Settings > Enable S3 Recording under Stream Recording.

Step 2: Adding the HLS Upload Settings

In this step, we’ll add the necessary flags to enable uploading HLS Files to S3 Buckets in real time.

• Navigate to the Application Settings section on the dashboard and select Advanced under the drop-down bar.

• Locate the setting hlsHttpEndpoint and set it to:

hlsHttpEndpoint=http://localhost:5080/LiveApp/hls-upload

Here, LiveApp is the application name and you can replace it with your preferred application.

• It can also be set as:

hlsHttpEndpoint=https://{SERVER_DOMAIN}:5443/{APPLICATION_NAME}/hls-upload

• Once the HLS files start getting uploaded to the S3 bucket, you will see such log excerpts on the server side
  

2023-08-24 16:02:55,703 [java-sdk-progress-listener-callback-thread] INFO  io.antmedia.servlet.UploadHLSChunk - File uploaded to S3 with key: streams/blogpost-hls-upload-2023-08-24_16-02-52.970.m3u8
2023-08-24 16:02:55,734 [java-sdk-progress-listener-callback-thread] INFO  i.a.storage.AmazonS3StorageClient - File uploaded to S3 with key: streams/blogpost-hls-upload-2023-08-24_16-02-52.970000000000.ts
2023-08-24 16:02:55,734 [java-sdk-progress-listener-callback-thread] INFO  io.antmedia.servlet.UploadHLSChunk - File uploaded to S3 with key: streams/blogpost-hls-upload-2023-08-24_16-02-52.970.m3u8
2023-08-24 16:02:57,630 [http-nio2-0.0.0.0-5080-exec-9] INFO  i.a.storage.AmazonS3StorageClient - File  upload has started with key: streams/blogpost-hls-upload-2023-08-24_16-02-52.970000000001.ts

Step 3: Customizing the real-time HLS uploads

Now that we have learned to upload HLS files to the S3 bucket in real-time, let’s learn about the best practices and settings to make the most of this feature.

• If you want to store all the .ts files for a streamId, it is important to set the below properties:
  

hlsflags=+append_list
hlsPlayListType=event
deleteHLSFilesOnEnded=false

• If using the same streamId and you need to prevent the HLS files from getting overwritten, you can enable the data and time for HLS files:
  

addDateTimeToHlsFileName=true

Using this property, when the same streamId is used, the existing HLS files will not be overwritten as they will have different timestamps.

Conclusion

Congratulations!

So by using this feature, we can now say goodbye to post-stream waiting as you now have the power to upload HLS files to S3 buckets in real-time. Whether it’s AWS S3, OVH object storage, Digital Ocean Spaces, Wasabi storage, Google Cloud Storage, or Min.io, this capability unlocks instant content delivery over S3.

Starting from version 2.6.2, engage your audience with captivating experiences as your stream unfolds. Prepare your S3 bucket, follow the simple steps, and empower your live streaming like never before.

Seamlessly share and archive content while connecting with viewers while in the moment.

For any questions that you may have, please don’t hesitate to reach us and our community on GitHub https://github.com/orgs/ant-media/discussions

Also, don’t forget to visit Ant Media Server Marketplace to check the latest plugins and applications designed for your use case.