Real-time video streaming: an introduction for AUVSI members

The success or failure of a project is often down to how much you understand the subject. Concepts that work on paper might not do so in reality. Furthermore, you may not even be aware of possibilities that you could consider.

The prospect of integrating live video into a project can be daunting to those with little or no experience of the subject in general or even to those “in the know” if they are unaware of the specialist techniques available for specific situations.

This article, in the form of an introduction to video streaming, aims to guide those who wish to explore the possibilities of live video streaming in harsh environments, over wireless networks and expensive airtime applications.

1. Definition of “Video” and its relationship with bandwidth

Video is simply a series of image frames made up of pixels. The more pixels in a frame, the more detail is apparent. The more frames per second (fps), the smoother the video looks to the viewer.

To illustrate, a standard household TV uses around 25fps; that is, 25 frames are shown per second, and the transmission is seamless. Viewers will not notice the transition between frames; therefore, the video looks smooth.

Slowing the frame rate to 16fps, the human eye still perceives the video as smooth; however, slowing further to 8-15fps will make the footage noticeably jerky as the gaps in frames can be seen by the eye.

Why is this important? Video data is transmitted over networks using bandwidth. Each frame of video contains an amount of data measured in Kilobits (Kbs) or Megabits (Mbs). The more frames you send every second, the more data is involved. The data transmission rate is important as the network used for transmission has to have enough bandwidth available to carry this data.

If you have less bandwidth on a network than you are trying to send each second, the data is delayed, and the video takes longer to arrive at the viewer, resulting in a video feed that lags behind real-time.

2. What is Bandwidth

Bandwidth is the data transfer speed or capacity of a network measured in Kilobits per second (Kbps) or Megabits per second (Mbps).

Airtime bills are often calculated by how much bandwidth you have used, so the higher your usage, the higher the bill. This is an essential factor in budgeting any project using live video streaming, particularly on cellular and satellite networks.

The smaller the video data can be made, the less bandwidth is needed to send it. If you need less bandwidth to send the video, the network doesn’t have to be as fast. Bandwidth can be costly so, by requiring less bandwidth, you can save money.

Finally, bandwidth may be too low for standard video solutions to even operate. There is a significant difference between specialist applications designed for these environments compared with the majority of standard video cameras and systems on the market. Don’t assume you cannot have a successful operational result until you have explored these specialist technologies.

Specialist applications are required for harsh environments or areas of poor bandwidth availability

3. Compression

Video compression is the process of encoding video data to reduce its volume.

Video data in its raw format is large and unwieldy. Much information is stored in each video frame, so compression techniques exist to make these frames smaller.

How do you make video smaller in order to transmit more effectively?

There are many different compression techniques on the market – some efficient, some less so.

Whilst it is true that the more the video is compressed, the less bandwidth is required to stream, the downside to video compression is that the more compression, the more degraded the image quality as some of the data is lost in the process.

Most modern video cameras output video in a compressed format using algorithms that lose as little data as possible whilst maintaining as much high-quality video image as possible. This is ok for image quality, but it does not take into account latency or bandwidth because, for the most part, these cameras are used on wired CCTV networks which don’t have any speed restrictions.

However, a different approach is needed on bandwidth-constrained or costly networks such as cellular or satellite. The compression process in these situations must try to squeeze the video as small as possible and as fast as possible.

The clever part about specialised video compression techniques is that you can achieve the optimum ratio of fit-for-purpose video images at the lowest possible video size without wholescale image degradation beyond acceptable levels. The ability to achieve this makes the difference between success and failure of these applications.

4. Importance of Latency

Latency is a synonym for “delay” and in video applications can be described as the delay from when something happens in the field to someone being able to see that image on a screen remotely.

Real-time applications need very low latency to succeed; otherwise, critical decisions are being made on information that happened a few seconds ago instead of on live information.

In a control room situation, operators need immediate response when using joystick/PTZ equipment, and low latency levels are required as standard. Any delay longer than a second or so can render an operating system ineffective. In most cases, specialist applications can achieve even lower latency levels of just a few milliseconds. This enables many more applications to be successful such as controlling remote-operated vessels or vehicles, even UAVs.

Video bandwidth and compression is critical in the pursuit of low latency. The video data rate must be less than the capacity on the network to be transmitted with no or minimal delay. If you try to send more than the capacity available on the network, then delay will occur as the network ‘funnel’ cannot transmit all the data in real time.

5. Reliability over Distributed Networks

Distributed networks have many advantages, but reliability or predictability is sometimes not one of them. Connections can be sketchy, and bandwidth availability or rate is not guaranteed.

Solutions that connect and communicate successfully across these networks have usually been designed specifically for this purpose. Adaptability is critical, and adaptive video compression solutions and bespoke real-time transmission protocols that connect/auto-connect/reconnect will give you reduced downtime without manual intervention.

Compressing the video to match the network speed is crucial but delivering that video reliably across the network is equally crucial. Standard video cameras and systems use standard protocols for delivering video designed for high bandwidth wired networks. They may not work so well in bandwidth-constrained or unreliable networks. Specialist protocols are needed to navigate these scenarios and have been developed to solve these specific problems.

6. Keep it Simple

Getting live-video streaming right is much more complex than it first appears, and it is always better to consult an expert, preferably one with experience in your field or in something you feel could be adapted.

Simplicity must also be considered. Setting up and operating any video product should be easy, saving time, effort, and resources, enabling you to focus on the operation/project at hand rather than struggle with technology. It is a complex subject, and you can quickly find yourself lost with unreliable and ineffective systems.

Getting a specialist involved at the beginning of your project will give you the best outcome. Retrofitting video-streaming into a project can always be done but working together from the early stages will negate unnecessary modifications when incorporating live video streaming.

7. Solutions for Real World Problems

Some traditional markets/areas have live video-streaming integrated as part of their model, primarily site security and CCTV. These applications are also based mainly on high-capacity wired network architectures, thus removing most of the need for specialist solutions.

However, many more have not adopted video so far, for a range of reasons – from perceived high cost to lack of expertise or the fact that the area in which they operate was previously considered impossible for live streaming.

With the advent of IoT (Internet of Things) and remotely operated assets, the possibilities of live video are being understood and attempted by more markets. Many still make the choice of adopting standard video technology deployed in CCTV and as a result, often fail to get the results they wanted or expected. However, as we have shown, by adapting specific video techniques, live video applications could now become a viable and successful option.

For example, streaming from remote and uncrewed vessels such as Mayflower Autonomous ship and Ocean Aero’s Triton – to autonomous robots like the Glocal Robotics‘ “Thalamus”, to integration on UAVs such as the German Drones Songbird 150, have all been made possible by deploying these techniques where standard systems failed.

Songbird 150 : Image Copyright German Drones

The Triton : Image copyright Ocean Aero

Thalamus : Image Copyright Glocal Robotics

Mayflower Autonomous Ship : Image copyright Promare/IBM

8. How much does this cost?

Firstly “cost” should be defined as the “total cost of ownership” rather than the cost of any hardware and software purchased.

The lowest cost option on paper may cost you a lot more in the long run when you add on the cost of data/bandwidth charges, human intervention, technical hours to solve issues, potential delayed launch, and downtime affecting every aspect of your project.

• Image quality
  There is no point in paying too much for high-quality images you do not need, nor do you want to spend so little that the images you receive are useless. Ensure that the system you are considering is adaptive and gives you operational-quality images at the right price. Ask for a demonstration of the equipment and see how it operates and the level of images you would expect to receive at any given bandwidth.
• Bandwidth
  Are the levels of compression sufficient to keep data costs within budget? Does the system have bandwidth efficiency at the heart of everything it does? Can it still operate in the harshest bandwidth environments, or does it fail? Ask for estimates, examples of previous projects, customer references, and case studies to support the claims.
• Reliable and Fit for Purpose
  Is the solution proposed fit for purpose? This will be guaranteed if you have involved the provider early in your project. However, it is worth asking questions about operating temperatures, terrains and conditions. What happens if/when it disconnects or malfunctions – what support mechanisms do they/you have in place? Ask how much human intervention might need to be budgeted to keep it operational.
• Implementation
  The last thing you need in any project at any stage is an unnecessary complication. Look for remotely installed software systems, perhaps with off-the-shelf hardware to keep costs down. Ask how long the proposed solution will take to deploy initially and in-field conditions if that is your intended use. Ask about additional options, such as multi streams from a single hardware, integration with third-party hardware and systems, and if there are other features you may not have thought about.
• Support and functionality
  A well-designed system will need minimal maintenance and support. However, having a knowledgeable and available partner to support you along the journey is crucial. A specialist with knowledge and experience can advise and guide you through. They can also listen to your requirements and adapt systems to fit your needs perfectly. Ask about these levels of support, not forgetting about all important firmware upgrades to keep your software secure and current.