Axis Zipstream Technology

Axis Zipstream technology makes it possible to use higher resolution and increase forensic usability, while reducing storage costs. The intelligent compression method ensures that important image details get enough attention in the video stream, while unnecessary data is removed.

Most networked video surveillance systems today are limited by bandwidth and storage for the recorded video. Zipstream is a radically improved, standard-compatible video encoder implementation that lowers bandwidth and storage requirements by an average of 50% or more when compared to standard compression. Important details and motion are preserved at high video quality, while the Axis-unique compression enhancement can filter the rest of the image information harder to make optimal use of the available bandwidth.

Zipstream consists of a collection of algorithms that analyze the video stream in real time:

• Dynamic ROI (regions of interest) — this identifies regions of interest based on objects, people, or motion in the scene, and applies the correct level of compression from a forensic perspective.

• Dynamic GOP (group of pictures) — this algorithm makes the camera send bandwidth-intense I-frames less frequently when there is no motion in the scene.

• Dynamic FPS (frames per second) — this reduces the bitrate when there is little or no motion in the scene. The camera captures and analyzes video at full frame rate, but unnecessary frames are not encoded.

Zipstream is continuously being improved and given additional features. Since its introduction to the market in 2015, Zipstream enhancements include PTZ camera functionality, support for 4K Ultra HD, multi-megapixel, and 360-degree-panoramic cameras, dynamic FPS limitation, and dynamic FPS frame skipping. A recent update was adding a profile that optimizes the video stream for storage. This enables more advanced video compression techniques while also making it easier and more intuitive to use Zipstream.

Zipstream now has AV1 support in cameras based on system-on-chip ARTPEC-9. These cameras have support for AV1 and H.265 in parallel with H.264 to enable flexible migration over an extended transition period. ARTPEC-9 can deliver multiple streams simultaneously up to the maximum performance limit of the chip.

Camera technologies such as sensors, optics, and embedded image processing have evolved rapidly over the last decade. This enables video with higher resolutions, frame rates, and dynamic range, where more scene details can be captured. As a result, video evidence and forensic analysis is more reliable today, but only when it is possible to retrieve the video from the right place, at the right time, and with the right quality. And due to the higher bitrate, the requirements on storage and bandwidth have increased.

Axis Zipstream technology is optimized for video surveillance, and is a standard-compatible video encoder implementation that is radically more efficient than standard encoders. It lowers bandwidth and storage requirements by an average of 50% or more. Zipstream is a collection of intelligent compression algorithms that ensure that important details in the image get enough attention in the video stream, while unnecessary data is removed. Zipstream is continuously improved with additional dynamic features.

Surveillance video needs to be processed before it can be efficiently stored. This is done using video compression algorithms that encode video data by reducing and removing redundant information. These algorithms locate regions in the video that have already been transferred, so that redundant sending in the next image frame can be avoided. The algorithms also identify places in the video where details can be removed without reducing the visual quality.

State-of-the-art video compression methods that function well together are grouped into an international standard, which is a video stream syntax created for storing, sharing, and viewing video.

The most used video compression standard today is H.264, which is efficient enough to reduce several days of surveillance video to fit on a single SD card.

AV1 is a modern and open-source friendly standard because it is license-free to Alliance for Open Media (AOM). AV1 is predicted to play a significant role in future security surveillance where more solutions need cloud integration.

H.265 was meant to replace H.264, but licensing issues made it hard for companies to use it widely. As a result, it has been difficult for hardware vendors to preinstall client decoders, and also too complex for end users to implement them on their own.

The different video encoder standards do not stipulate the actual video compression method; only the syntax and the method to perform playback is standardized. This enables improved video encoding solutions to be created while keeping the file format for interoperability (decoder compatibility). Zipstream is a more effective implementation of a native H.264/AV1/H.265 video encoder for surveillance applications. It includes various methods, unique to surveillance, that enable networked cameras to produce video at significantly lower bitrates.

Axis Zipstream technology is a collection of algorithms that lets the camera analyze the video stream in real-time. Motion and details of interest are preserved with the given video quality, while the Axis-unique method can filter other areas more aggressively for optimal use of the available bandwidth.

Zipstream is in no way a replacement for High Efficiency Video Coding (HEVC)/ITU Telecommunication Standardization Sector (ITU-T) H.265, which was jointly developed by ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG), or for the AV1 standard that was developed by Alliance for Open Media (AOM). Zipstream is a video encoder enhancement, which can be applied to many video compression standards, including H.264, AV1, or H.265, with minor adaptations.

Zipstream can be requested automatically from many VMS solutions. You can also enable Zipstream by configuring it in the cameras. This can be efficiently managed in AXIS Device Manager.

You can configure the different parts of the Zipstream algorithm individually or use storage profile, which automatically configures Zipstream to optimize the video for storage.

• Classic profile: The default profile, which lets you control major parts of the Zipstream algorithm individually.

• Storage profile: A profile that configures Zipstream so that the video is optimized for storage and later access.

When a video stream from the camera is requested, a stream profile parameter can be added to ask for a storage optimized stream. This is the preferred way to enable storage profile, but it requires VMS integration. If this is not available, a configuration interface in the camera can force all streams to use storage profile.

Zipstream adapts the compressed video stream based on four factors:

• Scene motion

• Scene content

• Ambient light level

• Configuration options

Configuration options that affect Zipstream:

• Compression

• Group of pictures (GOP) length

• Frame rate

• Strength (Zipstream effort level)

• Dynamic GOP

• Dynamic GOP limitation

• Dynamic frames per second (FPS)

• Dynamic FPS limitation

• Dynamic FPS frame skip mode

The strength parameter defines the effort level for Zipstream, as follows:

All strength parameter settings are compatible with all existing software applications, while still reducing the bitrate.

Other parameters can be configured as follows:

By default, network cameras that support Zipstream are configured with the strength parameter 10 and dynamic GOP/FPS disabled. The default setting is compatible with all existing applications, while still reducing the bitrate.

The bitrate reduction can be derived from either the dynamic ROI of Zipstream or its dynamic GOP or dynamic FPS.

Dynamic ROI (Region of Interest)

Through real-time analysis, the dynamic ROI identifies regions of interest based on objects, people, or motion in the scene and applies the correct level of compression from a forensic perspective. This process is performed for all image content, resulting in a totally flexible dynamic ROI. The ROI will automatically expand, shrink, change shape, split, merge, disappear, and reappear depending on content, for the benefit of tuning the instant bandwidth.

Since it is unknown in which parts of the image relevant information may appear, Zipstream prepares the system for unexpected events. This dynamic automatic ROI is much more convenient than other traditional ROI implementations in which the region is set manually.

Dynamic GOP (group of pictures)

With dynamic GOP, the camera will send bandwidth-intensive I-frames less frequently when there is no motion in the scene. Video from typical surveillance scenes with limited motion can be compressed to an extremely low bitrate with no loss of detail. This algorithm performs a real-time adaptation of the GOP length on the compressed video, according to the amount of motion. Note that not all clients or VMS may support smooth playback of video with this algorithm enabled, even though the compressed video stream conforms to the H.264 standard.

Dynamic FPS (frames per second)

The dynamic FPS reduces the bitrate by avoiding unnecessary encoding of video frames, which is done by omitting them from the stream. A static surveillance scene will be encoded at a radically reduced frame rate, even though the camera is capturing and analyzing video at the full frame rate. As scene motion is used as a control variable, a small moving object far away might not render at full frame rate. Objects approaching the camera increase the frame rate to capture every important detail. The number of delivered frames per second is restricted automatically by the camera, and this saves a substantial amount of data in many scenes.

The dynamic FPS limitation parameter can be used to configure a lower limit of the dynamic FPS. A dynamic frame rate between the stream fps and the configured minimum fps will then be selected, enabling use with supporting systems with minimum-fps requirements, as well as with systems that require a higher fps.

Some video management systems might not support smooth playback of video with dynamic frame rate, even though the compressed video stream conforms to the H.264/AV1/H.265 video standards. In these cases, disabling the frame skipping (setting the dynamic FPS frame skipping mode to “empty”) makes it possible to still use dynamic FPS. The video frame rate will vary while the full stream frame rate is maintained. Disabled frame skipping works as a compatibility mode that allows all users to benefit from the dynamic FPS, even though the bitrate saving will be smaller than when frame skipping is enabled.

Legal requirements may prevent the use of dynamic frame rate in some surveillance cases. By choosing the correct minimum-fps value, the dynamic FPS algorithm can still be used.

The main use case of Axis video products is to record video for storage and be able to access it later. Storage profile in Zipstream minimizes the bitrate while maximizing the evidence value for that use case. Using the preconfigured profile, the camera automatically enables the specific Zipstream algorithm that is most suitable for that type of camera, and uses more advanced video encoding tools. The profile is different in different cameras depending on their capabilities and the result can differ between camera types.

Storage profile utilizes a new GOP structure, with up to two bi-directional frames (B-frames) per P-frame, that saves bitrate by being able to use future information in the video encoding. The number of B-frames is dynamically changed, as there are situations where using B-frames would increase the bitrate. Beyond the bitrate reduction, the B-frames will introduce a latency impact of 1/fps per B-frame. This means that for, for example, a 25 fps video, an extra 80 ms of latency will be added to the video when using storage optimized video. Note that H.264 Baseline profile does not support B-frames and will be overridden to H.264 High profile if storage profile is used.

Dynamic GOP will always be used in storage profile, while the other Zipstream configurations remain as they are.

If the camera is overloaded by too many stream requests, storage profile videos will be prioritized. This is to ensure the preservation of their evidence value.

Zipstream reduces the average bitrate by using real-time scene information. The total bitrate reduction can be estimated by evaluating the bitrate savings for each algorithm independently, and combining the results. Expected bitrate reductions are shown in the table below. Note that all examples and figures in this section were created using H.264 compression.

The graphs below use dynamic GOP and plot the instantaneous bitrate from a video with four different motion scenarios A, B, C and D. The top graph has Zipstream disabled. The middle graph has a low Zipstream effort level, and the bottom graph has a high effort level configured. All streams are variable bitrate (VBR) streams with GOP length=32. Each I-frame update is clearly visible as a spike in bitrate, which can be read on the vertical axes. The bitrate reduction is represented by the areas shaded in grey.

The example time periods in the figure above highlight the behavior of Zipstream under different conditions:

A: Time period with brief and small movements. The motion is detected, and adding bits in that region can preserve the quality of the moving part of the video.

B: This period of larger and longer motion needs more space, but it is still possible to save storage for this motion, since the dynamic ROI detects areas where non-prioritized information can be removed.

C: Periods with no motion are detected and the dynamic GOP algorithm prevents unnecessary I-frame updates.

D: Period with small and extended motion.

The example in the figure above shows the behavior of Zipstream in four different motion scenarios (E, F, G and H), with dynamic FPS enabled:

E: With motion in the scene, the camera produces data at 30 fps.

F: When the motion decreases, the frame rate drops substantially. The bitrate decreases when the frame rate is reduced, as less data is transferred.

G: During a period with no motion in a completely static scene, the frame rate drops to almost zero between I-frames. Sparsely spread I-frame updates are the only bitrate source.

H: When motion is again detected, the camera immediately returns to 30 fps.

The original compression parameter is still used when Zipstream is enabled. This parameter controls the amount of compression applied to important forensic details. Compression is usually set to 30 and this value is recommended also when Zipstream is enabled.

The bitrate controller built into the encoder can be combined with Zipstream to enforce a maximum bitrate (MBR) limit. MBR is a variable bitrate (VBR) configuration that includes an upper limit to protect the system from temporary bandwidth spikes. However, the MBR limit must be sufficient to capture the details of moving objects in the scene to enable the full potential of Zipstream and VBR.

To limit the bitrate for increased storage time, cloud-connected cameras or cameras with edge storage should be configured with the strength parameter set to 30 (effort level High) and dynamic GOP enabled. This setting is suitable to combine with motion detection triggering and/or MBR systems where the bitrate is allowed to adapt to changes in complexity. Edge storage is a capability in Axis network cameras and video encoders that enables video recording directly to an onboard SD card or a network-attached storage device (NAS).

The dynamic GOP and dynamic FPS algorithms can be used simultaneously for increased bitrate reduction. If the VMS or other client software cannot handle the varying GOP length, set a shorter maximum GOP length, or disable dynamic GOP altogether. If the software cannot handle the varying frame rate, disable the dynamic FPS frame skipping or set a minimum allowed dynamic FPS.

This chapter presents the measured bitrate reduction for different types of surveillance scenes.

This section presents scenes where Zipstream can reduce storage needs. The measured total bitrate reduction is shown together with the Zipstream effort level and whether dynamic GOP and dynamic FPS were enabled or not.

	Zipstream strength: Low Dynamic GOP: Off Dynamic FPS: Off Total bitrate reduction: 25%

	Zipstream strength: High Dynamic GOP: On Dynamic FPS: Off Total bitrate reduction: 50%

	Zipstream strength: High Dynamic GOP: On Dynamic FPS: Off Total bitrate reduction: 90%

Zipstream strength: Extreme Dynamic GOP: On Dynamic FPS: On Total bitrate reduction: 73%

Zipstream strength: Extreme Dynamic GOP: On Dynamic FPS: On Total bitrate reduction: 99.7%

Zipstream strength: Extreme Dynamic GOP: On Dynamic FPS: Off Total bitrate reduction: 85%

Enabling Zipstream storage profile can save a lot of storage compared with the default Zipstream setting. Even for scenes with a lot of motion, storage profile can further reduce the bitrate because new compression tools are used. Dynamic GOP (and dynamic FPS) does not matter much in these scenes because there is motion all the time. If the motion calmed down, dynamic GOP would provide additional savings.

	Zipstream storage profile Enabled Additional bitrate reduction*: 40% * compared with Zipstream default configuration (strength: Low, dynamic GOP: Off, dynamic FPS: Off)

	Zipstream storage profile Enabled Additional bitrate reduction*: 33% * compared with Zipstream default configuration (strength: Low, dynamic GOP: Off, dynamic FPS: Off)

	Zipstream storage profile Enabled Additional bitrate reduction*: 32% * compared with Zipstream extreme configuration (strength: Extreme, dynamic GOP: On, dynamic FPS: On)

The algorithm for PTZ cameras enables Zipstream to reduce the bitrate even when the camera is panning, tilting, or zooming. The algorithm reduces the bitrate in real-time by automatically updating the dynamic ROI that preserves important image details. To further improve PTZ usability and reduce system requirements, a dynamic bitrate controller has been added to avoid bandwidth peaks caused by camera movements. It does this by reducing the general video quality while preserving reference points that the operator can use for navigation, in order to maintain the orientation and the tracking of important objects during rapid camera movements.

Storage profile works with PTZ cameras but may introduce some latency in the live view mode.

In a PTZ camera, the dynamic ROI algorithm compensates for both scene motion and camera motion simultaneously. During camera movements, some areas of the video are identified as more important and prioritized, while other areas are compressed more to reduce bandwidth usage. This part of the algorithm reduces the average bandwidth and storage, while preserving forensic details.

Even with the enhanced dynamic ROI enabled, a panning, tilting, and zooming camera requires more bandwidth than a fixed camera. This is because new information is captured at a very high rate during the camera’s rapid repositioning. However, since motion blur reduces the video quality anyway, a dynamic bitrate controller algorithm can be used to automatically reduce the bitrate and avoid bandwidth peaks triggered by camera motion. A PTZ camera typically performs panning, tilting, and zooming within a fraction of a second. As soon as the camera stops again, the bitrate controller immediately restores the bitrate to deliver optimal video quality.

The dynamic bitrate controller reduces the requirements on the entire system, such as transmission equipment (switches and routers), storage (recording servers and disk size), and viewing devices (computers and decoders). This means that remote PTZ cameras can be operated using a less complex transmission channel, while still preserving their benefits and flexibility.

The example in the figure below plots the instantaneous bitrate from a video with four different motion scenarios (J, K, L, and M). The upper graph shows the result when Zipstream is disabled. In the lower graph, Zipstream for PTZ is enabled. All streams are VBR streams with the GOP length=32. The instantaneous bitrate (the yellow areas) can be read on the vertical axes.

J: Initially, the PTZ camera is motionless in its overview position. The standard Zipstream algorithm is saving considerable amounts of storage as the camera is completely still. Suddenly the camera captures a small amount of motion.

K: The operator pans and zooms the camera to get higher resolution footage of the motion event. During the rapid repositioning, the dynamic bitrate controller achieves a substantial bitrate reduction.

L: The PTZ camera records the event in high quality video. The standard Zipstream algorithm automatically reduces the bitrate in un-prioritized areas of the image.

M: After the event, the operator pans and tilts to view a larger area to search for similar events. The video quality is automatically adjusted to match PTZ movements.

Zipstream can be enabled also for the products with the highest need for bitrate reductions: 4K and multi-megapixel cameras. While these high-resolution cameras are extremely efficient at capturing forensic details, they have been expensive to use due to the high storage requirements. Today, Zipstream can analyze a 4K stream in real time to reduce transmission and storage.

Panoramic cameras are fixed cameras that provide wide-area coverage – from 180º to 360º with a single camera. They are often used in surveillance, especially for monitoring activity and detecting incidents in large areas, for tracking the flow of people, and for improving area management. New panoramic camera models combine wide-area coverage with multi-megapixel resolution, and provide de-warped images with a high level of detail. Zipstream supports these cameras for all panoramic view options and can significantly reduce storage requirements.

In cameras with the system-on-chip ARTPEC-9, Zipstream supports hardware-accelerated AV1 video encoding based on the AV1 format released by AOM. Even though it is completely new for the security use case, AV1 is the video encoder expected to eventually replace H.264 in popularity thanks to low bitrate, new features, and wide client decoder support.

AV1 is likely to become the preferred video encoder standard for cloud solutions, and also prove valuable for on-prem solutions that require cloud integration or remote video access for mobile users. With its rapidly expanding ecosystem, AV1 is optimized for low footprint but can also scale to custom hardware when offloading is needed.

Zipstream supports the global video encoding standard H.265. However, H.265 was developed for noiseless broadcast video and is not yet fully adapted for video surveillance, where difficult lighting conditions are common. Also, H.265 ecosystem support is still limited and lacking real progress.

Zipstream for H.265 is delivered with the same tools and benefits as the initial H.264 version, but with even lower bitrates for complex scenes. H.265 is very efficient for encoding moving objects with a lot of detail, but in some cases Zipstream with H.264 could still deliver lower bandwidth.

Zipstream can, depending on camera SoC, provide H.264, H.265, and AV1 support in parallel in the same camera, without requiring reconfiguration or complicated system setups. True multi-streaming with selectable codec and configurations per stream enables all these types of video to be transmitted or stored, for maximum flexibility. This multi-codec approach is central for making the transition period between standards as smooth as possible.

Camera surveillance systems require the bitrate to be reduced, while at the same time maintaining the image quality. Even the slightest deviation must be detected, and it must be possible to perform advanced forensic work after an incident. Zipstream enables continuous recordings due to the low bitrate used for static scenes.

For AXIS Camera Station Edge an even lower bitrate is desirable, because system cost and ease-of-installation are priorities. The aim is to save video of sufficient quality on cost-efficient edge storage. However, video quality needs to be reduced in a controlled manner, in order to easily find and understand the course of events. Zipstream reduces the number of missed triggers by allowing longer recording segments for each motion-triggered event without generating excessive data.

Zipstream is relevant for users who wish to reduce the cost of storage or network loads. In any video surveillance system, reducing storage needs directly results in a lower total cost, independent of system size or storage solution. With Zipstream, less storage is needed per recorded minute. This enables increased retention time, resolution, or number of cameras, without having to increase the storage space.

Cameras that use Zipstream and AV1 are very attractive for cloud solutions, which require efficient, low-bitrate video encoding. Because AV1 is natively supported by many mobile devices, computers, and web browsers without needing plugins, Zipstream with AV1 integrates seamlessly with cloud provider toolboxes. Additionally, AV1 can be used with WebRTC for realtime, low latency, high resolution video streaming with significantly lower bitrate compared with traditional H.264 encoding.

Axis recommends using networked video with a variable bitrate (VBR), where the quality adapts to the scene content in real time. Using a constant bitrate (CBR) as a storage reduction strategy is not recommended, as cameras delivering CBR video may have to discard important forensic details in critical situations due to the bitrate limit.

Zipstream makes it possible for the system installer to continue using VBR, with or without a limit, for optimal video quality while reducing the storage requirements. In this way the system can keep delivering high quality video. Important forensic details such as faces, tattoos, and clothing patterns are isolated and preserved, while irrelevant parts such as white walls, lawns, and vegetation are smoothed out.

If a storage solution or the network requires an absolute upper bandwidth limit, Zipstream is compatible with MBR, a method that protects the system from temporary bandwidth spikes.

AOM: Alliance for Open Media

AV1: AOMedia Video 1

CBR: Constant bitrate

FPS: Frames per second

GOP: Group of pictures

HEVC: High Efficiency Video Coding

IEC: International Electrotechnical Commission

ISO: International Organization for Standardization

ITU: International Telecommunication Union

ITU-T ITU: Telecommunication Standardization Sector

MBR: Maximum bitrate

MPEG: Moving Picture Experts Group

NAS: Network-attached storage

PTZ: Pan-tilt-zoom

ROI: Region of interest

SoC: System on chip

VBR: Variable bitrate

VCEG: Video Coding Experts Group or Visual Coding Experts Group

VMS: Video management system