Thermal cameras

• What can thermal cameras do?
  Thermal cameras detect the thermal radiation (heat) that all objects with a non-zero temperature emit. With the ability to pick up small temperature differences and convert them into a visual image, these cameras can distinguish persons and vehicles at very large distances. They keep on performing even in complete darkness and regardless of lighting conditions, camouflaging, vegetation, difficult weather, or other conditions where a visual camera would be insufficient.

• What are they used for?
  Thermal cameras are widely used in perimeter protection systems. Live video from a thermal camera can reveal individuals around critical locations long before a visual camera has detected anything unusual. The thermal images are automatically analyzed directly in the camera, and the security system can be set up to respond in various ways. It can trigger automatic audio alerts in loudspeakers to actively deter the intruders, email alerts to security personnel, and pan and zoom the system’s visual cameras to capture and record ordinary video footage in which the intruders can be identified.

  Thermal cameras are also installed to monitor the temperature of industrial processes. They can be used to find heat leaks in buildings or determine whether a vehicle was recently used.

  It is generally not possible to identify specific individuals from thermal images alone. This makes thermal cameras a valuable option for surveillance in locations where privacy is especially important, such as schools.

• NETD is a measure of thermal sensor accuracy
  A thermal sensor’s ability to detect very small differences in thermal radiation can be characterized by its NETD (noise equivalent temperature difference) value. In general, the smaller the NETD, the better the sensor. However, cameras should not be rated by comparison of NETD specifications only, due to the lack of a standardized measurement protocol.

• Rules of thumb for installation guidance
  Johnson’s criteria describe the relation between the minimum required resolution and the expected detection range, based on whether you want to be able to detect, recognize, or identify vehicles or individuals. Another tool is the nomograph, which shows graphically the relation between detection range and the focal length of the camera lens for specific resolution requirements. However, actual results may differ depending on weather conditions. Also, if analytics applications are used, they may require a larger number of pixels in order to function than what these rules of thumb suggest.

• Environmental impact on detection
  Rain, fog, and smog reduce the detection range. The rate of thermal radiation attenuation depends on the size and concentration of the particles or water droplets in the air. But a thermal camera’s range is, for most cases, much less affected by such phenomena than a visual camera is. Especially in moderate fog or smoke, thermal cameras detect objects that would be completely invisible to a visual camera.

Thermal cameras create images based on the infrared radiation that is emitted from all objects as a function of their temperature. The ability to detect small temperature differences makes these cameras excellent at distinguishing persons obscured by complex backgrounds or hidden in deep shadows. Vehicles and other objects are also easily detected, day and night and irrespective of the light conditions.

This white paper discusses the benefits of thermal cameras and their use with video analytics in perimeter protection. It describes how thermal camera performance can be measured and how the detection range is based on the focal length of its lens combined with the desired level of accuracy. We also provide some insights into how weather conditions affect performance, and what to consider when you are about to install a thermal camera.

Thermal cameras are used in a wide range of security applications, such as perimeter protection around industrial sites, airports, and power plants. Live video from a thermal camera can inform a security operator about a person walking among the cars in a parking lot long before a visual camera has detected the movement. Their outstanding detection capabilities also make thermal cameras valuable in search and rescue operations.

Thermal images alone are, in general, not enough to identify individuals. Thermal cameras are therefore a good choice in many situations where privacy is an issue. In many countries, you must have permission from the authorities to record video in public areas. It is often easier to get permission for thermal cameras than visual cameras because individuals in the scene cannot be identified.

Compared to visual cameras, thermal cameras provide more reliable detection and shape recognition. This is achieved by combining high image contrast with motion detection. As a result, the false alarm rate can be kept down, with fewer unnecessary responses and actions by personnel.

The thermal information provided by a thermal camera also makes it possible to monitor processes and detect abnormal behavior when temperatures change. For example, thermal cameras can be used to find heat leaks in buildings or determine whether a car has been driven recently.

Axis thermal cameras provide discreet and cost-effective detection and can significantly enhance building security and emergency management. The built-in intelligence of the camera together with added video analytics create a solution where the video surveillance system automatically performs an analysis of the captured video. Thermal cameras help distribute this analysis to visual cameras in the IP system, for example, in perimeter protection.

Axis analytics applications for perimeter protection provide a highly effective system that automatically detects and responds to intruders. You can decide which types of responses you want. When someone enters a predefined area within the camera’s field of view, a thermal camera can, for example, automatically trigger email alerts to security staff and at the same time trigger a PTZ (pan-tilt-zoom) camera to supply visual video. This makes it possible to acknowledge suspicious activity already before intrusion and to visually verify what is going on before taking relevant action. The camera can also use edge-to-edge technology to activate a loudspeaker to scare off intruders.

Axis analytics applications for perimeter protection are edge-based. This means that they are embedded in the cameras, which is where the analysis takes place. Since the video is not sent to any central server for analysis, the system is flexible and scalable and costs can be kept down.

NETD is the most common measure of classifying the performance of a thermal sensor, and even of whole thermal camera systems. NETD is short for noise equivalent temperature difference. This defines the sensor’s noise threshold, i.e., NETD represents the temperature difference required to produce a signal equal to the noise threshold.

NETD essentially determines the sensor’s ability to distinguish very small differences in thermal radiation in the image. The smaller the NETD, the better the sensor. With a NETD of, for example, 50 mK (millikelvin), a sensor can detect only those temperature differences that are larger than 50 mK while smaller differences will disappear in the noise.

Comparing specified NETD values between different cameras can be problematic. The values may have been calculated using different methods or under different conditions, for example in different ambient temperatures, using different integration times, or with different optical F-numbers. Specified NETD values also usually do not include spatial noise. This means that the NETD can be low even though the image is quite noisy due to fixed and quasi-fixed spatial noise.

Actual camera performance is affected by many factors other than the NETD value of its sensor, and the best camera does not necessarily have the smallest NETD. For example, NETD does not take into consideration how well in focus a camera is; a camera out of focus can still have a good NETD value. Thus, one thermal camera should not be chosen over another based only on a comparison of their specified NETD values.

At Axis, we measure the NETD of thermal cameras according to a common approach, as described in this section.

A thermal camera system with F/1.0 optics is used. The target is a blackbody of good quality. Most image processing steps (such as linear and non-linear signal transferring, sharpening, and local image enhancement) are bypassed, while non-uniform correction, flat field correction, and noise filtering are performed.

Datasets are collected at blackbody temperatures of 20 °C, 25 °C, and 30 °C.

For both 20 °C and 30 °C, a sequence of 100 frames is collected. The average of these two datasets is calculated for each pixel, producing two average frames — one at 20 °C and one at 30 °C. By further subtracting these two frames from each other and dividing with the temperature difference (i.e., dividing with 10 °C), we get the average response frame of the thermal camera system.

For 25 °C, a dataset of 200 sequential frames is collected. The standard deviation of each individual pixel of these 200 frames is calculated and stored in a frame. This frame of pixel standard deviation values is divided with the average response frame. The result is averaged and multiplied with 1000 to yield the NETD value in mK (millikelvin).

The resolution that is required for detection is stated in pixels and determined by means of Johnson’s criteria. This is a method developed in the 1950’s for predicting the performance of sensor systems. The American scientist John Johnson measured the ability of observers to identify scale model targets under various conditions and came up with criteria for the minimum required resolution. These criteria provide a 50% probability of an observer distinguishing an object at the specified level.

The object can be a person, typically defined with a critical width of 0.75 m(2.46 ft) or a vehicle, typically defined with a critical length of 2.3 m(7.55 ft). For a thermal sensor, the temperature difference between the object and its background needs to be at least 2 °C(3.6 °F) according to Johnson’s criteria.

The levels of Johnson’s criteria used for Axis thermal cameras are:

• At least 1.5 pixels are needed for detection, meaning that the observer can see that an object is present.

• At least 6 pixels are needed for recognition, meaning that the observer can distinguish the object, for example, a person in front of a fence.

• At least 12 pixels are needed for identification, meaning that the observer can distinguish an object and object characteristics, for example, a person holding a crowbar in his hand.

Johnson’s criteria were developed under the assumption that visible information was processed by a human observer. If the information instead is processed by an application algorithm, there will be specific requirements on the number of pixels needed on the object for reliable operation. It should be noted that even if a human observer would be able to detect the object, the application algorithm may need a larger number of pixels at a given detection range to work properly.

For finding the required number of pixels at a given range, a nomograph is a practical tool. This is a two-dimensional diagram that explains the relation between the focal length of the lens, the number of pixels across the object, and the range.

For example, if we know both the number of pixels required and the distance at which we need to be able to recognize an object, it is possible to calculate which lens or camera to use. Equally, if the camera and the number of pixels required are known, the distance at which the camera can detect an object is indicated by the nomograph.

The nomograph example indicates that if the focal length of the camera is 60 mm, the object will be recognizable (6 pixels across the object) at 300 m(328 yd) (point A). If only detection is required (1.5 pixels across the object), the range will instead be 1200 m(1,312 yd) (point B).

It is essential to remember that Johnson’s criteria are valid only in ideal conditions. The weather conditions on site will affect the detection range of both the human eye, a visual camera, and a thermal camera. The detection range of a thermal camera is usually less influenced by the weather, for example on a misty day, than the range of a visual camera.

The detection range exemplified in the nomograph of the previous section ideally requires a temperature difference of 2 °C(3.6 °F) between the targeted object and the background. Weather conditions can have a negative effect on the thermal image by leveling out temperature differences, but advanced image processing such as local contrast enhancement helps the camera distinguish objects from the background even when the temperature difference is small.

The two most important environmental factors that affect the image of an object in the camera are absorption and scattering. They reduce the thermal radiation that reaches the camera, thereby reducing the distance at which the camera can detect an object. Scattering has a greater effect on the loss of thermal energy than absorption.

Water vapor (H₂O) and carbon dioxide (CO₂) in the air are the primary causes of absorption. During absorption, the heat radiated from the object is absorbed by water vapor and carbon dioxide and loses some of its energy before reaching the camera. The water vapor content of the air affects image quality even in sunny and clear weather, when the water vapor content can be high.

On a day when the water vapor content is low, less thermal radiation is absorbed by the water molecules, allowing more thermal radiation to reach the thermal camera. This results in better image quality compared to a day when the water vapor content is higher.

During scattering, the thermal radiation from the object is dispersed when it hits particles in the air. The loss of radiation is directly related to the size and concentration of the particles, droplets, or crystals that constitute polluting, condensing, or precipitating conditions such as fog, smog, haze, rain, or snow.

Fog appears when water vapor in the air condenses into water droplets. The droplet sizes vary with different kinds of fog. Dense fog consists of bigger water droplets and thus scatters thermal radiation more than light fog does. In addition, fog scatters thermal radiation more than both smog and haze do, because of the greater size and concentration of water droplets in fog.

Axis thermal cameras work primarily in the long-wavelength infrared (LWIR) wavelength range. In general, the transmission of LWIR wavelengths is considerably better in conditions with airborne particles, such as fog and smoke, compared to visible wavelengths. In most cases, the ‘short’ visible wavelengths are absorbed and scattered by the particles to a higher degree than the LWIR wavelengths are. This decreases the detection range of the visual cameras compared to the thermal ones. A person that is clearly visible with a thermal camera in foggy weather might be invisible to a visual camera.

One way to classify fog is the system used by the International Civil Aviation Organization (ICAO). Its categories are defined by the visual range in each type of fog. The table below lists these categories and also the approximate detection range of LWIR wavelengths for each class.

It is evident from the table that for lighter fog (of classes I and II), the LWIR range is much longer than the visual range. For denser fog (of class III), however, even the LWIR wavelengths are absorbed and scattered. In this condition, there is almost no difference in range between visual and thermal cameras.

The table should be used only as an estimation. The actual detection range of a camera depends on other factors as well, such as physical objects in the scene, the temperature difference between the object and its background, and the physical installation.

Even though raindrops are larger than fog droplets, their concentration is lower. This means that rain does not scatter thermal radiation as much as fog does. The level of scattering during snowfall is somewhere in between the range of fog and rain. Sleet or wet snow has a scattering level more similar to rain, whereas dry snow is more similar to fog. Examples of approximate attenuations in different weather conditions are shown in the table.

For example, a thermal network camera with a 60 mm lens (as exemplified in the nomograph shown earlier in this document) will have a range of 300 m (328 yd) with 6 pixels on the targeted object on a clear day. On a foggy day, the attenuation will be 10 dB/km or 1 dB/100 m, giving an attenuation of 3 dB in total. The 3 dB attenuation means that only 50% of the emitted energy from the object will reach the thermal sensor, resulting in a lower input signal. A lower input signal will give a noisier image, since the signal-to-noise ratio decreases. To some extent, image processing will compensate for this, but the image will still contain less information and therefore look flatter. Its contrast will be lower which will make it more difficult to distinguish between, for example, foliage and flat surfaces in the image background. Signal attenuation will degrade camera performance and the reliability of integrated video analytics applications.

Therefore, installations where one single camera is working close to its maximum performance should be avoided. A better option is to use several cameras to cover the given distance. This will safeguard reliable operation by meeting the required amount of pixels on target and also ensure that the emitted energy from the object is sufficient.

Rain and wet snow not only scatter radiation, but also level out temperature differences in the image background. A level background temperature results in decreased background contrast for a thermal camera.

While scattering means that less energy reaches the camera sensor, the levelled out background temperature does not influence the sensor. However, since the contrast of the image will be lower, it will be more difficult to distinguish details in the background and the image will look flatter. It will still be easier for a thermal camera to detect a person, since the contrast between the warm person and the cold background will be higher.

On a cloudy day, the contrast in the background will be lower in the same way, whereas on a sunny day it will increase. Temperature differences increase because things with different surface material will be heated at different rates.

When installing a thermal network camera, there are some things to consider. To achieve the best results when detecting people, the temperature of the background of the monitored object should be as even as possible, and it should be colder or warmer than a typical person that may appear in the scene. This way a person will stand out from the background.

There should be a free line of sight from the camera to the region of interest, without anything disturbing or blocking the view. The scene should have one or a few easily recognizable objects, for example, a chimney against the sky, or a building. A chimney in use will be warm, and a building is almost always leaking some indoor heat.

Make sure that the scene does not contain any tree branches, flags, or similar objects that move in to and out of the scene when it is windy. The camera should be mounted as firmly as possible, and clear and sharp edges should be kept at a distance from the intended scene. A sharp edge just outside the scene may trigger a false motion alarm if the camera sways in the wind and moves the viewed scene over the edge. Since the camera is moving, it will interpret the changed image as movement in the scene, even though nothing but the camera has actually moved.

Thermal cameras with support for electronic image stabilization are less affected by vibration. However, these factors should still be considered when installing a thermal camera, to optimize camera performance.