Using intelligent cameras for smart security and elevated surveillance

This article looks at the range of technologies essential to building today’s unobtrusive and low-power yet high-performance AI cameras at the edge.

Public safety and infrastructure security are fuelling the adoption of intelligent IP cameras with embedded AI, which can detect and analyze events and behaviors, and send real-time alerts to first responders.

These cameras protect people and property, enhance worker safety, unlock business insights, and drive productivity gains. In a typical security scenario, AI helps detect anomalies like a person entering a restricted area or unusual behaviour and reports it to the system.

While security is often the primary driver for deploying surveillance systems, they are increasingly being used to provide additional analytics that can unlock business insights and operational efficiencies to retailers, transport hubs and factories. In car parks, for example, AI uses surveillance data, checks whether a vehicle has paid the parking fee and allows it to enter the parking lot. Simultaneously, it provides statistical analysis about how many vehicles entered, how long they stayed and much more.

Fig.1 shows more applications for both security and improved business insights.

Figure 1: The growing surveillance market is driven by more than just security

Giving these cameras the ability to see, sense, and think at the intelligent edge calls for several core technologies. One way of summarising these technologies is as below:

• Ultra-low power microcontrollers with embedded AI for audio and image event detection while providing secure node authentication and data protection.
• Connectivity: Edge-to-enterprise Industrial Ethernet and Power over Ethernet (PoE) solutions enable seamless connectivity.
• Intelligent motion: Cameras must rapidly target an event or intruder, and then smoothly and noiselessly track and record movement with maximum captured image and video quality.
• 24 GHz radar provides augmented vision capabilities to overcome harsh weather conditions, minimize false alarms and enabling fast intruder detection.
• Robust protection from cyber-attacks using physically unclonable function (PUF) technology, implemented using ChipDNA semiconductor hardware.

This approach can be illustrated by Fig. 2, which shows one possible implementation of these technologies, using Analog Devices semiconductor hardware.

Figure 2: Analog Devices’ elevated surveillance hardware components

Small andlow-power

Today, intelligent cameras demand compactness, featherweight design, and minimal power consumption. The intelligent edge and intelligent motion components are particularly important in achieving these objectives, so we take a closer look at these in the content that follows.

We also discuss key security concepts and hardware, as these are vital to secure camera operation.

Enabling the intelligent edge with optimized power economy: using the MAX 78002

While AI is the key to modern surveillance camera performance, it tends to require extreme computational and electrical horsepower. This just adds to the proliferating demands of the other components, such as the PTZ motors.

Analog Devices addresses the AI part of the power equation with their MAX78000 AI microcontroller family. These devices enable neural networks to execute at ultra-low power and live at the edge of the IoT. They combine the most energy-efficient AI processing with Analog Devices' proven ultra-low power microcontrollers. The hardware-based convolutional neural network (CNN) accelerator enables applications to execute AI inferences while spending only microjoules of energy.

The MAX78000 features an Arm® Cortex®-M4 with FPU microcontroller for efficient system control with an ultra-low power deep neural network accelerator. A RISC-V core is also integrated and can execute application and control codes as well as drive the CNN accelerator.

The CNN engine has a weight storage memory of 442KB and can support 1-, 2-, 4-, and 8-bit weights (supporting networks of up to 3.5 million weights). The CNN weight memory is SRAM-based, so AI network updates can be made on the fly. The CNN engine also has 512KB of data memory. The CNN architecture is highly flexible, allowing networks to be trained in conventional toolsets like PyTorch® and TensorFlow®, then converted for execution on the MAX78000 using tools provided by Analog Devices.

In addition to the memory in the CNN engine, the MAX78000 has large on-chip system memory for the microcontroller core, with 512KB flash and up to 128KB SRAM. Multiple high-speed and low-power communications interfaces are supported, including I2S and a parallel camera interface (PCIF).

Fig.3 shows the top-level architecture of the MAX78000.

Figure 3: MAX78000 architecture

Our article ‘The modern challenges of image recognition and facial recognition – and how AI is facilitating tiny, competitive edge-based solutions’ describes the MAX78000 and its evaluation kit, and how to use these to build Face Identification models.

This article focuses on a different aspect of the MAX78000: how its different clocks and operating modes can be jointly configured to optimise power and performance. A brief description of these is followed by an example showing optimized power and performance figures for a Face ID demo application which shows the identification of subjects using facial recognition on the MAX78000.

Clocks: The MAX78000 includes multiple configurable clocks used by different peripherals. The user can configure the clock sources as needed to select the optimum combination of performance and power efficiency. The selected System Oscillator (SYS_OSC) is the clock source for most internal blocks. The following oscillator sources are available and can be selected as SYS_OSC:

• Internal Primary Oscillator (IPO) – 100MHz:
  IPO is the fastest frequency oscillator and draws the most power. When enteringlow-power Mode (LPM), this oscillator can be powered down.
• Internal Secondary Oscillator (ISO) – 60MHz:
  This is alow-power internal secondary oscillator that is the Power-On Reset default for SYS_OSC.
• Internal Nano-Ring Oscillator (INRO) – 8kHz-30kHz:
  INRO is an ultra-low power internal oscillator that can be selected as the SYS_OSC and is always enabled. The frequency is configurable to 8kHz, 16kHz, or 30kHz (default).
• Internal Baud Rate Oscillator (IBRO) – 7.3728MHz:
  This is a verylow-power internal oscillator that can be selected as SYS_OSC. This clock can optionally be used as a dedicated baud rate clock for the UARTs.
• External Real-Time Clock Oscillator (ERTCO) – 32.768kHz:
  ERTCO is an extremelylow-power internal oscillator that can be selected as the SYS_OSC. The ERTCO can optionally use a 32.768kHz input clock or an 8kHz independent nano-ring oscillator instead of an external crystal. This oscillator is the default clock for the Real-Time Clock (RTC) and is enabled automatically as soon as the RTC is enabled.

Operating modes: The MAX78000 includes multiple operating modes and the ability to fine tune power options to optimize performance and power. Table 1 shows the operating modes available, and their impact on the various MAX78000 internal components.

Table 1: MAX78000 operating modes

Power consumption in different modes: In developing an application, the user can switch the MAX78000 to different operation modes and schedule the tasks accordingly to save power. An understanding of power consumption in each operation mode, as well as the time needed to boot up or to wake up, helps the user select the appropriate mode and plan the duty cycle of the tasks.

CNN power: The core of an AI application is the inference, performed by the CNN accelerator. Depending on the application, the inference can happen continuously on incoming data or periodically at certain time intervals. The CNN inference can start once the input data is ready, or in FIFO mode where it starts as soon as enough data to begin the process is stored in the FIFO.

Application power: When developing an application, it is important to choose the operation mode of each core, clock frequency, and duty cycle. Typically, applications running on Arm can be executed at a higher frequency (100MHz, vs. 50MHz APB clock) and have more optimized code, which can result in lower overall energy used. If there are parallel tasks that can be assigned to both Arm and RISC-V cores, it is important to use the proper operation mode for each core.

Case Study - FaceID Demo on Arm and CNN: This demo application demonstrates the identification of subjects using facial recognition on the MAX78000. The FaceID CNN model generates an embedding of the size 512 from live images captured from the EV kit camera. The distance of the embedding from the embeddings of known subjects is calculated, and if it is greater than a threshold, the best candidate is selected. The database of known embeddings is created from pictures of the subjects and is integrated with the firmware. The inference is executed on a 160x120 face box of each image. To enhance the performance of identification, the inference is executed three times on a frame as well as each time the face box is slightly moved on the picture.

In this case study, RISC-V is off, and the entire task is driven by Arm and executed sequentially. The Arm goes to SLEEP mode during inference and UPM or STANDBY between frames for about half a second. Table 2 summarizes the execution time and power of the main functions of the FaceID.

Table 2: Power and execution time of different tasks in faceID (arm and CNN)

Intelligent motion control for PTZ cameras

The challenge of the motion control subsystem is to maximise the quality of captured images and video. This calls for rapid transitions to new target positions and then smooth continuous variable speed tracking. The operation should also be noiseless to avoid attracting attention from the camera’s target.

Stepper motors are the preferred technology as they are of low cost with simpler system architecture. However, these motors create key challenges related to smoothness and precision of movement, as well as motor vibrations.

Stepper motors have an inherently ‘jerky’ motion, which leads to uneven movements, and vibrations in the camera. The need for precise movement requires a higher resolution than that available from a typical stepper motor of, say, 200-step resolution. Meanwhile, MOSFET switching causes audible noise.

This ‘Jerky’ motion of the motor, which is mainly evident at low to medium speeds, can be countered with gearing – but gearing is expensive, and may amplify stepper motor resonance points. Adding 16X microstepping can make motion smoother but it can be improved.

Trapezoidal ramps – accelerate to the desired value, then decelerate to zero to stop - are often used for motion profiles, but these can lead to sudden stops and starts, making the video feed jumpy and possibly unwatchable.

Improvements can be made by using higher resolution 256X microstepping with suitable chopper algorithms. Along with higher precision, vibration is reduced. Even smoother movement can be achieved using more precise MOSFET switching schemes such as StealthChop™.

Precise movement to a target position for camera tracking can be achieved using more complex ramps like sixPoint™ or eightPoint™ rather than the simple linear ramp – see Fig.4. Multiple acceleration phases help to reduce jerk, so images and video remain clear. Additionally, the camera can move faster without loss of image or video quality. The idea is to come as close as possible to S-shaped (Sinusoidal) ramping, depending on available computer power. Dedicated motion controllers are available for this.

Figure 4: Liner ramping vs eightPoint ramping

Audible noise can be reduced by using a precise MOSFET switching algorithm. Different types of current control and voltage control algorithms are available. The more advanced algorithms can also reduce motor vibration and interference with the video feed.

Analog Devices provides packaged motion control solutions for stepper motors in PTZ cameras, available at different levels of integration. The driver system contains the predriver and MOSFET stages. The cDriver contains both motion controllers and drivers in one package, where a microcontroller sets up the motion controller and manages step generation while the driver takes care of the current regulation.

At the higher end is the full MotionCookie™ modular solution, which contains microcontroller, power circuitry and interfaces.

Motion control technology from ADI Trinamic includes solution for stepper motors as well as brushless DC (BLDC) and brushed DC (BDC) motors – from mW to kW sizes, and voltages from battery 1.8V to 80V.

One of Analog Devices’ most recent voltage control algorithms is called StealthChop™. This is a voltage-based PWM chopper, which modulates the current based on the duty cycle. It reduces audible noise dramatically and is optimized for low to medium velocities. It leads to the smoothest movements and a clean sine wave for silent motor operation.

StallGuard2™ is a back EMF detection algorithm that is mainly used for detecting stalls in the motor. StallGuard2 can detect if the back EMF changes due to load change or some other scenario, and, with proper tuning, load changes beyond a threshold size will raise a flag to trigger any responsive action needed; the motor can be stopped in one full step if necessary.

StallGuard2 allows designers to build applications without end stops for homing, reducing cost. It can also provide a system health check, highlighting any adjustments needed. Additionally, it can indicate any obstacles, essentially adding a diagnostic/failure function to the system.

Semiconductor implementation of these techniques

The ADI Trinamic TMC5072 cDriver chip is an integrated motor driver and motion controller solution for CCTV cameras, pumps and other automated equipment applications. The device has two integrated SixPoint™ ramp controllers, microstepping indexers, the sensorless stall detection technology StallGuard2™ and the completely noiseless current control mode StealthChop™ and is intended to drive two bipolar stepper motors.

The output driver blocks consist of low RDSon N-Channel power MOSFETs configured as full H-bridges to drive the motor windings. The TMC5072 is capable of driving up to 1.5A of current from each output (with proper heatsinking). TMC5072 is designed for a supply voltage of 5-28V. For high-reliability requirements, an Encoder interface provides feedback on the system status. The device provides a SPI interface for configuration and diagnostics and a step and direction interface.

Figure 5: TMC5072 block diagram

Security

Security is fundamental to the integrity of a surveillance network. Cryptography can provide the necessary protection, as it deploys flexible and effective tools to counter a myriad of potential security threats that embedded electronic systems face. There are a variety of hardware and software approaches for implementing crypto solutions. It is generally understood that a hardware-based solution (i.e., a dedicated security IC) is the most effective formulation for the root of trust and the way to provide the countermeasures and protection that prevent numerous types of common attacks.

Attacks on Security ICs: Assuming a security IC-based protection solution, there are two general categories of attack scenarios: non-invasive and invasive.

Non-invasive attacks consist of operational measurements, sometimes combined with other externally applied stimuli, in an effort to obtain cryptographic keys or other sensitive data. Examples of such efforts include differential or simple power/electromagnetic analysis (DPA/SPA/DEMA/SEMA) or the inducing of fault states through voltage glitching, extreme thermal conditions, or laser and timing attacks. While the non-invasive attack threats are technically complex to address, there are established circuits and algorithmic countermeasures that are proven effective in protecting the security IC and sensitive stored data from being compromised.

Invasive attacks on a security IC consist of direct die-level circuit probing, modification, deprocessing, and reverse engineering, again with the objective of compromising the solution by obtaining keys, disabling functionality, or completely reverse engineering the design to a netlist for reproduction. The skill set and required tools are more complex than in the non-invasive scenarios, but they do exist and are commonly used to attack the security ICs that protect high-value assets.

PUF technology solution: Physically unclonable function (PUF) technology has emerged to provide strong protection against invasive threats. PUF is derived from the complex and variable physical/electrical properties of ICs. Because PUF depends on random physical factors (unpredictable and uncontrollable) that exist natively and/or are incidentally introduced during a manufacturing process, it is virtually impossible to duplicate or clone. PUF technology natively generates a digital fingerprint for its associated security IC, which can be utilized as a unique key/secret to support cryptographic algorithms and services including encryption/decryption, authentication, and digital signature.

Semiconductor cryptographic coprocessor solution: Analog Devices offers a semiconductor device which implements PUF technology – the DS28S60 cryptographic coprocessor, which together with the MAXQ1065 cryptographic controller, can solve edge node security challenges around data security, IP protection, secure connectivity and device authentication.

The DS28S60 integrates Analog Devices patented ChipDNA™ feature, a physically unclonable function (PUF) to provide a cost-effective solution with the ultimate protection against security attacks. Using the random variation of semiconductor device characteristics that naturally occur during wafer fabrication, the ChipDNA circuit generates a unique output value that is repeatable over time, temperature, and operating voltage.

Attempts to probe or observe ChipDNA operation modifies the underlying circuit characteristics, preventing discovery of the unique value used by the chip's cryptographic functions. ChipDNA output is utilized as key content to cryptographically secure all device-stored data.

Figure 6: DS28S60 block diagram

Conclusion

Sophisticated, unobtrusive, low-power surveillance cameras that meet today’s security demands are, as Fig. 2 shows, dependent on a wide range of technologies, plus advanced semiconductor devices that implement them. The primary drivers are the edge intelligence, intelligent motion, and secure aspects that this article has discussed. However, vision augmentation, flexible connectivity and power delivery, although not discussed here, are also important.

Yet, as Fig. 2 also reveals, Analog Devices has solutions for all these technologies in terms of depth of understanding, available hardware, and evaluation kits.

All this technology is available from Newark, backed by engineers ready to discuss your application with you and help you find your best way forward.