|
The demand for camera-based driver assistance systems in the automobile industry continues to grow. Traffic sign recognition is an application that is receiving much attention because of its potential for improving the safety of drivers and passengers on the highway.
Such image processing is still a high-end option, and will require joint promotion by government and auto industry leaders to develop a mass market opportunity. The technology is an example of the image recognition that can be applied to many areas of vehicular and industrial design.
Traffic sign recognition is based on pattern recognition algorithms that are well understood by machine vision specialists. There is no doubt that the classic pattern-recognition algorithms are capable of straining the capabilities of even advanced general purpose microprocessors. For this type of signal processing application, it is advantageous to select a processor with an architecture that is designed to efficiently deal with the particular computational and I/O challenges inherent in processing a continuous video stream in search of traffic sign images. It is essential that the algorithms are also coded for maximum speed and efficiency.
These articles detail a camera-based traffic-sign recognition system implemented on the Blackfin® processor, a low-power and low-cost embedded DSP from Analog Devices. We will demonstrate the design of an efficient software architecture, and show how a filter block algorithm can be used to implement the sign recognition application. The results are compared, and it is then shown what steps can be undertaken in order to make optimal use of the available computing performance.
A camera-based driver assistance system
A typical system for camera-based pattern recognition consists of a video camera and a monitor that are both connected to the I/O interfaces of a processor that controls the operation of the system and performs the required signal processing. External SDRAM is usually required to provide adequate memory space for the video processing required to extract road sign messages from the input video stream. The figure below shows the block diagram of such a system. Although the display may not be deployed in the final application, it is a helpful tool for application development and presentation.
Image sensor
Leading auto makers have qualified CMOS image sensors as video sources for driver assistance systems. At present, most cameras capture gray scale images of VGA size (640 x 480 pixels) with up to 10-bit grayscale depth. The frame rate is generally 30 frames per second and the sensitivity of CMOS sensors allows use in low light and at night as well as in the glare of bright sun and oncoming lights.
In future applications, sensors with a larger frame size and color capability will be required. The camera in this example design delivers color images transmitted in the luminance/chrominance YUV differential image format. Most sensors are driven by an external clock generator and provide the image data over a synchronous parallel data bus with a width of up to 10 bits. Two additional signals allow synchronization to the line length and the image height.
TFT display
LCD TFT displays, which have become common in the automotive dashboard, require RGB image data. Every display pixel requires input values for the three display colors (red, green, and blue). This mismatch in image formats between capture and display is common in video systems and well known color"space conversion algorithms switch between the RGB and YUV formats. The diagram below shows the connection of a TFT display to the processor.
View a full-size image
Multi-core embedded processor speeds signal processing
The central processor of the example driver assistance system is the Blackfin BF561. The Blackfin architecture was developed for signal and image processing and offers arithmetic and logic units (ALUs) which permit the parallel execution of instruction streams on multiple pieces of data. As mentioned previously, using a processor designed for signal processing applications assures that the filter and pattern matching algorithms run efficiently and flexibly due to the efficient signal processing instruction set and hardware features. The BF561 is a dual core processor with two symmetric processor cores and thus twice as much raw computing capacity is available for the application.
We have already mentioned the parallel I/O ports that are used for the capture and display video interfaces. In addition, Blackfin processors provide a large number of other interfaces such as I2C and NAND flash interface for common system level interface and interfaces for operation in specific application environments, such as automotive systems where the CAN bus, and MOST bus are usually needed.
Software architecture
We are using the term software architecture to refer to the software that controls the system. It refers to the control code that contains the state machine of the system. The control code configures all interfaces; calls the required filter blocks in a predetermined order; starts the direct memory access (DMA) data transactions; and synchronizes events that pause or repeat processing.
Processor cores
The processor cores execute the control software and perform mathematic calculations. Each Blackfin processor core has two ALUs, allowing up to four calculations to be performed simultaneously. Read and write operations retrieve the data needed for calculation from memory, and store the results. A maximum width instruction is shown below. This instruction simultaneously performs two independent multiplications per processor core with subsequent additions and two load operations:
If, because of address conflicts, this maximum width instruction cannot be scheduled, parts of the parallel instruction can be performed. Here are a few examples:
The types of operations that can be performed in parallel are numerous, with some dependencies on hardware implementation.
|
