同高科技

The green stereo box is the detection of real cars on the ground. Yellow is the point cloud displayed by the lidar. The pink point cloud is generated by an independently trained depth estimator and is located outside the green box, so it is very inaccurate. Through end-to-end depth estimation and 3D target object detection, a more accurate blue point cloud is obtained. (Photo source: The paper "End-to-end "virtual Lidar" for image-based 3D target object detection)

According to Maims Consulting, Tesla’s founder Elon Musk’s attitude against the use of LiDAR for autonomous vehicles is well known. He believes that relying on lidar is like walking on crutches. There is no plan to use lidar in Tesla. However, this view is still controversial, and its future performance remains to be judged by history. High tech

    Tesla now respects a technology called "virtual Lidar" or "pseudo-LiDAR" (pseudo-LiDAR). This technique involves building tools to collect camera images (three-dimensional or two-dimensional) and calculate the distance information for each pixel in the image. Lidar determines the distance to each pixel by calculating the time required for the light pulse to reach the target object and return. High tech

        At the Scaled Machine Learning conference in February this year, Tesla shared how to obtain the accuracy of traditional lidar with just a few cameras. The 3D rendering of the visual information received by the visual sensor (camera), and the use of artificial intelligence (AI) software to match lane line, traffic, pedestrian and other information, and finally make the vehicle make a decision. At present, Tesla has taken computer vision to an unprecedented level, not only capable of analyzing images, but also analyzing individual pixels in images. Over time, this may be able to replicate most of the functions of traditional lidar, which means that all software solutions that have been developed for lidar processing can be used, or Tesla’s technical level in 3D object detection will be further improved. . High tech

On the other hand, humans can estimate the distance through the brain. We know how big the target objects are and how they move, so we know how far away they are. At the same time, humans also have some "talents", such as the stereoscopic properties of the human eye, but only for medium distances. Another talent is "motion parallax". When the line of sight moves laterally in the field of view, the movement direction and speed of the object are different, which also provides us with information to judge the distance. High tech

     In this way, the human brain is fully qualified for this task. In fact, the distance can be estimated by closing one eye while driving. At present, people are trying to build machine learning technology through neural networks to judge the distance from images. This is the "virtual laser radar" technology. High tech

     The earliest concept of "virtual lidar" was traced back to a technical paper from Cornell University in 2018, which proposed a new method to shorten the performance gap between pure vision technology architecture and lidar. High tech

The paper changes the 3D information presentation form of the stereo camera target detection system, converts the image-based stereo vision data into a 3D point cloud similar to that produced by lidar, and switches to the final view format through data conversion. High tech

In the two years since then, Cornell University and others have successively published papers on visual depth estimation, target recognition, and 3D Packing based on this method. Some researchers have found that after using its new method, the performance of the camera in the detection of target objects is close to that of lidar. Analyzing the image captured by the camera with a bird's-eye view can increase the accuracy of target detection by a factor of two, making the stereo camera a viable alternative to Lidar at a much lower cost. Tonggaotong High-tech Technology

    Training "virtual lidar" is not very difficult, but usually, training needs to provide enough labeled images. A test car may be equipped with expensive lidar, so you can drive around to obtain training data combined with lidar "ground real" distance data. The real distance calculated by the lidar displays a large number of images in the neural network, so that it can calculate the distance by itself. This technique is a variant of "unsupervised learning" because it does not require manual labeling of the data in advance, which is much lower than the cost of supervised learning. Therefore, if the neural network develops well, it should be like this. At the same time, the simulator data can also be trained to improve the model. High tech

Another effective training method relies on objects in the real world that change distance in a predictable way. For example, when you see an object moving along a path allowed by physics, your estimate is likely to be correct. However, if you see an object moving in a certain space in an impossible motion, you know that it is wrong. High tech

The problem with neural networks is that they tend to look at single-frame images rather than moving images like humans do. In fact, looking at still images alone, humans will have more errors. I believe that machine learning technology will overcome this over time. The problem is that we must achieve high reliability to stand out. At the same time, it also needs to have the ability to process things that have never been seen, and this ability will be a huge challenge for neural network technology. For example, if you are driving and there is an object on the road ahead, you need to know how far it is from you as quickly and accurately as possible. If the object is a car, you know the size of the car, so you can quickly determine its distance. Similarly, if a car rolls over, the training database may never have encountered such an event. For a random object, you want to know whether it is a large object in the distance or a small object in the vicinity? The only way is to look at its relationship with the geometry of the road. This situation is more complicated. High tech

If the above problems can be solved, then they must have a tool that can capture camera images and can also generate a "3D point cloud" generated by lidar, and because the camera is cheap, its cost is much lower. At the same time, the tool is able to do this at long distances. Generally, the detection distance of the lidar is only about 120 m, and the better one is about 240 m. As we all know, the human eye can see the distance up to 1600 m. High tech

Ironically, developers dedicated to lidar technology have built systems that rely on these point clouds and spent a lot of time perfecting them. If a "virtual lidar" system can be used to generate high-quality point clouds, people can use it immediately. Those who have always wanted to use "virtual lidar" technology have no similar experience in using this form of data. Instead, they plan to combine other elements of the visual system (divide the image into different objects and classify them) with distance estimation. But for now, they may not have the ability to achieve the breakthrough they expect. High tech

On the other hand, companies that use lidar will say, “It’s great, you can finally replace expensive lidar with cheaper technology.” However, if you are a fan of lidar manufacturers (such as Ford, Cruise, Waymo, and Aurora) , They may feel a lot of money wasted. High tech

The obvious point is that you need to know the distance of all objects on the road, and you must complete the distance estimation correctly and quickly. The news has repeatedly reported that Tesla's self-driving cars crashed into trucks, guardrails, and stalled vehicles in the front lane, because these vehicles were blocked by a car that suddenly drove away. When an obstacle on the road is suddenly sensed by a sensor on the car, you need to know how far it is from you, and the data is highly reliable, so you can initiate emergency braking. Lidar almost works this way, but computer vision does not. "Virtual Lidar" technology is the key to solving this problem. But at present, most companies plan to use lidar to solve this problem, they know that lidar is effective, and they want to see it become cheaper. High tech

Of course, if Tesla can solve this problem internally, it will not be shared with others (although public demonstrations may cause other companies to develop the same technology). Tesla's perception research and development team is also trying to develop a tool to match distance estimation with image classification, rather than generating lidar-style point clouds. This technology is not a "virtual lidar", but if the reliability is high, it will be equally effective. High tech