Categories
AI

The problem with self-driving cars

We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 – 28. Join AI and data leaders for insightful talks and exciting networking opportunities. Register today!


Aka: Why this burning money pit has failed to produce meaningful results for decades.

The future is here, and it looks nothing like we expected. As we approach the 10-year anniversary of Alexnet, we have to critically examine the successes and failures of machine learning.

We are looking out from a higher plateau.

We have achieved things in computer vision, natural language processing and speech recognition that would have been unthinkable just a few years ago. By all accounts, the accuracy of our AI systems exceeds the wildest imaginations of yesteryear. 

And yet, it’s not enough. 

We were wrong about the future. Every prediction about self-driving cars has been wrong. We are not living in a future of autonomous cyborgs, and something else has come into focus. 

Augmentation over automation.

Humans crave control. It is one of our deepest, most instinctual desires. There is no world where we give it up. One of the biggest misunderstandings of the AI community today is that people become comfortable with automation over time. As the reliability of automated solutions is proven, the microwave background comfort of society steadily rises.

This is false.

The history of technology is not the history of automation. It is the history of control and abstraction. We are tool-builders, so uncomfortable with experiences beyond our control that for thousands of years we developed entire civilizations and mythos around the movement of the heavens. So it is with all technology.

And so it is with AI.

Since the early days, the problem with self-driving cars has been obvious: there’s no control. When we look at the successful implementations of self-driving cars — now several years old — we see lane assist and parallel parking. We see situations and use cases where the control pane between human and machine is obvious. In all other situations, where the goal has been the pursuit of mythical level 5 autonomy, self-driving cars have failed miserably.

Technology is not the bottleneck.

In 1925 we had a radio-controlled car navigating the streets of New York City through a busy traffic jam without a driver behind the wheel. At the 1939 World’s Fair, Norman Geddes’ Futurama exhibit outlined a plausible smart highway system that would effectively use magnetized spikes — like electromagnetic fiducials — embedded in the road to guide cars. He predicted that autonomous cars would be the dominant form of transportation by the 1960s.

Of course, he was wrong too.

Not about the technology though. No, “smart highways” have been tremendously successful and straightforward where they’ve been implemented. Even without additional infrastructure, we’ve got self-driving cars today that are more than capable of driving as safely as humans. Yet, even with more than $80 billion flowing into the field from 2014 to 2017, we have no self-driving cars. For reference, the $108 billion the U.S. federal government committed to public transit over a 5-year period was the largest investment the country has ever made in public transportation.

The difference of course, is that I can actually ride a train.

The problem, fundamentally, is that nobody has bothered to think about the new control panes that we’re trying to enable. The question was never about automating driving. That’s a myopic, closed-minded way of thinking. The question is about how to transform the transit experience.

Cars suck.

They’re big, loud, smelly and basically the most inefficient form of transportation someone could imagine. They’re the most expensive thing a person owns after their home, but they don’t create value. It’s not an asset that anybody wants to own, it’s an asset that people have to own. It’s a regressive tax that destroys the planet and subsidizes the highways that blight our cities. It’s an expensive, dangerous hunk of metal that sits unused in an expensive garage nearly 100% of the time.

Cars suck.

And making them self-driving solves approximately none of these problems. That’s the problem. When we spend too much time focusing on the quasi-mythical state of full automation, we ignore the impactful problems that sit in front of us. Uber was successful because you could call a car with the press of a button. Leases are successful, despite the cost, because it’s a different control pane for the car. These are new transit experiences.

So, where’s the actual opportunity?

I think that companies like Zoox have an interesting and compelling thesis. By focusing on the rider experience, and critically by designing a highly novel interface for teleguidance, I think they have a real shot of delivering something useful out of the self-driving car frenzy. I think it’s important to realize, though, that their teleguidance system is not some temporary bridge to get from here to there. The teleguidance system and its supporting architecture is arguably a more defensible breakthrough for them than any algorithmic advantage. That, combined with a model that eliminates ownership delivers a compelling vision. Of … ya know … a bus.

Don’t be distracted.

I haven’t used Zoox’s teleguidance system. I don’t know for certain that it is more efficient than driving, but at least they’re pointed in the right direction. We have to stop thinking about self-driving cars as fully autonomous. When level 5 autonomy is always right around the corner, there’s no need to think about all the messy intermediate states. The truth is that those messy intermediate states are the whole point.

This is the crux of the problem with self-driving cars.

If you’re an investor looking for the first company that’s going to “solve” self-driving cars, you’re barking up the wrong tree. The winner is the company that can actually deliver improved unit economics on the operation of a vehicle. Until we solve that problem, all of the closed track demos and all of the vanity metrics in the world mean nothing. We’re dreaming about the end of a race when we haven’t even figured out how to take the first step.

And the barrier isn’t machine learning.

It’s user experience. 

Slater Victoroff is founder and CTO of Indico Data.

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.

If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.

You might even consider contributing an article of your own!

Read More From DataDecisionMakers

Repost: Original Source and Author Link

Categories
Computing

This Is What the Self-Driving Apple Car May Look Like

Thanks to several 3D concept renders, we now know what the future self-driving Apple Car might look like.

Vanarama, a British car-leasing company, took inspiration from other Apple products, as well as Apple patents, in order to accurately picture the rumored Apple car.

Image source: MacRumors

Although Apple has revealed very little about the self-driving car it is allegedly working on, Vanarama claims to have based the renders on patents filed by Apple itself. The design of the car takes after current Apple products, such as iPhones and MacBooks, and incorporates their current style into the coupe SUV model pictured above. Aside from the images, Vanarama has shared a fully interactive 3D concept render that lets the user explore both the exterior and the interior of the car.

The design places a lot of emphasis on the comfort of use and is much different from the cars we see every day, although it’s not too different from Elon Musk’s Tesla. It especially resembles the Tesla Cybertruck, but with a sleeker design without the sharp edges of the Tesla. The interior of the car includes several parts inspired by Apple products, such as the door handles that resemble iPhone buttons.

Vanarama’s render showcases a pillarless design that makes the car easier to get in and out of when both sets of doors are open. The seats are fully rotatable, which allows for the front seats to be turned to face the back seat. The inclusion of coach doors comes from another Apple patent. The car offers ample space for passenger movement when boarding and for loading larger items into the vehicle.

The interior of the rumored Apple Car.
Image source: MacRumors

Apple had also filed a patent for an intelligent automated assistant for the car, and thus, Vanarama included Siri. The assistant is built into the steering column alongside the customizable dashboard and navigation screen.

The renders were first shared by MacRumors. Such a spacious design that promotes freedom and comfort is definitely plausible for the Apple car, as the company seems to have settled on the self-driving technology and might even remove both the steering wheel and the pedals.

The car would rely on hands-off driving and would likely include an iPad for the users to interact with. However, Apple is reportedly still considering adding a steering wheel that would allow the passengers to take over the car in the case of emergencies.

It’s hard to tell whether Vanarama’s design is close to what Apple is planning, but it’s certainly inspired by real Apple patents. According to Bloomberg, Apple may be considering a design that resembles the Lifestyle Vehicle from Canoo, where passengers sit along the sides of the car, facing each other.

Although an exciting prospect, the Apple Car is still a long way from being confirmed or released. The current goal is for the car to be launched sometime in 2025, but at this point in development, delays are very possible.

Editors’ Choice




Repost: Original Source and Author Link

Categories
Game

‘PUBG Mobile’ update adds a self-driving Tesla Model Y

PUBG Mobile probably isn’t the first game you’d expect to have an electric vehicle tie-in, but it’s here all the same. Krafton and Tencent Games have rolled out a 1.5 update for the phone-focused shooter that includes a raft of not-so-subtle plugs for Tesla and its cars. Most notably, you can find a Model Y on Erangel that can drive itself when you activate an autopilot mode on the highway —not that far off from the real Autopilot mode.

You’ll also find a Gigafactory on Erangel where you can build the Model Y by activating switches, and self-driving Semi trucks roam around the map dropping supply crates when you damage the vehicles. No, despite the imagery, you can’t drive a Cybertruck or Roadster (not yet, at least).

The additions are part of a larger “technological transformation” for Erangel that includes an overhaul of the buildings and new equipment, including an anti-gravity motorcycle.

As is often the case, you shouldn’t expect these updates in regular PUBG — the battle royale brawler for consoles and PCs has a more realistic atmosphere. The PUBG Mobile update is really a not-so-subtle way for Tesla to advertise its EVs in countries where it doesn’t already have strong word-of-mouth working in its favor.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.

Repost: Original Source and Author Link

Categories
Tech News

Self-driving cars don’t need LiDAR

What is the technology stack you need to create fully autonomous vehicles? Companies and researchers are divided on the answer to that question. Approaches to autonomous driving range from just cameras and computer vision to a combination of computer vision and advanced sensors.

Tesla has been a vocal champion for the pure vision-based approach to autonomous driving, and in this year’s Conference on Computer Vision and Pattern Recognition (CVPR), its chief AI scientist Andrej Karpathy explained why.

Speaking at CVPR 2021 Workshop on Autonomous Driving, Karpathy, who has been leading Tesla’s self-driving efforts in the past years, detailed how the company is developing deep learning systems that only need video input to make sense of the car’s surroundings. He also explained why Tesla is in the best position to make vision-based self-driving cars a reality.

A general computer vision system

Deep neural networks are one of the main components of the self-driving technology stack. Neural networks analyze on-car camera feeds for roads, signs, cars, obstacles, and people.

But deep learning can also make mistakes in detecting objects in images. This is why most self-driving car companies, including Alphabet subsidiary Waymo, use lidars, a device that creates 3D maps of the car’s surrounding by emitting laser beams in all directions. Lidars provided added information that can fill the gaps of the neural networks.

However, adding lidars to the self-driving stack comes with its own complications. “You have to pre-map the environment with the lidar, and then you have to create a high-definition map, and you have to insert all the lanes and how they connect and all the traffic lights,” Karpathy said. “And at test time, you are simply localizing to that map to drive around.”

It is extremely difficult to create a precise mapping of every location the self-driving car will be traveling. “It’s unscalable to collect, build, and maintain these high-definition lidar maps,” Karpathy said. “It would be extremely difficult to keep this infrastructure up to date.”

Tesla does not use lidars and high-definition maps in its self-driving stack. “Everything that happens, happens for the first time, in the car, based on the videos from the eight cameras that surround the car,” Karpathy said.

The self-driving technology must figure out where the lanes are, where the traffic lights are, what is their status, and which ones are relevant to the vehicle. And it must do all of this without having any predefined information about the roads it is navigating.

Karpathy acknowledged that vision-based autonomous driving is technically more difficult because it requires neural networks that function incredibly well based on the video feeds only. “But once you actually get it to work, it’s a general vision system, and can principally be deployed anywhere on earth,” he said.

With the general vision system, you will no longer need any complementary gear on your car. And Tesla is already moving in this direction, Karpathy says. Previously, the company’s cars used a combination of radar and cameras for self-driving. But it has recently started shipping cars without radars.

“We deleted the radar and are driving on vision alone in these cars,” Karpathy said, adding that the reason is that Tesla’s deep learning system has reached the point where it is a hundred times better than the radar, and now the radar is starting to hold things back and is “starting to contribute noise.”

Supervised learning

The main argument against the pure computer vision approach is that there is uncertainty on whether neural networks can do range-finding and depth estimation without help from lidar depth maps.

“Obviously humans drive around with vision, so our neural net is able to process visual input to understand the depth and velocity of objects around us,” Karpathy said. “But the big question is can the synthetic neural networks do the same. And I think the answer to us internally, in the last few months that we’ve worked on this, is an unequivocal yes.”

Tesla’s engineers wanted to create a deep learning system that could perform object detection along with depth, velocity, and acceleration. They decided to treat the challenge as a supervised learning problem, in which a neural network learns to detect objects and their associated properties after training on annotated data.

To train their deep learning architecture, the Tesla team needed a massive dataset of millions of videos, carefully annotated with the objects they contain and their properties. Creating datasets for self-driving cars is especially tricky, and the engineers must make sure to include a diverse set of road settings and edge cases that don’t happen very often.

“When you have a large, clean, diverse datasets, and you train a large neural network on it, what I’ve seen in practice is… success is guaranteed,” Karpathy said.

Auto-labeled dataset

With millions of camera-equipped cars sold across the world, Tesla is in a great position to collect the data required to train the car vision deep learning model. The Tesla self-driving team accumulated 1.5 petabytes of data consisting of one million 10-second videos and 6 billion objects annotated with bounding boxes, depth, and velocity.

But labeling such a dataset is a great challenge. One approach is to have it annotated manually through data-labeling companies or online platforms such as Amazon Turk. But this would require a massive manual effort, could cost a fortune, and become a very slow process.

Instead, the Tesla team used an auto-labeling technique that involves a combination of neural networks, radar data, and human reviews. Since the dataset is being annotated offline, the neural networks can run the videos back in forth, compare their predictions with the ground truth, and adjust their parameters. This contrasts with test-time inference, where everything happens in real-time and the deep learning models can’t make recourse.

Offline labeling also enabled the engineers to apply very powerful and compute-intensive object detection networks that can’t be deployed on cars and used in real-time, low-latency applications. And they used radar sensor data to further verify the neural network’s inferences. All of this improved the precision of the labeling network.

“If you’re offline, you have the benefit of hindsight, so you can do a much better job of calmly fusing [different sensor data],” Karpathy said. “And in addition, you can involve humans, and they can do cleaning, verification, editing, and so on.”

According to videos Karpathy showed at CVPR, the object detection network remains consistent through debris, dust, and snow clouds.

Karpathy did not say how much human effort was required to make the final corrections to the auto-labeling system. But human cognition played a key role in steering the auto-labeling system in the right direction.

While developing the dataset, the Tesla team found more than 200 triggers that indicated the object detection needed adjustments. These included problems such as inconsistency between detection results in different cameras or between the camera and the radar. They also identified scenarios that might need special care such as tunnel entry and exit and cars with objects on top.

It took four months to develop and master all these triggers. As the labeling network became better, it was deployed in “shadow mode,” which means it is installed in consumer vehicles and run silently without issuing commands to the car. The network’s output is compared to that of the legacy network, the radar, and the driver’s behavior.

The Tesla team went through seven iterations of data engineering. They started with an initial dataset on which they trained their neural network. They then deployed the deep learning in shadow mode on real cars and used the triggers to detect inconsistencies, errors, and special scenarios. The errors were then revised, corrected, and if necessary, new data was added to the dataset.

“We spin this loop over and over again until the network becomes incredibly good,” Karpathy said.

So, the architecture can better be described as a semi-auto labeling system with an ingenious division of labor, in which the neural networks do the repetitive work and humans take care of the high-level cognitive issues and corner cases.

Interestingly, when one of the attendees asked Karpathy whether the generation of the triggers could be automated, he said, “[Automating the trigger] is a very tricky scenario, because you can have general triggers, but they will not correctly represent the error modes. It would be very hard to, for example, automatically have a trigger that triggers for entering and exiting tunnels. That’s something semantic that you as a person have to intuit [emphasis mine] that this is a challenge… It’s not clear how that would work.”

Hierarchical deep learning architecture

Tesla’s self-driving team needed a very efficient and well-designed neural network to make the most out of the high-quality dataset they had gathered.

The company created a hierarchical deep learning architecture composed of different neural networks that process information and feed their output to the next set of networks.

The deep learning model uses convolutional neural networks to extract features from the videos of eight cameras installed around the car and fuses them together using transformer networks. It then fuses them across time, which is important for tasks such as trajectory-prediction and to smooth out inference inconsistencies.

The spatial and temporal features are then fed into a branching structure of neural networks that Karpathy described as heads, trunks, and terminals.

“The reason you want this branching structure is because there’s a huge amount of outputs that you’re interested in, and you can’t afford to have a single neural network for every one of the outputs,” Karpathy said.

The hierarchical structure makes it possible to reuse components for different tasks and enable feature-sharing between the different inference pathways.

Another benefit of the modular architecture of the network is the possibility of distributed development. Tesla is currently employing a large team of machine learning engineers working on the self-driving neural network. Each of them works on a small component of the network and they plug in their results into the larger network.

“We have a team of roughly 20 people who are training neural networks full time. They’re all cooperating on a single neural network,” Karpathy said.

Vertical integration

In his presentation at CVPR, Karpathy shared some details about the supercomputer Tesla is using to train and finetune its deep learning models.

The compute cluster is composed of 80 nodes, each containing eight Nvidia A100 GPUs with 80 gigabytes of video memory, amounting to 5,760 GPUs and more than 450 terabytes of VRAM. The supercomputer also has 10 petabytes of NVME superfast storage and 640 tbps networking capacity to connect all the nodes and allow efficient distributed training of the neural networks.

Tesla also owns and builds the AI chips installed inside its cars. “These chips are specifically designed for the neural networks we want to run for [full self-driving] applications,” Karpathy said.

Tesla’s big advantage is its vertical integration. Tesla owns the entire self-driving car stack. It manufactures the car and the hardware for self-driving capabilities. It is in a unique position to collect a wide variety of telemetry and video data from the millions of cars it has sold. It also creates and trains its neural networks on its proprietary datasets, its special in-house compute clusters, and validates and finetunes the networks through shadow testing on its cars. And, of course, it has a very talented team of machine learning engineers, researchers, and hardware designers to put all the pieces together.

“You get to co-design and engineer at all the layers of that stack,” Karpathy said. “There’s no third party that is holding you back. You’re fully in charge of your own destiny, which I think is incredible.”

This vertical integration and repeating cycle of creating data, tuning machine learning models, and deploying them on many cars puts Tesla in a unique position to implement vision-only self-driving car capabilities. In his presentation, Karpathy showed several examples where the new neural network alone outmatched the legacy ML model that worked in combination with radar information.

And if the system continues to improve, as Karpathy says, Tesla might be on the track of making lidars obsolete. And I don’t see any other company being able to reproduce Tesla’s approach.

Open issues

But the question remains as to whether deep learning in its current state will be enough to overcome all the challenges of self-driving. Surely, object detection and velocity and range estimation play a big part in driving. But human vision also performs many other complex functions, which scientists call the “dark matter” of vision. Those are all important components in the conscious and subconscious analysis of visual input and navigation of different environments.

Deep learning models also struggle with making causal inference, which can be a huge barrier when the models face new situations they haven’t seen before. So, while Tesla has managed to create a very huge and diverse dataset, open roads are also very complex environments where new and unpredicted things can happen all the time.

The AI community is divided over whether you need to explicitly integrate causality and reasoning into deep neural networks or if you can overcome the causality barrier through “direct fit,” where a large and well-distributed dataset will be enough to reach general-purpose deep learning. Tesla’s vision-based self-driving team seems to favor the latter (though given their full control over the stack, they could always try new neural network architectures in the future). It will be interesting to how the technology fares against the test of time.

This article was originally published by Ben Dickson on TechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech, and what we need to look out for. You can read the original article here.



Repost: Original Source and Author Link

Categories
AI

The lessons we learn from self-driving will drive our robotics future

Where does your enterprise stand on the AI adoption curve? Take our AI survey to find out.


Robotics is entering an exponential growth phase. There are increasingly new and diverse applications for robots, both the inspiring and the mundane. Just within the context of the COVID-19 pandemic response, robots have been deployed in novel ways — disinfecting public spaces, handling infectious materials, and providing medical care to patients.

But the horizon for new robotics applications is ever expanding, and it is AV (autonomous vehicle) development that will further accelerate this growth. Why? Because the challenge that self-driving cars present is the same challenge that acts as a barrier for most other kinds of robots. The AV industry, with its concentration of talent, infrastructure, and capital, is primed to meet this challenge.

The autonomy challenge

Even as the use of robots has become more widespread, its applications have remained somewhat limited. For decades, one-armed giants performed highly scripted tasks and were built for a single purpose, like spot welding or adding threads to the end of a pipe. They were not flexible enough to perform a variety of tasks or respond well in unstructured environments. Even when deployed in less structured environments, like those used in surgical settings or even aerial drones, robots have functioned primarily as a remote-controlled extension of a human actor, with limited autonomy.

AVs, on the other hand, inherently require a great deal of autonomy; there is literally no human being behind the wheel, and the stakes are high. AVs need the ability to sense, plan, and act in highly dynamic, unstructured environments such as the chaotic streets of San Francisco. They need to respond to humans — other drivers, pedestrians, cyclists, that guy on a motorized skateboard — and make collaborative decisions with them.

Consider one of the common yet more challenging traffic scenarios that humans regularly encounter: a four-way stop. Despite the laws that govern how drivers should stop and proceed in their turn, the reality is that most of the time, people navigate these intersections via nonverbal communication with each other. They make eye contact, nod, wave each other on. Without the capacity to communicate using these cues, an AV must still decipher the intent of other drivers and communicate its own — for instance, creeping forward slowly to convey its intent to proceed through the intersection — all while obeying traffic laws and making safety-critical decisions. This choreography cannot be scripted in advance. AV decision-making must conform to human-like social expectations in real time based on the current situation and potential evolution of all the relevant actors in the scene, including itself, for some time into the future.

The crux of the challenge involves making decisions under uncertainty; that is, choosing actions based on often imperfect observations and incomplete knowledge of the world. Autonomous robots have to observe the current state of the world (imperfect observations), understand how this is likely to evolve (incomplete knowledge), and make decisions about the best course of action to pursue in every situation. This cognitive capability is also essential to interpersonal interactions because human communications presuppose an ability to understand the motivations of the participants and subjects of the discussion. As the complexity of human–machine interactions increases and automated systems become more intelligent, we strive to provide computers with comparable communicative and decision-making capabilities. This is what takes robots from machines that humans supervise to machines with which humans can collaborate.

Where human-robot collaboration can take us

As robotics has grown as an industry, costs have fallen, enabling adoption across a broad variety of contexts. In some cases, the technology is familiar but the application is novel. While drones aren’t new, companies deploying them to inspect power lines or to collect information for insurance claims is. Same for the one-armed giants now employed as hotel concierges or baristas instead of spot welders.

Commerce has benefited greatly from automation. Materials handling in particular has been ripe for automation via self-guided vehicles, largely because it’s such a dangerous sector for human workers. Robots equipped with lidar, cameras, and a bevy of other sensors — like those that enable AVs’ perception systems — can safely and quickly navigate loading docks and factory floors while avoiding collisions with workers. These robots, however, still rely on a fairly structured and predictable environment (markers on the ground help them navigate) and lack dynamic responsiveness. During the last few years, some have argued that injuries in some fulfillment centers have resulted from robots moving at a faster pace than the humans working alongside them.

Robotics in healthcare environments has become commonplace, too. Robot-assisted surgical systems like Intuitive’s da Vinci are used in 90% of prostatectomies instead of traditional laparoscopic tools. But robots are increasingly valuable not just in the operating room but throughout hospitals and nursing homes, especially in the context of the COVID-19 pandemic. Robots are helping caregivers lift patients and performing other tasks as well as providing social interaction to the elderly. Robotics have increasingly been used with children as well, not just as trendy tech toys but legitimate STEM educational tools. Research into the treatment of children with autism using emotive robots has gained traction in recent years.

AV development is key

With more players in the field and increasing adoption, the $100+ billion global robotics sector has been growing by leaps and bounds, and according to IDC is expected to triple by the end of 2021. Much of this can be attributed to driver-assistance technologies now common in new vehicles, especially those at the higher end of the market. Companies developing fully autonomous technology, however, are poised to push the robotics envelope in the automotive industry and beyond.

As AV companies meet the challenge of human-robot collaboration at the level required to bring self-driving vehicles to market, the horizon for leveraging these solutions for other robotics applications only expands. Like a chess grandmaster, an AV must consider multiple possible moves and countermoves both for itself and other traffic participants and then make safety-critical decisions in a noisy and rapidly changing environment. It needs to take into account context like traffic laws and local norms; driving in a city like Houston is not the same as navigating Hong Kong. And a successful AV has to communicate its goals and its intent to humans in a way that feels natural and intuitive.

Developing the kind of decision-making needed for AVs to succeed will unlock complex “critical thinking” for other robotic applications, allowing a greater degree of autonomy and human-robot collaboration in both new and familiar use cases. Physical agents that can autonomously generate engaging, life-like behavior will lead to safer and more responsive robots. The shift from humans supervising robots to collaborating with them is the way forward for both AVs and the sector at large.

Rashed Haq is Vice President of Robotics at Cruise.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link

Categories
AI

Tesla AI chief explains why self-driving cars don’t need lidar

Where does your enterprise stand on the AI adoption curve? Take our AI survey to find out.


What is the technology stack you need to create fully autonomous vehicles? Companies and researchers are divided on the answer to that question. Approaches to autonomous driving range from just cameras and computer vision to a combination of computer vision and advanced sensors.

Tesla has been a vocal champion for the pure vision-based approach to autonomous driving, and in this year’s Conference on Computer Vision and Pattern Recognition (CVPR), its chief AI scientist Andrej Karpathy explained why.

Speaking at CVPR 2021 Workshop on Autonomous Driving, Karpathy, who has been leading Tesla’s self-driving efforts in the past years, detailed how the company is developing deep learning systems that only need video input to make sense of the car’s surroundings. He also explained why Tesla is in the best position to make vision-based self-driving cars a reality.

A general computer vision system

Deep neural networks are one of the main components of the self-driving technology stack. Neural networks analyze on-car camera feeds for roads, signs, cars, obstacles, and people.

But deep learning can also make mistakes in detecting objects in images. This is why most self-driving car companies, including Alphabet subsidiary Waymo, use lidars, a device that creates 3D maps of the car’s surrounding by emitting laser beams in all directions. Lidars provided added information that can fill the gaps of the neural networks.

However, adding lidars to the self-driving stack comes with its own complications. “You have to pre-map the environment with the lidar, and then you have to create a high-definition map, and you have to insert all the lanes and how they connect and all the traffic lights,” Karpathy said. “And at test time, you are simply localizing to that map to drive around.”

It is extremely difficult to create a precise mapping of every location the self-driving car will be traveling. “It’s unscalable to collect, build, and maintain these high-definition lidar maps,” Karpathy said. “It would be extremely difficult to keep this infrastructure up to date.”

Tesla does not use lidars and high-definition maps in its self-driving stack. “Everything that happens, happens for the first time, in the car, based on the videos from the eight cameras that surround the car,” Karpathy said.

The self-driving technology must figure out where the lanes are, where the traffic lights are, what is their status, and which ones are relevant to the vehicle. And it must do all of this without having any predefined information about the roads it is navigating.

Karpathy acknowledged that vision-based autonomous driving is technically more difficult because it requires neural networks that function incredibly well based on the video feeds only. “But once you actually get it to work, it’s a general vision system, and can principally be deployed anywhere on earth,” he said.

With the general vision system, you will no longer need any complementary gear on your car. And Tesla is already moving in this direction, Karpathy says. Previously, the company’s cars used a combination of radar and cameras for self-driving. But it has recently started shipping cars without radars.

“We deleted the radar and are driving on vision alone in these cars,” Karpathy said, adding that the reason is that Tesla’s deep learning system has reached the point where it is a hundred times better than the radar, and now the radar is starting to hold things back and is “starting to contribute noise.”

Supervised learning

The main argument against the pure computer vision approach is that there is uncertainty on whether neural networks can do range-finding and depth estimation without help from lidar depth maps.

“Obviously humans drive around with vision, so our neural net is able to process visual input to understand the depth and velocity of objects around us,” Karpathy said. “But the big question is can the synthetic neural networks do the same. And I think the answer to us internally, in the last few months that we’ve worked on this, is an unequivocal yes.”

Tesla’s engineers wanted to create a deep learning system that could perform object detection along with depth, velocity, and acceleration. They decided to treat the challenge as a supervised learning problem, in which a neural network learns to detect objects and their associated properties after training on annotated data.

To train their deep learning architecture, the Tesla team needed a massive dataset of millions of videos, carefully annotated with the objects they contain and their properties. Creating datasets for self-driving cars is especially tricky, and the engineers must make sure to include a diverse set of road settings and edge cases that don’t happen very often.

“When you have a large, clean, diverse datasets, and you train a large neural network on it, what I’ve seen in practice is… success is guaranteed,” Karpathy said.

Auto-labeled dataset

With millions of camera-equipped cars sold across the world, Tesla is in a great position to collect the data required to train the car vision deep learning model. The Tesla self-driving team accumulated 1.5 petabytes of data consisting of one million 10-second videos and 6 billion objects annotated with bounding boxes, depth, and velocity.

But labeling such a dataset is a great challenge. One approach is to have it annotated manually through data-labeling companies or online platforms such as Amazon Turk. But this would require a massive manual effort, could cost a fortune, and become a very slow process.

Instead, the Tesla team used an auto-labeling technique that involves a combination of neural networks, radar data, and human reviews. Since the dataset is being annotated offline, the neural networks can run the videos back in forth, compare their predictions with the ground truth, and adjust their parameters. This contrasts with test-time inference, where everything happens in real-time and the deep learning models can’t make recourse.

Offline labeling also enabled the engineers to apply very powerful and compute-intensive object detection networks that can’t be deployed on cars and used in real-time, low-latency applications. And they used radar sensor data to further verify the neural network’s inferences. All of this improved the precision of the labeling network.

“If you’re offline, you have the benefit of hindsight, so you can do a much better job of calmly fusing [different sensor data],” Karpathy said. “And in addition, you can involve humans, and they can do cleaning, verification, editing, and so on.”

According to videos Karpathy showed at CVPR, the object detection network remains consistent through debris, dust, and snow clouds.

tesla object tracking auto-labeling

Above: Tesla’s neural networks can consistently detect objects in various visibility conditions.

Image Credit: Logitech

Karpathy did not say how much human effort was required to make the final corrections to the auto-labeling system. But human cognition played a key role in steering the auto-labeling system in the right direction.

While developing the dataset, the Tesla team found more than 200 triggers that indicated the object detection needed adjustments. These included problems such as inconsistency between detection results in different cameras or between the camera and the radar. They also identified scenarios that might need special care such as tunnel entry and exit and cars with objects on top.

It took four months to develop and master all these triggers. As the labeling network became better, it was deployed in “shadow mode,” which means it is installed in consumer vehicles and run silently without issuing commands to the car. The network’s output is compared to that of the legacy network, the radar, and the driver’s behavior.

The Tesla team went through seven iterations of data engineering. They started with an initial dataset on which they trained their neural network. They then deployed the deep learning in shadow mode on real cars and used the triggers to detect inconsistencies, errors, and special scenarios. The errors were then revised, corrected, and if necessary, new data was added to the dataset.

“We spin this loop over and over again until the network becomes incredibly good,” Karpathy said.

So, the architecture can better be described as a semi-auto labeling system with an ingenious division of labor, in which the neural networks do the repetitive work and humans take care of the high-level cognitive issues and corner cases.

Interestingly, when one of the attendees asked Karpathy whether the generation of the triggers could be automated, he said, “[Automating the trigger] is a very tricky scenario, because you can have general triggers, but they will not correctly represent the error modes. It would be very hard to, for example, automatically have a trigger that triggers for entering and exiting tunnels. That’s something semantic that you as a person have to intuit [emphasis mine] that this is a challenge… It’s not clear how that would work.”

Hierarchical deep learning architecture

Tesla neural network self-driving car

Tesla’s self-driving team needed a very efficient and well-designed neural network to make the most out of the high-quality dataset they had gathered.

The company created a hierarchical deep learning architecture composed of different neural networks that process information and feed their output to the next set of networks.

The deep learning model uses convolutional neural networks to extract features from the videos of eight cameras installed around the car and fuses them together using transformer networks. It then fuses them across time, which is important for tasks such as trajectory-prediction and to smooth out inference inconsistencies.

The spatial and temporal features are then fed into a branching structure of neural networks that Karpathy described as heads, trunks, and terminals.

“The reason you want this branching structure is because there’s a huge amount of outputs that you’re interested in, and you can’t afford to have a single neural network for every one of the outputs,” Karpathy said.

The hierarchical structure makes it possible to reuse components for different tasks and enable feature-sharing between the different inference pathways.

Another benefit of the modular architecture of the network is the possibility of distributed development. Tesla is currently employing a large team of machine learning engineers working on the self-driving neural network. Each of them works on a small component of the network and they plug in their results into the larger network.

“We have a team of roughly 20 people who are training neural networks full time. They’re all cooperating on a single neural network,” Karpathy said.

Vertical integration

In his presentation at CVPR, Karpathy shared some details about the supercomputer Tesla is using to train and finetune its deep learning models.

The compute cluster is composed of 80 nodes, each containing eight Nvidia A100 GPUs with 80 gigabytes of video memory, amounting to 5,760 GPUs and more than 450 terabytes of VRAM. The supercomputer also has 10 petabytes of NVME superfast storage and 640 tbps networking capacity to connect all the nodes and allow efficient distributed training of the neural networks.

Tesla also owns and builds the AI chips installed inside its cars. “These chips are specifically designed for the neural networks we want to run for [full self-driving] applications,” Karpathy said.

Tesla’s big advantage is its vertical integration. Tesla owns the entire self-driving car stack. It manufactures the car and the hardware for self-driving capabilities. It is in a unique position to collect a wide variety of telemetry and video data from the millions of cars it has sold. It also creates and trains its neural networks on its proprietary datasets, its special in-house compute clusters, and validates and finetunes the networks through shadow testing on its cars. And, of course, it has a very talented team of machine learning engineers, researchers, and hardware designers to put all the pieces together.

“You get to co-design and engineer at all the layers of that stack,” Karpathy said. “There’s no third party that is holding you back. You’re fully in charge of your own destiny, which I think is incredible.”

This vertical integration and repeating cycle of creating data, tuning machine learning models, and deploying them on many cars puts Tesla in a unique position to implement vision-only self-driving car capabilities. In his presentation, Karpathy showed several examples where the new neural network alone outmatched the legacy ML model that worked in combination with radar information.

And if the system continues to improve, as Karpathy says, Tesla might be on the track of making lidars obsolete. And I don’t see any other company being able to reproduce Tesla’s approach.

Open issues

But the question remains as to whether deep learning in its current state will be enough to overcome all the challenges of self-driving. Surely, object detection and velocity and range estimation play a big part in driving. But human vision also performs many other complex functions, which scientists call the “dark matter” of vision. Those are all important components in the conscious and subconscious analysis of visual input and navigation of different environments.

Deep learning models also struggle with making causal inference, which can be a huge barrier when the models face new situations they haven’t seen before. So, while Tesla has managed to create a very huge and diverse dataset, open roads are also very complex environments where new and unpredicted things can happen all the time.

The AI community is divided over whether you need to explicitly integrate causality and reasoning into deep neural networks or if you can overcome the causality barrier through “direct fit,” where a large and well-distributed dataset will be enough to reach general-purpose deep learning. Tesla’s vision-based self-driving team seems to favor the latter (though given their full control over the stack, they could always try new neural network architectures in the future). It will be interesting to how the technology fares against the test of time.

Ben Dickson is a software engineer and the founder of TechTalks, a blog that explores the ways technology is solving and creating problems.

This story originally appeared on Bdtechtalks.com. Copyright 2021

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member



Repost: Original Source and Author Link

Categories
AI

Regulators know teleoperation is key for self-driving vehicles to succeed

Elevate your enterprise data technology and strategy at Transform 2021.


The first article in this series highlighted one of the biggest and most revolutionary changes ever to come our way — autonomous vehicles (AVs). AV technology will mark the first time in history that transportation is possible without requiring any active human labor.

Autonomous Driving (AD) is made possible by the harnessing of cutting-edge sensors, advanced computing, and sophisticated algorithms. Nevertheless, while some claims are being made that AVs are ready for prime time, they are premature. There remain significant kinks in the technology. Hence, regulatory bodies are closely monitoring the development of such systems in order to ensure public safety.

41 U.S. states have approved development of self-driving cars

No fewer than 41 U.S. states have enacted legislation or issued executive orders regarding the development of self-driving cars, but the bottom line for activation is “not yet.” For now, there must be a human in the loop.

The missing link, of course, is teleoperation. Numerous countries and states are paving the way. The United Kingdom, Sweden, Japan, California, Michigan, and Texas are just some of the places where teleoperation is mandatory for any operation of autonomous vehicles for testing purposes or pilot programs. With trailblazers like these, surely the need, acceptance, and demand for teleoperation and its legislation is expected to spread.  This happened in a major way last month when Germany took the lead in this roster of teleoperation trailblazers.

Germany authorizes commercial deployment of autonomous vehicles (AVs); teleoperation required

On May 28, the German Bundesrat approved the new Level 4 AV law. Level 4 refers to levels 0-5 of autonomy in vehicles as defined by SAE International, a global standards developing organization for engineering professionals. Levels 4 and 5, being the highest two, are the only levels that do not require active participation from an in-cabin human driver. In essence, the German government passed a law stating that AVs can be commercially deployed on the roads of Germany. As part of this groundbreaking initiative, officials mandated that humans must still be able to intervene, from a distance. They legislated teleoperation into law.

Sometimes legislation is written in ambiguous terms, leading to confusion and abuse. Fortunately this is not the case with Germany, where the new law stipulates that:

  1. Manual remote driving is not allowed.
  2. Only indirect controls are allowed. This is when the autonomy system offers one of a menu of choices for the human to choose from, or where the human gives “high-level” commands.
  3. The Self Driving System (SDS)/Autonomous Driving System (ADS) is always in control of the driving task, and is always responsible for driving. Even after a command is given by a human operator, the SDS/ADS decides when and how to execute it safely.
  4. Teleoperation is intentionally called “Technical Supervision (Technische Aufsicht),” to make the role distinction clear.
  5. The SDS/ADS always decides when it needs support from the Control Center (CC) and initiates a teleoperation session. This only happens after the vehicle performs a minimal risk maneuver [MRM] and brings the vehicle to a safe stop with hazard lights on, aka a minimal risk condition [MRC]. There is one exception: A teleoperation session can be initiated from the CC in order to issue a safe stop command only in special circumstances, including a cybersecurity attack.
  6. There is no longer a requirement for each vehicle to be monitored 1-on-1 by a human. Humans can monitor on an as-needed basis.
  7. A continuous data link through cellular network(s) is mandatory.

Legislation for AV technology is beginning to catch up

Germany has always been a leader and an innovator in the automotive sector. This legislation demonstrates how the country continues to lead the pack. Surely other governments will take their cues from Germany and will begin to adopt very similar laws. Legislation has been lagging behind AV technology, but it’s starting to catch up.

Changes like these tell us a few things. First and foremost, it tells us that governments are really taking autonomy seriously and understand its importance. Furthermore, it tells us that this is all happening in real time. We are yet another step closer to self-driving cars. Finally, it tells us that the future will have to incorporate teleoperation. The importance of a human having the ability to intervene in critical situations is of great importance, and all companies dealing with autonomous vehicles will have to adopt this technology.

Amit Rosenzweig is founder and CEO of Ottopia, a teleoperation software company.

 

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link

Categories
AI

What Waabi’s launch means for the self-driving car industry

Elevate your enterprise data technology and strategy at Transform 2021.


It is not the best of times for self-driving car startups. The past year has seen large tech companies acquire startups that were running out of cash and ride-hailing companies shutter costly self-driving car projects with no prospect of becoming production-ready anytime soon.

Yet, in the midst of this downturn, Waabi, a Toronto-based self-driving car startup, has just come out of stealth with an insane amount of $83.5 million in a Series A funding round led by Khosla Ventures, with additional participation from Uber, 8VC, Radical Ventures, OMERS Ventures, BDC, and Aurora Innovation. The company’s financial backers also include Geoffrey Hinton, Fei-Fei Li, Peter Abbeel, and Sanja Fidler, artificial intelligence scientists with great influence in the academia and applied AI community.

What makes Waabi qualified for such support? According to the company’s press release, Waabi aims to solve the “scale” challenge of self-driving car research and “bring commercially viable self-driving technology to society.” Those are two key challenges of the self-driving car industry and are mentioned numerous times in the release.

What Waabi describes as its “next generation of self-driving technology” has yet to pass the test of time. But its execution plan provides hints at what directions the self-driving car industry could be headed.

Better machine learning algorithms and simulations

According to Waabi’s press release: “The traditional approach to engineering self-driving vehicles results in a software stack that does not take full advantage of the power of AI, and that requires complex and time-consuming manual tuning. This makes scaling costly and technically challenging, especially when it comes to solving for less frequent and more unpredictable driving scenarios.”

Leading self-driving car companies have driven their cars on real roads for millions of miles to train their deep learning models. Real-road training is costly both in terms of logistics and human resources. It is also fraught with legal challenges as the laws surrounding self-driving car tests vary in different jurisdictions. Yet despite all the training, self-driving car technology struggles to handle corner cases, rare situations that are not included in the training data. These mounting challenges speak to the limits of current self-driving car technology.

Here’s how Waabi claims to solve these challenges (emphasis mine): “The company’s breakthrough, AI-first approach, developed by a team of world leading technologists, leverages deep learning, probabilistic inference and complex optimization to create software that is end-to-end trainable, interpretable and capable of very complex reasoning. This, together with a revolutionary closed loop simulator that has an unprecedented level of fidelity, enables testing at scale of both common driving scenarios and safety-critical edge cases. This approach significantly reduces the need to drive testing miles in the real world and results in a safer, more affordable, solution.”

There’s a lot of jargon in there (a lot of which is probably marketing lingo) that needs to be clarified. I reached out to Waabi for more details and will update this post if I hear back from them.

By “AI-first approach,” I suppose they mean that they will put more emphasis on creating better machine learning models and less on complementary technology such as lidars, radars, and mapping data. The benefit of having a software-heavy stack is the very low costs of updating the technology. And there will be a lot of updating in the coming years as scientists continue to find ways to circumvent the limits of self-driving AI.

The combination of “deep learning, probabilistic reasoning, and complex optimization” is interesting, albeit not a breakthrough. Most deep learning systems use non-probabilistic inference. They provide an output, say a category or a predicted value, without giving the level of uncertainty on the result. Probabilistic deep learning, on the other hand, also provides the reliability of its inferences, which can be very useful in critical applications such as driving.

“End-to-end trainable” machine learning models require no manual-engineered features. This means once you have developed the architecture and determined the loss and optimization functions, all you need to do is provide the machine learning model with training examples. Most deep learning models are end-to-end trainable. Some of the more complicated architectures require a combination of hand-engineered features and knowledge along with trainable components.

Finally, “interpretability” and “reasoning” are two of the key challenges of deep learning. Deep neural networks are composed of millions and billions of parameters. This makes it hard to troubleshoot them when something goes wrong (or find problems before something bad happens), which can be a real challenge in critical scenarios such as driving cars. On the other hand, the lack of reasoning power and causal understanding makes it very difficult for deep learning models to handle situations they haven’t seen before.

According to TechCrunch’s coverage of Waabi’s launch, Raquel Urtasan, the company’s CEO, described the AI system the company uses as a “family of algorithms.”

“When combined, the developer can trace back the decision process of the AI system and incorporate prior knowledge so they don’t have to teach the AI system everything from scratch,” TechCrunch wrote.

self-driving car simulation carla

Above: Simulation is an important component of training deep learning models for self-driving cars. (credit: CARLA)

Image Credit: Frontier Developments

The closed-loop simulation environment is a replacement for sending real cars on real roads. In an interview with The Verge, Urtasan said that Waabi can “test the entire system” in simulation. “We can train an entire system to learn in simulation, and we can produce the simulations with an incredible level of fidelity, such that we can really correlate what happens in simulation with what is happening in the real world.”

I’m a bit on the fence on the simulation component. Most self-driving car companies are using simulations as part of the training regime of their deep learning models. But creating simulation environments that are exact replications of the real world is virtually impossible, which is why self-driving car companies continue to use heavy road testing.

Waymo has at least 20 billion miles of simulated driving to go with its 20 million miles of real-road testing, which is a record in the industry. And I’m not sure how a startup with $83.5 million in funding can outmatch the talent, data, compute, and financial resources of a self-driving company with more than a decade of history and the backing of Alphabet, one of the wealthiest companies in the world.

More hints of the system can be found in the work that Urtasan, who is also a professor in the Department of Computer Science at the University of Toronto, does in academic research. Urtasan’s name appears on many papers about autonomous driving. But one in particular, uploaded on the arXiv preprint server in January, is interesting.

Titled “MP3: A Unified Model to Map, Perceive, Predict and Plan,” the paper discusses an approach to self-driving that is very close to the description in Waabi’s launch press release.

MP3 self-driving neural networks probablistic deep learning

Above: MP3 is a deep learning model that uses probabilistic inference to create scenic representations and perform motion planning for self-driving cars.

The researchers describe MP3 as “an end-to-end approach to mapless driving that is interpretable, does not incur any information loss, and reasons about uncertainty in the intermediate representations.” In the paper researchers also discuss the use of “probabilistic spatial layers to model the static and dynamic parts of the environment.”

MP3 is end-to-end trainable and uses lidar input to create scene representations, predict future states, and plan trajectories. The machine learning model obviates the need for finely detailed mapping data that companies like Waymo use in their self-driving vehicles.

Raquel posted a video on her YouTube that provides a brief explanation of how MP3 works. It’s fascinating work, though many researchers will point out that it not so much of a breakthrough as a clever combination of existing techniques.

There’s also a sizeable gap between academic AI research and applied AI. It remains to be seen if MP3 or a variation of it is the model that Waabi is using and how it will perform in practical settings.

A more conservative approach to commercialization

Waabi’s first application will not be passenger cars that you can order with your Lyft or Uber app.

“The team will initially focus on deploying Waabi’s software in logistics, specifically long-haul trucking, an industry where self-driving technology stands to make the biggest and swiftest impact due to a chronic driver shortage and pervasive safety issues,” Waabi’s press release states.

What the release doesn’t mention, however, is that highway settings are an easier problem to solve because they are much more predictable than urban areas. This makes them less prone to edge cases (such as a pedestrian running in front of the car) and easier to simulate. Self-driving trucks can transport cargo between cities, while human drivers take care of delivery inside cities.

With Lyft and Uber failing to launch their own robo-taxi services, and with Waymo still away from turning One, its fully driverless ride-hailing service, into a scalable and profitable business, Waabi’s approach seems to be well thought.

With more complex applications still being beyond reach, we can expect self-driving technology to make inroads into more specialized settings such as trucking and industrial complexes and factories.

Waabi also doesn’t make any mention of a timeline in the press release. This also seems to reflect the failures of the self-driving car industry in the past few years. Top executives of automotive and self-driving car companies have constantly made bold statements and given deadlines about the delivery of fully driverless technology. None of those deadlines have been met.

Whether Waabi becomes independently successful or ends up joining the acquisition portfolio of one of the tech giants, its plan seems to be a reality check on the self-driving car industry. The industry needs companies that can develop and test new technologies without much fanfare, embrace change as they learn from their mistakes, make incremental improvements, and save their cash for a long race.

Ben Dickson is a software engineer and the founder of TechTalks. He writes about technology, business, and politics.

This story originally appeared on Bdtechtalks.com. Copyright 2021

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link

Categories
AI

The success of self-driving vehicles will depend on teleoperation

Elevate your enterprise data technology and strategy at Transform 2021.


There is a lot of chatter right now about self-driving vehicles, which is understandable, as we stand on the threshold of an entire new world of driving. It’s not just the dream of self-driven vehicles, but also of green cars and trucks that glide silently without emitting any climate- threatening pollutants.

The cutting edge electric vehicle (EV) and autonomous vehicle (AV) technologies are virtually symbiotic, with a great deal of collaboration between the two emerging sectors. Yet, EVs actually are way ahead of autonomous driving in terms of fulfilling their vision in the real world. And it’s not just because people are gun-shy when it comes to tooling around in a driverless car. An empty driver’s seat is perhaps all fine and well on a straight road with almost no traffic and no sudden surprises. But when it comes to real-life traffic, there’s still no substitute for a hands-on driver. A human behind the wheel can see and properly respond to unexpected problems; autonomous technology has a long way to go before it can match wits with a human in such situations.
For the time being, self-driving technology falls short of the vision. Among the stumbling blocks are imperfect sensors, lack of mathematical models that can successfully predict the behavior of every road user, the high cost of collecting enough data to train algorithms, and, of course, passenger inhibition.(When elevators were introduced in the 1850s, it took years for most people to overcome their fear of the new contraptions.)

Today’s self-driving vehicles operate on a complex marriage of camera, lidar, radar, GPS and direction sensors. Combined, these are expected to deliver an all-situation, all-weather answer empowering an AV to see all, anticipate everything, and guarantee safe delivery to one’s destination.

Only they don’t. And they can’t.

Poor visibility, temporary detours, and lane-changes are challenges the current technology cannot process to provide an all-weather, all-situation solution. On-board computers, algorithms, and sensors are just not good enough today to deliver instant, and possibly life-saving, reactions.

While AVs have learned to recognize a fixed traffic light and a moving pedestrian, they cannot yet adapt to all new situations, especially at high speeds and in complex urban environments.
It’s no wonder that government regulators are reluctant to clear AVs for prime time. There is just not enough data to insure confidence in an AV’s ability to prevent damage, injury, and death. Understandably, consumer confidence is also just not there yet.

These are just some of the reasons why teleoperation is necessary to make the era of AVs practical. Simply put, teleoperation is the technology that allows one to remotely monitor an AV, take control as needed, and solve problems quickly and remotely.

With teleoperation, a single controller positioned at a distance from, say, a fleet of robo-taxis can observe each vehicle in real time and override their autonomy as necessary or provide much-needed input. When the problem is solved, the AV continues on its autonomous way. In effect, the remote “driver” takes over or issues commands only when human intervention is needed. And he or she can monitor and handle a number of vehicles simultaneously. The teleoperator is also there to speak with passengers who might be concerned about why the AV is taking a few extra seconds to cross an intersection.

Problem solved? Not so fast.

Because not all teleoperation technology is created equal. Guiding a vehicle remotely requires an ability to transfer, with as close to no delay as possible, information between the vehicle and the teleoperation center. Continuous and reliable two-way data streaming, regardless of changing network conditions, is absolutely critical, and a big challenge of its own. And yet, neither 4G LTE nor WiFi are equipped to support such high bandwidth and low latency communication, especially from a vehicle in motion. It will be a while before 5G will be the universal standard, and even with that network upgrade the challenges will remain.

Another obstacle is one that’s measured in milliseconds. Even with the strongest data connection, there is still a split-second difference between what is happening on the road and what the teleoperator can see. This drastically affects their ability to react.

The human factor (or UX, for user experience) is also an issue. The way different teleoperators perceive driving environments varies, and is different from that of an in-vehicle operator. It is insufficient to merely receive a video feed and follow through with commands. Tools that help create situational awareness are necessary — for example, an overlay to employ sensor systems and translate their information into recommended decisions.

The final bogeyman is hackers, the devils who delight in throwing in a virtual spanner that can ruin one’s day. When a bank account is hacked, money is lost. When a teleoperation post is hacked, lives can be lost, as safety and ingenuity are compromised.

Clearly, all of these issues require solutions that depend on deep innovation in multiple technologies. Details on the teleoperation solution will be explored in succeeding articles. Stay tuned.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link

Categories
AI

In a year of major shifts, the self-driving car market is consolidating

Join Transform 2021 this July 12-16. Register for the AI event of the year.


News broke this week that Woven Planet, a Toyota subsidiary, will acquire Level 5, Lyft’s self-driving unit, for $550 million. The transaction, which is expected to close in Q3 2021, includes $200 million paid upfront and $350 million over a five-year period.

Toyota will gain full control of Lyft’s technology and its team of 300. Lyft will remain in the game as a partner to Toyota’s self-driving efforts, providing its ride-hailing service as a platform to commercialize the technology when it comes to fruition.

The Toyota-Lyft deal is significant because it comes on the back of a year of major shifts in the self-driving car industry. These changes suggest the autonomous vehicle market will be dominated by a few wealthy companies that can withstand huge costs and very late return on investment in a race that will last more than a few years.

The costs of self-driving car technology

Costs remain a huge barrier for all self-driving car projects. The main type of software powering self-driving cars is deep reinforcement learning, which is currently the most challenging and expensive branch of artificial intelligence. Training deep reinforcement learning models requires expensive compute resources. This is the same technology used in AI systems that have mastered complicated games such as Go, StarCraft 2, and Dota 2. Each of those projects cost millions of dollars in hardware resources alone.

However, in contrast to game-playing AI projects, which last between a few months to a few years, self-driving car projects take several years—and maybe above a decade—before they reach desirable results. Given the complexities and unpredictability of the real world, designing and testing the right deep learning architecture and reward, state, and action space for self-driving cars is very difficult and costly. And unlike games, the reinforcement learning models used in driverless cars need to gather their training experience and data from the real world, which is fraught with extra logistical, technical, and legal costs.

Some companies develop virtual environments to complement the training of their reinforcement learning models. But those environments come with their own development and computing costs and aren’t a full replacement for driving in the real world.

Equally costly is the talent needed to develop, test, and tune the reinforcement learning models used in driverless cars.

All of these expenses put a huge strain on the budgets of companies running self-driving car projects. According to reports, the sale of Level 5 will cut Lyft’s net annual operating costs by $100 million. This will be enough to make the company profitable. Uber, Lyft’s rival, also sold its driverless car unit, Advanced Technologies Group (ATG), in December because it was losing money.

So far, no company has been able to develop a profitable self-driving car program. Waymo, Alphabet’s self-driving subsidiary, has launched a fully driverless ride-hailing service in parts of Arizona. But it is still losing money on the project and is in the process of expanding the service to other cities in the U.S.

Driverless cars are not ready for primetime

Not long ago, it was generally believed that self-driving cars were a solved problem and it would only take a couple of years of development and training to get them ready for production. Several companies had hailed launching robo-taxi services by 2018, 2019, and 2020. A few carmakers promised to make full self-driving cars available to consumers.

But we’re in 2021, and it’s clear that the technology is still not ready. Our deep learning algorithms are not on par with the human vision system. That’s why many companies need to use complementary technologies such as lidars, radars, and other sensors. Added to that is precision mapping data that provide the car with exact details of what it should expect to see in its surroundings. But even with all these props, we haven’t reached self-driving technology that can run on any road, weather, and traffic condition.

The legal infrastructure for self-driving cars is also not ready. We still don’t know how to regulate roads shared by human- and AI-driven cars, how to determine culpability in accidents caused by self-driving cars, and many more legal and ethical challenges that arise from removing humans from behind steering wheels.

In many ways, the self-driving car industry is reminiscent of the early decades of AI: The technology always seems to be right around the corner. But the end goal seems to be receding as we continue to approach it.

The self-driving car market is consolidating

What does this all mean for companies that are running self-driving car projects? Many more years and billions of dollars’ worth of investment in developing a technology that doesn’t seem to get off the ground.

This will make it very difficult for companies that don’t have a highly profitable business model to engage in the market. And this includes ride-hailing services, which are under extra pressure due to the coronavirus pandemic. Startups that are living on VC money will also be hard-pressed to deliver on timelines that are shaky at best.

Lyft’s sale to Toyota is part of a growing trend of self-driving car projects and startups gravitating toward deep-pocketed automotive or tech giants.

Waymo will continue to operate and push forward for self-driving technology because its parent company has a long history of funding moonshot projects, most of which never reach profitability. Amazon acquired Zoox last year. Apple is considering creating its own electric self-driving car. And Microsoft is casting a wide net in the market, investing in several self-driving car projects at the same time.

Traditional carmakers are also becoming big players in the market. Argo AI is backed by Ford and Volkswagen, both of whom have a major stake in the future of self-driving cars. General Motors owns Cruise. Hyundai has poured $2 billion into a joint self-driving car venture with green tech startup Aptiv. And Aurora, the company that acquired Uber’s ATG, is developing partnerships with several automakers.

As the self-driving car industry shifts from hype to disillusionment, the market is slowly consolidating into a few very big players. Startups will be acquired, and we can probably expect one or more mergers between big tech and big automotive. This is going to be a race between those who can withstand the long haul.

This story originally appeared on Bdtechtalks.com. Copyright 2021

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link