Self Driving Cars are At A Transition Point
Cruise leaves the game as Waymo and Tesla ramp up
GM Cruise is no longer building a robotaxi, as of this past week. This is a huge disappointment (although not necessarily a surprise), as they were long one of the apparent leaders in the race to build a robotaxi business, and their cars were at one point a pretty common sight around San Francisco.
Not to say some, like George Hotz of Comma, haven’t predicted this outcome. I’ll be honest, I am not one of those: I was always pretty optimistic about Cruise, and thought their aggressive deployment strategy would give them the edge.
This unfortunately has been coming for a while. GM Cruise canceled its inspiring Origin robotaxi earlier this year, and had to ground operations last year. This came on the heels of a stop to all self driving operations late in 2023, after a particularly terrible incident involving a pedestrian being dragged under a Cruise robotaxi and receiving significant injuries.
The Self-Driving Future Is Here in SF
Even as Cruise pulls out of the robotaxi race, we see Waymo pulling ahead on pure self driving. At this point, Waymo averages at least 17,000 miles per critical intervention, with planned rollouts in cities like Tokyo, and offering an incredible experience in cities like SF.
Let’s take a look at what the field looks like.
What does the self-driving landscape look like?
There are a few key players in self-driving right now:
As stated above, Waymo is the leader in reliable on-road autonomy, with 17,000 miles-per-intervention.
On the other hand, Tesla is clearly the leader in scaling production, with millions of semi-autonomous cars using Tesla Autopilot — the big question is how far are they from Waymo levels of reliability.
Zoox remains in the game, with a smaller fleet but deployments in Las Vegas and a ramp-up in San Francisco.
Wayve is the wildcard; a new(-er) AI-first company with a distinctive approach and some really interesting ideas representing impressive technological advances.
There are some other minor players - Comma has its OpenPilot; companies like Nuro and Stack AV are still moving along. Aurora has focused on self-driving trucking. Overland has its own unique take, with self-driving for offroad vehicles.
And there are the Chinese players like Baidu. China’s self-driving scene, like its humanoid robotics and AI scenes, seems ruthlessly competitive and quite impressive, with car rides being available for as little as 50 cents in Wuhan. Companies like AutoX can also run taxis without safety drivers in major cities like Shenzhen.
What does your robotaxi look like?
We’ve seen a couple distinctive visions: the “personal” model, which is what Tesla is pitching above and what Waymo first pitched when it was the Google Self-Driving Car project.
Others have pitched coach-style seating, which takes more advantage of the fully-autonomous nature of the self-driving car and has some other advantages as well (such as four-way symmetry, which could potentially keep repair costs lower).
Fans of the Tesla and Google approach would argue that most trips on ride-hailing apps like Uber are single-occupancy, so why build a bigger, heavier, and more expensive car in the first place?
How this works out economically will depend on whe
How Do These Systems Work?
There are two main sets of approaches.
The DARPA Teams
I’m calling a set of companies “the DARPA teams” after the DARPA Urban Challenge in 2007, which launched the careers of many of the founders, engineers, and scientists involved in this set of companies. What I really mean is “pre-deep-learning robotics” companies — although of course they all have incredibly strong deep learning teams.
This pool includes Zoox, Waymo, and most of the defunct self-driving companies like Argo AI. These companies have a heritage that dates back before deep learning -- in Waymo’s case, long before it. These companies make liberal use of lidar and cameras, and have a classic hierarchical robotics stack:
Isolated detection and tracking teams, which are responsible for producing information for decision making processes
Multi-level motion planning, with route planning, street-level motion planning, and controls teams
The modularity of this approach makes it easy to analyze. You can test each individual component, train specific models, do targetted data collection.
So, from a “software engineering management” perspective, the DARPA approach is really appealing. The problem is that it’s expensive, requires a huge headcount, and is often slow to iterate.
What if planning needs some new feature exposed by detection? Suddenly you have cross-team dialogues and code reviews and extra procedures. Your new feature might add to
And that’s not to mention that each level of indirection between motion-level decision making and perception adds to potential errors, which means if you do this, you have to do it really well and study it carefully (which Waymo does, but arguably Uber, Cruise and others who tried this approach did not).
The End To End Modellers
Tesla: As always, Tesla is kind of its own thing. It used to be a fairly traditional stack by all appearances, but based on cameras only; in the last year has switched to an “end to end” approach which has dramatically superior capabilities to the old one. They have massive amounts of driving data to leverage.
There’s a lot of potential here, by removing all of the system complexity inherent in the DARPA approach. But of course, this makes the relationship between training the end-to-end model and deploying it much less direct. Many people report occasional regressions in Tesla’s self driving. While these kinks will be ironed out, this is an inevitable result of training and deploying large end-to-end models like this (we occasionally see similar things with ChatGPT!).
Wayve: based on self-supervised learning and world models. They seem to be tracking directly for the end-to-end step, leveraging their expertise in things like world models, as well as partnerships, to make up for the massive data gap between them and Tesla.
So When Will I Have My Self Driving Car?
Currently, in my opinion, we have mostly a two way race, and that’s because Waymo and Tesla have done a fantastic job mitigating the risk of developing this technology. Waymo has taken a very cautious approach, expanding first in easy environments in Phoenix, using safety drivers, and more recently writing a recipe for expanding to new cities.
Tesla, on the other hand, has a massive pool of devoted drivers and built-in manufacturing capacity. Tesla can also test Autopilot in a wider variety of environments, since they always have a safety driver (for now): the owner of the vehicle.
It looks like, for now, Waymo will keep adding cities, one by one. The question is if they can scale. While some claim Waymo has positive unit economics, I’m personally a bit skeptical. I am optimistic, though, that their recent strategic partnership with Hyundai will result in mass-produced robotaxis in the not-too-distant future.
My big question with Waymo is if they can expand to cities with less-pleasant weather. This is why thier plans for Tokyo are such a big deal: Tokyo’s weather is pretty similar to the US east cost, with cold, wind, and occasional snow. If Waymo can conquer that, it unlocks some of the most lucrative taxi markets in the world.
The big question with Tesla is: when can they take the driver out of the car? Autopilot is incredible in that it works pretty well basically everywhere, but pretty well isn’t good enough to remove the steering wheel. The FSD Community Tracker reports an amazing 66% of drives with no interventions, something like an average of 43 miles per disengagement — which is, to be clear, an incredible feature for a consumer car, but pretty far from Waymo’s 17,000.
So which happens first: will Waymo and its partners mass produce and get their cars out to more major cities, or will Tesla roll out a robotaxi? To me, it seems the likely answer is “a bit of both:” lots of people will drive their Tesla around during their day-to-day lives, but take a Waymo every time they, say, fly to San Francisco.
To bring us back to the beginning, though: the global transportation market is something like $18 trillion. This will be increasingly autonomous throughout the next decade, and increasingly it seems clear those autonomous capabilities will be captured by only a couple companies with the long-term vision to follow through (Waymo and Tesla, yes, but potentially also Wayve, Zoox, Baidu, etc.). It seems insane to me to bow out of the race at this late a date.
Thanks for sharing! I think the problem is similar to that of delivery robots, the business model is capped by the hourly rate that a driver costs, and many aspects of a real driver are difficult to replace like keeping the car clean and sanitary, helping with luggage, and - most importantly - providing accountability. The latter will need to be absorbed by the companies themselves. There is a use case where autonomous taxis can be offered when and where people don't want to work, but these are really economical edge cases.
"a pedestrian being dragged under a Cruise robotaxi and receiving significant injuries." - Well that is a scary thought if the sensors on the car don't pick up the disturbance and it just keeps going and going.
The concept is a tough ask, when a 1% error rate is extremely impressive from a technological standpoint but horrible from a safety standpoint.