Gary Zhou 
Self-driving cars are as fascinating as they are challenging to develop. The sensors, compute, actuation all need to continuously work together to understand the surrounding environment and respond in real-time. And when it all works as expected, we will save lives and change society for the better.
Back in 2013, Tesla, while still struggling, decided to take on this challenge. All in the midst of posting huge operating losses, struggling to manufacture cars and scale production. It’s worth considering:
What was Tesla’s strategy, and why?
How has it evolved over the years?
Fast forward to today and Tesla has a large fleet on the road with self-driving features. Not to mention, Tesla has one of the highest attach rate for self-driving features, the most diverse self-driving fleet by geography, and one of the most diverse self-driving fleets by vehicle type.
So, how does a struggling automaker achieve such impressive results in less than a decade?
It all boils down to two of Tesla’s biggest strengths, including Elon Musk’s first principles-based thinking and Tesla’s sheer execution prowess. Those and five core pillars to which Tesla’s success can be attributed.
1. One architecture — across all vehicles
Starting October 2016, all Tesla vehicles sold are equipped with the hardware necessary for full self-driving — sensors including camera, radar, GPS, ultrasonics, and the onboard computer.
This is counterintuitive to how the automotive industry works, where every last bit of margin is squeezed. Why equip vehicles with extra hardware that the customer specifically didn’t ask for? Turns out, there are quite a few benefits.
Opening the door to software revenues
This is a very common strategy in the tech world — supply the hardware for cheap and milk revenues by selling software. It’s why Amazon sold echo devices for under $50. But the benefits of this strategy in the automotive world are only magnified. As you can always buy another smart speaker, but your car stays with you much longer.
Say the full self-driving hardware costs Tesla around $1000 per car. ($600 for compute, $400 for sensors + wiring). At 400k vehicles per year, the total cost to Tesla is $400M. And the full self-driving package is priced at $7k. So the break-even point is at $400M/$7k ~ 57k vehicles or an attach rate of 57k/400k ~14%.
Per analysts, Tesla easily beats that rate today. And it seems things will only improve: the price of self-driving vehicles will only increase with time. It’s already up to $8k as of July 2020.
What’s more, as Tesla ships more vehicles, the cost of full self-driving hardware per vehicle will drop, too. With time, as Tesla’s software matures, more Tesla owners will purchase some variant of the full self-driving package from the Tesla app.
Easier to develop, debug and deploy software
Just ask any Software Engineer who had to flash upgrades across different hardware skews: a single architecture streamlines development and testing and makes it easier to roll out more upgrades, more frequently.
Compare this to an automaker with different brands, models, skews each with its own version of custom self-driving hardware and software. Simplicity is a key competitive advantage.
Streamlining operations
The story continues. Fewer distinct components implies streamlined supply chains that are both resilient and easier to manage. Buying more helps Tesla ride down the cost curve and negotiate better prices with suppliers.
Continuous improvement
There’s no concept of a model year when it comes to Tesla vehicles. Tesla does not, for instance, wait to bundle all the new features and lump it into next year’s Model S.
Instead, Tesla continuously improves its products and processes so that the customer always gets the latest and greatest of what’s possible. And this helps Tesla in building self-driving technology.
Say, for instance, Tesla needs to update the firmware on a certain vehicle part for it to better co-operate in building and/or testing self-driving software. Or perhaps, Tesla wants to update their vehicle’s sensor configuration with better sensors or upgrade the on-board computer. Such changes, once decided, are instantly rolled into production vehicles. This keeps Tesla’s vast fleet agile and its engineers always occupied with the latest and greatest datasets.
2. Tesla’s fleet
As of July 2020, Tesla probably has close to a million vehicles on the road in major parts of North America, Europe, and Asia. And every time an owner drives home and sits down to eat dinner, the vehicle is busy sending back mountains of data of all kinds back to Tesla. And Tesla has over time probably mastered the art of sorting through this data pile and making it immensely useful to not just this owner, but its entire fleet.
Clearly, there are more positives here than meets the eyes.
Data across vehicle models
Tesla has a diverse fleet of vehicles, all the way from a sedan to an SUV. And such a fleet helps Tesla capture different perspectives of the world. Because a camera mounted on the Model X, sees the world slightly differently than a corresponding camera on the Model 3.
This is useful for two reasons. First, augmenting data sets with different perspectives makes the neural network more robust. And that implies the network is more likely to accurately detect and classify objects in places and contexts it’s not seen before. Second, such a network will probably more easily transfer when used in a new perspective, i.e. the corresponding camera on the Tesla Cybertruck.
Real-world examples
Self-driving algorithms test and train the best with real-world examples. And Tesla certainly has one of the richest data sets.
In 2019, Waymo drove 20 million miles on public roads. Since equipping every vehicle sold with full self-driving hardware, Tesla has sold more than half a million vehicles. Say conservatively, each of them drove ~8k miles in 2019. This accumulates to 500k * 8k = 4 billion real-world miles.
Let’s say every mile Waymo drove was useful in both developing and testing self-driving algorithms. This implies that to be on par with Waymo, Tesla has to only make 0.5% (20M/4B) of its 4 billion miles useful. And Tesla is adding ~100k vehicles every quarter to this fleet. Which means they accumulate an additional 100k*(8k/4) = 200 million miles of driving data each quarter. The number of vehicles and miles driven, both, are only growing.
The counter-argument, of course, is that such a comparison is a gross oversimplification. After all, it’s hard to quantify the definition for a useful mile.
But what’s important to point out is that there’s an order of magnitude at play here. And my calculations are very conservative. Tesla already has 3 billion miles with AutoPilot.
Besides, there are other noteworthy advantages. By consistently driving more, Tesla is more likely to encounter corner cases and train its software to handle those. As a result, Tesla’s software matures faster.
Moreover, most of Waymo’s miles come from cities in the US. Whereas drivers have driven Teslas on all kinds of roads and terrains in different countries. So the day Tesla claims it has achieved its goal of full self-driving, it would have realized it on a far larger geographic footprint.
Low light presents a huge challenge to perception systems. Things get even more interesting when headlights are the only external source of light. Understanding the sensitivity and tolerance of neural networks to such conditions is important in making progress.
In the real world, people drive in all kinds of ways that lead to close encounters. Such cases are very useful to capture and solve for. Something you cannot easily get with a smaller fleet or in simulation. Lyft’s L5 self-driving division has a similar strategy. They tap into Lyft’s ride-sharing data to do just that.
Shadow mode
Tesla’s fleet is not just a massive data-gathering machine, it’s also the world’s largest testing platform for self-driving software.
Whenever Tesla has a new piece of software, they deploy it on their fleet in what’s called the shadow mode. In this mode, the new software will activate and execute, but it won’t actually be allowed to drive the vehicle. Instead, the computer simply notes down what the software would have done. And engineers are able to later access such information to determine if the software is ready to be deployed.
This is a huge advantage for engineers at Tesla. To see their code behave in real-world conditions is valuable information. And such a useful diagnostic mechanism helps engineers churn out better code in less time.
Reducing operating expenses
It costs a lot of time and money, to operate and maintain a fleet of vehicles for collecting data and testing algorithms. To achieve full self-driving capability, we need to repeat this process across different scenarios, including vehicle type; time of day; weather conditions, type of road, traffic pattern, geography, etc.
Tesla owners are literally getting this job done for free. It helps Tesla focus more on analyzing and utilizing the data than worry about gathering it.
Human driving data
There’s plenty for software and algorithms to learn when it comes to driving cars. And we can teach them by showing how it’s done. Tesla has access to tons of sensor recordings (input to the algorithm) and the corresponding human driving commands (expected output from the algorithm) of braking, acceleration, and steering, over billions of miles. A fairly powerful supercomputer could probably train an AI to learn how to drive just by watching these recordings.
COVID proof?
It’s 2020 and we now live in a drastically different world. Every self-driving company has had to alter its fleet operations.
The responses range the gamut — 1) temporary pause; 2) re-purpose fleet; 3) resume safe operations. Tesla no doubt will be impacted. It’s safe to assume most Tesla owners are working from home and driving a lot less. But based on Musk’s tweet, it seems like Tesla is recovering fast.
Time will tell if this is just a temporary recovery. But even if you cut usage by 50%, it’s still billions of real-world miles.
3. Full self-driving
Tesla is focused on the future — Full self-driving.
Not partial. Not assisted. Not conditional. Not constrained. Not compromised.
This is an important distinction. One that often gets them in trouble. As their cars today very much need human input at all times. While the long term payoff is huge, Tesla has clearly chosen the most difficult path here. It’s like saying we’ll reuse rockets by vertically landing them on tiny barges floating in the middle of the ocean. Ok, maybe not that hard!
But having clearly defined this end goal is advantageous.
Every self-driving application
Tesla is set up to compete in every possible self-driving application out there, as opposed to having a narrow focus, which may only help excel in one specific application of self-driving technology.
Tesla is poised to benefit from self-driving technology in many ways.
- Logistics: Tesla has more than 2000 orders for its semi-trucks.
- Robotaxis: Tesla has already made public their plans for launching a robotaxi network.
- Last-mile delivery: Tesla van or maybe a compact car?
- Consumer vehicles: Models S, 3, X, Y, Cybertruck.
- And whatever this is — link.
Addressing the long tail
The 80/20 rule aptly describes the self-driving problem. Computers have to be trained for numerous corner cases and rare events to be able to drive safely. By opting to work on the full self-driving problem, Tesla has got an early start to knocking down this long tail. For instance, a task as simple as detecting a stop sign, gets increasingly difficult when you account for all the ways it can present itself — occluded, in heavy rain/snow, on a digital display, manually held, on a toll both, on a school bus, in different languages, etc. And Tesla’s strategy allows it to create the largest dataset of stop signs and train its networks.
4. Vertical Integration
Musk’s vision and scrappiness bring in clear-as-glass clarity, ruthless prioritization, and humbleness to accept mistakes and correct them fast. Tesla’s journey to building self-driving systems is a great example of seeing this play out. Over time, a slew of decisions have helped bring a lot of development in house.
AI in-house
Back in 2017, Tesla decided to build its own AI chips, further integrating hardware and software development in house. Tesla realized that software is no longer structured lines of code. Rather it’s all about gathering and managing datasets and tuning recipes to fit the silicon constraints. Tesla has been on this path for a while now.
Cross-functional collaboration
Owning the factory, manufacturing processes, and overall product design has its benefits. I imagine software, electrical, and mechanical engineers seamlessly co-ordinating between Palo Alto and Fremont to build self-driving systems for operating in the real world.
This means that at a fundamental level, the self-driving software, vehicle operating system, battery management software and manufacturing processes are in sync.
Charging infrastructure
A truly self-driving car should be conscious of the state of its battery at all times, plan trips accounting for charging stops, park itself and not expect a human to plug or unplug it from an EV charger.
Owning the charging infrastructure gives Tesla this advantage.
The Tesla app
To truly own the end-to-end self-driving stack, you’ve got to own the user interface.
The Tesla App, which already does quite a few things, is the perfect fit. In the future, I won’t be surprised if this app provides a seamless interface to rent out your Tesla, trade energy with the grid, buy insurance, and have a “surprise me” feature to take you autonomously on a weekend road trip.
5. The Chinese market
China is the most important automotive market and by extension, very important for self-driving technology. It’s also a very different market when it comes to the pace of innovation, social acceptance of technology, and the regulatory climate.
To succeed, every automaker needs to have a China strategy. The usual norm is to partner with local automotive and tech companies in exchange for rights to build plants, sell vehicles, and develop and test self-driving technology. All that is easier said than done.
But Tesla has managed to pull off the unthinkable–ramping up a factory in Shanghai in less than a year. This implies Tesla now gets to rinse and repeat its self-driving strategy in the world’s largest electric vehicle market. It gets to build a fleet, collect data, tune its models, and continuously deploy and test.
Final thoughts
Self-driving is an AI problem. AI problems are solved with data. Data demands infrastructure at scale, both on the road and in the cloud. And Tesla has been patiently putting all the right pieces in place.
Original Publication by Venkatesh Rao at Medium.
Want to buy a Tesla Model 3, Model Y, Model S, or Model X? Feel free to use my referral code to get some free Supercharging miles with your purchase: http://ts.la/guanyu3423
You can also get a $100 discount on Tesla Solar with that code. Let’s help accelerate the advent of a sustainable future.
