Why Silicon Valley's Optimization Mindset Sets Us Up for Failure

That mindset is inculcated early in the training of technologists. When developing an algorithm, computer science courses often define the goal as providing an optimal solution to a computationally-specified problem. And when you look at the world through this mindset, it’s not just computational inefficiencies that annoy. Eventually, it becomes a defining orientation to life as well. As one of our colleagues at Stanford tells students, everything in life is an optimization problem.

The problem here is that goals such as connecting people, increasing human flourishing, or promoting freedom, equality, and democracy are not goals that are computationally tractable. Technologists are always on the lookout for quantifiable metrics. Measurable inputs to a model are their lifeblood, and the need to quantify produces a bias toward measuring things that are easy to quantify.

The bottom line is that technology is an explicit amplifier . It requires us to be explicit about the values we want to promote and how we trade-off between them, because those values are encoded in some way into the objective functions that are optimized. And it is an amplifier because it can often allow for the execution of a policy far more efficiently than humans. For example, with current technology we could produce vehicles that automatically issue speeding tickets whenever the driver exceeded the speed limit