Issue 0005: Gradient Descent and the Danger of Local Maxima

Issue 0005: Gradient Descent and the Danger of Local Maxima

The Trek: 18,215 Feet Up, 10,000 Feet Down

In 2011, my team did the Everest Base Camp trek while we were in Nepal. We framed it to our command as an “endurance and team-building event”, which was technically true. It was also secondary that this endurance and team-building event took place in one of the most beautiful parts of the world, where people pay money to take their vacations.

Everest Base Camp sits at 17,598 feet. We started at Lukla, which sits at 9,383 feet. The trek to base camp took six days, including a two-day acclimation stop in Namche Bazaar. That works out to a daily average of 1,369 feet of net elevation gain, which sounds like almost nothing.

Of course, the Himalayas are not a single steady slope. The trek involves constantly going up and over one peak, descending into a valley, and then climbing up and over the next. Throughout the trek you routinely encounter false peaks. Visually, you think you are coming to the top of a mountain. When you get there, you realize that what looked like a summit was concealing a much larger one. Then, when you get to the top of that mountain, you have to descend again into another valley before climbing something taller still.

In total, the trek involves about 18,215 feet of total ascent and roughly 10,000 feet of total descent to achieve a net climb of 8,215 feet. During the ascent to base camp, you spend more than a third of your time going downhill.

The trek is mapped to get you to base camp. That means the next best step is not always the one that takes you to a higher elevation. You have to spend considerable time descending so you can ascend the next higher peak. And you have to repeat that process dozens of times.

I have been thinking about this idea in a different context, which is what I want to explore.

The Map: Gradient Descent and Local Maxima

In machine learning, gradient descent is the workhorse optimization technique that powers many ML algorithms. It runs against a loss function: a measure of how wrong a model's predictions are against its training data. The algorithm adjusts the model's parameters in the direction that most reduces the loss, then takes another step, then another. It stops when a minimum value has been reached and the next step would result in an increase in the loss function.

The mirror algorithm, gradient ascent, does the same thing in reverse for problems where the goal is to maximize rather than minimize and is called a reward function; the local-optimum trap still applies and is direction-agnostic.

To make that concrete: for a linear regression, the loss function is typically mean squared error, the average of the squared differences between predicted values and actual values. The algorithm adjusts the coefficient on each input variable to reduce that average, then re-checks, then adjusts again. The result is a line that roughly approximates the data.

There is variation across ML algorithms in which parameters get adjusted and which loss function is being minimized:

• Linear regression adjusts coefficients on input variables to minimize prediction error.
• Logistic regression adjusts feature weights to minimize classification error.
• Neural networks adjust millions or billions of weights to minimize prediction or generation loss.
• Transformer-based LLM models adjust attention weights to minimize next-token prediction loss.

The sophistication grows across that list. The underlying motion does not. The system learns from its own errors. Each step is small, local, and tied to the information immediately available.

The trap, well documented in any introductory machine learning textbook, is the local maximum. Imagine a hilly terrain. You are trying to climb to the highest peak, but you are surrounded by fog. If you always take the nearest uphill step, you may reach the top of a hill and stop there. Every move nearby is downhill. By the only measure available to you, you have reached the top.

But across the valley is a much higher mountain.

A model trained naively will get stuck on a local maximum and conclude that it has done its job. The fix, in ML, involves things like randomized restarts, momentum terms, and occasional larger steps that deliberately move against the immediate gradient to escape a hill that is not the mountain.

When you’re running a company, the same thing can happen.

The Terrain: What This Looks Like Inside an Operating Company

The company that has worked on its dashboards for two years and now has the cleanest reporting suite in the portfolio. The board deck looks great. The CEO can pull any metric in 30 seconds. When you ask what decision the dashboards have changed in the last six months, the answers get vague.

The COO has cut average meeting time by 40%, ships two new ops scorecards a quarter, and has the most responsive Slack culture you've seen in a company. The same company is missing forecasts by 5 to 7% every quarter, and the misses are coming from the same three drivers as last year.

The sales org has tightened the deal-stage definitions, the CRM hygiene, the weekly pipeline reviews. Activity is up 30%. But, the close rate is unchanged.

These are all real improvements. Each one would show up as a positive step on whatever local gradient the function is being measured against. But, are any of them moving the strategic position of the business?

This is the local maximum problem in operating form. Many companies do not stall because they stop improving. They stall because they improve the wrong thing for too long. The work is getting more efficient. The metrics are getting better. The dashboards are getting cleaner. The strategic terrain underneath has not moved.

Local optimization	Strategic failure mode
More efficient meetings	No clearer decisions
Better dashboards	No improved operating judgment
Faster ticket resolution	No reduction in root causes
More sales activity	No stronger positioning
More automation	No better process architecture
More AI tools	No compounding data advantage

The reason this is so hard to see from inside is that every measurement available to the operator confirms progress. Every nearby move is uphill. The fog hides the larger mountain they need to summit.

Most Companies Are Not On a Local Peak

The strongest argument against this framing is that most companies do not have a local-maximum problem. They have a "we are not even climbing the hill we are on" problem. They have execution debt, broken cadences, decisions that nobody owns, and they would benefit from a year of disciplined gradient descent without any meta-level questioning of where they are headed.

Once they reach the top of the hill they’re climbing, they’ll be able to scan the terrain and identify any taller hills or mountains nearby.

That is real. The "strategic descent" framing could be the kind of thing that gets used to justify a $500k consulting engagement, or to talk a board out of pressing on missed quarters, or to give a CEO permission to chase the next shiny object instead of finishing the current one. That can happen, too.

The test is not whether the company is improving. Almost every operating company is improving at something. The test is whether the improvements are compounding into a strategic position or not.

What compounding looks like in practice: a new dashboard surfaces a question the company could not previously ask. The question produces a decision. The decision improves an outcome that the dashboard then measures, and the loop tightens.

A new sales motion produces a tighter qualification frame, the tighter frame produces a better-fit customer cohort, the better-fit cohort produces case studies that change the next quarter's positioning. A company on the right hill shows this kind of trajectory across functions, where the gains in one place make the next move in another place easier or more valuable.

A company on a local peak shows the opposite. Each round of improvement requires more effort to produce a smaller marginal gain. The gains do not propagate beyond the function that produced them. The dashboards get better and the decisions stay the same. The sales activity goes up and the close rate stays flat. The cyber posture hardens and the board still cannot tell you what the program does for the enterprise.

If your operating improvements are compounding, keep climbing. If they are not, the question is no longer "how do we get more efficient." The question is whether the hill is the right hill.

The two postures require different work. Operational improvement runs on cadence, accountability, and discipline. Strategic improvements often require temporary inefficiency: slower growth, fewer clients, fewer logos, more time in the rough. Conflating them is how you end up with a company that is both optimized and stuck.

Optionality: The Asymmetric Way Out

In machine learning, the practical fixes for the local maximum problem look like randomization, momentum terms, larger occasional steps, and ensemble methods that train multiple models from different starting points. In strategy, the equivalent approach has a name: optionality.

Nassim Taleb's framing is that some bets are convex (the downside is bounded but the upside is asymmetric) and some are concave (the upside is capped but the downside is unbounded). A pure gradient-descent strategy is concave in the long run. It locks the company into the hill it is already on and forecloses the possibility of a larger move. Optionality is the deliberate refusal to let the current gradient define the entire search space.

• Local optimization: what is the next measurable improvement?
• Optionality: what small bets could expose us to much larger upside?

This is the same tension that reinforcement learning calls exploitation versus exploration.

Exploitation means using what currently works. Exploration means trying uncertain actions that may reveal better possibilities. A system that only exploits gets stuck on a local peak. A system that explores too much never compounds anything and burns itself out chasing novelty. The art is balancing both, and the practical implication for an operating company is that exploration has to be funded as a deliberate line item, not as the slack that happens to be left over after the operating cadence runs.

In practice this looks like this: a slice of the calendar reserved for talking to customers outside the current ICP, a small budget for prototypes that may not ship, an outside advisor whose job is to surface what the company is not currently asking about, a quarterly meeting that exists specifically to question the existing roadmap rather than execute against it.

These are not efficient activities. They are deliberately inefficient, because efficiency on the current gradient is exactly what produces the local-maximum trap.

Google famously had “20% time”, where they allowed their employees to work on whatever they wanted with whoever they wanted. This program produced AdSense, Google News, and Google Translate. (after the company went public, the focus on quarterly earnings - local maximum - led the employees to code it “120% time”).

The Adjustment: Questions for Your Own Terrain

A few diagnostics I have found useful when sitting with an operator who senses they have been climbing the wrong hill.

1. What was the strategic position of the business 24 months ago, and what is it today? If the position is the same and the operating function is dramatically more efficient, the company has been doing gradient descent on a local peak.

2. What is the biggest improvement you have shipped in the last 12 months, and what decision did it change? If you cannot name a decision the improvement changed, the improvement is local.

3. What would you have to stop doing to move position? If the answer is "nothing, we just need to do everything we are doing better," the company is probably on a local peak.

What temporary inefficiency would the company need to accept to move to a higher peak? Slower close rate this quarter so the sales team can rebuild positioning. Lower forecast accuracy for two months while the data spine gets rebuilt. Fewer dashboards while the underlying definitions get reworked.

If no temporary inefficiency is on the table, no strategic move is on the table.

Dead Reckoning and Terrain Association

This newsletter is called Dead Reckoning because the discipline of navigating without observable terrain features turns out to map onto a lot of strategic situations. Dead reckoning is what soldiers and sailors do when the terrain has few identifiable features: they infer position from movement and correction until they reach a terrain feature they can recognize. Terrain association is the deliberate use of features in the environment (ridgelines, streams, road junctions, saddles between peaks) to verify or correct your position.

Gradient descent is pure dead reckoning. The algorithm has no broader visibility. It cannot see the larger terrain. It can only feel the slope it’s on and step in the direction of greatest improvement. That works when the landscape is smooth and convex, which it sometimes is. It fails when the landscape has multiple peaks and valleys, which it usually does in real markets, careers, and operating companies.

Terrain association is what the strategic operator adds on top. Not every step needs to be uphill. The route to a higher peak is judged against the larger map, not against the immediate gradient. The operator doing this well is constantly cross-referencing the felt local conditions against the strategic terrain they are trying to cross. The operator who is not is just walking uphill in the fog.

The dangerous thing about a local maximum is that it feels like success. Every nearby move confirms you are on top. Strategy begins with the suspicion that the hill you have climbed may not be the mountain.

On the trek, the route was mapped, and the route included lots of ups and downs. Eighteen thousand feet of ascent. Ten thousand feet of descent. The next best step is not always the one that takes you higher.

Sometimes you need to climb the hill you’re on to get a better view of the terrain. Sometimes you’d be better off descending now to climb the taller mountain.