Charles Cushing

Observations — 0001 // AI transformation

Here’s a roundup of some thoughts and observations from this week.

Written entirely by hand — I just like em dashes.

If we don’t know each other, you can learn about me here.

Everyone wants to use more AI. Not everyone has been equally successful.

Watching this unfold in StartOps, I got frustrated because I couldn’t understand why. More casual chats and 30-minute catchups weren’t revealing any new info.

So, I started doing NDA-signed, screenshare-on, deep-dive interviews with StartOps members to understand the blockers to their AI transformation.

After a dozen calls, I’m still at the foot of the mountain of understanding.

But some interesting patterns are already falling out. Given how fast the space is moving, I figured better to share early-and-uncertain than never at all.

Here’s my recap so far:

The 3 levels of AI adoption

I can now put companies in one of three buckets of AI transformation within a few minutes of starting a call:

Level 1: Text-in, text-out

The company is mostly using browser-based tools like ChatGPT and Claude.ai. The main use cases are document drafting, research, contract review and editing.

Level 2: Decentralized automation

A cohort within the company has figured out how to automate stuff.

This usually looks like skills + connectors, maybe some light vibecoded apps.

Most of the time it’s under 10% of the employees doing this, and often fewer.

Some of the more advanced teams I talked to are already hitting the challenges of this model. Decentralization can lead to brittleness, redundancy, and lack of observability.

(Interestingly, it seems like much of the AI training market is optimizing for this outcome, despite the obvious shortcomings vs. Level 3.)

Level 3: Centralized company-as-code

Someone at the company — a passionate vibecoder, or the engineering team — has started building a central hub for company apps and agents.

This can take many shapes, from a shared git repo for skills, to a home-grown ERP, to a full-on agent coordination layer.

The bigger point: the company has begun to represent itself as software (whether they recognize it or not).

I’m biased, but my view is that this pattern will win in the long run (and is the only way to compete in the Latency Game — more on that later.)

Bottlenecks on the way to Level 3

For the companies that are failing to get to Level 3, I’ve categorized the bottlenecks as I see them.

Bottleneck 1: Conviction

This is the meta-bottleneck. All downstream bottlenecks reduce to this in some way.

Simply put: the costs of AI transformation are often immediate and steep. The payoffs can be distant and hard to quantify. Bridging the gap takes conviction.

In my sample set, the companies seeing success with AI were the ones with the highest implicit conviction in the future payoff.

Some people were targeting specific, tangible results: faster data mining, reduced SaaS spend, reduced headcount as they grew.

But in most cases, company disposition towards AI could best be characterized as “vibes”.

Weirdly, this may actually be the right approach.

The companies that were struggling to invest more were looking for justification in dollars and cents: “I pay a guy in the Philippines $3.5k/month — so what if I save him 10 hours/week?”

The companies investing more were naturally technophiles. They had a history of making tech investments, building-vs-buying, taking data seriously. They saw AI as a natural extension of their technological leverage.

“Future technological leverage” is hard to quantify. It takes a high degree of internal confidence to set that as a goal. (To underscore the point: zero of the companies I talked to could articulate their internal payoff function or ROI calculation.)

Finally — one specific manifestation of the Conviction bottleneck is a question that came up in nearly every call: “What workflows are other people doing with AI?” This question belies the worldview beneath it — that AI is a way to swap the source of labor within a company, vs. changing how companies themselves are built.

My working theory is that some companies understand that they are playing the Latency Game, and some don’t. More on this below.

Bottleneck 2: Infra

To work effectively, agents need infra:

  1. A place to run
  2. Clean context to run with

Let me break down each:

A place to run

Above, I called out the pattern that Level 2 companies had decentralized infra, while Level 3 companies were centralized.

This simple distinction matters immensely, because every skill, every vibecoded app, every dashboard and every agent workflow needs a place to run.

If a company doesn’t have a default answer, the answer gets invented each time.

In my survey, I’ve seen Level 2 companies running vibecoded apps from every imaginable platform; skills being re-invented across teams; dashboards running on localhost instead of shared.

This pattern introduces friction on every level — deciding where to home something, sharing it, permissioning it, auditing it.

The Level 3 alternative is to commit to a central app to run across the company.

Some people are using off-the-rack solutions for this, like Notion; many companies are (justifiably!) scared of lock-in, so are building their own, or taking a wait-and-see approach.

Clean context to run with

I’m developing the view that many AI adoption failures are just data organization failures in disguise.

A tale of two cities, from my interviews:

Company A

Company B

Company A is at Level 1. Company B is at Level 3, despite being maybe 50x bigger in revenue and complexity.

Having a ‘company hub’ app can help, as a place to host internal apps, dashboards, and agents. But there is also often a parallel investment that needs to be made in data infra.

Crossing the chasm

The infra bottleneck is no joke. And it has one other nasty feature: it’s chasm-shaped.

Jumping 90% of the way across a chasm doesn’t help you — you either make it to the far lip, or you don’t. By the same token (no pun intended), an agent missing 10% of its essential context is almost worse than no agent at all.

This is where the Conviction bottleneck comes back to bite. Getting good AI infra in place can be a massive and expensive undertaking. If you don’t see the long term payoff, you won’t make it across the chasm.

A note on build vs. buy

One of the key themes that emerged across the calls was ‘build vs. buy’. Do we buy an agent-native ERP, or try to build it in house? Can we vibecode our inventory planning app, or would something off-the-shelf work better?

There are even many companies now offering agents-as-a-service specifically for ops: Prysmic, Hazel, Handled, Deliberate and others are vying to become the agent layer for ecommerce.

Candidly — across the companies I talked to, the answer is still “it depends”, and I don’t yet have a great razor for deciding which is the best path for a given situation.

Pinning this for a future post.

Bottleneck 3: Talent × Time

Implementing AI only takes three ingredients: time, talent, and tokens. Of those, I’ve seen talent and time form the main bottlenecks.

These calls have made me appreciate the huge skill shortage of agentic engineers working in-house at ecommerce companies. (Probably less true for tech companies.)

That said, a weird feature of AI is that time is freely convertible into talent. An avid novice can ramp up quickly on how to use AI, by simply asking AI how to get better.

And many companies actually DO have a subset of employees who are wildly excited about AI and doing more with it.

So one of the most puzzling findings from my calls: almost no companies are giving their power-user employees enough time to build.

Across the board, AI transformation is being viewed like any other corporate L&D — something you learn about on the side of your actual job. “We don’t have enough time” was the frustrated refrain I heard from many AI power users, ranging from low-level ops analysts to COOs.

Many of these folks were using their nights and weekends to build the leverage they wanted for the weekdays.

Only one company I interviewed ‘did the right thing’ and converted an eager Chief of Staff into a dedicated internal AI builder.

This baffles me. There are companies spending multiple FTE salaries/year on employee AI training; yet won’t allocate a single dedicated FTE to actually building the AI infra they need.

Another interesting pattern has been the reach for outside consultants. Many teams are doing this, given a perceived lack of internal talent or time. I don’t have hard numbers on this, but my sense is that hiring an external consultant might end up costing 10x for the same result as a dedicated in-house person.

Finally — token costs came up in almost no calls. There simply isn’t enough AI usage in the companies I interviewed for it to be the gating factor yet. (Again, probably more true in ecommerce than tech.)

Again, this reduces to the Conviction bottleneck. AI transformation requires high upfront cost for long term compounding — and without deep conviction most companies choke on the upfront cost. I believe this is a huge mistake that will catch many companies flat-footed in 6-12 months.

My suggestion for most teams: if you have someone who is eager to do more agentic engineering, make that their full time job and farm the rest out downstream.

The prize

So in summary: the ecommerce companies I talked to vary wildly in their AI adoption success. And even the successful ones seem to be there more because of their natural disposition vs. a particular goal.

I think there is a prize, though. A successful AI transformation means you get to win — or at least not lose — the Latency Game.

But what is the Latency Game? That’s the question I’ll take up next time.

Get future essays

One email when a new piece goes up.