[Medium Article] 12 Years in Data & AI: Lesson #2 — How Organizational Structure Determines Data & AI Success
date posted: 2026-Mar-18, last edit date: 2026-Mar-18
Originally published on Medium, March 18, 2026.
Introduction
I’ve had the opportunity to work across different organizational structures for Data & AI. Over time, something became apparent to me:
Most organizations are trying to fix Data & AI with tools, while the real constraint is structure.
And structure determines what your data team is allowed to become.
And if you sit with that long enough, you arrive at something uncomfortable:
Most Data & AI failures are not technical. They are structural — and they were predictable.
This isn’t theoretical.
I’ve seen talented teams rendered invisible by structure — and average teams punch far above their weight simply because they were positioned correctly.
In this article, I’ll unpack the three most common structures, and what each one of them actually optimizes for.
1. Data under Business → The Short-Term Factory
When the data function sits within a business unit, it gains immediate proximity to demand. Priorities are clear. Stakeholders are accessible. Feedback loops are fast.
This sounds like an advantage — and for a while, it is.
But proximity to demand becomes a trap.
Under this structure, the focus shifts heavily toward lagging indicators:
- sales, conversions, revenue
These matter — but they are outputs, not investments.
On the other hand, the work that builds long-term, compounding value — leading indicators:
- customer experience quality, reusable data products, predictive capability
gets chronically deprioritized because it doesn’t show up in next quarter’s numbers.
What erodes first is invisible:
- documentation, data modeling discipline, pipeline re-usability
These feel optional… until they aren’t.
By the time the technical debt becomes visible, the team is too busy servicing requests to fix it.
The deeper issue is incentive design. In this structure, the message is clear:
Deliver now. Improve later.
And “later” rarely comes.
Over time, data becomes a production line for sales justification — not a strategic capability.
If your data team is always busy, always firefighting, and nothing compounds — you’re likely in this structure.
2. Data under IT → Technically Correct, Commercially Irrelevant
The IT-led structure is the mirror image of the first. Here, things are done “properly”:
- architecture reviews, governance frameworks, engineering standards
The work has discipline — and that discipline has real value.
But something critical is missing: business gravity.
Without a direct line to strategic priorities, the team defaults to what it can control:
- infrastructure, tooling, internal standards
Rather than what the organization actually needs, priorities drift toward what IT understands — not what the business is trying to achieve.
The result is a familiar pattern:
- technically excellent systems that nobody uses
- pipelines that were never asked for
- dashboards that solve the wrong problem
This is not a talent failure. It is a positioning failure. Teams in this structure often develop a compliance mindset — focusing on whether something was done correctly, rather than whether it was worth doing at all.
That distinction is subtle — but it is everything.
3. Hub-and-Spoke → The Balancing Act
The hub-and-spoke model is the most effective structure I’ve seen. Not because it eliminates the tensions above — but because it acknowledges them directly. The core insight is simple:
Data has two genuinely competing needs: Control and Context.
- The hub provides control: governance, shared platforms, standards, talent development, architectural coherence
- The spokes provide context: domain expertise, stakeholder alignment, prioritization, execution
Neither can replace the other.
A hub without spokes becomes IT. Spokes without a hub become fragmentation and data politics.
But here is where most organizations get it wrong:
They treat this as a design decision — not an operating discipline.
Hub-and-spoke is not something you implement once. It is something you continuously manage. The tension between:
- central vs local
- standardization vs flexibility
- long-term investment vs short-term delivery
never disappears. It must be actively managed.
Organizations that succeed are comfortable operating inside that tension.
Organizations that fail try to eliminate it — either by centralizing everything back into IT, or by letting autonomy spiral into inconsistency.
The model works when the tension is managed. It fails when someone tries to resolve it.
The Trade-offs
Every structure rewards a behavior — and quietly punishes another:
So the question is not:
“What is the best structure?”
The real question is:
“What trade-offs can our organization afford — and which ones will quietly kill us?”
Because the failure modes don’t show up immediately. They compound — as technical debt, talent attrition, missed opportunities, and eventually, AI initiatives that scale the wrong thing.
A Note on the Role of the CDO
In every structure I’ve worked within, the Chief Data Officer role has been consistently misunderstood — by boards, executives, and sometimes even by CDOs themselves.
The CDO is not:
- a data owner, a governance enforcer, an AI sponsor
The real role is to hold tension at the organizational level.
Between:
- short-term delivery and long-term value
- business urgency and technical reality
- autonomy and coherence
And increasingly:
The willingness to say no to AI when AI is not the answer.
Because when the data foundation is weak, AI doesn’t solve the problem. It amplifies it.
The CDO who understands this is not focused on “doing AI.”
They are focused on building the conditions under which AI can actually work.
If your structure is broken, AI will not fix it. It will scale the dysfunction — faster, and in front of more people.
Closing Thought
Large organizations rarely fail at Data & AI because they lack tools, models, or talent.
They fail because structure shapes behavior — quietly, persistently, and in ways no technology investment can override.
Structure defines:
- what gets prioritized
- what gets ignored
- and ultimately, what Data & AI are allowed to become
The good news is that structure is a choice.
But it only changes when leadership is willing to examine the incentives it has already created — and ask, honestly:
What behavior are we actually rewarding?
Until that happens, no platform roll-out, no AI strategy, and no transformation program will change the outcome.