Skip to content

Build cluster · Data, analytics, BI

Technology Rewired: Data and Analytics

Data and analytics moved from on-prem warehouses and desktop BI to cloud warehouses, ELT pipelines, and self-serve dashboards. AI added natural language queries. Agents now write SQL, build pipelines, and answer business questions directly, which makes trusted metric definitions and data quality the real bottleneck.

The shift: Questions get answered in plain English; analysts own definitions and trust.

Verified
2.3

Supervised agents today
3.2 in five years

How has the data and analytics team changed across five eras?

  1. Era 1 · On-prem

    Before 2005

    0.3

    Data lived in on-prem warehouses that a central BI team owned. Business users filed a request, waited weeks, and got a report built by a specialist who knew the schema. Analysts spent most of their time in spreadsheets, reconciling one system's numbers against another's.

    The warehouse team was small and powerful, and every question went through them.

  2. Era 2 · SaaS and cloud

    2005 to 2020

    1.0

    Cloud warehouses, ELT tools, and self-serve BI created a new role, the analytics engineer, who owned the models between raw tables and dashboards. Every department got its own analyst and its own dashboards.

    The result was sprawl. Two teams could show two different numbers for the same metric, and the argument about whose was right ate more time than the analysis. The discipline that mattered was data governance, and most teams came to it late.

  3. Era 3 · AI-assisted

    2020 to 2024

    1.8

    Natural language query features and SQL copilots made analysts faster and let some business users ask simple questions on their own. Answers were only as good as the underlying models and definitions, and a confident wrong number travelled faster than a slow right one.

    The analyst's day moved from writing queries to checking them.

  4. Era 4 · Agentic

    2024 onward

    2.3

    Agents now write and run SQL, build pipeline code, and answer ad hoc questions in chat with charts attached. A data team of a few analysts and an analytics engineer can cover requests that used to need a queue. The open problems are trust and access: which tables an agent may read, and whether its answer used the approved metric definition.

    The same plumbing feeds models. How you prepare and index data decides what an LLM can retrieve, which I cover in efficient data processing for LLMs and semantic search.

  5. Era 5 · Next 5 years

    2026 to 2031

    3.2

    My bet: the dashboard stops being the main interface, and data teams spend their time on semantic layers, quality, and access policy rather than report building. Analysts become editors and auditors of agent answers, and the scarce skill is knowing when a number is wrong.

Which data and analytics software is being rewired?

No deep dives yet. The first ones are in the queue below.

Coming next

  • BI and dashboards
  • Data warehouses
  • ETL and ELT
  • Data catalogs
  • Product analytics
  • Customer data platforms
  • Spreadsheets
  • Data quality

Also wired into this category: Customer Onboarding Email.