home
/
blog
/
reporte-difusion-ia

A Methodological Critique of Measuring Global AI Diffusion

10 minutes
AI
Data Science
Microsoft
Papers
In January 2026, Microsoft released the AI Diffusion Report 2025 (H2) to gauge global AI adoption during the second half of 2025. I offer a brief commentary on the generalization of user behavior across all platforms based solely on Microsoft ecosystem telemetry.

Contents

Why Measuring "Everything" Leaves Us Measuring Nothing

In the current race to quantify the impact of Artificial Intelligence, we find ourselves flooded with metrics promising a comprehensive overview of technological adoption.
On January 8, 2026, the Microsoft AI Economy Institute published the AI Diffusion Report 2025 (H2). You can find the official PDF version here: [1], or view the web version at: [2]. The paper underpinning this document was published on ArXiv as: "Measuring AI Diffusion: A Population-Normalized Metric for Tracking Global AI Usage" (Misra et al, 2025), available here: [3].
While the effort to standardize metrics is commendable, the study suffers from a methodological weakness (in my view): by attempting to scale proprietary Windows telemetry data to infer global behavior, the study sacrifices internal validity in favor of a questionable generalist reach.
Below, I analyze why extrapolating internal Windows user data and assuming statistical independence introduces critical noise into these measurements. Therefore, we will focus on the paper [3], which contains the study's methodological details, rather than the corporate documents published by Microsoft (based on the paper), which I consider to be polished and sanitised to an impressive degree.
Basically, two main points stand out to me:
  1. That based on Windows telemetry, it is inferred that users on other platforms exhibit identical behaviors regarding AI usage and adoption (Population Homogeneity).
  2. That the way users engage with desktop devices is independent (unrelated) to how they use mobile devices (Statistical Independence).
The data presented in the Microsoft AI Diffusion Report 2025 H2 should be interpreted strictly as adoption indicators within the Windows ecosystem—a job they did brilliantly. However, in my opinion, they do not represent the totality of global AI usage, especially in research, software development, and mobile-first markets. By trying to measure everything, they have left us measuring nothing.

Where Does the Data Come From?

The data stems exclusively from "anonymized" internal telemetry from Microsoft users, specifically Windows desktop/tablet versions.

AI Usage Source

The first filter is that telemetry comes from users who have opted in to share data at some point (e.g., error reporting or diagnostic feedback). Furthermore, only users active for at least 90 minutes per week are considered. The data is derived from searches and pages visited in Microsoft browsers, installed software, etc.

Population Source

To calculate the share, they cross-reference telemetry with:
  • The number of active Windows devices in each country.
  • Statistics on working-age population and Internet access (likely from sources like the World Bank or ITU, although the paper focuses on its own "Population with PC" estimate).
They lack direct data from mobile operating systems (Android/iOS) and other desktop or laptop operating systems. They infer this based on the percentage of Windows users relative to the total users on sites like StatCounter.
Telemetry data sent by Windows users can be viewed in the Diagnostic Data Viewer. Microsoft assures this data is anonymized.

Telemetry data sent by Windows users can be viewed in the Diagnostic Data Viewer. Microsoft assures this data is anonymized.

The Problem of Population Homogeneity

The study uses the Windows ecosystem telemetry as a base proxy. To estimate the total population of AI users in a region, they apply a normalization based on Market Share.
The implicit logic can be summarized in the following equation:
Ntotal=Monthly Active DevicesMarket Share=Some proportionN_{total} = \frac{\text{Monthly Active Devices}}{\text{Market Share}} = \text{Some proportion}
Where Monthly Active Devices represents devices detected by Microsoft, and Market Share is the proportion of the Desktop and Tablet market using Microsoft. By active devices, they refer, as mentioned, to users:
  • On Windows with at least 90 minutes of activity per month,
  • Who have accepted to share their information. It does not state whether this accounts for other Windows products like GitHub or Visual Studio.

Why This Metric Fails

This formula assumes population homogeneity. It takes for granted that "AI usage density" is constant across all operating systems; that is, it assumes a macOS or Linux user behaves identically to a Windows user.
The paper assumes that behavior on Windows (mostly corporate/office) is extrapolatable to the whole world.
For example, a Linux user running Ollama in their terminal or using a Python script calling the OpenAI API is invisible to this paper's methodology because they do not generate traditional "web visits" nor do they use the Windows telemetry stack.
The demographic reality of the tech sector suggests otherwise:
  • The Developer Bias: According to the Stack Overflow Developer Survey 2025[4], nearly 50% of professional developers use Unix-based systems (macOS or Linux) as their primary environment.
  • Data Science and ML: Community surveys (such as those from Kaggle) show that Linux environments dominate Machine Learning and model inference workflows.
  • JetBrains State of Developer Ecosystem Report 2025[5]: This survey indicates that 85% of developers use AI to write code, and 62% use at least one AI-based assistant, agent, or code editor.
By dividing Windows telemetry (which undoubtedly dominates the general consumer and administrative/corporate market with ~72% share) by its market share, the study statistically dilutes the Power Users residing in other ecosystems—essentially, the users who are likely developing the bulk of AI worldwide.
They are trying to measure the AI "gold rush" by only looking at who buys standard picks and shovels, ignoring the engineers using heavy machinery on other platforms.
While it is true that StackOverflow ([4]) and JetBrains ([5]) surveys have their own bias—being voluntary surveys taken by users whose profiles diverge from what the AI Diffusion Report seeks—we certainly cannot ignore these numbers, much less generalize that Windows users adopt AI in the same way as Linux/macOS users.
For instance, no distinction is made between a user on their work computer (corporate user) and that same person at home using their personal Windows device completely differently (let alone if they use other operating systems). This person would be counted twice, and their "AI adoption" would surely look different when analyzing data across their devices.

The Trap of Statistical Independence

Another weak point that caught my eye is how the authors aggregate users from different platforms (Desktop / Mobile), assuming them to be independent.
To calculate the percentage of unique users utilizing AI, the study assumes statistical independence between usage events. Mathematically, they postulate that the joint probability is the product of the marginal probabilities:
P(Desktop∩Mobile)≈P(Desktop)⋅P(Mobile)P(Desktop \cap Mobile) \approx P(Desktop) \cdot P(Mobile)
This assumption ignores basic human behavior. Technology adoption has a strong positive correlation.
  • Reality: A user utilizing AI tools on their desktop (e.g., Copilot or ChatGPT Web) has a very high probability P(M∣D)P(M|D) of having an AI app on their mobile—often the same one.
  • Consequence: By assuming independence (zero correlation), the model drastically underestimates the intersection (Desktop∩MobileDesktop \cap Mobile).
This leads to a systematic "double counting" error. And (unintentionally?) they artificially inflate the estimated number of unique users. In technical terms, the estimator is not independent and identically distributed (i.i.d.).

Blindness to Embedded AI

Another critical flaw in data collection is the lack of granularity regarding professional tools. The study seems to rely on "visits to AI domains" or "opening an AI application on my system." However, the current revolution isn't just about visiting a website; it's about integration (bots, local LLMs, RaspberryPi, Python projects, etc.). Using AI via API within Integrated Development Environments (IDEs) or terminals (CLI) does not necessarily generate the "web footprint" the study tracks. For Windows telemetry, opening VS Code counts merely as opening a text editor. The study does not detail mechanisms to detect if that user is massively utilizing GitHub Copilot within the editor.
An engineer spending 8 hours a day pair-programming with AI in VS Code could be counted as a "Non-AI user" if they don't explicitly visit the ChatGPT website (on their Edge browser in Windows OS). Once again, the metric penalizes technical sophistication.

The Good

  • Population Normalization: Unlike web metrics (which count "visits"), they attempt to measure "people." The effort to divide AI usage by the "working-age population" of each country is a necessary step forward.
  • Global Reach: They cover over 100 countries. Most reports (Stanford AI Index, etc.) tend to have a massive bias toward the US and Europe. This report shines a light on overlooked markets (referring to the Global South and Global North).
  • Acknowledging the Divide: Although their exact numbers may be doubtful, the trend is correct: infrastructure (PC + fast Internet) and socio-economic conditions are the biggest bottlenecks for AI in the Global South, not a lack of interest (though this isn't necessarily deduced from this study, in my view).
The main critique of this paper isn't that Microsoft's data is bad—in fact, it's one of the richest telemetry sources in the world, dominating >70% of the market—but that manipulating it to encompass platforms outside their control weakens its generalization.
A study limited to "AI Diffusion in the Windows Ecosystem" would have been preferable. This would have guaranteed perfect Internal Validity, serving as a robust indicator for the corporate market.
By attempting to measure everything through linear extrapolations over heterogeneous populations, we end up with a metric that, while ambitious, obscures the true dynamics of Artificial Intelligence adoption.
They are measuring who uses Microsoft, not necessarily who uses AI.

AI User Share Metric

They elaborate on a metric called AI User Share, defined as the percentage:
AI User Share=(% of Microsoft Users That Use AI)=(% of Population With a Desktop Device)=(× Mobile Scaling Factor) \begin{aligned} \text{AI User Share} &= (\% \text{ of Microsoft Users That Use AI})\\ &= (\% \text{ of Population With a Desktop Device})\\ &= (\times \text{ Mobile Scaling Factor})\\ \end{aligned}
To obtain the absolute number of AI users, this value is multiplied by the working-age population [3].
Example of the AI User Share metric intended to reflect AI adoption globally relative to the working-age population (15 to 65 years old).

Example of the AI User Share metric intended to reflect AI adoption globally relative to the working-age population (15 to 65 years old).

To conclude, and getting a bit suspicious (and bold), reading the paper left me with the following questions (in addition to the question marks raised above):
  • Are they seeking to influence public policy in these countries to maintain their corporate hegemony?
  • Are they turning a methodological weakness into corporate marketing?
  • What happens with users who use Windows for work but use a different system at home, or behave completely differently?
  • Does Microsoft want to be seen as the global AI leader, knowing they are losing the race for AI tool innovation?
  • Was it just a mistake and nothing more?
Ultimately, the paper (and the report) are useful for spotting relative trends (who is growing the fastest), but I would mistrust the absolute numbers. The real percentage is likely higher if we consider "Shadow AI" (use of non-corporate tools), agnostic AI systems, and the actual mobile ecosystem.
✌🏼

References

Cargando notas...

Other posts: