The Moment Search Ended
For a long time, we have taken the act of search for granted. When something comes to mind, we open a search box, type in keywords, scan the results, and click the link that seems most relevant. This process has become so familiar that it is no longer a deliberate behavior but almost a reflex. Search results were merely a starting point, and the real information existed in the next step—inside the websites themselves. In that sense, search has always been based on movement. The process of starting from a results page and navigating across multiple web pages to gather information has been at the core of how the web functions.
But this assumption is quietly breaking down. Recent generative AI–based search features no longer send users to websites. Instead, they provide complete, self-contained answers directly on the results page. Users no longer need to click links—and increasingly, they have no reason to. Search is no longer a starting point; it has become a destination. At first glance, this may seem like a simple UX improvement. In reality, it represents a fundamental shift in the purpose of search itself.
The importance of this change lies not just in technological advancement. The moment search ends, the flow of the web ends with it. The traditional web operated on connections—links that led from one page to another, from one site to the next. But AI-based search breaks these connections. Users no longer move; they consume all information within a single interface. In this structure, websites are no longer the final destination of information but are reduced to raw material that AI systems reference.
At this point, we are faced with an uncomfortable question. Is search still a tool for navigating the web, or has it become a new interface that replaces it? There is no clear answer yet, but the current trajectory is unmistakable. Search is no longer a gateway to the web—it is evolving into a layer that absorbs it. And this transformation is happening far faster than most people realize.
How the Web’s Revenue Model Has Been Sustained
To understand why this shift is perceived as a problem, we must first examine how the web has been sustained. The web appears “free” on the surface, but in reality, it operates on a highly structured economic model. Blogs, news sites, technical documentation, and community posts are all created through someone’s time and effort. The cost of producing this content is typically recovered through traffic. Users visit sites, read pages, view ads, or subscribe—this is how revenue is generated.
At its core, the model is simple: traffic equals value. The more visitors a site attracts, the more ad revenue it earns, the more subscribers it gains, and the stronger its brand becomes. For website operators, the most important metric is not just the quality of content itself, but how effectively that content drives visits. Search engines play a critical role in this system—they are the most powerful source of traffic distribution.
Because of this, an implicit contract has long existed between search engines and publishers. Publishers provide high-quality content, and search engines connect that content to users. Search engines deliver traffic through links, and publishers monetize that traffic. This structure has remained relatively stable over time. While there have been challenges such as algorithm changes and SEO competition, the fundamental flow itself has not been disrupted.
However, this system depends on a key assumption: users must visit websites. The moment this assumption breaks, the entire structure begins to collapse. If users can obtain all necessary information directly from search results, there is no longer a need to visit websites. Content continues to be produced, but the traffic it once generated disappears. As a result, the revenue model collapses as well.
What matters here is that this issue is not limited to large publishers or media companies. Personal blogs, technical documentation sites, and developer communities all rely on the same structure. In other words, when traffic declines, it is not just individual businesses that are affected—it is the entire web ecosystem. Understanding this is essential to recognizing that what is happening is not a simple technological trend, but a structural crisis.
What AI Search Has Changed
So how exactly has generative AI–based search altered this structure? On the surface, the change seems minor—just an additional summary at the top of search results. But in reality, the way information is delivered has fundamentally changed. Traditional search presented multiple links, requiring users to choose and explore. AI search, on the other hand, integrates information from multiple sources into a single, complete answer.
This difference is more significant than it appears. In the traditional model, users visited multiple sites, understood different contexts, compared perspectives, and made judgments. Information was distributed, and users had to assemble it themselves. AI search eliminates this process. Since the result is already structured, users no longer need to explore. Information consumption becomes faster and more efficient—but a critical shift occurs in the process.
Information is consumed, but traffic is not generated. AI systems draw from various websites to generate answers, yet users do not visit the original sources. Links may still exist, but they become optional. In most cases, users feel they have already received sufficient information and do not click further. In this model, content producers still exist, but the value they create is no longer directly rewarded.
This transformation is not just a UX change—it restructures the entire flow of the web. Search engines are no longer intermediaries; they become endpoints. All information is consumed within that endpoint. Websites lose their role as independent destinations and begin to function as internal components of AI systems. The web shifts from a space directly accessed by users to a data layer that AI systems learn from and reference.
Although this structural shift is still in its early stages, its direction is clear. Search is evolving toward delivering complete answers, and in the process, the role of websites is steadily diminishing. If this trend continues, we will move from an era of “exploring the web through search” to an era of “consuming everything within search.” And at this point, regulation, conflict, and competing interests inevitably begin to emerge.
Why the CMA Stepped In — The Moment Regulation Appears
At this point, a natural question emerges. If the market is changing this rapidly, how far should that change be allowed to go? It becomes necessary to decide whether this should be viewed simply as technological innovation or as something that disrupts market order. This is where the UK Competition and Markets Authority (CMA) enters the picture. The CMA is not merely an institution that mediates competition between companies; it intervenes when there is a risk that the structure of the market itself could be distorted by a single entity. In this case, what they focused on was not just a feature change, but the fact that search—a core piece of infrastructure—was being restructured by one company’s AI strategy.
One of the concepts used by the CMA is “Strategic Market Status.” This does not simply refer to companies with high market share, but to those with the power to reshape the rules of the market itself. Google effectively holds this position in search. The problem is that this dominance has become even more powerful when combined with the new layer of AI. The authority to decide how search results are displayed, how content is summarized, and how user behavior flows are designed is concentrated within a single company. This is no longer just a competition issue—it creates a structure in which access to information itself becomes dependent on the design choices of that company.
As a result, the CMA began to consider not just recommendations, but regulatory measures in the form of “Conduct Requirements.” These include three key pillars: fairness, choice, and transparency. Fairness aims to prevent imbalances between publishers and platforms. Choice ensures that both users and content providers can determine their own position. Transparency requires that the entire process be externally observable. All three are either insufficient or largely absent in the current AI search structure.
Seen in this context, regulation is not emerging to suppress technology, but rather to catch up with a structural change that has already occurred. The market has already shifted, user behavior has already changed, and platforms are already absorbing value in new ways. The CMA’s intervention is not an attempt to reverse this transformation, but to prevent it from tilting too far in a single direction. And at this point, one of the more concrete solutions proposed is the concept of “opt-out.”
Why the Opt-Out Proposal Emerged — and Why It Is Weak
The CMA’s proposed “opt-out” approach appears, on the surface, to be a reasonable solution. It allows publishers to choose whether their content can be used in AI summaries or generative features. It seems to guarantee a minimum level of choice, offering a compromise that reduces regulatory intensity while addressing the issue. It does not directly restrict technology, but instead grants content providers a degree of control.
However, this proposal does not address the underlying structure—and that is precisely the problem. In the current AI search model, search and AI functionality are effectively bundled together. If a publisher wants to appear in search results, they are inevitably exposed to AI usage through the same crawling and indexing process. The opt-out mechanism operates on top of this bundled structure. In other words, it appears to offer choice, but in reality forces publishers into a trade-off where they must give up something important.
At its core, the choice is simple: give up traffic, or provide content to AI. Even if it is technically possible to remain visible in search while opting out of AI, the boundary is defined by the platform. The publisher’s control is limited, and their actual influence remains minimal. Ultimately, opt-out presents itself as a form of choice, but in practice it functions as a constrained option within conditions set by the platform.
Another critical issue is transparency. When a publisher chooses to opt out, it is difficult to verify how their content is actually handled within AI systems. There is no clear standard for what data has been used for training, how it is reflected in summaries, or how previously collected data continues to be used. In such a situation, opt-out risks becoming a formal option without real control.
In the end, this proposal does not solve the problem—it merely makes it appear manageable. The structure remains unchanged, and the platform’s power is intact. The only difference is the addition of a button that allows publishers to “step out.” But as discussed earlier, the core issue is not the presence of a button, but how the structure itself is designed. This is why, rather than welcoming the proposal, the industry has begun to demand more fundamental changes—most notably, the separation of crawlers.
Publishers’ Demand — The Fundamental Call for Crawler Separation
The demand for “crawler separation” from publishers is not a simple request for technical improvement. It is a demand to dismantle the current structural coupling. In today’s system, a single crawler explores the web, builds search indexes, and simultaneously feeds data into AI training and summarization processes. In such a structure, content providers have no control over which parts of the system their data is used for. Once collected, the data is repurposed across multiple functions, with boundaries defined entirely within the platform.
This is why major publishers such as the BBC, The Guardian, and the Financial Times have made a clear demand: separate the crawler used for search from the crawler used for AI. This is not technically infeasible. In fact, companies like OpenAI and Anthropic already operate different crawlers for different purposes. Such separation allows content providers to make more explicit choices about how their data is used.
The importance of crawler separation lies in how it fundamentally changes the level of choice. If opt-out is a constrained option within the platform, crawler separation enables externally controllable choices by structurally dividing the system. Publishers could remain visible in search while withholding data from AI, or allow AI usage only under specific conditions. In other words, content providers gain the ability to clearly define how their data is used.
Another critical aspect of this demand is verifiability. If crawlers are separated, external organizations or regulators can confirm whether that separation is actually being implemented. This moves beyond technical implementation—it becomes a structural mechanism for establishing trust. In contrast, when a single crawler performs all roles, such verification is effectively impossible.
Ultimately, the demand for crawler separation is not just a request to “split functionality.” It is a call to redefine the relationship between content providers and platforms within the web ecosystem. And the very fact that such a demand has emerged reveals how heavily the current structure is tilted in one direction. At this point, the conflict is no longer about technology or features—it is shifting into a question of power and control.
The Platform’s Response — Why Google and Microsoft Refuse
A natural question follows from the previous discussion: if the demand is this clear, why do platforms refuse to accept it? On the surface, the answer seems simple. It is costly and inefficient. Google, for example, argues that separating crawlers would lead to duplicated data storage, increased infrastructure costs, and reduced overall system efficiency. Technically, this claim is not entirely wrong. Crawling the same web twice and maintaining two separate data pipelines would indeed introduce additional cost and complexity.
However, this explanation is not sufficient. If we look at this issue only through the lens of cost or efficiency, we miss the core point. What truly matters is how control over data is maintained. In the current structure, data collected by a single crawler can be reused in multiple ways—search indexing, recommendation systems, AI summaries, and model training all rely on the same data pool. This structure is not only efficient; it maximizes the utility of data. And the broader that utilization becomes, the stronger the platform’s competitive advantage.
The moment crawlers are separated, this structure breaks. Data becomes segmented by purpose, and its usage scope becomes restricted. This is not just a technical change—it weakens the platform’s absolute control over data. If publishers can choose which data to allow and which to deny, platforms can no longer freely exploit all collected data. Ultimately, the refusal to separate crawlers is less about cost and more about a strategic decision to preserve data dominance.
What is particularly interesting is Microsoft’s position. Despite being a competitor to Google, it takes a similar stance on this issue. The reason is simple: Bing operates on the same structural model for expanding AI functionality. The integration of data collection and AI utilization into a single pipeline is not unique to one company—it is a common architecture across modern AI platforms. Breaking this structure is not just a problem for Google; it would fundamentally change how AI platforms as a whole operate.
In the end, the platform response may appear as a technical rebuttal, but in reality, it reflects deeper structural interests. Behind the argument of efficiency lies the intention to maximize data utilization, and behind the logic of cost lies the strategy to preserve the existing system. At this point, the debate moves beyond technical considerations and becomes increasingly about who controls data and who captures its value.
The Core of the Conflict — Content vs Platform
Stepping back, it becomes clear that this debate is not about specific features or policies. It is a structural conflict between those who produce content and the platforms that distribute and reconstruct it. When the web first emerged, content and platforms were relatively loosely connected. Content existed independently as websites, and platforms served to discover and connect that content. In this structure, both sides depended on each other while maintaining a certain balance.
However, the introduction of AI has begun to break this balance. Platforms no longer simply connect content—they collect it, reconstruct it, and deliver it as a complete, unified output. In this process, the original content may not even be directly exposed to users. Content still exists, but its value is consumed in a transformed form within the platform. In this structure, the entity that reconstructs content captures more value than the one that creates it.
It is not enough to describe this as “AI using content.” More accurately, AI transforms content into interchangeable input data. Individual articles, posts, and documents are no longer consumed as independent units; they are absorbed as fragments of data. These fragments are gathered from multiple sources and integrated into a single answer, during which the original context and individual value are diluted. As a result, users receive information without recognizing where it came from or the context in which it was created.
The most significant problem here is the asymmetry in value distribution. Creating content requires time and cost, but summarizing and reconstructing it can generate greater value at a lower cost. In other words, the cost remains with content producers, while revenue shifts toward platforms. This imbalance lies at the heart of the current conflict. Publishers are not merely concerned about declining traffic—they are pointing to a structural unfairness.
At this point, the debate expands beyond technology into the realm of economic structure. For the web to sustain itself, content producers must continue to exist. But if the current trajectory continues, the incentives to produce content will inevitably decline. Platforms require more data, yet the entities that generate that data become weaker. Unless this contradiction is resolved, the system cannot remain stable in the long term. And this is where the issue evolves from a conflict between companies into a question of the sustainability of the entire web ecosystem.
The Real Impact on Personal Blogs and Developers
To fully understand this issue, we need to bring it down to a more practical level. So far, the discussion has focused on conflicts between large publishers and platforms, but in reality, the earliest impact is likely to be felt by personal blogs and developer communities. Anyone running a technical blog or consistently publishing content on a specific topic has likely already noticed it: traffic from search is steadily declining.
This shift is fundamentally different from typical algorithm changes. In the past, under SEO-driven competition, producing better content or structuring it more effectively could still yield rewards. But in an AI-centric search model, this assumption begins to break down. Even if high-quality content is created, if it is summarized and consumed directly within search results, fewer users will visit the original page. The link between content quality and traffic weakens.
In this environment, personal blogs face a new dilemma. There is still a clear reason to produce high-quality content, but if that content no longer leads to direct visits, the motivation itself can erode. Advertising revenue becomes harder to sustain, and building a subscription model carries high barriers to entry. As a result, many personal blogs may find themselves in a position where the rewards no longer justify the cost of maintenance. This is not just about declining traffic—it threatens the sustainability of content creation itself.
Developer communities face a similar impact. When searching for solutions to technical problems, users increasingly obtain sufficient answers from AI-generated summaries. As a result, direct visits to forums or blog posts decrease. This changes how content within communities is consumed and affects both participation and contribution. In the long term, it may even reshape how knowledge itself is accumulated.
In this way, the expansion of AI search goes far beyond improving the search experience—it fundamentally transforms the structure of creation and sharing on the web. This change has already begun and is likely to accelerate. At this point, we return to the original question: can the web absorb this transformation and find a new balance, or will it be restructured into something entirely different? Answering this question is now the next stage of the discussion.
Is the Web Disappearing, or Being Reconstructed
Following the trajectory so far, we arrive at a naturally extreme question. Can the web survive this transformation, or will it gradually disappear? This may sound exaggerated, but it is in fact a very real concern. The changes we are witnessing do not simply improve features—they are altering the fundamental principles on which the web operates. In the past, there have been multiple platform shifts, yet each time the web adapted and survived. The rise of search engines, the expansion of social media, and the transition to mobile all reshaped the web’s structure, but none replaced it entirely.
However, this shift is different in nature. Previous platforms controlled the “entry points” to the web but still preserved the flow of sending users back to it. Search engines provided links, and social media created pathways that led to content. In contrast, AI-based search breaks this flow. Users no longer move to the web; they consume everything within the platform itself. This difference may seem subtle, but in reality, it weakens the very reason the web exists. If websites are no longer destinations, then the question becomes: where does their value come from?
That said, it would be premature to conclude that the web will simply disappear. The web is not just an interface—it is the underlying infrastructure where information is created and stored. No matter how advanced AI becomes, the data it relies on must originate somewhere. And that place is the web. In other words, the web may retreat from the visible layer where users directly interact, but rather than vanishing, it is more likely to shift into an invisible infrastructure.
This transformation changes the form of the web, not its existence. The real issue is how value is redistributed during this transition. As the web moves into the infrastructure layer, platforms take ownership of the direct interface with users. Content producers are no longer directly connected to users, but only indirectly through platforms. This is not merely a UX change—it fundamentally alters how the value of content is evaluated and rewarded.
What matters here is how we interpret this shift. Viewing the web as disappearing is an oversimplification. On the other hand, assuming nothing is changing ignores reality. A more accurate perspective is that the web is being restructured into a completely different role than before. And this restructuring is still in its early stages—the final outcome is not yet determined.
Conclusion — Where Content Goes in the AI Era
From everything discussed so far, one thing becomes clear. We are not merely experiencing a new feature—we are passing through a turning point in how information is created and consumed. Search is no longer a pathway to the web; it is becoming a complete experience in itself. This shift provides convenience for users, but at the same time, it destabilizes the foundation of the web ecosystem. At the center of this tension lies the relationship between those who produce content and the platforms that reconstruct it.
This tension cannot be resolved in the short term. Regulators, publishers, and platforms are all moving in different directions. The opt-out proposal from the UK Competition and Markets Authority is only a starting point, and it is far from sufficient to solve the problem. More fundamental demands, such as crawler separation, have yet to be accepted, while platforms continue to show strong resistance to changing the current structure. This situation is likely to produce further conflicts and negotiations.
From the perspective of content creators, what can be expected? The long-held assumption that “good content generates traffic” may no longer hold as an absolute truth. Content remains important, but the pathways through which it reaches users are changing. In this environment, creators must redefine how their value is established. Strategies that rely solely on search visibility are increasingly likely to face limitations.
At the same time, we must not forget a crucial fact. While AI appears to generate content on its own, it actually operates based on existing content. In other words, content creation cannot disappear entirely. The real question is how that creation will be rewarded and sustained within a new structure. Until there is a clear answer to this, the current conflicts will inevitably continue.
Ultimately, all of the debates explored in this article converge on a single question: who owns content in the AI era, and who should receive the value generated from it? This is not just a technical discussion—it is a defining question that will shape the future of the web.
From here, the next stage of discussion naturally follows. What strategies should we adopt in this transformation? How should personal bloggers, developers, and content creators respond to this structural shift? And is it possible to create new forms of connection that do not rely on platforms? In the next article, we will explore these questions in greater detail, focusing on what it truly means to create content in the age of AI.
