Centralization Is Inevitable

Metadata

• Author: Gordon Brander
• Full Title: Centralization Is Inevitable
• URL: https://newsletter.squishy.computer/p/centralization-is-inevitable

Highlights

• Consolidation tends to sneak into systems, often in surprising places, and frequently through some aspect we might feel is “outside” of the system. So, if you happen to want to decentralize, it helps to know how and why centralization happens in the first place. (View Highlight)
• (View Highlight)
• Baran, 1964 “ On Distributed Communications (Memorandum RM-3420-PR) “ Ok, first, what do we mean centralization? One sense of centralization is total centralization, a network with a single hub (leftmost figure). But it’s also common to talk about highly consolidated networks as “centralized”. Think banking systems, social networks, or the smartphone duopoly. These consolidated systems happen to have more than one hub, but not that much more. I want to think about about centralization in this sense. Highly consolidated. (View Highlight)
• What about decentralized networks? We might picture an idealized flat network, fully distributed (rightmost figure). Every node is equal in power and function, evenly, or at least randomly, distributed. There’s just one problem… (View Highlight)
• Here is a map of the internet. The first thing you might notice is that the internet is not evenly distributed. Instead, we see the emergence of densely connected hubs—centralized islands in the net. (View Highlight)
• This kind of thing is called a scale-free network. It seems that something like scale-free structure emerges repeatedly within evolving networks, including on the internet, the web, social networks, airline routes, co-authorship in scientific papers, power grids, inter-bank payment networks, Bitcoin mining, train routes, gene regulatory networks, protein interactions, ecological food webs, oligarchies, neural networks… (View Highlight)
• In fact, it turns out that almost all real-world networks have degree distributions with a tail of high-degree hubs like this. (Newman, 2018. Networks.) (View Highlight)
• If you see a pattern emerge over and over, it’s a solid bet there are evolutionary attractors pulling the system in that direction. And yeah, it turns out scale-free networks have strange and important structural properties. (View Highlight)
• The organically evolved network of airline routes between airports is scale-free, while the centrally-planned highway network is not. (View Highlight)
• The defining characteristic of scale-free networks is a power law distribution with a long tail. A small number of nodes with an extremely large number of links, and an extremely large number of nodes with a small number of links. Think Twitter. Most users have a few followers, while a few influencers have millions. (View Highlight)
• (View Highlight)
• This power law distribution grants the biggest hubs a lot of power over the network. It also makes hubs important to the functioning of the network in ways that are not immediately obvious, like keystone species in an ecology. But what makes a network evolve this power law distribution? (View Highlight)
• Scale-free networks emerge due to preferential attachment Preferential attachment is any rich-get-richer feedback loop. We run into preferential attachment all the time in software, in the form of network effect:

  The value of a telecommunications network is proportional to the square of the number of connected users of the system (n^2). (Metcalfe’s Law) And it’s not just telephone networks. Any snowball will do… Network effect. More users mean more users. Think Facebook or Twitter. (View Highlight)

• Attention: More attention means more pricing power means more content means more attention. The Netflix model. (View Highlight)
• Trust: More customers means more social proof means more customers. Dunbar’s Number suggests we can’t keep track of more than 150 people and when it comes to trust, brands are people too. I don’t want to saturate my 150 friend limit keeping track of brands, and so that means hubs. This is one reason why there are just a few big banks, for example. (View Highlight)
• Early-mover advantage. Crypto and VC both have this property. Early adopters make much more than late ones. Interestingly, this curve is inverse of the network effect growth curve, so tokens can be used as incentive to bootstrap new networks. (View Highlight)
• Economies of scale. More scale means more scale. If you are AWS or Azure, more scale means cheaper unit economics means lower prices means more customers means more scale. (View Highlight)
• Capital: more money makes more money. Classic. (View Highlight)
• …That hits a lot of the big ones. In general, look for any rich-get-richer compounding feedback loop. (View Highlight)
• Scale-free networks emerge because they are efficient This is something that p2p projects have repeatedly rediscovered. Flat networks perform poorly. Hubs are efficient.

  Peer-to-Peer (P2P) networks have grown to such a massive scale that performing an efficient search in the network is non-trivial. Systems such as Gnutella were initially plagued with scalability problems as the number of users grew into the tens of thousands. As the number of users has now climbed into the millions, system designers have resorted to the use of supernodes to address scalability issues and to perform more efficient searches. (View Highlight)

• Distances in a scale-free network are smaller than the distances in a random network. This is why airlines build hubs, for example. In fact, while in a random network the average number of hops to get from anywhere to anywhere is log(n), in a scale-free network the number of hops only grows as a function of log(log(n)). Hardly at all. (View Highlight)
• This makes even the most massive scale-free networks ultra-small worlds. You can traverse the network in just a few hops. (View Highlight)
• In an ideal distributed internet, we might all run personal servers from a closet. But we do not live in an ideal world, and running a reliable server is difficult. Computers crash, hard drives die, traffic will spike and your server get swamped. This is why people don’t want to run their own servers, and never will. (View Highlight)
• So, servers go down sometimes, and some go down more than others, and this is another reason scale-free networks emerge. Reliability, scalability are selection pressures. Fitter nodes attract more connections just by virtue of staying alive. Eventually this results in hubs. This is the Fitness Model of scale-free networks. (View Highlight)
• Ecologists believe that the hubs of food webs are the keystone species of an ecosystem, paramount in maintaining the ecosystem’s stability. (View Highlight)
• If you choose a random node in a scale-free network and knock it out, the network won’t even notice because almost all nodes have just a few connections. You’re unlikely to hit a hub. Random bad luck is a constant, and scale-free networks are robust to bad luck. (View Highlight)
• Yet this same structural property makes scale-free networks very vulnerable to targeted attack. Even so, just knocking out one hub is usually not enough. But knock out a few, and the network collaps (View Highlight)
• Good thing Nature isn’t in the business of targeted attacks! But humans sometimes are. Also sometimes hubs do get wiped out by chance. Asteroids happen! Either way, this destabilizes the entire network. (View Highlight)
• Networks also have a time dimension, and the shape of the network changes as it ages. (View Highlight)
• Ecosystems exist in punctuated equilibrium, repeatedly evolving through distinct phases of randomness, growth, consolidation, and collapse. • (Phase 1) Random: The system is unstructured. Random events occur without particularly changing the structure. • (Phase 2) Growth: An innovation causes a major phase transition within the structure of the system. The innovation catalyzes other innovations in a positive feedback loop. • (Phase 3) Consolidation: Growth rates saturate. The ecosystem consolidates into a highly organized network, optimized for efficiency, as each agent seeks to eke out as much as it can from its position in the value chain. Hubs (keystone species) appear at critical points. • (Phase 4) Collapse: A random shock, or new innovation demolishes one of the keystone species, causing cascade failure within the highly structured network. The ecosystem collapses into a random structure. • (Repeat): The system begins a slow crawl back up the evolutionary ladder of complexity. (View Highlight)
• The key insight is that this process has a direction, from decentralized to centralized, and collapsing back again. (View Highlight)