Essays

We’ve been hearing about a particular kind of promise for a while now and it has started to become more mainstream recently, especially after LLMs. The promise that an artificial entity referred to as super intelligent AI, an upgraded version of artificial general intelligence is a decade if not few years away. One which will have more compute capacity at its disposal than that all of the humanity’s brain processing capacity combined and would not cost us the entire planet to run it. When it’s finally here, the world would be a different place assuming everyone has access to it. Optimists say if widely shared and well governed, it could make humanity more egalitarian by giving everyone access to world class education, healthcare, scientific knowledge and productivity reducing advantage of wealth or geography. But if controlled by few governments or organisations, it could result in concentration of wealth, influence, mass surveillance and may provide decision making power in the hands of few elites. Anyway, we’ll focus mostly on the good part today i.e. the part where everyone becomes a builder, an entrepreneur and the collective productivity capacity of human civilisation multiples, not by little but by orders of magnitude because for the first time in the history as nobody is held back by not knowing how ...

For a long time, I assumed self help books were simply motivational advice repackaged for each new generation, but the more I looked at them, the more I began to see something else: Self help literature is less about individuals and more about civilizations, and every era produces its own version of the ideal human that society is trying to manufacture at scale and a lot of it can be tied to finding meaning across different periods in history (will write about it soon) ...

A commonly shared image online (not attached here) compares the number of parameters in a large language model to the number of neurons in the human brain, i.e. 86 billion. The intent is probably to illustrate scale, but the analogy is broken at the root, let me tell you why A parameter, which is a connection weight is a static number, frozen the moment inference from a model begins while a neuron is a computational device in itself, connected to thousands of other neurons. Each neuron integrates all its inputs, computes across time, has memory and contains dendrites that act as sub-processors. So the map isn’t just off by a scaling factor, it’s broken at the root as I said earlier, because it compares a living i.e. changing thing to a fixed one. A more careful version of the comparison would map parameters to synapses instead, and even that fails, because a biological synapse is plastic and dynamic. It strengthens, weakens and rewires as the brain computes. A parameter? Just a number at inference ...