Essays

For a long time, I assumed self help books were simply motivational advice repackaged for each new generation, but the more I looked at them, the more I began to see something else: Self help literature is less about individuals and more about civilizations, and every era produces its own version of the ideal human that society is trying to manufacture at scale and a lot of it can be tied to finding meaning across different periods in history (will write about it soon) ...

A commonly shared image online (not attached here) compares the number of parameters in a large language model to the number of neurons in the human brain, i.e. 86 billion. The intent is probably to illustrate scale, but the analogy is broken at the root, let me tell you why A parameter, which is a connection weight is a static number, frozen the moment inference from a model begins while a neuron is a computational device in itself, connected to thousands of other neurons. Each neuron integrates all its inputs, computes across time, has memory and contains dendrites that act as sub-processors. So the map isn’t just off by a scaling factor, it’s broken at the root as I said earlier, because it compares a living i.e. changing thing to a fixed one. A more careful version of the comparison would map parameters to synapses instead, and even that fails, because a biological synapse is plastic and dynamic. It strengthens, weakens and rewires as the brain computes. A parameter? Just a number at inference ...