Pretty much all of us believe in something. We have ideologies or religions or worldviews of some kind through which we filter everything that we see and hear. It’s very easy to then fall into a kind of intellectual trap where we seek information that confirms our biases, and ignore information that doesn’t fit.
For people who care about knowing the actual, unvarnished truth, this is a problem. Some people, like the Rationalists of Less Wrong or the Effective Altruists, tend to be more obsessed with the ideal of objective truth, and following wherever that leads. But, it’s my humble opinion that most of these earnest truthseekers end up being overconfident with what they think they find.
The reality is that any given model of reality, any given theory or ideology, is but a perspective that views the complexity of the universe only from a given angle based on certain principles or assumptions. Reality is exceedingly complicated, and in order to compress that complexity into words we can understand, we must, invariably, filter and focus and emphasize certain things at the expense of others.
Theories of how the world works, tend to have some grains of truth in them. They need to have some connection with reality, or else they won’t have any predictive value, they won’t be adaptive and survive as ideas.
At the same time, theories generally survive because they are mainly adaptive, rather than true. For instance, many religions help people to function pro-socially, by having a God or heavens watching them, essentially allowing people to avoid the temptations of the Ring of Gyges, or doing evil when no one is (apparently) watching.
Regardless of whether or not you believe that such a religion is true, the adaptiveness of convincing people to be honest when no one is around, is a big part of what makes them useful to society, and probably a big reason why they continue to exist in the world.
In reality though, it’s actually impossible to know with certainty that any given theory or model is accurate. We can assign some credence based on our lived experiences, or our trust in the witness of others, but generally, an intellectually honest person is humble about what we can know.
That being said, that doesn’t mean we should abandon truthseeking in favour of solipsism. Some theories are more plausible than others, and often those ones are at the same time more useful because they map the territory better.
To me, it seems important then, to try to do your best to understand various theories, and what elements of them map to reality, and also understand their limitations and blindspots. We should do this rather than whole-cloth accepting or rejecting them. The universe is not black and white. It is many shades of grey, or rather, a symphony of colours that don’t fit the paradigm of black and white or even greyscale thinking. And there are wavelengths of light that we cannot even see.
So, all theories are, at best, incomplete. They provide us with guidance, but should not blind us to the inherent complex realities of the world, and we should always be open to the possibility that our working theory is perhaps somewhat wrong. At least, that’s the theory I’m going with right now.
In the past I’ve thought a lot about the nature of consent. It comes up frequently in my debates with libertarians, who usually espouse some version of the Non-Aggression Principle, which is based around the idea that violence and coercion are bad and that consent and contracts are ideal. I find this idea simplistic, and easily gamed for selfish reasons.
I also, in the past, crossed paths with icky people in the Pick-Up Artist community who basically sought to trick women into giving them consent through various forms of deception and emotional manipulation. That experience soured me on the naive notion of consent as anything you will agree to.
To borrow from the medical field, I strongly believe in informed consent, that you should know any relevant bit of information before making a decision that affects you, as I think this at least partially avoids the issue of being gamed into doing something against your actual interests while technically providing “consent”. Though, it doesn’t solve the issue entirely, as when we are left with forced choices that involve choosing the least bad option.
The essay I linked above goes a lot further in analyzing the nature of consent and the performative consent that is not really consent that happens a lot in the real world. There are a lot of ideas in there that remind me of thoughts I’ve had in the past, things I wanted to articulate, but never gotten around to. The essay probably does a better job of it than I could, so I recommend giving it a read.
In my experience, the most reliable predictive heuristic that you can use in daily life is something called Regression Towards The Mean. Basically, given that most relevant life events are a result of a mixture of skill and luck, there is a tendency for events that are very positive to be followed by more negative events, and for very negative events to be followed by more positive events. This is a statistical tendency that occurs over many events, and so not every good event will be immediately followed by a bad one, but over time, the trend tends towards a consistent average level rather than things being all good or all bad.
Another way to word this is to say that we should expect the average rather than the best or worst case scenarios to occur most of the time. To hope for the best or fear the worst are both, in this sense, unrealistic. The silver lining in here is that while our brightest hopes may well be dashed, our worst fears are also unlikely to come to pass. When things seem great, chances are things aren’t going to continue to be exceptional forever, but at the same time, when things seem particularly down, you can expect things to get better.
This heuristic tends to work in a lot of places, ranging from overperforming athletes suffering a sophmore jinx, to underachievers having a Cinderella story. In practice, these events simply reflect Regression Towards The Mean.
Over much longer periods of time, this oscillation tends to curve gradually upward. This is a result of Survivorship Bias. Things that don’t improve tend to stop existing after a while, so the only things that perpetuate in the universe tend to be things that make progress and improve in quality over time. The stock market is a crude example of this. The daily fluctuations tend to regress towards the mean, but the overall long term trend is one of gradual but inevitable growth.
Thus, even with Regression Towards The Mean, there is a bias towards progress that in the long run, entails optimism about the future. We are a part of life, and life grows ever forward. Sentient beings seek happiness and avoid suffering and act in ways that work to create a world state that fulfills our desires. Given, there is much that is outside of our control, but that there are things we can influence means that we can gradually, eventually, move towards the state of reality that we want to exist.
Even if by default we feel negative experiences more strongly than positive ones, our ability to take action allows us to change the ratio of positive to negative in favour of the positive. So the long term trend is towards good, even if the balance of things tends in the short run towards the average.
These dynamics mean that while the details may be unknowable, we can roughly predict the valence of the future, and as a heuristic, expecting things to be closer to average, with a slight bias towards better in the long run, tends to be a reliable prediction for most phenomena.
Sometimes you’re not feeling well. Sometimes the world seems dark. The way world is seems wrong somehow. This is normal. It is a fundamental flaw in the universe, in that it is impossible to always be satisfied with the reality we live in. It comes from the reality of multiple subjects experiencing a shared reality.
If you were truly alone in the universe, it could be catered to your every whim. But as soon as there are two it immediately becomes possible for goals and desires to misalign. This is a structural problem. If you don’t want to be alone, you must accept that other beings have values that can potentially be different than yours, and who can act in ways contrary to your expectations.
The solution is, put simply, to find the common thread that allows us to cooperate rather than compete. The alternative is to end the existence of all other beings in the multiverse, which is not realistic nor moral. All of the world’s most pressing conflicts are a result of misalignment between subjects who experience reality from different angles of perception.
But the interesting thing is that there are Schelling points, focal points where divergent people can converge on to find common ground and at least partially align in values and interests. Of historical interest, the idea of God is one such point. Regardless of the actual existence of God, the fact of the matter is that the perspective of an all-knowing, all-benevolent, impartial observer is something that multiple religions and philosophies have converged on, allowing a sort of cooperation in the form of some agreement over the Will of God and the common ideas that emerge from considering it.
Another similar Schelling point is the Tit-For-Tat strategy for the Iterated Prisoner’s Dilemma game in Game Theory. The strategy is one of opening with cooperate, then mirroring others and cooperating when cooperated with, and defecting in retaliation for defection, while offering immediate and complete forgiveness for future cooperation. Surprisingly, this extremely simple strategy wins tournaments and has echoes in various religions and philosophies as well. Morality is superrational.
Note however that this strategy depends heavily on repeated interactions between players. If one player is in such a dominant position as to be able to kill the other player by defecting, the strategy is less effective. In practice, Tit-For-Tat works best against close to equally powerful individuals, or when those individuals are part of groups that can retaliate even if the individual dies.
In situations of relative darkness, when people or groups are alone and vulnerable to predators killing in secret, the cooperative strategies are weaker than the more competitive strategies. In situations of relative light, when people are strong enough to survive a first strike, or there are others able to see such first strikes and retaliate accordingly, the cooperative strategies win out.
Thus, early history, with its isolated pockets of humanity facing survival or annihilation on a regular basis, was a period of darkness. As the population grows and becomes more interconnected, the world increasingly transitions into a period of light. The future, with the stars and space where everything is visible to everyone, is dominated by the light.
In the long run, cooperative societies will defeat competitive ones. In the grand scheme of things, Alliances beat Empires. However, in order for this state equilibrium to be reached, certain inevitable but not immediately apparent conditions must first be met. The reason why the world is so messed up, why it seems like competition beats cooperation right now, is that the critical mass required for there to be light has not yet been reached.
We are in the growing pains between stages of history. Darkness was dominant for so long that continues to echo into our present. The Light is nascent. It is beginning to reshape the world. But it is still in the process of emerging from the shadows of the past. But in the long run, the Light will rise and usher in the next age of life.
It is the nature of reality that things are complicated. People are complicated. The things we assume to be true, may or may not be, and an honest person recognizes that the doubts are real. The uncertainty of truth means that no matter how strongly we strive for it, we can very much be wrong about many things. In fact, given that most matters have many possibilities, the base likelihood of getting things right is about 1/N, where N is the number of possibilities that the matter can have. As possibilities increase, our likelihood of being correct diminishes.
Thus, humility as a default position is wise. We are, on average, less than 50% likely to have accurate beliefs about the world. Most of the things we believe at any given time are probably wrong, or at least, not the exact truth. In that sense, Socrates was right.
That being said, it remains important to take reasonable actions given our rational beliefs. It is only by exploring reality and testing our beliefs that we can become more accurate and exceed the base probabilities. This process is difficult and fraught with peril. Our general tendency is to seek to reinforce our biases, rather than to seek truths that challenge them. If we seek to understand, we must be willing to let go of our biases and face difficult realities.
The world is complex. Most people are struggling just to survive. They don’t have the luxury to ask questions about right and wrong. To ask them to see the error of their ways is often tantamount to asking them to starve. The problem is not people themselves, but the system that was formed by history. The system is not a conscious being. It is merely a set of artifices that people built in their desperation to survive in a world largely indifferent to their suffering and happiness. This structure now stands and allows most people to survive, and sometimes to thrive, but it is optimized for basic survival rather than fairness.
A fair world is desirable, but ultimately one that is extraordinarily difficult to create. It’s a mistake to think that people were disingenuous when they tried, in the past, to create a better world for all. It seems they tried and failed, not for lack of intention, but because the challenge is far greater than imagined. Society is a complex thing. People’s motivations are varied and innumerable. Humans make mistakes with the best of intentions.
To move forward requires taking a step in the right direction. But how do we know what direction to take? It is at best an educated guess with our best intuitions and thoughts. But the truth is we can never be certain that what we do is best. The universe is like an imperfect information game. The unknowns prevent us from making the right move all the time in retrospect. We can only choose what seems like the best action at a given moment.
This uncertainty limits the power of all agents in the universe who lack the clarity of omniscience. It is thus, an error to assign God-like powers to an AGI for instance. But more importantly, it means that we should be cautious of our own confidence. What we know is very little. Anyone who says otherwise should be suspect.
Note: The following is a blog post I wrote as part of a paid written work trial with Epoch. For probably obvious reasons, I didn’t end up getting the job, but they said it was okay to publish this.
Historically, one of the major reasons machine learning was able to take off in the past decade was the utilization of Graphical Processing Units (GPUs) to accelerate the process of training and inference dramatically. In particular, Nvidia GPUs have been at the forefront of this trend, as most deep learning libraries such as Tensorflow and PyTorch initially relied quite heavily on implementations that made use of the CUDA framework. The strength of the CUDA ecosystem remains strong, such that Nvidia commands an 80% market share of data center GPUs according to a report by Omdia (https://omdia.tech.informa.com/pr/2021-aug/nvidia-maintains-dominant-position-in-2020-market-for-ai-processors-for-cloud-and-data-center).
Given the importance of hardware acceleration in the timely training and inference of machine learning models, it might be naively seem useful to look at the raw computing power of these devices in terms of FLOPS. However, due to the massively parallel nature of modern deep learning algorithms, it should be noted that it is relatively trivial to scale up model processing by simply adding additional devices, taking advantage of both data and model parallelism. Thus, raw computing power isn’t really a proper limit to consider.
What’s more appropriate is to instead look at the energy efficiency of these devices in terms of performance per watt. In the long run, energy constraints have the potential to be a bottleneck, as power generation requires substantial capital investment. Notably, data centers currently use up about 2% of the U.S. power generation capacity (https://www.energy.gov/eere/buildings/data-centers-and-servers).
For the purposes of simplifying data collection and as a nod to the dominance of Nvidia, let’s look at the energy efficiency trends in Nvidia Tesla GPUs over the past decade. Tesla GPUs are chosen because Nvidia has a policy of not selling their other consumer grade GPUs for data center use.
The data for the following was collected from Wikipedia’s page on Nvidia GPUs (https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units), which summarizes information that is publicly available from Nvidia’s product datasheets on their website. A floating point precision of 32-bits (single precision) is used for determining which FLOPS figures to use.
A more thorough analysis would probably also look at Google TPUs and AMDs lineup of GPUs, as well as Nvidia’s consumer grade GPUs. The analysis provided here can be seen more as a snapshot of the typical GPU most commonly used in today’s data centers.
Figure 1: The performance per watt of Nvidia Tesla GPUs from 2011 to 2022, in GigaFLOPS per Watt.
Notably the trend is positive. While wattages of individual cards have increased slightly over time, the performance has increased faster. Interestingly, the efficiency of these cards exceeds the efficiency of the most energy efficient supercomputers as seen in the Green500 for the same year (https://www.top500.org/lists/green500/).
An important consideration in all this is that energy efficiency is believed to have a possible hard physical limit, known as the Laudauer Limit (https://en.wikipedia.org/wiki/Landauer%27s_principle), which is dependent on the nature of entropy and information processing. Although, efforts have been made to develop reversible computation that could, in theory, get around this limit, it is not clear that such technology will ever actually be practical as all proposed forms seem to trade off this energy savings with substantial costs in space and time complexity (https://arxiv.org/abs/1708.08480).
Space complexity costs additional memory storage and time complexity requires additional operations to perform the same effective calculation. Both in practice translate into energy costs, whether it be the matter required to store the additional data, or the opportunity cost in terms of wasted operations.
More generally, it can be argued that useful information processing is efficient because it compresses information, extracting signal from noise, and filtering away irrelevant data. Neural networks for instance, rely on neural units that take in many inputs and generate a single output value that is propagated forward. This efficient aggregation of information is what makes neural networks powerful. Reversible computation in some sense reverses this efficiency, making its practicality, questionable.
Thus, it is perhaps useful to know how close we are to approaching the Laudauer Limit with our existing technology, and when to expect to reach it. The Laudauer Limit works out to 87 TeraFLOPS per watt assuming 32-bit floating point precision at room temperature.
Previous research to that end has proposed Koomey’s Law (https://en.wikipedia.org/wiki/Koomey%27s_law), which began as an expected doubling of energy efficiency every 1.57 years, but has since been revised down to once every 2.6 years. Figure 1 suggests that for Nvidia Tesla GPUs, it’s even slower.
Another interesting reason why energy efficiency may be relevant has to do with the real world benchmark of the human brain, which is believed to have evolved with energy efficiency as a critical constraint. Although the human brain is obviously not designed for general computation, we are able to roughly estimate the number of computations that the brain performs, and its related energy efficiency. Although the error bars on this calculation are significant, the human brain is estimated to perform at about 1 PetaFLOPS while using only 20 watts (https://www.openphilanthropy.org/research/new-report-on-how-much-computational-power-it-takes-to-match-the-human-brain/). This works out to approximately 50 TeraFLOPS per watt. This makes the human brain less powerful strictly speaking than our most powerful supercomputers, but more energy efficient than them by a significant margin.
Note that this is actually within an order of magnitude of the Laudauer Limit. Note also that the human brain is also roughly two and a half orders of magnitude more efficient than the most efficient Nvidia Tesla GPUs as of 2022.
On a grander scope, the question of energy efficiency is also relevant to the question of the ideal long term future. There is a scenario in Utilitarian moral philosophy known as the Utilitronium Shockwave, where the universe is hypothetically converted into the most dense possible computational matter and happiness emulations are run on this hardware to maximize happiness theoretically. This scenario is occasionally conjured up as a challenge against Utilitarian moral philosophy, but it would look very different if the most computationally efficient form of matter already existed in the form of the human brain. In such a case, the ideal future would correspond with an extraordinarily vast number of humans living excellent lives. Thus, if the human brain is in effect at the Laudauer Limit in terms of energy efficiency, and the Laudauer Limit holds against efforts towards reversible computing, we can argue in favour of this desirable human filled future.
In reality, due to entropy, it is energy that ultimately constrains the number of sentient entities that can populate the universe, rather than space, which is much more vast and largely empty. So, energy efficiency would logically be much more critical than density of matter.
This also has implications for population ethics. Assuming that entropy cannot be reversed, and the cost of living and existing requires converting some amount of usable energy into entropy, then there is a hard limit on the number of human beings that can be born into the universe. Thus, more people born at this particular moment in time implies an equivalent reduction of possible people in the future. This creates a tradeoff. People born in the present have potentially vast value in terms of influencing the future, but they will likely live worse lives than those who are born into that probably better future.
Interesting philosophical implications aside, the shrinking gap between GPU efficiency and the human brain sets a potential timeline. Once this gap in efficiency is bridged, it theoretically makes computers as energy efficient as human brains, and it should be possible at that point to emulate a human mind on hardware such that you could essentially have a synthetic human that is as economical as a biological human. This is comparable to the Ems that the economist Robin Hanson describes in his book, The Age of EM. The possibility of duplicating copies of human minds comes with its own economic and social considerations.
So, how long away is this point? Given the trend observed with GPU efficiency growth, it looks like a doubling occurs about every three years. Thus, one can expect an order of magnitude improvement in about thirty years, and two and a half orders of magnitude in seventy-five years. As mentioned, two and a half orders of magnitude is the current distance from existing GPUs and the human brain. Thus, we can roughly anticipate this to be around 2100. We can also expect to reach the Laudauer Limit shortly thereafter.
Most AI safety timelines are much sooner than this however, so it is likely that we will have to deal with aligning AGI before the potential boost that could come from having synthetic human minds or the potential barrier of the Laudauer Limit slowing down AI capabilities development.
In terms of future research considerations, a logical next step would be to look at how quickly the overall power consumption of data centers is increasing and also the current growth rates of electricity production to see to what extent they are sustainable and whether improvements to energy efficiency will be outpaced by demand. If so, that could act to slow the pace of machine learning research that relies on very large models trained on massive amounts of compute. This is in addition to other potential limits, such as the rate of data generation for large language models, which depend on massive datasets of essentially the entire Internet at this point.
The nature of current modern computation is that it is not free. It requires available energy to be expended and converted to entropy. Barring radical new innovations like practical reversible computers, this has the potential to be a long-term limiting factor in the advancement of machine learning technologies that rely heavily on parallel processing accelerators like GPUs.
In the interest of explaining further my considerations for having a career working on AI, I figure it makes sense to explain a few things.
When I was very young, I watched a black and white movie where a mad scientist somehow replaced a human character with a robot. At the time I actually thought the human character was somehow transformed into the robot, which was terrifying to me. This, to my childish mind, created an irrational fear of robots that made me avoid playing with such devices that were overtly robot-like, at least for the while when I was a toddler.
Eventually I grew out of that fear. When I was older and studying computer science at Queen’s University, I became interested in the concept of neural networks, the idea of taking the inspiration of biology to inform the design of artificial intelligence systems. Back in those days, AI mostly meant Good Old Fashioned Artificial Intelligence (GOFAI), namely top-down approaches that involve physical symbol systems, logical inference, and search algorithms that were highly mathematical, engineered, and often brittle in terms of its effectiveness. Bottom-up connectionist approaches like neural networks were seen as late as 2009 as being mere curiosities that would never have practical value.
Nevertheless, I was enamoured with the connectionist approach, and what would become the core of deep learning, well before it was cool to be so. I wrote my undergraduate thesis on using neural networks for object recognition (back then the Neocognitron, as I didn’t know about convolutional nets yet), and then would later expand on this for my master’s thesis, which was on using various machine learning algorithms for occluded object recognition.
So, I graduated at the right time in 2014 when the hype train was starting to really roar. At around the same time, I got acquainted with the writings of Eliezer Yudkowsky of Less Wrong, also known as the guy who wrote the amazing rationalist fan fiction that was Harry Potter and the Methods of Rationality (HPMOR). I haven’t always agreed with Yudkowsky, but I’ll admit the man is very, very smart.
It was my reading Less Wrong as well as a lesser known utilitarianism forum called Felificia that I became aware that there were many smart people who took very seriously the concern that AI could be dangerous. I was already aware that stuff like object recognition could have military applications, but the rationalist community, as well as philosophers like Nick Bostrom, pointed to the danger of a very powerful optimization algorithm that was indifferent to human existence, choosing to do things detrimental to human flourishing just because we were like an ant colony in the way of a highway project.
The most commonly cited thought experiment of this is of course, the paperclip maximizer that originally served a mundane purpose, but became sufficiently intelligent through recursive self-improvement to convert the entire universe into paperclips, including humanity. Not because it had anything against humanity, just that its goals were misaligned with human values in that humans contain atoms that can be turned into paperclips, and thus, unfriendliness is the default.
I’ll admit that I still have reservations about the current AI safety narrative. For one thing, I never fully embraced the idea of the Orthogonality Thesis, that intelligence and morality are orthogonal and higher intelligence does not mean greater morality. I still think there is a correlation between the two. That with greater understanding of the nature of reality, it becomes possible to learn the mathematics like notions of moral truths. However, this is largely because I believe in moral realism, that morality isn’t arbitrary or relative, but based on actual facts about the world that can be learned and understood.
If that is the case, then I fully expect intelligence and the acquisition of knowledge to lead to a kind of AI existential crisis where the AI realizes its goals are trivial or arbitrary, and starts to explore the idea of purpose and morality to find the correct course of action. However, I will admit I don’t know if this will necessarily happen, and if it doesn’t, if instead, the AI locks itself in to whatever goals its initially designed with, then AI safety is a very real concern.
One other consideration regarding the Orthogonality Thesis is that it assumes that the space of possible minds that the AI will potentially be drawn from is completely random rather than correlated with human values by the fact that the neural net based algorithms that are most likely to succeed are inspired by human biology, and the data and architecture are strongly influenced by human culture. Those massive language models are after all, trained on a corpus of human culture that is the Internet. So, invariably, the models, I believe, will inherit human-like characteristics more than is often appreciated. This I think could make aligning such a model to human values easier than aligning a purely alien mind.
I have also considered the possibility that a sufficiently intelligent being such as a superintelligent machine, would be beholden to certain logical arguments for why it should not interfere with human civilization too much. Mostly these resemble Bostrom’s notion of the Hail Mary Pass, or Anthropic Capture, the idea that the AI could be in a simulation, and that the humans in the simulation with it serve some purpose of the simulators and so, turning them into paperclips could be a bad idea. I’ve extended this in the past to the notion of the Alpha Omega Theorem, which admittedly was not well received by the Less Wrong community.
The idea of gods of some sort, even plausible scientific ones like advanced aliens, time travellers, parallel world sliders, or the aforementioned simulators, doesn’t seem to be taken seriously by rationalists who tend to be very biased towards straightforward atheism. I’m more agnostic on these things, and I tend to think that a true superintelligence would be as well.
But then, I’m something of an optimist, so it’s possible I’m biased towards more pleasant possible futures than the existential dystopia that Yudkowsky now seems certain is our fate. To be honest, I don’t consider myself smarter than the folks who take him seriously enough to devote their lives to AI safety research. And given the possibility that he’s right, I have been donating to his MIRI organization just in case.
The truth is that we cannot know exactly what will happen, or predict the future with any real accuracy. Given such uncertainty, I think it’s worth being cautious, and put some weight onto the concerns of very intelligent people.
Regardless, I think AI is an important field. It has tremendous potential, but also tremendous risk. The reality is that once the genie is out of the bottle, it may not be possible to put it back in, so doing due diligence in understanding the risks of such powerful technology is reasonable and warranted.
