It's not enough to do the right thing unless you have an unusually good concept of what's right.
The folk conception of morality doesn't constrain behavior enough. People can see themselves as doing the right thing even while they behave ineffectually. Morality, as popularly conceived, is not a principle which leads people to prioritize efforts with a high return on investment. Morality, as we're used to thinking about it, offers no criticism of the unambitious man who is in all other ways virtuous.
Morality, as it should be conceived, tells us to not sit idly when we don't need to rest. Morality, when taken seriously, tells us to more fully employ ourselves in the pursuit of the good. Morality, when distilled to the conceptual potency of self-consistency, is manic to the degree that mania gets shit done.
....
That last paragraph is what I feel like saying to myself when I'm not getting enough done. And maybe it has some truth. But also, maybe the pressing issue of moral progress is not that we need to squeeze our hearts harder to wring out ever last drop of motivation. Rather, the issue we should consider presently is that we now have opportunities before us which are historically great, and yet our shared, historically-adapted sense of obligation doesn't seem to grok that the cognitive investment of evaluating policies can eclipse the muscular investment of working the earth and wiping the tears from each human eye in a K-mile radius.
The historical man of good character didn't have the economic advantages that we do. As Industry has raised our sense of what is possible, so too it should raise our sense of what achieved outcomes are morally sufficient.
It's still not enough to do the right thing unless you have an unusually good concept of what's right. But maybe good outcomes come more from financial consequentialism than from the emotional disposition characterized by mania and desperation and passion and tireless sacrifice. Or maybe that's my stupid immoral heart asking not to be wrung any tighter.
Priorities
I once spent an evening with a demented elderly woman who thought that she was still a hospital administrator. She was very adamant that something needed to be done or someone needed to be called to deal with the clear and present crisis of all the suffering patients. There weren't any patients around her. But she wasn't wrong.
The Religious Cognition Of Atheists
Religions exist across cultures and one explanation for their existence is that humans have cognitive architectures tending to confabulate and preserve the stuff of religions. I think I first heard it from Sapolsky that purifying rituals of religions might have been historically innovated by people with minds tending toward OCD and that the ideas of unseen spirits might have come from people tending toward schizophrenia. Given that some abnormal behaviors and beliefs are historically innovated, to understand the existence of religions we then need to look to more neurotypical incentives for why religious content is widely culturally adopted. Maybe individuals have needs for tribal culture which religious content provides to them, or maybe some religion-stuff is the right amount of weird and emotionally compelling to give people a sense of insightfulness or personal meaning, or maybe some religion stuff is mememtically virulent (which is the mind projected way of saying that that humans copy and repeat some things even without seeing value of doing so), or maybe part of the reason for individual adoption of religion is gullibility or acceptance of social authority, and probably other stuff.
So maybe the genesis of a religion proceeds like this: innovation of compelling nonsense by somewhat abnormal persons followed by pragmatic adoption by neurotypical people who are in need of some emotionally compelling weirdness.
This is a nice narrative, but I think it's too easy on the normies. Watching people talk about their religion (or politics) is almost as terrifying to me as watching people who have been confabulated by dementia or traumatic brain injuries. Neurotypical religious thought might be normal, but it is not sound. The fuel which feeds the fire of religion isn't "pragmatic interest in communal narrative and ritual" or anything nearly so rational, it's the fact that people are cognitively misshapen and dysfunctional, universally, pathetically.
If the scary, sad, gross cognitive failures of neurotypical religious thought are human universals, as religion is a human universal, then we atheists should not be so proud as to think ourselves spared. We have the same disease, even if some of our symptoms have cleared.
This in mind, I wanted to identify some of the failures of human cognition which are associated with religions cross-culturally. Maybe I should have just started with a list of biases from Kahneman and Tversky, but that's not what I did.
One failing of religious thought is compartmentalization, which is mostly to say that people practicing a religion are unable to scrutinize the beliefs of the religion in a way that can successfully produce doubt. Questions of internal consistency can be raised, but somehow these questions don't seem to bear much on whether the person will continue to retain the conceptual edifice as a whole. Further, the edifice can be maintained in the mind with almost no explanatory or predictive connection to reality. For many people, "the suffering observed in the world" doesn't seem to enter the radar of "facts which might challenge a belief in a beneficent god". The philosophical discourses on the question of evil, plentiful as they may be, are not enough to discount this huge daily disconnect between the beliefs and sense data available to neurotypical religious people. I propose that any topic which you think would be controversial to bring up in polite multicultural company, such as cryonics or antinatalism or artificial singletons in my own case, are worth special scrutiny, of the sort that could successfully produce doubt, should the topic merit doubt. If this doesn't sound like an immense epistemic burden, think about the level of reflective scrutiny that it would take for a religious person to abandon their faith in the absence of new social incentives.
So much as the attribution of sacredness to a thing is associated with an unwillingness to quantify the value of the thing or to contemplate trades of the thing against mundane things, all sacredness attribution is a kind of compartmentalization and is deserving of special scrutiny. Maybe you know the dollar value of a human life, but there's probably something out there which you value emotively but still treat in an economically incoherent way. Spooky stuff, right? Someone get Night Vale on the line.
I want to write up the other religion-associated cognitive failings that I identified tonight in long-form, but it's getting late and I have a thing to do tomorrow, so in the interest of posting something at all, which has not been my strength, I'm going to paste the remainder of my twitter tweets on the subject with very little editing.
Another cognitive failing of religious thought is overestimating the causal power of categorization judgments? Or, no, I want to identify something more general than that. What's going on in humans such that they think prayer is a thing worth doing? We overestimate the power of linguistic utterances? That covers more behavior but it's less cognitively abstract. Maybe the core failure is that humans expect reality to condition its behavior socially on our social displays. This is kind of like attributing animacy widely to reality.
For the next failure of religious cognition that I want to talk about, consider divination, numerology, omens, and delusions of reference. Part of what's going on with these is an expectation of a natural cosmic order where the world is self-similar or self-describing, maybe? Divination and numerology are schools of thought for deriving predictions from principles that (the practitioners believe) operate throughout the universe structurally (like a folk notion of physics, but phrased directly in terms of the concepts and objects of daily experience, instead of being phrased in terms of particle-field-shaped math). So that's interesting.
"Reality is self-similar" is, I think, a good guess; it feels close to one of the mistaken inductive principles which is common to lots of religious cognition cross-culturally, but it's also not quite the principle that I want to identify. Maybe the problem with it is that divination isn't just like reading the structure of the heavens in a spilled drop of cola, it's not just seeing the self-similar structure, but it also has a social component of requesting information before possibly receiving it. Like religious people think that the cosmos knows things and may choose to reveal info them if it thinks that they're ready? And the rituals and superstitions of divination are like a cargo cult for getting the attention of that Knowing Natural Order? Does that make sense? When people look for spiritually guiding signs in their daily lives, those signs are like communicative events to them. I think that makes sense. But to the degree that I see personally significant messages in the mundane events of my life, those signs don't really feel communicated to me, so maybe "there is a knowledgeable natural order which has a plan for you" isn't one of the core universal religious mental failings that I need to identify and be vigilant against in myself. I'll keep looking for other magical thinking that might be cognitively upstream of delusions of reference and which plausibly applies to me and other neurotypical people. And the same for prayer, because "expecting reality to respond socially to your social displays" isn't quite as persuasive an account of prayer as I'd like. That was the previous failure that I identified, remember? After compartmentalization and sacredness.
Finally, Zodiacs seem like evidence of another culturally pervasive religion-associated cognitive failure. Maybe the genesis of zodiacs happens like this: people have a general over-reliance on explanations of behavior in terms of stable personality traits, and some people prone to apophenia notice false patterns in who gets stuck with which stable traits (again, which traits the people may not have stably), and then people generally suck at knowing when to abandon named/reified concepts, and so the confabulated patterns of explaining how traits are allocated stay in discussion and eventually become common knowledge.
Nothing in this explanation of zodiacs really rings of magical cognition. Maybe that's because zodiacs aren't all that religion-bound, or maybe it's because I haven't done a good job of finding a supernatural-inductive-bias-like cognitive failing which is upstream of people caring about zodiacs and which is also plausibly at work in my own mind to my detriment. Something something swearing, something something did you know that I hate the Myers–Briggs Type Indicator?
That's the end for tonight, folks. Be good to yourselves.
Edit 1: I just heard about thought-fusion beliefs as studied in abnormal psychology. One example is a person thinking that {an event is more likely to happen because they've been imagining the event happening}. Another class of thought-fusions beliefs is like when a person feels that it's roughly as blameworthy to perform a socially prohibited action as it is to imagine doing so. Are those two examples of religion-like cognition? I think they might be. I think thought fusion might be a kind of widely operating inductive bias which partly explains the cross-cultural genesis and retention of religious content and which also might afflict the minds of atheists. Rad. And maybe the first of those two thought fusion beliefs, where motivated focus on a topic alters the expectation that it will happen, maybe that's also part of the explanation of prayer. Prayer is a way to convince yourself of a desirable fiction through motivated focus. I like that explanation a lot. It's definitely something I can be on the guard against in myself.
Edit 2: I would like to add "believing things because they sound cool or beautiful" as a universal human cognitive failing that manifests equally in religious and non-religious people, and which might support the generation and adoption of religions cross culturally. I think that's an important one and it deserves a short name so that it's easier to call up from the depths of memory. I hesitate to call it a kind of optimism bias, because some things that sound cool are very pessimistic.
Perhaps nearby to "believing things because they sound cool or beautiful", there's a bias of believing things which flatter our self image in ways that we can believe. "I'm the most beautiful man on earth" falls flat as flattery, but "I'm one of a chosen people" is vague enough that everyone can believe it of themselves and not be disturbed that the tribe on the other side of the river also believes it. So that's one more thing to guard against.
I'm wondering now if my purpose in writing this post, deep down, was to find cognitive biases that I could want to be vigilant against. A lot of the Tversky and Kahneman stuff has a ring of either "this won't replicate" or "this is only called a bias when it happens to lead to bad outcomes, and isn't architecturally unsound", and it's hard to care about those. By finding biases that I can associate with religion, maybe I'm just trying to motivate myself to be unlike a negative role model. Is that bad? Is that what one of my negative role models would do? I don't know.
Edit 1: I just heard about thought-fusion beliefs as studied in abnormal psychology. One example is a person thinking that {an event is more likely to happen because they've been imagining the event happening}. Another class of thought-fusions beliefs is like when a person feels that it's roughly as blameworthy to perform a socially prohibited action as it is to imagine doing so. Are those two examples of religion-like cognition? I think they might be. I think thought fusion might be a kind of widely operating inductive bias which partly explains the cross-cultural genesis and retention of religious content and which also might afflict the minds of atheists. Rad. And maybe the first of those two thought fusion beliefs, where motivated focus on a topic alters the expectation that it will happen, maybe that's also part of the explanation of prayer. Prayer is a way to convince yourself of a desirable fiction through motivated focus. I like that explanation a lot. It's definitely something I can be on the guard against in myself.
Edit 2: I would like to add "believing things because they sound cool or beautiful" as a universal human cognitive failing that manifests equally in religious and non-religious people, and which might support the generation and adoption of religions cross culturally. I think that's an important one and it deserves a short name so that it's easier to call up from the depths of memory. I hesitate to call it a kind of optimism bias, because some things that sound cool are very pessimistic.
Perhaps nearby to "believing things because they sound cool or beautiful", there's a bias of believing things which flatter our self image in ways that we can believe. "I'm the most beautiful man on earth" falls flat as flattery, but "I'm one of a chosen people" is vague enough that everyone can believe it of themselves and not be disturbed that the tribe on the other side of the river also believes it. So that's one more thing to guard against.
I'm wondering now if my purpose in writing this post, deep down, was to find cognitive biases that I could want to be vigilant against. A lot of the Tversky and Kahneman stuff has a ring of either "this won't replicate" or "this is only called a bias when it happens to lead to bad outcomes, and isn't architecturally unsound", and it's hard to care about those. By finding biases that I can associate with religion, maybe I'm just trying to motivate myself to be unlike a negative role model. Is that bad? Is that what one of my negative role models would do? I don't know.
news_2016-11
On 2016-11-24 I began an effort to learn to enjoy news reports. I'll be updating this post throughout the remainder of November with whatever stories I judge to be something other than ... gosh, I don't know a word that's hateful enough to describe most news reports. Epistemic poison? I probably won't be linking the original articles, because the original articles are seriously not worth sharing beyond the summaries that I'm providing here. Have I mentioned that I do not yet like news reports?
The Economist: Syrian government bombs rebels.
CNN: Japan and South Korea are now sharing military intelligence.
Bloomberg: The Philippine economy is doing poorly.
Bloomberg: In an effort to account for untaxed black market money, the Indian Government demonitizes its largest bank notes and issues a new series of bank notes which can be exchanged for the old ones.
The Guardian: Hurricanes continue to exist in the tropics.
The Guardian: There's a big fire in Haifa, Israel's third-largest city.
The Guardian: Peace negotiations between the Colombian government and FARC, a revolutionary socialist guerilla force, which have been in progress since 2012 are still in progress.
The Guardian: Thousands of Rohingya Muslim refugees fleeing oppressive social conditions in Myanmar since 2015 continue to exist.
NY Times: People continue to kill and die in the Iraqi civil war.
That's it so far. Maybe I should make a list of all the sites that I looked at in order to show how very many fail to host a single story of greater merit than "A person said that they wouldn't like it if a thing happened that wasn't in their interest".
The Economist: Syrian government bombs rebels.
CNN: Japan and South Korea are now sharing military intelligence.
Bloomberg: The Philippine economy is doing poorly.
Bloomberg: In an effort to account for untaxed black market money, the Indian Government demonitizes its largest bank notes and issues a new series of bank notes which can be exchanged for the old ones.
The Guardian: Hurricanes continue to exist in the tropics.
The Guardian: There's a big fire in Haifa, Israel's third-largest city.
The Guardian: Peace negotiations between the Colombian government and FARC, a revolutionary socialist guerilla force, which have been in progress since 2012 are still in progress.
The Guardian: Thousands of Rohingya Muslim refugees fleeing oppressive social conditions in Myanmar since 2015 continue to exist.
NY Times: People continue to kill and die in the Iraqi civil war.
That's it so far. Maybe I should make a list of all the sites that I looked at in order to show how very many fail to host a single story of greater merit than "A person said that they wouldn't like it if a thing happened that wasn't in their interest".
Questions, declarations, demands
Questions are approximately statements of ignorance, but they're better at getting people to respond. Why? Consider:
S1. Declaration of ignorance
Alani: Oh no, here come's Hukov. He's such a prick.
Hukov: Hi, Alani. I don't know how the world would look if participatory consent were a socially overvalued moral construct. Idk if you know either.
Alani: Hukov, I don't care about you or what you know. Go away.
S2. Inquiry
Alani: Oh no, here comes Hukov. He's such a prick.
Hukov: Hi, Alani. What if Jesus were a raisin?
Alani: I ....hm....what if Jesus were a raisin? Let's talk this out.
...
So maybe one reason that questions are more effective than declarations of ignorance in getting an informative response is that questions, by skipping the mention of the speaker and their mental state, give the audience fewer opportunities to decide they're not interested.
Another reason is that questions are kind of like demands for answers, and people have a quick mental reaction that promotes compliance with demands.
If a demand is roughly a statement of the speaker's desire for the audience to perform a behavior, then maybe a question also encodes a little bit of a preference for the audience to respond, and a little bit of an assumption by the speaker that they have social authority to voice their preference with a reasonable expectation that the audience will comply.
Neat.
S1. Declaration of ignorance
Alani: Oh no, here come's Hukov. He's such a prick.
Hukov: Hi, Alani. I don't know how the world would look if participatory consent were a socially overvalued moral construct. Idk if you know either.
Alani: Hukov, I don't care about you or what you know. Go away.
S2. Inquiry
Alani: Oh no, here comes Hukov. He's such a prick.
Hukov: Hi, Alani. What if Jesus were a raisin?
Alani: I ....hm....what if Jesus were a raisin? Let's talk this out.
...
So maybe one reason that questions are more effective than declarations of ignorance in getting an informative response is that questions, by skipping the mention of the speaker and their mental state, give the audience fewer opportunities to decide they're not interested.
Another reason is that questions are kind of like demands for answers, and people have a quick mental reaction that promotes compliance with demands.
If a demand is roughly a statement of the speaker's desire for the audience to perform a behavior, then maybe a question also encodes a little bit of a preference for the audience to respond, and a little bit of an assumption by the speaker that they have social authority to voice their preference with a reasonable expectation that the audience will comply.
Neat.
Habits, Perceived Affordances, and Interrupted Causal Contingencies
D asked me to fix the toilet. The chain that allows the handle outside to tank to raise the flapper within the tank (the toilet's epiglottis) had become detached, and she couldn't figure out how to reattach it. I sat down to examine the internals of toilet tank, closed the valve on the water supply line, and placed my hands into the tank water.
"Eeh! Gosh, that's cold!" I exclaimed.
"Well," D said, "you already thought to close the valve on the water supply line. Why don't you flush the remaining water out of the toilet tank?"
Right! Yes! That's what I ...almost did? Why didn't I do that? I depressed the lever handle, ...
And the toilet didn't flush because the chain that allows the lever handle outside the tank to raise the flapper had become detached. That's the thing that I was trying to fix, remember? Pretty funny.
Or, no, probably not funny, but maybe it's the sort of thing I could write a post about? Yes! Done.
"Eeh! Gosh, that's cold!" I exclaimed.
"Well," D said, "you already thought to close the valve on the water supply line. Why don't you flush the remaining water out of the toilet tank?"
Right! Yes! That's what I ...almost did? Why didn't I do that? I depressed the lever handle, ...
And the toilet didn't flush because the chain that allows the lever handle outside the tank to raise the flapper had become detached. That's the thing that I was trying to fix, remember? Pretty funny.
Or, no, probably not funny, but maybe it's the sort of thing I could write a post about? Yes! Done.
Of Confidence, Understanding, and Conclusions
Confusion and understanding are important concepts in rationality and metaethics. The concepts of conclusions and resolved beliefs are similarly important. These concepts appear in the written works of our culture much as they appear in our daily language: without precise definition or explicit connection to the mathematics of probability theory. One may wonder, what is the connection, if any, between epistemic confidence and understanding, or between confidence and resolved belief? Below I will outline some guesses.
What if like:
You're chatting with a peer. They ask a question and you pause for just a beat before replying. I propose that the condition of your mind during that pause is nearly the same as the mental condition of confusion. The difference is that the condition of a mental pause is almost imperceptively brief, while confusion persists for noticeable and even uncomfortable spans of time. Confusion is a condition where the mind is generating and discarding ideas, having not yet settled on an idea which is good enough to justify ending the search. Confusion doesn't have to be preceded by a surprise, and not all uncertainty is accompanied by confusion. Where in a Bayesian mind do we place confusion? We place it not in uncertainty, and not in surprise, but in the failures of attempted retrieval and production of ideas, theories, strategies. On the time scale of milliseconds, these failures are mental pauses. On the time scale of moments and minutes and more, these failures are called confusion.
So much as the subjective feeling of confusion comes from noticing one's own failure at cognitive production, so contrariwise the feeling of understanding comes from noticing a productive fluency; noticing that one is quickly able to generate ideas, hypotheses, strategies in a domain, and to feel confident that the produced cognitive contents are valuable, and that the value justifies terminating a mental procedure of search or construction.
From the idea of terminating a mental train of thought, we naturally come to the concept of conclusions. I once wrote, "Our beliefs gain the status of conclusions when further contemplation of them is no longer valuable." There are many reasons why you might disvalue further contemplation of an idea! You could have other things which you prioritize thinking about! You could expect that there are no further reasons which could change your belief! You could notice a limit on your reasoning, like an inability to quickly find reasons which would change your belief, while still accepting that such reasons might exist! There could be unpleasant environmental consequences of continuing to contemplate a topic, like if someone didn't want you to think about the thing and they could tell when you were doing so!
So conclusions are merely those beliefs that you maintain when a train of thought terminates, and which you're likely to recall and make future use of. Resolved beliefs are different! I know that "resolved belief" isn't a standard term in the rhetoric of rationality or metaethics, but maybe it should be. Resolved belief is the great promise of maturity! A resolved belief is a conclusion where you have a reasonable expectation that no further reasons exist that would change your mind. The conservation of expected evidence means that something has gone wrong in your mind if you expect that your belief will change some specific way after seeing some unknown evidence, but it is still entirely rational for you to expect that your beliefs will change in some unknown direction after seeing unkown evidence. If you expect that your beliefs will change a lot (in some unknown direction!) that just means you're about to see strong evidence. One way to phrase this situation is to say that the question whose truth value you are contemplating is about to be strongly tested.
I'm not totally sure that updating on lots of bits of evidence is the same as reaching a resolved belief. It seems like a different thing to say, "this issue is probably settled hereafter" than to say, "our inquiry changed our beliefs drastically". If the latter is to say that an idea has been tested, then the former is to say that the idea perhaps can be tested no further: that there is no test the observation of whose result will cause us to update our beliefs. The relationship between these two sounds like something that mathematicians have probably already figured out, either in analyzing converge of beliefs in learning algorithms, or perhaps in logic or metamathematics, where settled conclusions, once tested now untestable, are very salient. Unfortunately, I'm too stupid for real math. Some quantities that seem relevant when making a judgement of whether you've dealt with most of the important possibilities are 1) the conceptual thoroughness of your investigations, per genus et differentiam, 2) the length of time that you've spent feeling confident about the topic and similar topics, and maybe 3) the track record of other people who thought they had investigated thoroughly or who thought that the issues were settled. For some issues, maybe you can brute force an answer by looking at how your beliefs change in subjectively-possible continuations of your universe? That's probably not a topic about which I can think productively.
On the topic of other maybe-relevant math that I'm too stupid to understand, I'm reminded of the idea of the Markov blanket of a variable in a probabilistic causal model. Within the pretense of validity of a given causal model, if you condition your belief about a variable X on the states of a certain set of nearby variables, the Markov blanket of X, then your belief about X shouldn't change after conditioning on the states of any variables outside of the blanket. Unlike in the case of beliefs that are settled by logical proof, I don't think it's the case that the untestable belief in X after having conditioned on its Markov blanket will necessarily be confident. Possibly that's not the case with proof either when you start talking about undecidability and self-reference. I guess the best question my curiosity can pose right now about resolvedness of belief is whether convergence of belief in probabilistic logical induction is related to an independence result like which obtains when you condition on the Markov blanket of a causal variable.
Learning rate adaptation is another math thing in the semantic neighborhood of resolved belief, but it's not a thing Bayesians often talk about, from what I've seen.
But so then:
The feelings of confusion and understanding both accompany our judgements about our individual abilities to quickly produce valuable cognitive contents in a conceptual domain. In familiar domains, we find that some beliefs and valuations, "conclusions", are reliably useful after being reproduced - retrieved or rederived. Confusion and understanding of a deeper kind pertain to a specific kind of conclusion that I've called a resolved belief, somewhat neglecting the notion of resolved valuations. In my terminology, we say that a belief is resolved to the extent that we expect that there are no facts we could learn which would change our minds about the belief. As I have introduced the notion, probability theory does not require that a belief which no longer seems testable need be confident. Still, I have a strong intuition that resolved beliefs and valuations should generally be and will generally be confident, in just the same manner that I become more confident that I know the contours of a face after doing a portrait study. While I think that resolved beliefs will generally be confident, contrarily, it's abundantly clear that one's confidence in a belief or valuation alone cannot justify an expectation that the thing won't change in the future. I don't yet know what theoretically justifies a judgement that an issue is settled, but maybe it has something to do with belief convergence in the mathematics of learning. The concept of metaethically resolved valuation might then be viewed as a variety of resolved belief in the domain of self-characterization. I hope that other philosophical questions can also be translated into terms where the mathematics of learning is clearly relevant, so that philosophical questions can potentially be resolved just as much as can computational ones or physical empirical ones. Resolved belief is the great promise of maturity, and it's high time that we figured out what it means.
Addendum: Hrm, the papers I've seen so far about posterior convergence make it seem like statisticians care about such things for pretty different reasons than I do. The usual situation presented in the literature on convergence of parameter estimates is like, "You get a sequence of percepts of the thing, and the noise has a certain shape. Depending on the shape, we can show how fast your guesses become better." Adapting that to be relevant to confusion, I'd want to frame attempts at cognitive production as noisy percepts of something. That's not the worst idea, but it doesn't seem to capture most of my idea of the source of failures of cognitive production. Maybe the literature on decreasing KL divergence in model selection will help to clear things up.
*distant voice*: Aren't you forgetting about Solomonoff induction?
Oh. Yeah. That's probably the topic I should review first.
What if like:
You're chatting with a peer. They ask a question and you pause for just a beat before replying. I propose that the condition of your mind during that pause is nearly the same as the mental condition of confusion. The difference is that the condition of a mental pause is almost imperceptively brief, while confusion persists for noticeable and even uncomfortable spans of time. Confusion is a condition where the mind is generating and discarding ideas, having not yet settled on an idea which is good enough to justify ending the search. Confusion doesn't have to be preceded by a surprise, and not all uncertainty is accompanied by confusion. Where in a Bayesian mind do we place confusion? We place it not in uncertainty, and not in surprise, but in the failures of attempted retrieval and production of ideas, theories, strategies. On the time scale of milliseconds, these failures are mental pauses. On the time scale of moments and minutes and more, these failures are called confusion.
So much as the subjective feeling of confusion comes from noticing one's own failure at cognitive production, so contrariwise the feeling of understanding comes from noticing a productive fluency; noticing that one is quickly able to generate ideas, hypotheses, strategies in a domain, and to feel confident that the produced cognitive contents are valuable, and that the value justifies terminating a mental procedure of search or construction.
From the idea of terminating a mental train of thought, we naturally come to the concept of conclusions. I once wrote, "Our beliefs gain the status of conclusions when further contemplation of them is no longer valuable." There are many reasons why you might disvalue further contemplation of an idea! You could have other things which you prioritize thinking about! You could expect that there are no further reasons which could change your belief! You could notice a limit on your reasoning, like an inability to quickly find reasons which would change your belief, while still accepting that such reasons might exist! There could be unpleasant environmental consequences of continuing to contemplate a topic, like if someone didn't want you to think about the thing and they could tell when you were doing so!
So conclusions are merely those beliefs that you maintain when a train of thought terminates, and which you're likely to recall and make future use of. Resolved beliefs are different! I know that "resolved belief" isn't a standard term in the rhetoric of rationality or metaethics, but maybe it should be. Resolved belief is the great promise of maturity! A resolved belief is a conclusion where you have a reasonable expectation that no further reasons exist that would change your mind. The conservation of expected evidence means that something has gone wrong in your mind if you expect that your belief will change some specific way after seeing some unknown evidence, but it is still entirely rational for you to expect that your beliefs will change in some unknown direction after seeing unkown evidence. If you expect that your beliefs will change a lot (in some unknown direction!) that just means you're about to see strong evidence. One way to phrase this situation is to say that the question whose truth value you are contemplating is about to be strongly tested.
I'm not totally sure that updating on lots of bits of evidence is the same as reaching a resolved belief. It seems like a different thing to say, "this issue is probably settled hereafter" than to say, "our inquiry changed our beliefs drastically". If the latter is to say that an idea has been tested, then the former is to say that the idea perhaps can be tested no further: that there is no test the observation of whose result will cause us to update our beliefs. The relationship between these two sounds like something that mathematicians have probably already figured out, either in analyzing converge of beliefs in learning algorithms, or perhaps in logic or metamathematics, where settled conclusions, once tested now untestable, are very salient. Unfortunately, I'm too stupid for real math. Some quantities that seem relevant when making a judgement of whether you've dealt with most of the important possibilities are 1) the conceptual thoroughness of your investigations, per genus et differentiam, 2) the length of time that you've spent feeling confident about the topic and similar topics, and maybe 3) the track record of other people who thought they had investigated thoroughly or who thought that the issues were settled. For some issues, maybe you can brute force an answer by looking at how your beliefs change in subjectively-possible continuations of your universe? That's probably not a topic about which I can think productively.
On the topic of other maybe-relevant math that I'm too stupid to understand, I'm reminded of the idea of the Markov blanket of a variable in a probabilistic causal model. Within the pretense of validity of a given causal model, if you condition your belief about a variable X on the states of a certain set of nearby variables, the Markov blanket of X, then your belief about X shouldn't change after conditioning on the states of any variables outside of the blanket. Unlike in the case of beliefs that are settled by logical proof, I don't think it's the case that the untestable belief in X after having conditioned on its Markov blanket will necessarily be confident. Possibly that's not the case with proof either when you start talking about undecidability and self-reference. I guess the best question my curiosity can pose right now about resolvedness of belief is whether convergence of belief in probabilistic logical induction is related to an independence result like which obtains when you condition on the Markov blanket of a causal variable.
Learning rate adaptation is another math thing in the semantic neighborhood of resolved belief, but it's not a thing Bayesians often talk about, from what I've seen.
But so then:
The feelings of confusion and understanding both accompany our judgements about our individual abilities to quickly produce valuable cognitive contents in a conceptual domain. In familiar domains, we find that some beliefs and valuations, "conclusions", are reliably useful after being reproduced - retrieved or rederived. Confusion and understanding of a deeper kind pertain to a specific kind of conclusion that I've called a resolved belief, somewhat neglecting the notion of resolved valuations. In my terminology, we say that a belief is resolved to the extent that we expect that there are no facts we could learn which would change our minds about the belief. As I have introduced the notion, probability theory does not require that a belief which no longer seems testable need be confident. Still, I have a strong intuition that resolved beliefs and valuations should generally be and will generally be confident, in just the same manner that I become more confident that I know the contours of a face after doing a portrait study. While I think that resolved beliefs will generally be confident, contrarily, it's abundantly clear that one's confidence in a belief or valuation alone cannot justify an expectation that the thing won't change in the future. I don't yet know what theoretically justifies a judgement that an issue is settled, but maybe it has something to do with belief convergence in the mathematics of learning. The concept of metaethically resolved valuation might then be viewed as a variety of resolved belief in the domain of self-characterization. I hope that other philosophical questions can also be translated into terms where the mathematics of learning is clearly relevant, so that philosophical questions can potentially be resolved just as much as can computational ones or physical empirical ones. Resolved belief is the great promise of maturity, and it's high time that we figured out what it means.
Addendum: Hrm, the papers I've seen so far about posterior convergence make it seem like statisticians care about such things for pretty different reasons than I do. The usual situation presented in the literature on convergence of parameter estimates is like, "You get a sequence of percepts of the thing, and the noise has a certain shape. Depending on the shape, we can show how fast your guesses become better." Adapting that to be relevant to confusion, I'd want to frame attempts at cognitive production as noisy percepts of something. That's not the worst idea, but it doesn't seem to capture most of my idea of the source of failures of cognitive production. Maybe the literature on decreasing KL divergence in model selection will help to clear things up.
*distant voice*: Aren't you forgetting about Solomonoff induction?
Oh. Yeah. That's probably the topic I should review first.
Subscribe to:
Posts (Atom)