Short Thought: Implications of Higher-Order-Thought Consciousness

David: Eliezer, I’ve read Good and Real, agree with you on topics as varied as Everett andBayesian rationalism; but I still don’t “get” your theory of consciousness. For example, a human undergoing a state of blind uncontrollable panic is no more capable of reflective self-awareness or any other form of meta-cognition than a panicking pig. The same neurotransmitter systems, same neurological pathways and same behavioural responses are involved in the panic response in both pigs and humans. So why is the human in a ghastly state of consciousness but the pig is just an insentient automaton?

Eliezer: One person’s modus ponens is another’s modus tollens: I’m not totally sure people in sufficiently unreflective flow-like states are conscious, and I give serious consideration to the proposition that I am reflective enough for consciousness only during the moments I happen to wonder whether I am conscious. This is not where most of my probability mass lies, but it’s on the table. I think I would be equally surprised to find monkeys conscious, or people in flow states nonsentient.

-David Pearce and Eliezer Yudkowsky, on Yudkowsky’s Facebook post, via this blog post. (I’ve highlighted the critical piece in bold.)

This has made me scared about Hansonian ems again. Also some other things.

Because it’s well known that flow states are really, really good at getting good performance, and in some cases also rapid improvement, in skills. (I’m thinking fire spinners, sometimes just referred to generally as flow arts, and musical practice.) And if that’s true, and they are also non-conscious/nonsentient, then even a mild em competition could result in the loss of literally everything that matters. Programming also has flow states, and if ems forked off nonsentient permanent flow-state forks to do further work and they stuck around for economic purposes, then we’ve lost the people.

(To say nothing of the potential issues this raises for hypnosis. Is my boyfriend nonsentient while I play with his head? If I make him forget what happened (very possible), have my actions lost all (consequentialist) moral relevance?)

I give serious consideration to the proposition that I am reflective enough for consciousness only during the moments I happen to wonder whether I am conscious

This on the other hand, is somehow darkly hilarious, because it would suggest that the only suffering that matters in and of itself is existential angst. Destroy Camus! He’s Worse Than Hitler!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s