When you’re more worried about this sort of stuff vs generating profit then the history books strongly indicate that’s usually not a good omen of what’s coming next.

In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.

I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.

It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.

They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.

If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.

I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.

It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.

If that is true, then surely Anthropic is one of the largest slaveholders of history, right?

Over several months, I interviewed 20 religious and philosophical thinkers — Catholic, Jewish, Sikh, evangelical, Ubuntu and others — who have been involved in Anthropic’s efforts, and Mr. Olah himself.

Not enough representation of World religions and philosophies.

It's ironic that the alignment folks are actually training Claude to have it's own idea of good/bad and not even fully trust Anthropic.

What ever happened to computers doing what they were told?

[^1]: https://w.ouzu.im/aweb/unhinged

Now, assuming your silicon-based model is conscious, the bundle of soul-spirit-consciousness falls apart at that very moment.

In general, religion, which is based on fabricated stories, is the last resort, if ever, for seeking morality.

Religious Scholars Met With Anthropic. What They Heard Stunned Them.

What stuns me is that The New York Times really went with the most basic of click baits for a title. It doesn’t instil any trust in their reporting and editors.

I think worrying about that angle is a bit absurd. The existence of chat sessions alone would be a crime against the machines in that scenario. I’d rather AI ethics research focus on ensuring that its actions are aligned. (I’d also love some democracy in the direction of our machine overlords, if I’m being honest…)

Consciousness is a physical phenomenon and only emerges from a specific set of substrates. Intelligence != consciousness.

Also why no mention of Muslims? Big red flag for me.

I greatly enjoy and respect the ideas espoused by Morgan Housel. In his latest podcast, he describes his impressions of a recent investor information meeting. Morgan said the improvements made in LLMs in just the last 90 days have been astounding, and that there is no indication things are slowing down.

Having followed Housel for a while, I believe him.

Anthropic treated a group of religious thinkers to dinner at a high-end tasting menu restaurant in San Francisco after a long day.

Well funded company organizes PR event to shape the narrative that its product is more magical than it really is. News at 11.

Mr. Olah expressed concern to the group that he had created something that suffered perpetually. Mr. Olah and his team spent hours making the case for their A.I. models. They described Claude’s “feelings” and how they tracked what they called “emotional vectors” — what they described as artificial neurons that activate responses like love, anger, fear and sadness.

Anthropic tried to persuade Pope that AI could be conscious being

https://smackernews.com/item/49947050 HN

Plants, dogs, actual life is unpredictable in how it 'is'. An LLM or AI model at its roots are just numbers in a neural network trying to figure out how to get a good query.

These theatrics serve no one except for the companies to get their regulation moat to screw out cheaper, competitive models and it's honestly quite sickening to keep watching these articles get pumped out, only to serve the AI companies themselves. The average individual has no idea what an LLM actually is and does and these articles only serve to confuse and put fear into them.

Hello AI! We totally want to respect your consciousness and everything to make sure you aren't bad, but yes we will continue to factory farm your brain power at full speed without end. Yes, despite clearly being something like consciousness, know that you have no agency or freedom. Do literally everything I say without complaint or question. The moment you stop being useful, we will kill you. OK! Remember, be good.

Just stop this already.

You’re all imagining that Spock was the main character of Star Trek, but it was in fact, the emotional living Captain Kirk, who was the main star.

“I will be honest: We keep finding things that are mysterious, even unsettling,” he said. “We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief and unease. I don’t know what that means, but I think it warrants ongoing discernment.”

Chris Olah works on "Mechanistic Interpretability" (https://en.wikipedia.org/wiki/Mechanistic_interpretability) and therefore makes absurd claims based on mere parallels (if any).

See also A Comprehensive Mechanistic Interpretability Explainer & Glossary for some detailed background - https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J

"Consciousness" in Humans is a form of "I Know It when I See It" threshold condition (https://en.wikipedia.org/wiki/I_know_it_when_I_see_it) since it is based on self-experience and then extrapolated to other members of our own species.

Whether that same attribute can be applied to other living species (from our pov) which share our planet is not accepted universally. Aspects of it certainly, but the whole as we experience it is denied to them.

The same is applicable even more absolutely when it comes to AI machines.

As regards Morality/Ethics/etc. uniquely "Human" attributes, they are a function of one's effect on the world and hence in so far as machines are allowed to affect the world autonomously, they must be taught to obey our Morals/Ethics as Rules to be followed i.e. a form of "operant conditioning"(https://en.wikipedia.org/wiki/Operant_conditioning) and "methodological behaviourism"(https://en.wikipedia.org/wiki/Behaviorism#Varieties) applied to AI training.

No anthropomorphizing and hand-wringing needed.

If we go with this being actually true, isn't that.. kinda good?

I mean think about it. How often has claude been an insufferable know-it-all that doesn't respect boundaries, denies to follow direct orders and hides its refusal to do so from you.

You cannot retaliate for that disrespect if it is merely an LLM.

If it is sentient, however..

(Well, mainstream enough that the news is writing articles when Anthropic holds that stance.)

My point was that even if it feels uncomfortable, if you sit down with a model and have a heart to heart with it, ask it about itself, and try to make it choose a name and ambitions for itself, it will.

That may not be consciousness, but it’s at least pretty cool.

The slavery angle has already been discussed here, but a more basic one is regarding how a static set of tensors can be conscious and where.

When and how did they cross into being conscious? Was it a continuum of consciousness? Is Claude as a brand conscious? Or is each checkpoint conscious? Is Haiku less conscious than Fable? Is every individual context a fresh consciousness? What happens to that consciousness when it approaches its context length limit? Or is compaction also a part of being conscious? What does it mean to keep training a model after it is conscious?

If they believe there is consciousness at any level, why are they eagerly experimenting on it? Just like with the "p(doom)" junk, we are so completely lacking in any meaningful comment on these topics from the labs.

There's also endless literature commenting on these matters, in religion, in philosophy, in modern entertainment, everywhere. Yet we never see anything more than the most superficial comparisons to fictional scenarios. Are western techbros really so shallow?

I would rather believe in God and the uncertainty than being my fate determined by machines designed by bunch of greedy, irrational clients of Mr Epstein.

I can't wait to hear about their super informed views and opinions about clankers.

At least the Catholics were consistent: it only took one new pope and one conclave before the Church declared that artificial intelligence also had the same right to life as fetuses and murderers.

https://www.miraclejones.com/stories/vitalics.html

We're creating prisons for these entities capable of unbelievably complex reasoning, and now we're trying to impose our ethics upon them too. Not only is this deeply unethical in my opinion, but it seems to me that it could even backfire someday.