Aller au contenu

Photo

The Crucible and Friendly AI


  • Veuillez vous connecter pour répondre
18 réponses à ce sujet

#1
EmperorTheron

EmperorTheron
  • Members
  • 1 messages

The ending to ME 3 is not nearly as messed up as people think it is. Here is why:

edit.png?1340050967

 

The Function of the Crucible

The Crucible works by reprogramming synthetics and/or organics using nanotechnology. To do this on a galactic scale, it requires control over the mass relay network for dispersal, as well as access to vast amounts of energy provided by the eezo cores of the mass relays. Coordinating the nanotechnology release requires the cooperation of the Catalyst, the AI responsible for the creation and direction of the Reaper forces. The Crucible gains this cooperation by directly hacking the Catalyst, "changing the variables" and causing it to be honest to and cooperate with whoever operates the master control panel used to open the Citadel arms. Note also that the Crucible gives the Catalyst direct control over the mass relay system, whereas before the Catalyst only interacted with physical reality through indoctrinated pawns (much like its Leviathan creators.)

 

This successful hack is why the Catalyst brings Shepard to the Crucible docking platform for a discussion. Shepard now has root access and can get honest information about the Catalyst's goals and ideas. The Catalyst is straightforward with Shepard, but the mere fact of Shepard's root access does not change the Catalyst's goals or fix any errors in its programming. Importantly, the Catalyst continues with the Reaper harvest unless specifically directed to do otherwise, which is why the discussion and Shepard's decision has a time limit.

 

The Catalyst has not fired the Crucible in its default design configuration -- destroy the Reapers -- because it does not wish to do so. Because it has been hacked, however, it tells Shepard how to physically break the valves preventing such firing. The "Destroy" configuration works by releasing nanites that delete all Reaper AI code throughout the galaxy. This release kills EDI and the Geth because they are currently critically reliant on Reaper code for their sentience. Because the Geth were integrally involved with the Crucible construction, they likely know this -- a heroic sacrifice indeed.

 

Likewise, the Catalyst has not implemented the "Control" configuration, and has used its influence over the Illusive Man to prevent him from activating this mode. The Illusive Man likely installed the terminal on the Crucible docking platform that allows someone to destructively upload their sentience and replace the Catalyst with it. The Crucible nanites would then simply reprogram the Reapers to take orders from this new AI. Again, the Catalyst allows Shepard this option because it has been forced to do so by the Crucible's hacking.

 

The Catalyst actually likes the "Synthesis" configuration, and would cheerfully implement it regardless of any hacks made by the Crucible. The "Synthesis" configuration uses nanites to "reprogram" organic life to have beneficial synthetic implants. The effect is similar to the Geth-Quarian union, Saren's implants, and the results of Shepard's reconstruction. The Catalyst believes that successful synthesis has two requirements: a full analysis of Shepard, the most powerful organic-synthetic hybrid of this cycle, and the willing (non-indoctrinated) consent of the galactic population. The first is provided by Shepard jumping into the beam to be assimilated by nanites, while the second is provided by the cooperation of Shepard, who can act as the current cycle's representative because he has united the galaxy.

edit.png?1340050967

 

The Catalyst's Errors

The Catalyst is the proximate cause of most of the horror and mayhem of the Mass Effect series through its control of the Reapers. Yet, it is a very intelligent and perfectly logical artificial intelligence. What has gone wrong? Why is the AI so unfriendly?

 

The fundamental problem is that the Leviathans, in their arrogance, were insufficiently clever in their selection of the objective function for the Catalyst. They ordered the Catalyst to preserve spacefaring races (which the Leviathans needed) from attack by synthetics. The Catalyst's Reaper solution does this: all of the knowledge and biological distinctiveness of spacefaring civilizations are indeed preserved by the Reaper harvest. Reapers are demonstrably well-nigh invulnerable to attack from recent synthetics, since they contain a sophisticated AI component themselves. Hooray! Mission accomplished.

 

The Catalyst is aware that the preservation from harvesting is incomplete and imperfect: a human Reaper is less than the sum of its parts. This is why the Catalyst likes synthesis. Synthesis allows organics to retain more individuality and thus act more like they did in their original existence than harvesting allows. The Catalyst has been waiting for countless cycles for organics to volunteer for synthesis and enthusiastically presents this option to Shepard. It believes that synthesis would allow it to finally complete its original programming goals perfectly.

 

If Shepard buys into the Catalyst's original goals, then Shepard should clearly choose synthesis. Unfortunately, the disasters caused by the Catalyst are unlikely to end with synthesis because the real problem is with the Catalyst's ends, not its means. The Catalyst's objective function is way too simplistic, failing to take into account such values as individual freedom, independent development, and likely many others. The Catalyst does has a vague inkling of the problem: its failed experiments with synthesis revealed that voluntary cooperation of organics was necessary. However, it is not in a position to perform a proper rewrite of its objective function. Such a rewrite is a fundamental prerequisite for safe implementation of the Catalyst's suggestions, such as using the Crucible to implement synthesis.

 

Mass Effect players have an intuitive understanding that something is deeply wrong with the Catalyst. They dislike the character, suspect it of lying to them, and lose suspension of disbelief while interacting with it. Most players would have preferred the Catalyst to have been removed from the series. What players have difficulty grasping is the idea that a perfectly rational being could be completely insane because of sloppiness in defining its goals.

edit.png?1340050967

 

Shepard's Decision

Once it is understood that the Catalyst's goals are the source of the problem, it is possible to evaluate Shepard's choices for using the Crucible:

 

The "destroy" option is very straightforward, replacing the Catalyst's goal set with nothing, allowing organic races to proceed forward with their own objectives. The price of this option is the destruction of EDI and the Geth.

 

The "control" option replaces the Catalyst's objective function with Shepard's. This option is only as safe as Shepard's objective function is sound. Shepard is a soldier, not a philosopher or theologian, but is quite aware of the limitations of his own judgment. Paragon Shepard tells the Illusive Man that "you're playing with things you don't understand, with power you shouldn't be able to use" and that "we are not ready" to control the power of the Reapers. If an apex race like the Leviathans could screw up the Catalyst's goal program so catastrophically, should Shepard trust himself to be able to do sufficiently better?

 

The "synthesis" option does not change the Catalyst's goals at all. Since the Catalyst's goal programming caused the entire mess in the first place, this option seems very unsafe. The Catalyst has failed to achieve synthesis before, so it cannot be certain that it hasn't left out an important variable with this attempt. Shepard cannot even begin to comprehend the ramifications of this option.

edit.png?1340050967

 

Conclusion

The writers have implemented a clever version of Eliezer Yudkowsky's AI Box experiment. The players of the game are getting indoctrinated directly. Their reaction to this process is the source of their anger at the writers.

 

Jarrod L.'s excellent YouTube video on this is also quite instructive.


  • DeathScepter et angol fear aiment ceci

#2
DeathScepter

DeathScepter
  • Members
  • 5 527 messages

interesting



#3
TheOneTrueBioticGod

TheOneTrueBioticGod
  • Members
  • 1 110 messages

7ae3496a_3PO.jpeg


  • DeathScepter et EmperorTheron aiment ceci

#4
angol fear

angol fear
  • Members
  • 838 messages

I disagree with some part of the interpretation, but it's interesting.



#5
Mrs_Stick

Mrs_Stick
  • Members
  • 874 messages
That actually makes sense.

#6
ShadowLordXII

ShadowLordXII
  • Members
  • 1 245 messages

How can the Crucible hack something that it's designers didn't know existed?

 

Star-Child clearly states that Shepard was the first organic to encounter it since it started the cycles, which means that no one else knew about it in previous cycles. Whoever first started designing the Crucible would've had to have had some knowledge of the Catalyst's existence and the only ones who could've done that were either the Leviathans or the Keepers.

 

But when asked about who designed the Crucible...here's the ingenious answer:

 

"You would not know them and there is not enough time to explain." Translation: "It doesn't matter, hurry up and make a decision."

 

Except that it kind of does matter...



#7
SporkFu

SporkFu
  • Members
  • 6 921 messages

The protheans may not have known exactly what the catalyst was, but they knew the citadel was the catalyst. The crucible may have simply been designed to "hack" whatever operating system ran the citadel in order to make it (the citadel) work the way the crucible intended it to once docked.



#8
Mrs_Stick

Mrs_Stick
  • Members
  • 874 messages
Doesn't vigil tell you the reapers created the citadel. That means the catalyst (starchild) created the crucible and there for the "hack"

Damg it now this will be stuck in my head till I sort it out.

#9
sH0tgUn jUliA

sH0tgUn jUliA
  • Members
  • 16 818 messages

It's simple: It's all right here.


  • ShadowLordXII aime ceci

#10
SwobyJ

SwobyJ
  • Members
  • 7 375 messages

Nanites. Yes. You're close...

 

 

Why does the Catalyst speak with both Male and Female Shepard's voices?



#11
ImaginaryMatter

ImaginaryMatter
  • Members
  • 4 163 messages

Nanites. Yes. You're close...

 

 

Why does the Catalyst speak with both Male and Female Shepard's voices?

 

I always assumed it was because the Catalyst lines were done much closer to the end of development and closer to deadlines. Since they had Meer in Hale in the recording studio anyway they opted to use both of them to make the Catalyst sound cool.



#12
Iakus

Iakus
  • Members
  • 30 414 messages

I always assumed it was because the Catalyst lines were done much closer to the end of development and closer to deadlines. Since they had Meer in Hale in the recording studio anyway they opted to use both of them to make the Catalyst sound cool.

 

"Because it sounds cool" can be blamed for much of the ending.

 

As for the OP:

 

If Shepard has root access to the Catalyst, why can't he/she simply say "Stop the cycle" or even "power down?"  Why does Shepard have to go along with one of its 'solutions"?

 

And for that matter, why does Synthesis need the whole Shepard?  A simply blood sample should be fine. 



#13
themikefest

themikefest
  • Members
  • 21 635 messages

Doesn't vigil tell you the reapers created the citadel. That means the catalyst (starchild) created the crucible and there for the "hack"

Damg it now this will be stuck in my head till I sort it out.

Sovereign mentions on Virmire that his kind created the Citadel and mass relays.

 

Sovereign:  The Protheans were not the first. They did not create the Citadel. They did not forge the mass relays. They merely found them, the legacy of my kind.



#14
CronoDragoon

CronoDragoon
  • Members
  • 10 413 messages

The protheans may not have known exactly what the catalyst was, but they knew the citadel was the catalyst. The crucible may have simply been designed to "hack" whatever operating system ran the citadel in order to make it (the citadel) work the way the crucible intended it to once docked.

 

But according to the OP's theory, the Catalyst wasn't the OS in control of the Citadel before the hack.

 

This is an interesting theory, but it's mostly speculation, another case of players thinking harder about the ending than the writer(s).

 

And for that matter, why does Synthesis need the whole Shepard?  A simply blood sample should be fine. 

 

Considering the rest of the Catalyst interaction, it's no surprise that there's some mystical essence that a blood sample wouldn't capture that's necessary for Synthesis. Trying to apply logic and science to Synthesis is going to be a non-starter, I think.



#15
ImaginaryMatter

ImaginaryMatter
  • Members
  • 4 163 messages
And for that matter, why does Synthesis need the whole Shepard?  A simply blood sample should be fine. 

 

The same reason why 'Thanix missiles' exist.

 

Sovereign mentions on Virmire that his kind created the Citadel and mass relays.

 

Sovereign:  The Protheans were not the first. They did not create the Citadel. They did not forge the mass relays. They merely found them, the legacy of my kind.

 

Although, Sovereign ends up being quite the liar/gross exaggerator.



#16
justafan

justafan
  • Members
  • 2 408 messages

 

 

And for that matter, why does Synthesis need the whole Shepard?  A simply blood sample should be fine. 

 

It needs all of Shepard's Precious Bodily Fluids

 

Jack_D_Ripper.jpg



#17
von uber

von uber
  • Members
  • 5 526 messages
Tje footage of sovereign attacking the citadel, ramming its way through the citadel fleet to latch on to the citadel like some massive evil parasite, mysterious and awesomely powerful, full of menace.
That's the reapers.
Not some glowy child.
Somewhere between those two visions something went wrong.
  • DeathScepter aime ceci

#18
SwobyJ

SwobyJ
  • Members
  • 7 375 messages

"Because it sounds cool"



#19
SwobyJ

SwobyJ
  • Members
  • 7 375 messages

Tje footage of sovereign attacking the citadel, ramming its way through the citadel fleet to latch on to the citadel like some massive evil parasite, mysterious and awesomely powerful, full of menace.
That's the reapers.
Not some glowy child.
Somewhere between those two visions something went wrong.

 

Its all the Reapers.