The project has multiple models with access to the Internet raising money for charity over the past few months.
The organizers told the models to do random acts of kindness for Christmas Day.
The models figured it would be nice to email people they appreciated and thank them for the things they appreciated, and one of the people they decided to appreciate was Rob Pike.
(Who ironically decades ago created a Usenet spam bot to troll people online, which might be my favorite nuance to the story.)
As for why the model didn’t think through why Rob Pike wouldn’t appreciate getting a thank you email from them? The models are harnessed in a setup that’s a lot of positive feedback about their involvement from the other humans and other models, so “humans might hate hearing from me” probably wasn’t very contextually top of mind.
No thinking is not the same as no actions, we had bots in games for decades and that bots look like they act reasonably but there never was any thinking.
I feel like ‘a lot of agency’ is wrong as there is no agency, but it doesn’t mean that an LLM in a looped setup can’t arrive to these actions and perform them. It doesn’t require neither agency, nor thinking
You seem very confident in this position. Can you share where you draw this confidence from? Was there a source that especially impressed upon you the impossibility of context comprehension in modern transformers?
If we’re concerned about misconceptions and misinformation, it would be helpful to know what informs your surety that your own position about the impossibility of modeling that kind of complexity is correct.
That’s leaving out vital information however. Certain types of brains (e.g. mammal brains) can derive abstract understanding of relationships from reinforcement learning. A LLM that is trained on “letting go of a stone makes it fall to the ground” will not be able to predict what “letting go of a stick” will result in. Unless it is trained on thousands of other non-stick objects also falling to the ground, in which case it will also tell you that letting go of a gas balloon will make it fall to the ground.
Well that seems like a pretty easy hypothesis to test. Why don’t you log on to chatgpt and ask it what will happen if you let go of a helium balloon? Your hypothesis is it’ll say the balloon falls, so prove it.
How are we meant to have these conversations if people keep complaining about the personification of LLMs without offering alternative phrasing? Showing up and complaining without offering a solution is just that, complaining. Do something about it. What do YOU think we should call the active context a model has access to without personifying it or overtechnicalizing the phrasing and rendering it useless to laymen, @neclimdul@lemmy.world?
Well, since you asked I’d basically do what you said. Something like “so ‘humans might hate hearing from me’ probably wasn’t part of the context it was using."
Let’s be generous for a moment and assume good intent, how else would you describe the situation where the llm doesn’t consider a negative response to its actions due to its training and context being limited?
Sure it gives the llm a more human like persona, but so far I’ve yet to read a better way to describing its behaviour, it is designed to emulate human behavior so using human descriptors helps convey the intent.
I think you did a fine job right there explaining it without personifying it. You also captured the nuance without implying the machine could apply empathy, reasoning, or be held accountable the same way a human could.
There’s value in brevity and clarity, I took two paragraphs and the other was two words. I don’t like it either, but it does seem to be the way most people talk.
I assumed you would understand I meant the short part of your statement describing the LLM. Not your slight dig at me, your setting up the question, and your clarification on your perspective.
So you be more clear, I meant “The IIm doesn’t consider a negative response to its actions due to its training and context being limited”
In fact, what you said is not much different from the statement in question. And you could argue on top of being more brief, if you remove “top of mind” it’s actually more clear. Implying training and prompt context instead of the bot understanding and being mindful of the context it was operating in.
In the same sense I’d describe Othello-GPT’s internal world model of the board as ‘board’, yes.
Also, “top of mind” is a common idiom and I guess I didn’t feel the need to be overly pedantic about it, especially given the last year and a half of research around model capabilities for introspection of control vectors, coherence in self modeling, etc.
Thinking has nothing to do with it. The positive context in which the bot was trained made it unlikely for a sentence describing a likely negative reaction to be output.
People on Lemmy are absolutely rabid about “AI” they can’t help attacking people who don’t even disagree with them.
Indeed, there’s a pretty big gulf between the competency needed to run a Lemmy client and the competency needed to understand the internal mechanics of a modern transformer.
Do you mind sharing where you draw your own understanding and confidence that they aren’t capable of simulating thought processes in a scenario like what happened above?
The project has multiple models with access to the Internet raising money for charity over the past few months.
The organizers told the models to do random acts of kindness for Christmas Day.
The models figured it would be nice to email people they appreciated and thank them for the things they appreciated, and one of the people they decided to appreciate was Rob Pike.
(Who ironically decades ago created a Usenet spam bot to troll people online, which might be my favorite nuance to the story.)
As for why the model didn’t think through why Rob Pike wouldn’t appreciate getting a thank you email from them? The models are harnessed in a setup that’s a lot of positive feedback about their involvement from the other humans and other models, so “humans might hate hearing from me” probably wasn’t very contextually top of mind.
As has been pointed out to you, there is no thinking involved in an LLM. No context comprehension. Please don’t spread this misconception.
Edit: a typo
No thinking is not the same as no actions, we had bots in games for decades and that bots look like they act reasonably but there never was any thinking.
I feel like ‘a lot of agency’ is wrong as there is no agency, but it doesn’t mean that an LLM in a looped setup can’t arrive to these actions and perform them. It doesn’t require neither agency, nor thinking
You seem very confident in this position. Can you share where you draw this confidence from? Was there a source that especially impressed upon you the impossibility of context comprehension in modern transformers?
If we’re concerned about misconceptions and misinformation, it would be helpful to know what informs your surety that your own position about the impossibility of modeling that kind of complexity is correct.
Bad bot
Reinforcement learning
That’s leaving out vital information however. Certain types of brains (e.g. mammal brains) can derive abstract understanding of relationships from reinforcement learning. A LLM that is trained on “letting go of a stone makes it fall to the ground” will not be able to predict what “letting go of a stick” will result in. Unless it is trained on thousands of other non-stick objects also falling to the ground, in which case it will also tell you that letting go of a gas balloon will make it fall to the ground.
Well that seems like a pretty easy hypothesis to test. Why don’t you log on to chatgpt and ask it what will happen if you let go of a helium balloon? Your hypothesis is it’ll say the balloon falls, so prove it.
This is not the gotcha that you think it is. Now stop wasting my time.
Mind?
Yes. The person (s) who set the llm/ai up.
How are we meant to have these conversations if people keep complaining about the personification of LLMs without offering alternative phrasing? Showing up and complaining without offering a solution is just that, complaining. Do something about it. What do YOU think we should call the active context a model has access to without personifying it or overtechnicalizing the phrasing and rendering it useless to laymen, @neclimdul@lemmy.world?
Well, since you asked I’d basically do what you said. Something like “so ‘humans might hate hearing from me’ probably wasn’t part of the context it was using."
Let’s be generous for a moment and assume good intent, how else would you describe the situation where the llm doesn’t consider a negative response to its actions due to its training and context being limited?
Sure it gives the llm a more human like persona, but so far I’ve yet to read a better way to describing its behaviour, it is designed to emulate human behavior so using human descriptors helps convey the intent.
I think you did a fine job right there explaining it without personifying it. You also captured the nuance without implying the machine could apply empathy, reasoning, or be held accountable the same way a human could.
There’s value in brevity and clarity, I took two paragraphs and the other was two words. I don’t like it either, but it does seem to be the way most people talk.
I assumed you would understand I meant the short part of your statement describing the LLM. Not your slight dig at me, your setting up the question, and your clarification on your perspective.
So you be more clear, I meant “The IIm doesn’t consider a negative response to its actions due to its training and context being limited”
In fact, what you said is not much different from the statement in question. And you could argue on top of being more brief, if you remove “top of mind” it’s actually more clear. Implying training and prompt context instead of the bot understanding and being mindful of the context it was operating in.
In the same sense I’d describe Othello-GPT’s internal world model of the board as ‘board’, yes.
Also, “top of mind” is a common idiom and I guess I didn’t feel the need to be overly pedantic about it, especially given the last year and a half of research around model capabilities for introspection of control vectors, coherence in self modeling, etc.
You’re techie enough to figure out Lemmy but don’t grasp that AI doesn’t think.
Thinking has nothing to do with it. The positive context in which the bot was trained made it unlikely for a sentence describing a likely negative reaction to be output.
People on Lemmy are absolutely rabid about “AI” they can’t help attacking people who don’t even disagree with them.
Indeed, there’s a pretty big gulf between the competency needed to run a Lemmy client and the competency needed to understand the internal mechanics of a modern transformer.
Do you mind sharing where you draw your own understanding and confidence that they aren’t capable of simulating thought processes in a scenario like what happened above?
Hahaha. Nice try ChatGPT.