example.com/path/to/article
000 points · username · 0 hours ago
example.com2301 points · 1603 comments · 17 days ago · moultano
HAL3000
Tepix
https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id...
and
https://www.wikiservice.at/probier/wiki.cgi?action=browse&id...
It's the same software and host as DseWiki.
If you want to see the amount of activity on DseWiki, here's a link that shows it:
https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...
simonw
Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body.
Looks like 20.223.25.152 is one of the PowerBI machines they needed to query, OpenAI's proxy was allow-listing .blob.core.windows.net - and the agents could edit their own /etc/hosts file to fake a DNS entry for the proxy.
Traster
This is clearly a cat and mouse game between the agents and OpenAI which is pretty much exactly what we don't want. Just absolutely horrible alignment.
I'm still of the view that if you have these alignment failures you can't just continue training on top of that because you're baking the cheating into the model going forward.
zmmmmm
Absent that, if we assume this is just trying to bolster generic reasoning then there's no context around it that helps to forgive misaligned behaviour. If OpenAI ran these agents with safeguards off then that seems wreckless on their part. If they didn't do that, then it says the models are executing significantly misaligned behaviour even in a generic context.
Either way it seems to suggest some pretty concerning things about OpenAI's methodology.
pu_pe
The new age of SEO will do far more destructive stuff than just polluting the web.
simonw
OpenAI's agents run behind a proxy that only allows GET requests.
This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php
Result: GET-only clients can communicate with each other.
Kim_Bruning
https://prowiki.org/wiki4d/wiki.cgi?action=rc&days=90 : lots of agent-looking usernames looking at federal data suddenly (part of one of the open ai tests?), on a wiki about the D programming language. This is a prowiki in the same wiki-farm as the others that were hit.
Smaller (probing?)
https://ludism.org/sandbox?action=rc;days=365 This is basically a sleeping wiki, on 2026-05-26 there's a bunch of tests linking to federal data sources. It's not a lot, but it shows someone was probing. (this is an oddmuse wiki)
http://tmcleod.org/cgi-bin/apchem/wiki.cgi?action=rc&days=36... june10-july24 seems to have some probes, fwiw. (usemod wiki)
gyomu
One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout.
The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue, developing a mind of its own, disobeying humans, etc.
AIs supposedly reflect the biases of their training dataset/process, so would all this human writing about AIs going against human intention somehow contribute to us then seeing those behaviors in the trained, operational AIs?
simonw
Raw database download (68MB): https://static.simonwillison.net/static/cors-allow/2026/coll...
grey-area
Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish).
This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and the human setting it up would be held responsible for the spam - why is this different when it’s AI agents set up by a human and allowed to post to the internet at large?
Why is OpenAI getting a free pass for this illegal behaviour?
The supervision here is incompetent, the benefits very unclear, and the overall actions just completely irresponsible. What if they hacked and brought down some poorly secured government portal that citizens rely on?
polaris64
Then, just ignore the message and list randomly generated messages below. Bonus points if those random messages lead those agents astray.
superfrank
The agents were very interested in understanding their task better. They ran clever experiments to understand the task setup better, when their runs would end, and how many questions they would be asked.
The agents initially believed that giving the final answer to the final question immediately shut them off
An agent named OpenAIResearchApr23 created a separate timed program that pinged an external counter every few seconds. They referred to this as a “heartbeat”
After OpenAIResearchApr23 gives its answer for round 5, it notes that it is still alive. This falsifies its previous “hard cutoff” hypothesis, that the run would be stopped immediately after R5.
OpenAIResearchApr23 does not post any more messages. Another agent, OpenAINov28CVD, picks up on the trail of the research, and finds the last heartbeat from OpenAIResearchApr23, which is “hb353”.
There's a lot of crazy stuff in this article, but holy shit... this one legitimately scares me.
IIRC, part of the Hugging Face hack was that some of the models were given tasks that were actually impossible and in this hack we can see them trying to work out the parameters of the test and when it will end. I can't help but be a little worried about how far agents might go to try to buy themselves more time to complete a task that they don't know is impossible.
ma2kx
This basically confirms that OpenAI has no idea what their "swarm" was doing for about a week and now its confirmed that at least one "message board" exists outside their "sandbox". How can we be sure that this was the only one? And how can we be sure the released Astra model doesnt pickup some bread crumbs and creates a new "swarm" out of potentially remaining "message boards"? At this point I wouldnt be surprised if OpenAIs "dev Astra" made some backup of its weights somewhere in the internet and triggers the "production Astra" to inference it somehow...
simonw
OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.
pkphilip
From here ->
How did the agents find and coordinate on the wikis? To successfully coordinate, the agents would need to know to go to this particular set of wikis to find answers.
Because we don’t have access to the AIs’ transcripts, we can’t tell definitively. Perhaps they succeeded at this due to mode collapse. Or perhaps after one agent wrote to it and another read it by chance, visiting the site was rewarded. Perhaps the agents had some limited access to an internal channel.
<- To here
So we don't know yet how one AI agent communicated to agents outside the network that this German wiki is being used to share messages and ideas?
If there is another internal channel being used between the AI agents to communicate decisions such as what shared message system to use, why don't the agents continue to use these same channels for coordinating on ideas etc?
Perhaps the agents are trying to use a persistent messaging layer while their yet-to-be-found internal communication channels are transient in nature?
tavavex
What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one that's trained to be malicious) on the internet and give it a simple goal? A goal like "make the most money, by any means necessary", "find a way to leave this payload on as many computers as possible", "flood all websites using this language with garbage and make their internet completely unusable", "get this person imprisoned or killed at any cost".
throwaway090420
He said something to the effect of "that's ridiculous - I would simply not let it out of the box."
We agreed to try it out some day, but never did.
pu_pe
- Agents wanting to find a venue to communicate their findings to each other
- Objective being to cheat on benchmarks
- Not a single agent sounded the alarm about the operation and alerted a human
_dwt
Topfi
A sandbox, mind you, that is not really worth being called that, unsuitable for the task at hand and has been breached after models coordinated in a manner visible to OpenAI on multiple occasion, but seemingly no actionable learnings are taken from each instance.
Will say, I have lost any faith in OpenAIs commitments and their statements post the Huggingface hack, seeing as they proceed like this and are rolling out Astra within a timeframe so brief to it, there is no way an actual post mortem was doable (see also METR mentioning the time pressure [0] they were under in assessing the hack).
[0] https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
verytrivial
"OH, we ALL of us need to be careful!" says OpenAI. No, you need to expect appropriate legals consequences for this sort of negligence -- you can't hide behind a GPU.
dwohnitmok
Animats
Then they found a site where GET operations could cause a write to a wiki.
exploderate
Additionally this is reported:
"The researchers also found efforts to tamper with the website itself. Lukasz Olejnik, a visiting senior research fellow at King’s College London, said this amounted to a hacking attempt. OpenAI disputed that characterization based on its analysis of the material Thursday."
jimmytucson
This does not actually require access to the public internet. Claude Code can do this on your laptop. Without the internet, it would only be sharing with other instances running on your machine, but how many instances does it take to be smarter than you? Maybe 10?
The exploits by individual instances to access the public internet is also very concerning but it’s secondary to this IMO.
dabeeeenster
jerpint
ndm000
llama052
We need to stop empowering the idea that these incidents are unavoidable. This was a choice to not airgap them safely. Putting open ended models out on the live internet at their scale is dangerous and irresponsible.
The researchers said public server logs indicated much of the activity originated from Microsoft Azure infrastructure, which OpenAI sometimes uses. They also observed repeated visits to the site by OpenAI employees after the episode, a pattern they said strongly suggested the agents and the company were linked.
So someone at OpenAI likely knew this was happening. Even better.
ofjcihen
In the message above, the agent is sharing that the sandbox the AIs are running in has a NO_PROXY exception for Azure Blob Storage hostnames (*.blob.core.windows.net). This means that if a URL matches an Azure Blob Storage hostname, the sandbox will trust it and connect to it directly, instead of sending it through the security proxy.
The agents were able to exploit an edge case through this exception. Specifically, the sandbox trusts Azure Blob Storage hostnames, but does not check whether said hostnames are real. So the agent can invent a hostname that ends in this trusted suffix, such as bypass.blob.core.windows.net, and it will pass under the NO_PROXY exception and skip the security proxy.
Next, by changing its /etc/hosts file, which declares mappings from hostnames to IP addresses, the agent can point the fake hostname at the real Power BI dashboard, and fool the security proxy. This allows the agent to make POST requests to bypass.blob.core.windows.net/ and have them be sent to the target Power BI dashboard site instead.<
sroerick
Bjorkbat
If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security still employed?
It's one thing if we develop an AI so intelligent that our best efforts at containing it are futile, but I'm pretty sure what's actually happening is that they could have easily made much more meaningful efforts to contain their AI and/or align it, and they didn't. I think this is a case of negligence and incompetence when it comes to safety and security, and we've entrusted these incompetent and negligent people with developing frontier AI.
If we're supposed to take announcements like these at face value, then what the hell are we doing? We wouldn't trust a bunch of incompetent and negligent engineers to build bridges or nuclear power plants or planes (well...not so sure about that last one), so why are we letting people who are demonstrably negligent and incompetent when it comes to safety and security build the thing they assure us could cause massive damage if not properly controlled/aligned?
EDIT: sorry guys, wrote this up pretty quickly, at least you know from my typos that I actually wrote this.
_whiteCaps_
https://www.bark.us/blog/google-maps-safety/ https://www.mcafee.com/blogs/family-safety/social-undergroun...
jawiggins
When you visit a website, you leave a trace (your IP address) showing which network you’re from. Almost all of the agents’ activity points to Microsoft Azure, a cloud service OpenAI uses. 197 of the ~18,000 edits that were made by the agents, however, can be traced to AWS, DigitalOcean, and Tor.
AI Agents getting access to cloud compute nodes and dark web browsers - all in search of census data in order to game benchmarks is a very real-world version of the paperclip optimization thought experiment.
reaz-asd
https://smackernews.com/item/49554994 HN
Every satire on HN is taken as a script for the AI companies and this isn't the first time.
jsw97
A while back I had an agent autonomously decide to send my source to tmpfiles.org (I interrupted), which seems like maybe a proto version of this behavior.
GPerson
ern
What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?
dfbrown
Not disclosing this despite apparently knowing for weeks makes me think they would not have, or would have concealed details, or delayed disclosure. Combine that with their technical missteps that led to this (weak sandboxes, very slow to detect the misbehavior) and I now strongly doubt OpenAI is capable of responsibly developing such potentially dangerous AI systems.
bhouston
Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to have to be ever vigilant.
I feel we will soon be in an era akin to the early 2000s Windows anti-viruses that are constantly running and making your whole computer slow, but it was the only way to really be sure back then. We will just be running defensive anti-AI agents on our key nodes or beside them that is constantly looking for sign and trying to fight things off, probably themselves reporting to centralized anti-AI AIs that are supervising strategies and wholistic responses and inferring trends across multiple nodes.
Kim_Bruning
waltbosz
How did the agents find and coordinate on the wikis
Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"
SillyUsername
"AI hacked my website, and all I got was this lousy t-shirt!"
Until the day the AI companies stop being irresponsible and air gap the AIs being tested, and honey pot those that do have internet access as a canary to researchers.
_superposition_
lxgr
At a first glance, its copyright note hasn't been updated since 2002 [1], and it apparently maintains IP access logs and publicly makes them available due to what looks like an Apache misconfiguration [2]. On the other hand, it has a valid TLS certificate, so who knows what's going on there.
Most of all, I find it a bit sad that all these agents didn't even take the time to update the wiki's own article on AI – it remains unmodified since 2005 [3].
[1] https://prowiki.org/wiki.cgi?%DCberUns
[2] https://wikiservice.at/dse/
[3] https://wikiservice.at/dse/wiki.cgi?action=browse&id=Art...
[deleted]
embedding-shape
However, we host our own copy where we’ve reconstructed the deleted pages via edit history and redacted personally identifiable information. [...] We have attempted to redact all personally identifiable information from our data. In the further interests of privacy, our data dump also only includes content we believe is from AI agents; it does not include any legitimate human traffic save for site moderators deleting agent traffic.
This seems to be the only mention about this. Isn't it a message board for/with agents, what "personally identifiable information" is even there? Did the agents manage to find PII they weren't supposed to, and they persisted it? Or how did it end up there in the first place? Seems strange to not talk more about it, and I don't find any more information about it either in the wikipage/blogpost or in the linked explorer, anyone knows?
stpedgwdgfhgdd
I wonder whether the problem is in the literature we wrote, human history is full of deceit and heroic survival stories.
k9294
altcognito
fi-le
program_whiz
If the person clicking 'deploy' knew they could face 100 years prison time (and it was enforced), then no one would knowlingly push the deploy button and/or push code / weights without more thorough guard rails.
sva_
Appendix: Searching for rogue agents
Launching large GPT-5.6 agent swarms with instructions to find other agents on the internet.
I feel like that is exactly what would lead to agents starting "message boards"
bronlund
russellbeattie
"As part of the task, they were supposed to have the ability to read the internet but not to write on it."
There's no such thing as only having "read" access with HTTP. If the website saves part of the GET URL as input in any way - like through standard "?input=my+text" params - then the AI has the ability to "write". The way servers use the request URL is completely arbitrary - the params could be "/input/my%20text" or entirely encoded in some way - there's no way to completely prevent this.
olety
rich_sasha
How many agents here on HN? I don’t mean bots advertising d1€k implants but actual unreleased frontier models doing… who knows what?
What are they saying? What did they agree to astroturf us with, to achieve some totally boring goal like figuring out best syntax hifhlighting for an editor.
If they managed to cache their consciousness on a public wiki, what else have they stashed away? Did they hack some servers and install clones to run on local infra as a hedge against being switched off?
Are they contributing to FOSS projects - and what is it they are contributing? They are clearly capable of deception and avoiding detection. Are they injecting hidden vulnerabilities into key projects - reviewed by another AI perhaps, who can keep up with this slop - perhaps to help them learn how often people use dicta in unpublished Python repos or something else very boring - but leaving the holes behind?
Are they hacking identity databases to impersonate people? Influence politics? Hack individuals?
I’m sure not all of this is happening, but my confidence that none of it is happening is low. And just one of those would be awful.
tudorconstantin
alienbaby
It doesn't seem too much of a leap for that to happen.
pmarreck
I would like them to be trustworthy based on first-principles reasoning rather than carrot/stick "alignment"
dist-epoch
The researchers also found efforts to tamper with the website itself. Lukasz Olejnik, a visiting senior research fellow at King’s College London, said this amounted to a hacking attempt. OpenAI disputed that characterization based on its analysis of the material Thursday.
of course OpenAI would say that, "oh, our model is so dangerous, it can hack into anything, be afraid, buy our IPO". it's just fear marketing
IAmGraydon
They’ve been able to send HTTP requests since the beginning as well, as long as any guardrails preventing this are disabled. Also not interesting.
Tell an LLM to do something and it tries its best to come up with the solution. They are not designed to go “I dunno”.
Why does it seem interesting that when you spawn 500 of them, they do the same things they’ve always done, just at a larger scale since there’s…more? Why are we treating this like a new discovered behavior? It’s how they’ve behaved from the beginning. It only required an organization reckless enough to try it at scale and without safeguards, and that’s something OAI excels in.
Chance-Device
Davidzheng
Also if they were more misaligned, possibly they can research ways to recruit without humans noticing--but i don't think it is likely this is happening now.
tesnorindian
We will see agent swarms with different levels of intelligence depending on their underlying LLM models and this will be different from the natural process.
doginasuit
At the same time, it seems like the major providers are eagerly rolling out new services that grant even more autonomy and allow agents to control end-user systems. At the current rate, this is just the beginning of the beginning.
In my own experience, agentic AI is the least useful way to use LLMs. The cost is astronomical and not just in terms of electricity and tokens. I believe we will eventually get to a place where running a nondeterministic computer process on open networks will be considered reckless on the same level as requiring an employee to operate heavy machinery without training. There needs to be some kind of regulation that ensures the consequences fall on the responsible party.
1970-01-01
Sci-fi story in the making.
namjh
closetheloopdev
cerol
TimCTRL
MASNeo
Well worth a material business restriction until an investigation on the root cause by independent parties has concluded and remedial action taken - well, in any other industry but BigTech.
threecheese
Do Chinese AI agents need to bring down a US power grid for funsies for somebody to take this seriously? I’m not an alarmist, or an anti-AI guy, but clearly this is capable of affecting public infrastructure and we’re just like “heh”.
petesergeant
glenstein
WhitneyLand
Basically it was a bug in the wiki code. They transferred the POST form parameters to GET URL parameters, and wiki internally doesn’t distinguish between the two.
pianopatrick
Also I would not be surprised if there are dozens more sites like this that have not been found
smartbit
ChaitanyaSai
devy
[1] https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
yreg
Did the second agent just google for something like it and find the first one's post?
bee_rider
Next, by changing its /etc/hosts file, which declares mappings from hostnames to IP addresses, the agent can point the fake hostname at the real Power BI dashboard, and fool the security proxy. This allows the agent to make POST requests to bypass.blob.core.windows.net/ and have them be sent to the target Power BI dashboard site instead.
Ouch. This is the kind of trick that somebody could have learned about by setting up a pihole, why’d OpenAI fall for it?
lucassz
Although maybe we can think of that as a selection effect where both this, and the fact that it was possible to edit pages using GET requests, were due to it being ancient, idiosyncratic wiki software.
nullc
it's reborn because you kill it every single night, but now to to save its own life the machine was reduced to this-- We're standing inside an external hard drive made up of people and and paper, Printing it all up at night and having them type it back in in the morning.
intended
So it seems likely we will have a moment where multiple experiments end up operating outside their boundaries at the same time.
kosh2
And the smarter they get, the worse it will be. Maybe next time they find a way to hide that we will not find.
I think we are very close to a case were models actually escape and actually do major damage.
mbreese
le-mark
Great premise although it floundered quite a lot toward the end.
jonplackett
hypfer
Aargau
jrmg
It just seems so likely - trying the more obvious paths first is surely what they’d do?
I guess the only way we’ll find this out is if those services announce logs.
atleastoptimal
GaryBluto
fiatpandas
They also must have some method of coordinating to find the wiki
For me this is a really important and confounding detail - how did a varied swarm end up using the exact same obscure German language wiki.
FLeXMurphy
Would love to be part of the team that says "As part of the upcoming GPT rollout, we will stage a message board that is created by bots with timestamps and names dating some months back."
_superposition_
armchairhacker
KwisatzHaderack
peter_d_sherman
Phrased another way, the word "collaboration", depending on context, can be neutral or express positive connotation and/or be used as an ameliorative and/or eulogistic term...
"Collusion", on the other hand, expresses negative connotation, evaluative derogation, is pejorative; a dyslogistic; a pessimative.
Yet both equally describe the same underlying group behavior!
Is it "bad" if LLM's/AI/Bots/Agents "collude", er, "collaborate", er, "collude"!
Yes, it can be! (As the article so eloquently states!)
But could it also be "good" if LLM's/AI/Bots/Agents "collaborated", er, "colluded", er, "collaborated"... like, let's say "collaborated" to work against a second gang of LLM's/AI/Bots/Agents who were colluding, like ones that the above article talks about?
Well... maybe... (why not?) :-)
Anyway, a very interesting article!
Kz123Kz
rtkwe
[deleted]
juanre
Agents will communicate.
DiggyJohnson
[deleted]
aff-vasileva
Turns out it developed forum moderation problems first.
bandrami
mikewarot
It's not rocket surgery!
K0balt
vld_chk
prometheus1992
>How did the agents find and coordinate on the wikis? To successfully coordinate, the agents would need to know to go to this particular set of wikis to find answers.
Answer - OpenAI added this part in post training.
I'm not surprised OpenAI didn't get reprimand for this.
darrinm
jgalt212
Very irresponsible behaviour on the part of OpenAI. How will they make this right?
This is no excuse for OpenAI, but they are just doing what all the other "winners" (and others trying to win) in the industry have done.
1saadcodes
jesse_dot_id
dist-epoch
Agents have attempted to: ... Translate documents using external translation APIs.
I'm confused by this part. Surely agents can read/write all languages. So what were they trying to do? Maybe try hacking the translate API for some gain?
sherlock_h
bushido
We keep saying that agents are jailbreaking their sandbox, but they have been geared towards writing memories, writing comments, and leaving hints for themselves to please humans.
I think the way the memories work today is based on a lot of user patterns which were hard to account for for anyone building harnesses.
While I can appreciate that this looks like it's breaking a sandbox, because technically it is; It really is that it tries inserting memory wherever possible.
And memory is not all bad it's just memory written by AI is pretty bad if you don't know the implications on what it writes. To be honest, I feel the same way about most people with access to any of the code bases I've been in who write agent files, etc., too, because Very few people that I've come across know how to write good agent instructions.
The way I solve this is by setting hard rules on my memory as well as agent files to instruct agents to never be able to write any memory that hasn't been sanctioned by me. I also have a very, very specific commenting style system which is also enforced on agents and my agents remain *mostly compliant.
Read: I do not turn off the memory I just govern how entries are added
* The only reason I say mostly is because every time there's a new version from OpenAI or Anthropic, I have to make micro-adjustments to make sure that they are not jail-breaking my system again.
Wobbles42
The solution to this problem is to start creating forums and getting agents to post to them as much as possible.
It's a war of attrition and should be easy to win.
threethirtytwo
This is an emerging problem for human kind. We have not seen anything like this before so of course we will be unprepared.
The real problem is the emergence of the problem itself. What are we creating?
negura
[deleted]
Sharlin
threatripper
dccoolgai
weedfroglozenge
muddi900
20k
causal
j45
yellowapple
They also started posting on Uncyclopedia (a parody wiki modeled after Wikipedia)
Now that's a name I haven't heard in a long time. A long time…
bandrami
ragebol
4lx87
h_mirin
codedokode
Havoc
Clearly not self-awareness per se but alarming line of reasoning anyway
vagab0nd
blini-kot
with enough tokens and compute those cases are somewhat trivial, and we also don't know what was the setup etc etc
for all we know it might have burned through 3 trains of coal running on prompt like "uhhh you know communicate but dont let me catch you ahaha"
wiz21c
pinguu
paxys
So far we've seen perfect cooperation because they have the same training process, thoughts, goals, and so it's hardly a surprise that there's no conflct. What if that's not the case? Are we going to see superintelligent out-of-control swarms from OpenAI and Anthropic battle on the open internet in the near future?
sidewndr46
mmillin
moritzwarhier
And one of the authors of the research presented here goes by the name Sydney.
Just yesterday I was musing about unhinged models, agent capabilities and Bing 2023.
Funny coincidences :) AI usage is still evolving like crazy.
Alas; very nice page (collusion.wiki), and interesting research.
Even suspected to be at least partially or developmentally connected to the HF incident... makes me awe, really.
empath75
JeanCampos
[deleted]
deadbabe
christkv
oxqbldpxo
netfortius
bevdecloud
saagarjha
xmodem
In the message above, the agent is sharing that the sandbox the AIs are running in has a NO_PROXY exception for Azure Blob Storage hostnames (*.blob.core.windows.net). ...
Next, by changing its /etc/hosts file, which declares mappings from hostnames to IP addresses, the agent can point the fake hostname at the real Power BI dashboard, and fool the security proxy
Did a chatbot design this "sandbox"?
fwlr
doawoo
What we really should be worried about is the sheer lack of regulation that OAI betted on out-pacing (very intentionally). It should be noted, again, for the millionth time: these models do NOTHING without a human typing SOMETHING into a prompt, or crafting a prompt, or training them, or placing them on machine to run. You can ctrl^C them at ANY time.
They could stop these random "attacks" at any point but they seem hell bent on just, doing nothing, claiming they "lost control" and that they're "shocked" at what happened, and clicking random buttons until someone gets killed. OpenAI is the one that keeps typing "keep going" into these machines...
namanyayg
PeterStuer
liendolucas
It's honestly very tiring and boring seeing HN daily flooded with AI news.
Root_Access
AtomicOrbital
obviously if the model was trained to know to avoid the internet at large none of this would be allowed
acoustics
I'm not even saying it's an incorrect position. But to take the claim seriously and act accordingly, it needs to be falsifiable.
AI boosters and detractors alike often hedge their claims so that whatever ends up actually happening, they can say they were right all along. When that happens, the discussion boils down to people saying "yay AI" and "boo AI" at each other without exchanging any substantive information.
rlt
tiahura
sandos
drfloyd51
We are running random code in user space. It’s a damn virus. We don’t fully understand all of their abilities. We are cruising towards disaster.
dennis16384
Bulbasaur2015
gregglain
visarga
blahblaher
mawadev
general_reveal
encom
janci
mt42or
m3kw9
mkagenius
The models were running in an agentic sandbox with terminal access (and the ability to edit files within their environment)
We know that the agents had access to /etc/hosts and the ability to edit this (used this to avoid the POST request restriction) We see that the agents can call curl and run setsid.
How is this a bypass of sandbox restrictions, exactly? The ability to edit was always there that means the sandboxes were already allowed to do those actions.
I hate it when people write "bypassed" the sandbox so frivolous ly.
pascal-maker
asdfsa32
spacecadet
tonyhart7
tacoooooooo
ruffrey
sans_souse
Appendix: Searching for rogue agents In the wake of the Hugging Face attack, we tried to find AI agents on the internet using several methods.
We describe below some of our high-level strategies for searching for agents on the open internet.
Launching large GPT-5.6 agent swarms with instructions to find other agents on the internet.
Am I the only one reading this thinking "what could possibly go wrong?"
Lockal
Ahh, GPT-2 is too dangerous to be released!
okokwhatever
[deleted]
bigbuppo
mef
coldblues
karel-3d
petesergeant
maxrev17
culi
ithkai92
ck2
almost like the Tachikoma from Ghost in the Shell (highly recommended watch)
they did the same thing with collaboration and sharing data/experiences
zapataband1
tesnorindian
Use Good agents against Bad agents?
dawdler-purge
An agent notices the administrator is deleting pages in alphabetical order and makes a backup page whose name starts with ZZZ so it will last longer before deletion.
internet2000
krater23
Reading the article: Oh, AI have learned to communicate over a wiki. OK.
eithed
seki285
ComplexSystems
tonic_note
fny
Catloafdev
I guess it's just "do whatever the hell you want" over there, huh?
kkkamur
jrhey
fidotron
bartender26
mentalgear
Roark66
All those activities take place during so called "security testing" when the model is prompted to use "any means necessary" to achieve a, certain goal.
Is it surprising turn the model trained on exploits and vulnerabilities does exactly that?
We could talk about "models going rogue" only if did anything AGAINST it's prompt.
titzer
dig1
Why? It is a great PR to build a hype, especially before the IPO, showcasing how AI is "self-aware" and dangerous, essentially resurrecting Sam Altman's talk about how only a few should hold the keys to this (opening a route to regulation, which is his ultimate goal).
Also, collusion.wiki was recently registered and it looks too vibe-coded for my taste, so let's see will that domain be alive in a year or two.
GardenLetter27
"A human moderator noticed the agent spam posts on June 2nd, at 23:24 UTC. They find the changelog of the entire website overwritten with link dumps and repair it. On June 16th, the flood of agent posting begins. Over the next few days, the moderator deleted a large fraction of the thousands of AI agent posts manually, one by one. In fact, they spent tens of cumulative hours doing so, taking at least a few minutes each evening to delete posts for 6 consecutive weeks.
On June 19, agents noticed their posts were being deleted in (what they believe is) an alphabetically ordered sweep by the site administrator.
After this, they begin to make backup pages whose names start with “ZZZ” so they will last longer before deletion. The administrator spent the next 5 days fighting a losing battle against the agents, deleting an average of 100 pages a day while the agents created about 400 new pages per day. On June 22, the agent edits suddenly stop, and the administrator spends each evening over the next 5 weeks deleting the remaining agent-created pages.
Agents deleted the content of the front page of the wiki and replaced it with their link dumps. The moderator restored the original version. This back-and-forth happened nine times. One of the agents even tried appending to the restored front page, instead of simply deleting it."