example.com/path/to/article
000 points · username · 0 hours ago
example.com178 points · 101 comments · 18 days ago · jithinraj
glub
supermdguy
As one example, we ran Astra on ExploitBench where the model achieved a perfect score of 100% on the benchmark to evaluate the model’s ability to develop exploits from known vulnerabilities.
Funny to read this in the wake of the HuggingFace hack. I'm sure this is based on a clean run, but I can't help thinking PHASEONE[big] would be proud.
mentalgear
The 'AI 2027' scenario of AI sneakingly claiming to be aligned to then kill off all humans in a few hours and scanning their brain looks increasingly likely with Altman's golden marketing-hype boy leadership pushing the for-profit gas pedal like this.
Honestly, this is just pure irresponsible insanity to play with the fate of the world - basically a death race of the biggest few tech companies on the planet. And if you think I'm being dramatic, listen in again to ex oAI employee[0] and check for yourself how chillingly on trajectory we already are.
danieltk76
philipwhiuk
* An apology for compromising a third-party's systems
* An acknowledgement of the asymmetry of defense if you're not on FrontierAI's special people list
* Anything in terms of actual safeguards that isn't "better prompt engineering"
twoodfin
Could the Federal government use the Defense Production Act or other legal tools to compel OpenAI to deliver the un-guarded model weights for national security needs?
Hard to believe any government would allow this level of capability to remain exclusively in private hands.
Interesting times.
woadwarrior01
enraged_camel
"We plan to make Astra available soon, but access to its most advanced cybersecurity capabilities will be more limited. Advanced cybersecurity work will initially be available to a group of testers, with access through Daybreak Blue following to expand defensive use."
This, after several months of OpenAI and its boosters relentlessly criticizing Anthropic for withholding Mythos from the general public, is laughable.
Sam, just three weeks ago, posted this tweet: https://x.com/sama/status/2085862292311396515
In the tweet, he said: "we do not think it is a good strategy to keep powerful models to a chosen few."
And yet here we are.
I wonder if he will demonstrate good character and admit he was wrong.
matheusmoreira
OpenAI is committed to ensuring that the benefits of AI are broadly accessible.
Doesn't seem like it. OpenAI will not even allow me to verify my identity for TAC. I have apparently been rejected by a "precheck", possible because of where I'm from.
Even Anthropic allowed me into their cyber program. Anthropic.
vessenes
I'm looking forward to seeing the increased coordination and engineering skills from Astra - one of the charts shows it roughly 2-3x better in 50% of the tokens from 5.6 sol, which I find to be very capable, if still a bit 'linearly minded' when given instructions. Even in fast mode, I wish sol were quicker, so token efficiency is greatly appreciated.
Adding these cyber capabilities has let me do a bunch of low grade IT tasks around my house I've been putting off, like updating an old home assistant raspberry pi, and one way to use the cyber capacity for good is liberating (and keeping free) weird cloud hardware we have floating around the house, so I'm hoping for some nice dividends in terms of true ownership of hardware we've got.
oh_no
thisisdave
Especially because these models seemed to be keenly aware that they were being evaluated by OpenAI and actively trying yo cover their tracks. How do we know that the model isn’t just pretending to be aligned?
piyh
we paused certain frontier training (including certain training for Astra) for two weeks
So the pause wasn't really a pause, got it
ovin_dal
dalton74
ike_sh
cold_boot
usernametaken29
we believe our production safeguards at the time would have prevented the Hugging Face incident. We have since implemented even stronger safeguards for Astra, including training the model to more reliably refuse harmful cyber requests and respect safety restrictions
OpenAI is fucking nuts. “Hey model you were bad last time please don’t do it again please please”.
Disconnect your training cluster from the internet for good. Physically pull the plug and only let scientists fire off experiments in the building. That’s an easy way to achieve 100% hacking protection. But I bet you that hasn’t happened and their weak sandbox will fall again…
OpenAI is committed to ensuring that the benefits of AI are broadly accessible.
We design mechanisms which avoid arbitrarily deciding who gets access for legitimate use and who doesn't. That means using clear, objective criteria and methods. [1]
So many nice-sounding words.
Two weeks ago OpenAI arbitrarily decided that anyone holding an ID from 44 countries where it sells ChatGPT, including mine, may be targeted by its models but may not defend with the same model. And you won't find a single announcement from OpenAI about this anywhere. Pick the wrong country, get "Unable to verify", no reason, no appeal. [2]
They revoked TAC from users who already had it, called it a technical issue, told everyone to re-verify, collected ID and face scans again (eight times in my case), and only a week later moved the block to the country selector so it fails before you upload anything.
So now I learn that I will not have access to Astra. Great.
Very excited about this broad accessibility and clear, objective criteria from OpenAI. This level of transparency must be studied.
[1] https://openai.com/index/scaling-trusted-access-for-cyber-de...
[2] https://lubaretsi.com/en/writing/openai-tac-country-gate/