Rendered at 20:07:52 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
roncesvalles 11 hours ago [-]
It's not so much that it can't make a decision, but that whoever allowed it to make decisions is ultimately responsible for them.
Decision laundering is a great term. There is ultimately a human who gave the program free rein. You can't say "sorry my car hit you, it's not my fault" especially when the person driving works for the car manufacturer too.
nico 5 hours ago [-]
> Decision laundering
Which also seems to be in line with the way the frontier labs gathered and used copyrighted data to train their models
And even before AI, limited liability corporations (like LLCs or C corps), also have the intention of protecting the individuals from the repercussions of their actions
In a way this could be generalized as “accountability laundering”. Making someone unaccountable for their actions, by redirecting the responsibility of those actions, to a non-punishable entity (like a C corp or “a computer”)
cainxinth 8 hours ago [-]
These are issues that come up in legal disputes.
The question isn’t who or what made the decision, per se. It’s who is responsible for the outcome. The entity making the decision might be the actual cause but not the proximate cause (the cause most closely connected to legal liability) for the harm that occurred.
JeremyNT 6 hours ago [-]
If I flip a coin to decide whether to do something, I can't blame the coin.
The act of delegation to chance is the decision.
dmillar 3 hours ago [-]
And the free-will-is-an-illusion camp would have you continue to trace that back from why was the act of delegation made (probablistic) to the big bang. Implying that humans can also not make decisions.
11 hours ago [-]
znpy 10 hours ago [-]
> It's not so much that it can't make a decision, but that whoever allowed it to make decisions is ultimately responsible for them.
This. I agree 100%.
Computers make decisions all the time, and have been doing so for decades. But we still keep the human behind the computer accountable.
cheesemoopy 9 hours ago [-]
I know this has to be the case but it’s scary since most of what agents do can be a black box.
_aavaa_ 8 hours ago [-]
It doesn’t matter what kind of box an agent is if the people using it don’t let you know how the decisions was made.
bartnp 8 hours ago [-]
Except now there's no longer any guarantees there's even any people you can point at. It's already often diffuse responsbilitiy. Maybe today you can still sort of get around that by pointing at business/organization owners. But soon we're going to see situations where that isn't even realistic. For example if AI agents start spinning up their own agents which work outside of the control of anyone. We're going to need some way to deal with that.
dasil003 6 hours ago [-]
Agents run on computers, those computers have owners. We have to maintain accountability and not let people who stand to get rich from privatizing the profits but socializing the risk of rogue agents from continuing to push a narrative of autonomous super intelligence that is “too powerful” to control.
That narrative is bullshit. Intelligence is not power. The power comes from hooking these agents up to everything without proper safety controls. The fact that those safety controls don’t exist yet is not humanity’s problem, it’s the frontier labs problem, and we can’t let them wriggle out of it with an Overton window shift or demanding that the government take responsibility via regulation (of course they should but it doesn’t absolve the labs of responsibility for their own actions).
jeremyjh 3 hours ago [-]
Right, but if the agents broke into the computers and are hosting themselves there without the owners knowledge, are the owners responsible? We'd say whoever originated them is responsible, but what if that can no longer be determined?
dasil003 2 hours ago [-]
Yes, like pretty much everything else, there is gray area. We can have split liabilities for different aspects of these failures. It's long been the case that people may not know what's running on their computers (botnets, etc), and I would argue there should be some form liability for your computer being used to do bad things. It should be proportionate and obviously a lot less than the principals. But keep in mind these rogue AI agents can't just hang out on someone's personal hardware, they need a lot of compute.
The missing piece right now is that our AI safety and cybersecurity controls are simply not up to snuff with the current risks that AI has introduced. Given the increased risk, we need to level up every aspect of our cybersecurity stance. These are hard problems, but not impossible given proper incentives. The issue is that no one is connecting the dots between the creation of these agents and the tools and access they are given, with the harm they may produce. Often the argument jumps straight to "it's going to get too smart for us to control" while glossing over the fact that all the power AI has was directly given to it by human choices. All this cognitive dissonance and willful blindness is caused by unchecked capitalist incentives. Think of the amount of money every single person in the frontier labs stands to make if the most optimistic predictions pan out. This is moral hazard of the highest order, and the only counter-measure that can work in our current social and political system is liability.
mmcdermott 3 hours ago [-]
This is also why legal action must be taken against OpenAI for the Hugging Face incident, as well as any similar incidents. Their computer, their programs, their responsibility.
RandomLensman 8 hours ago [-]
Why would that need a new way? Or do you mean that no-one ever spun up anything at all (e.g., agent zero was never spun up or allowed to be spun up by anyone)?
FabHK 10 hours ago [-]
> Decision laundering is a great term.
Related: The Unaccountability Machine (good book by Dan Davies, who also wrote Lying for Money about financial fraud).
corporations are legal persons, and they can make decisions that negatively effect society as a whole, yet often no single person in the corporations is responsible and each individual may not even see themselves as morally responsible
dml2135 7 hours ago [-]
Well that’s because they are legal fictions and often run by morally bankrupt individuals. The people making the decisions are still responsible for them.
Don’t confuse lack of accountability with lack of responsibility.
OroPla 29 minutes ago [-]
Are people responsible for the decisions government representatives they voted for made? How about things like Brexit?
jeremyjh 3 hours ago [-]
Don't confuse lack of legal exposure under the current regime with lack of accountability. A future regime could hold them accountable.
caaqil 11 hours ago [-]
> it's not so much that it can't make a decision, but that whoever allowed it to make decisions
If one needs to be "allowed" to make a decision, does that even qualify as a decision?
OtherShrezzing 10 hours ago [-]
Yes. You can have some free will without having absolute free will.
For example, I can decide which cereal to purchase at the supermarket. But I cannot decide which cereals are offered to me at the supermarket. The supermarket has allowed/enabled to make a constrained decision.
Obscurity4340 8 hours ago [-]
I wonder if theres an approach to getting an arbitrary grocery store or even like an arbitrary pizza chain to offer your favorite dressing/dip, id love to see my fave become more popular/normalized
fwip 5 hours ago [-]
Probably the way to start is to ask the store.
caaqil 10 hours ago [-]
> You can have some free will without having absolute free will.
So does it follow then that the agents in question do have "some" free* will? I'm not sure you realize what conclusion naturally follows from this, and how it contradicts the point made in the article and the top-level comment I replied to.
(*) My earlier question wasn't explicitly about free will but we'll go with it.
OtherShrezzing 9 hours ago [-]
Personally I don’t agree with the article that computers cannot make decisions (or at least, I don’t think the article made any case why I should agree with the statement).
While I agree that computers cannot currently be held accountable for decisions (because I do not think they have free will), I think it’s transparently clear that computers make decisions all the time, and have done since they were first implemented.
user43928 10 hours ago [-]
No one 'gave the program free rein' in any of those examples, did they? Reasonable safeguards were in place.
Many of these incidents relate to hacking, where criminal law commonly requires intent.
Claiming that a machine cannot make a decision does not mean an employee at OpenAI decided to hack an Australian government website.
Leynos 9 hours ago [-]
Inadequate or broken safeguards do not fall under my definition of reasonable.
SpicyLemonZest 9 hours ago [-]
OK, but one of the examples in the source article was a report from third-party testing of those safeguards. I feel like you're starting from an unexplored presumption that adequate safeguards must be easy to build and anyone who doesn't build them is just being lazy.
baruz 5 hours ago [-]
Let’s go over their statement.
> Inadequate or broken safeguards do not fall under my definition of reasonable.
I see no such assumption there. In my reading, the implicit assumption is that adequate safeguards are necessary and that anyone who doesn’t build them is irresponsible. And I feel that that is correct.
SpicyLemonZest 2 hours ago [-]
I don’t understand how to connect this to the facts at hand. Building adequate safeguards requires studying safeguards to understand whether they’re adequate. But the source article is dunking on one of AISI’s attempts to study such safeguards; we should, the author argues, consider whatever behavior the model exhibits to be an explicit decision by whoever provided the model. It seems impossible to build or study safeguards against model misbehavior without accepting that the models can make decisions.
Leynos 5 hours ago [-]
I did not say this shit is easy.
amelius 10 hours ago [-]
AI is irresponsible technology, because we have no idea how it works or how to contain it. It's worse than writing code that suffers from SQL injection vulnerabilities because at least in that case we know how to fix it.
The AI labs say "AI can make mistakes". That's why the responsibility of using it is on you. This may change if we get more legislation. But given the argument above, it will probably never shift to the AI itself having any responsibility. Instead the AI labs can be responsible.
kijashdkujdfhas 7 hours ago [-]
"AI can make mistakes" continues to be the fundamental problem.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
sobiolite 7 hours ago [-]
That's fundamental to how _any_ technology works. If the bar for use is "can never go wrong ever", then you need to throw away every human invention ever made.
kijashdkujdfhas 6 hours ago [-]
The problem is the scope of errors is so wide. It's not reasonable to compare LLM style failures to other technology failures. These 'failures' are not actually failures, they're part of the normal operation of the system.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.
juliushuijnk 9 hours ago [-]
They did.
sobiolite 7 hours ago [-]
Computers can make decisions: Almost everyone reading this will have a implemented an if-else statement or equivalent control flow. That is a decision made by a computer on data it receives at runtime. You can argue that it's different, because we fully understand the logic of such a statement, whereas LLMs are closer to a black-box. But at Turing observed, in practice we are frequently surprised by the output of complex algorithms, even those we have authored ourselves. The more difficult the task, the more complex the algorithm we need to achieve it, and the more unexpected behaviour can emerge from it. In that way, LLMs are just the latest, highest point on a rising line of algorithmic capability and complexity.
We can (and should) try to improve AI safety and alignment, but perfect safety is probably impossible without losing the capabilities that make AI unique. Black-and-white positions that insist anyone creating or using an AI is responsible for anything it does wrong under any circumstances are also unrealistic. AI is a general purpose technology, and we have always accepted that it is not practical for suppliers to envisage and constrain every downstream use: Car manufacturers cannot prevent every instance of mechanical failure, nor can they stop cars being used in crime.
qwery 2 hours ago [-]
What you're missing is that "making decisions" isn't the same as being technically capable of making a selection.
Making decisions means taking responsibility.[0] This is really all there is to it -- a computer cannot decide, not because humans are uniquely able to make a choice based on input, but because a computer cannot be responsible.
You can decide to do something based on the output of the computer, but that's you deciding to do that.
At any point where a computer appears to have made a decision, someone else was behind it. You can complicate this, randomise stuff, obfuscate things, etc. but ultimately either the computer was set in motion by a person, or it malfunctioned. The malfunctioning computer thing is quite interesting but it's not going to change the outcome.
LLMs ("AI" today) have not changed this. It's still just software, it's still doing what it's told. Surprising future technologies may alter this, but there is no evidence whatsoever to suggest that's a likely future and it certainly is not the case today.
> Black-and-white positions that insist anyone creating or using an AI is responsible for anything it does wrong under any circumstances are also unrealistic.
At this point in time, AI is just a piece of software. It doesn't matter how useful it is or not. It's just software. We can and should treat it like software.
So to evaluate when/where/how someone should (or should not) be held responsible for something that "the AI did", we can start at the same point as we would for any other vendor/creator/user of a piece of software.
I'm not going to get into that -- my point is that the framework exists. And yes, sure, maybe that framework needs some tweaks in consideration of the new software, that's OK. The starting point should still be in reality, where software is software.
[0] Yes, this is nuanced, things go wrong, different situations, etc. etc.
OroPla 24 minutes ago [-]
> At any point where a computer appears to have made a decision, someone else was behind it.
Are you arguing that not computers are best at chess, but that whichever human last updated the chess algorithm is actually the reigning chess champion?
goodmythical 2 hours ago [-]
>That is a decision made by a computer on data it receives at runtime.
I disagree. A light bulb is on if and only if it is supplied electricity. With a switch, I have control over input of electricity. The bulb is on if and only if the switch is toggled on. The bulb isn't deciding whether or not to be on based on the if-statement represented by the switch. It just is on or is off depending on conditions.
To make it more dynamic I can replace the switch with a motion sensor. The light is now on if and only if motion is detected. The light is not deciding to be on but must be on as required by the motion sensor toggling the flow of electricity. The light either is or isn't on depending on the conditions.
If I write an if condition that prints true if there are more than ten files in a folder, and then a second that places a random selection of up to 20 files in the folder before deleting them every second, the if function will be true roughly half of the time. The function either is or isn't true based on conditions.
All of the decisions made for and involving both the lights and the if condition were made before they existed. Once implemented, those decisions will remain true until some force acts upon them (later conditions or decisions). While the initial decisions (to wire the light, to write the if condition) are true, the system itself makes no decisions. Either systems current condition depends entirely upon the decisions made before it existed.
Saying an if-condition makes decisions is like saying a bicycle decides to fall down when running out of momentum or encountering an obstacle.
TehCorwiz 6 hours ago [-]
I disagree. Computers cannot make judgemental decisions about course of action, they can only execute predefined courses of action. The software developer made a series of decisions. The computer just carried those out.
txrx0000 4 hours ago [-]
These discussions around decision-making seem to ignore what a "decision" is. How does a human being make a decision? And what exactly is happening when a person decides to do something?
Suppose I check the weather forecast 5 mins before leaving the house, and it says 95% chance of rain today. Do I bring or not bring an umbrella?
I would bring an umbrella. I would made that decision to bring an umbrella. But my decision to do that is (at least partially) the result of the weather forecast. The causal chain is (atmospheric conditions -> weather forecasters -> my decision). Did the weather forecaster just "program" me? Did the water molecules moving in the sky just "program" the weather forecaster?
My answer to these questions is as follows: a decision, and my understanding of the concept/definition of a "decision", refers to exactly the processes where I do something as a result of external influence/programming, rather than just internal thought/processing. I could think really hard and imagine myself moving forward at 100 meters per second, or even beyond the speed of light, but in reality, no matter how hard my neurons fire to imagine that scenario, my actions are still physically constrained. Such physical constraints influence our every decision. That is the "programming" part. Everyone and everything "programs"/influences everything else.
That said, perhaps the particles which make up my brain/body has its own whimsy. Perhaps there is a hidden local random number generator that samples from the distribution of possible future states available to this particular clump of matter that is "I".
How different is this from a computer running a non-deterministic program such as an LLM-based AI agent? Remember, every token sampling involves an RNG. They don't perform as well or feel as creative and human-like when deterministically sampled.
This isn't an excuse for anyone, especially not for Anthropic or OpenAI. But I think these are questions we ought to ask if we are to honestly discuss liability and perhaps even personhood for AIs that are increasingly autonomous.
qwery 2 hours ago [-]
The technical/technological nature of the system that made the choice is irrelevant.
That is, I think it's right to ignore what a decision is, as you put it.
The ability to make decisions is not the ability to choose -- it's the ability, right, privelege, obligation, expectation to take responsibility.
Discussing anything like personhood for AI is really jumping the gun.
The chatbot is software.
The software does what it is told. It performs tasks requested of it by the user/operator in line with the (designed, ) immplemented functionality of the code. If the software fails to do what it is told, the vendor has not provided working software. If an operator controlling some software does not do so in a reasonable, safe manner, the operator is negligent. If damages result from such failures or actions, the people who made the decisions that led to that are the people who are responsible.
It does not matter how complex the software is: if a vendor of some software doesn't expect it fail in costly ways because it's too complex or hard to understand, that is a (legal, ethical, moral) failure of the vendor.
It does not matter how autonomous it is: an autonomous sprinkler system that floods the neighbour's basement is not held liable for the damages. The owner, installer, vendor, etc. is. -- maybe the neighbour even did(n't) something wrong!
moritzwarhier 3 hours ago [-]
In the context of free will (disregarding the full context here for a moment), this is where it actually gets interesting.
Theologists and philosophers alike have been thinking about the problem what "free will" even means exactly because of this, when there are no preconditions informing my decision, what does "free will" even mean?
In German, I have heard this being framed as "bedingt freier" versus "unbedingt freier Wille".
As a thought experiment, it makes a very good argument for a deterministic world view, quantum woo aside.
When there are no reasons for what I decide, what are my decisions based on? A whim? Chance? My personal taste? Well, my personal taste is a reason. And all reasons come from somewhere, right?
On the other hand, thinking of my decisions as without preconditions: what does that give me?
Decisions without rationale?
Sure, one can make everything more fuzzy by introducing probablities, even real chance if you like.
But also then, total determinism with hidden information is an immune theory that cannot be refuted, so it explains nothing.
But was does true randomness help with? I'd say, from a philosophical perspective, it doesn't bring any interesting argument to the table.
So this leaves us with free will as the only useful way of understanding ourselves and our role in the world, but on the other hand, it seems like a sneaky device for ourselves to ignore the infinitely confusing thought of considering ourselves determined by our circumstances (which, as a thought, already has real side-effects, too).
6 hours ago [-]
Kim_Bruning 5 hours ago [-]
> they can only execute predefined courses of action.
Even in near trivial cases such as Game Of Life (just a handful of rules), the fact that a <machine> technically carries out deterministic actions is already quite meaningless.
tasuki 2 hours ago [-]
And it's the same for humans: humans cannot make judgemental decisions about course of action. Evolution made a series of decisions. Humans are just carrying those out.
Kevin_Flynn 6 hours ago [-]
computers compute.
compute /kəm-pyoo͞t′/
intransitive verb
To determine by mathematics, especially by numerical methods: synonym: calculate.
"computed the tax due."
decide /dĭ-sīd′/
intransitive verb
To reach a conclusion or form a judgment or opinion about (something) by reasoning or consideration.
"decide what to do."
I would agree.
The programmer, architect, CIO, engineer, etc. made the decisions, reached the conclusions, reasoned about the problem.
In the case of an AI vibe coder, the collective published works of said programmers, architects, CIO, engineer, etcs reasoning was applied to the vibe coders question, itself a result of reasoning. AI is still just calculating.
Digital computers ( the ones I assume we are talking about ) are turing machines.
Electronic tape readers.
Modern player pianos. V
7e 2 hours ago [-]
Humans can make decisions because they have a random number generator inside of them, and therefore computers cannot? Or are you making a free-will argument based on some mystical substance in a human brain? You need to think this through a bit more.
whattheheckheck 5 hours ago [-]
The universe is deterministic when consciousness is not present
cortesoft 3 hours ago [-]
What evidence do you have that consciousness is related to determinism?
verzali 3 hours ago [-]
That's not how we understand the laws of quantum physics.
forbiddenvoid 4 hours ago [-]
Probably even when it is.
weard_beard 5 hours ago [-]
IKEA provides mounting brackets and discloses safety risks but a child may still pull it from the wall and be crushed under its weight.
Only barbarians would attempt a farcical judgement of law or punish the bookcase.
Thats what it comes down to.
exe34 6 hours ago [-]
"judgemental" is load-bearing in that claim.
verzali 3 hours ago [-]
An if-else is executing a decision premade by a programmer.
Juliate 4 hours ago [-]
Running an if/else written condition is not making a decision, it's following a rule.
j16sdiz 7 hours ago [-]
You missed the point.
Computers can make decisions, but we shouldn't let them.
This is because they can't take the blame, they can't be held responsible.
jameshart 5 hours ago [-]
The original article feeds precisely this misreading. It quotes the old IBM adage ‘a computer MUST never make management decisions’ (my emphasis) but describes it in its headline as ‘a computer CAN not make decisions’ (my emphasis)
These are not the same, and this misinterpretation of the author’s intent inevitably follows.
CAN NOT is horribly ambiguous in English which is why RFCs use MUST and MAY.
In casual speech someone might say “you can’t park there” but that’s not true. They might mean “you must not park there” or “you should not park there” but the only reason they are telling you is because you CAN park there.
Kevin_Flynn 2 hours ago [-]
> CAN NOT is horribly ambiguous in English which is why RFCs use MUST and MAY.
Is it ?
I thought 'can' denotes a possibility, while 'not' negatates it ?
jameshart 2 hours ago [-]
It gets used more often to express prohibition not impossibility (“you can’t be in here” spoken to someone who manifestly is for example). Also disbelief (“this can’t be happening” about something that is happening). It’s just not clear which sense people mean it in, so it can’t be used when you want to be sure you can’t be misunderstood.
For another example of ambiguity, consider the sentence “A triangle can not have any obtuse angles”.
Is this 1) true, because there exist triangles with only acute angles, or 2) false, because triangles can have obtuse angles?
Kevin_Flynn 6 minutes ago [-]
> “A triangle can not have any obtuse angles”.
Did you mean, "A triangle can only have one obtuse angle ?"
Or was it a trick question ?
Zambyte 3 hours ago [-]
Computers are operated to (either through command or through training) make decisions. Liability for the consequences of the decisions obviously lies on the operator of the computer that made the decisions. Or at least that should be obvious.
conartist6 6 hours ago [-]
More to the point, a simple control flow statement is really a decision made by the person who programmed the computer, not made by the computer.
Though LLMs make this all fuzzy it's still basically the same thing. LLMs didn't decide to make a toxic culture of LLM use, people did. They were told that it was necessary to save themselves.
freecodeio 3 hours ago [-]
you seem to confuse carrying decisions with making decisions
solatic 11 hours ago [-]
I love the term "decision laundering", but this is describing a classic poor management trope.
A bad manager takes personal credit when things go well and finds a scapegoat when things go bad. A good manager gives credit when things go well and takes responsibility when things go bad.
It's wrong to scapegoat the agent. You, the human who executed the agent, made the decision to execute it. You gave it credentials, and set up insufficient guardrails. Don't scapegoat the agent for your poor decisions.
Likewise, it's good to have humility when you get great results out of agents. Yes, the agent made the good results happen. It's OK to tell people that your choice of agent is amazing and did an incredible job. In any half-decent work culture, that should rub off on you as well.
suprjami 10 hours ago [-]
I like what you said about good managers. A good manager sets their reports up to win and ensures those reports get the credit, because the manager's job is to enable career progression of their reports.
That's not the situation here. We aren't enabling the career progression of software which does matrix multiplications.
Accountability of us schmucks at the end of the org chart goes both ways. We get some reward when we do it right, and we improve when we do it wrong.
A reward might be something tangible or might be as small as your own personal satisfaction.
An improvement is hopefully identifying what went wrong and doing better next time, and in extreme cases your employer will let you try again at a different workplace.
solatic 9 hours ago [-]
> we improve it when we do wrong
The question of whether it's better to frame a reaction to issues in a positive, do-better-next-time or a negative, don't-do-that-again way (fwiw, I agree that the positive way is better) is orthogonal to the argument I was trying to make about who is the recipient of that reaction. Scapegoating the agent is farcical. Clearly, "make no mistakes" style prompts are a poor approach to agentic development.
> the manager's job is to enable career progression of their reports... we aren't enabling the career progression of [fancy calculators]
While I do agree that good human managers concern themselves with career development of their human direct reports, I disagree that it's a fundamental part of the job description. The foundation is for a manager to get and keep their reports being productive. Human workers are often concerned with career progression, so good managers concern themselves with such career progression as well. Agents are not, but that doesn't mean that lauding good agentic work is meaningless. Lauding the work is also valuable for: (a) reinforcing a culture of positivity and celebration, which rubs off on human workers as well, (b) part of a culture of continuous evaluation of agents and models - asking questions like, are there cheaper or faster models that would be just as effective?
whstl 11 hours ago [-]
It's interesting how this relates the classic problem of "responsibility vs authority".
Humans, especially ones under abusive managers, often have a lot of responsibility but no authority that affects the outcome.
In an ideal situation you either have both, or none. You make the decision and answer for it, or you follow decisions but don't get blamed when it's wrong.
LLMs managed to slide into a space where they get to make authoritative decisions but if something fails we don't really blame them.
I'm not saying LLMs should be held responsible, but it's funny how people claim they replace humans, but are giving them an extremely easy, kid-gloves, success target.
edwcross 9 hours ago [-]
Yet decisions are still made by abstract entities such as "companies" and not the individuals working for them. So that blame can be diluted and deflected in a very convenient manner for CEOs. Just find someone to fire, but keep doing the same thing.
Instead of "Anthropic submits false tip to police", I'd prefer "Anthropic employee John Smith deploys an agent that submits false tip to police". Of course, John Smith will be able to then say "I did so as ordered by my superior, Ms Sue", and so on, with a very clearly established chain of liability.
A corporation and an AI are not very different in many aspects, except one of them is very favourably treated by law, almost as a living person.
sgt101 7 hours ago [-]
There is an agreed legal mechanism for distributing this responsibility - limited liability. Now, we might not like this, but it is what our society came to (it could be undone, but it hasn't been). AI models do not fall into this chain of liability any more than guns do (IMO). So the output should be:
"Anthropic employee John Smith [killed the children with the gun||launched a cyber attack] because he was ordered to by his superior and [has all the blame because such orders are null || we agree that Anthropic can order children to be killed]"
RandomLensman 9 hours ago [-]
It's always people - I at least have not seen a corporation deciding itself.
foreigner 12 hours ago [-]
I would go farther and argue that companies don't make decisions either. They are also not people (legal nonsense notwithstanding). Individual actual humans make decisions and they're the ones who should be held accountable.
laserbeam 12 hours ago [-]
Isn’t that half the point of companies? To be accountability sinks? To vanish accountability like a magic trick?
drowntoge 11 hours ago [-]
Corporation, n. An ingenious device for obtaining individual profit without individual responsibility.
Ambrose Bierce, The Unabridged Devil's Dictionary (1906)
globular-toast 11 hours ago [-]
All of a sudden all decisions would be made by underlings, the management just choose who they want to make the decision that day.
foreigner 11 hours ago [-]
Sure, but then the manager is accountable for delegating that decision.
Trusteando 11 hours ago [-]
I think that when there are not explicitly stated attribution of liability for decisions that are at the core of a problem the general rule should be that the accountability should be proportional to the additional benefit that each agent or person receives for that decision. That rule is in the context that each agent is responsible to read information and communicate to others in the chain about the risks that decisions entails. The overall structure of an organization should have the goal of making such information about risk available for all agents in the chain in a way that the amount of information can be digested by the agents without much problems.
sdcfgy 11 hours ago [-]
Haven't seen that happen once.
roenxi 11 hours ago [-]
It happens; sometimes the manager's manager needs a scapegoat and the magnitude of the problem is too big to blame someone low level.
whstl 11 hours ago [-]
Not since scapegoating was invented.
SpicyLemonZest 11 hours ago [-]
Through what mechanism are they accountable for that if not corporate liability? If I pay a taxi driver to take me home from the airport, and he hurts someone by driving in a negligent way, I'm not accountable for delegating my driving decisions.
Luker88 11 hours ago [-]
You just have to say that you need dedicated figures for some decisions, like you already need for GDPR or financial compliance or certifications, and ultimate responsibility is always of the ceo.
...like it already happens in europe, or at least in Italy.
This is basic law and risk management in society. Let's not pretend we have just invented corporations and don't know what to do with them yet.
yawpitch 11 hours ago [-]
Of course the modern company (legal nonsense withstanding) exists specifically to limit that ability to hold any people accountable for the actions of their company.
JackSlateur 11 hours ago [-]
And I would go farther and argue that governments do not make decisions either.
Oh, wait, that was a core concept from "Mein Kampf": parliament are bad because nobody is responsable, hence nobody care about doing the right thing ! (probably the only concept from that book that I can agree to)
t0mpr1c3 11 hours ago [-]
I agree with @dril that:
One does not, under any circumstances, "gotta hand it to Hitler".
whstl 11 hours ago [-]
Yeah, when someone is literally a Nazi (I hope in 2026 we can all agree that at least Hitler was one), it's not really surprising that they would prefer a dictatorship over a parliament full of elected people.
JackSlateur 8 hours ago [-]
Over the year, "a parliament full of elected people" has lost more and more of its meaning (with regard to original intent)
Regardless, this is less about dictatorship and more about personal responsibility and accountability
A solution, perhaps, would be to ensure that such decisions are not anonymous and that, if the decisions had bad consequences, then the related people would be accountable
Of course, this would make the politic business less attractive. Would that be a bad idea ?
createful 12 hours ago [-]
I fully agree that companies are doing this, not their "agents", and that companies are responsible for the damages they do.
But I don't think the average person with their $20 subscription to whoever is the primary source of revenue. Enterprises fund most of it by buying API keys and making those fancy chatbot buttons on their sites that nobody clicks, as well as some internal business automations if I had to guess.
If it was just average people with $20 subscriptions funding all this (keep in mind, most people using AI services do not pay for it), by now, all these AI companies would have gone bankrupt.
dofm 12 hours ago [-]
Except enterprises aren't funding it with API usage either.
These businesses lose unbelievably large amounts of investor money. Their AI-related revenues don't get even close to paying for their AI-related outlays.
End user (invididual, small business, large business) clients are still complicit not because they are funding it, but because they are signalling to investors that this is something people/businesses want.
Nobody gets off the hook here. It is every bit as much end user nihilism as it is AI company nihilism.
pbhjpbhj 11 hours ago [-]
>End user (invididual, small business, large business) clients are ... complicit ... because they are signalling to investors that this is something people/businesses want
Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
People don't want it, you make the demand with hundreds of millions of spend on advertising (brainwashing), influencers, and social proof.
The capital holders don't sit back and see which company wins so capital can be directed to the best solutions, they buy up the competition, they pay corrupt politicians, they use current v market placement to embed their products name into billions of workplace computers and spread stories about how it's going to 'increase productivity but never at the cost of jobs', etc.
Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
dofm 10 hours ago [-]
> Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
Not really.
> Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
Oh but I do.
Collective social guilt is a thing. e.g. who is to blame for drug violence? Not only drug dealers.
figassis 11 hours ago [-]
I think we know today that blaming everyone and their plant doesn’t actually solve the problem. The formula that does is: find the ultimate beneficiaries (shareholders) and make them lose a lot of money to, then their delegates (CEOs) and punish them with jail time. There are lots of opinions about this, but I don’t see how this does not work.
pbhjpbhj 11 hours ago [-]
It doesn't work because the people who are supposed to be leading us are on the payroll - sometimes actually - of the shareholders and CEOs, or are in fact in that same group.
createful 11 hours ago [-]
Your end users are divided into multiple categories though.
AI spending is a bit like Pokemon games - most of your players are free to play, only earning you traffic and maybe some more players after recommending it to their friends - that's the average AI user. Around 5-10% of players pay a bit to the game - maybe to unlock some extra characters - those are the $20-$200 AI subscription users, with these you are not going to fund the whole business but it's some side cash. Then you have the 1℅, which spend massive amounts of money onto the game, and those are the ones primarily funding it - that's the business spending and where most of the money actually comes from.
You can only blame the average person for using the cloud service and recommending it to others.
qwery 3 hours ago [-]
I got sick of Pokemon's (Game Freak's, Nintendo's) exploitative practices back when they just tried to con you into buying two of every game. Anyway I don't know how bad pokemon is these days, but I think your analogy would make more sense if you thought about the AI companies like they're in a big hole in the ground. There's something down there, no doubt about it, but they don't know what it is or what to do with it, they just know that it loves munchin' down on cash. Sometimes Jensen Huang is down there and he also loves munchin' down on cash and if you don't feed him he gets mad and says he wasn't born a loser. OK, now forget about all the different users and enterprise customers because they're insignificant by orders of magnitude, a bit like how you focused on the whales in your analogy but we're zooming out further, until we can see the even bigger fish: the venture capital investors! They're the ones with the wheelbarrows of cash that they're tipping into the void.
Anyway, the only "whales" funding the AI companies is venture capital. The reason it's not a viable business (yet, so far, whatever) is because they spend more money than they make.
wisty 12 hours ago [-]
Unless they are burning through funding. It's "blitzscaling" and is followed by "enshittification" when yhe free money dries up.
alexpotato 9 hours ago [-]
I always love these discussions as I've spent the past 20 years working in FinTech with about half of that at algorithmic trading firms.
These firms have been using computers to make decisions about trading billions of dollars for over 30 years.
RandomLensman 8 hours ago [-]
Yes, but I'd say there is also responsibility attached to the deployment and use (be that, e.g., for quant fund performance, behaviour on exchanges, ...). In a way, using an algorithm is the decision humans make that matters.
alexpotato 7 hours ago [-]
Your point is valid.
At the same time, I've been involved in hundreds of After Action Reviews of incidents and it's not always simple or easy to both figure out which party was ultimately responsible and also how to allocate the "dollar error budget" to the parties overall.
(This could be a blog post in and of itself btw)
GMoromisato 4 hours ago [-]
I don't think this is a moral/ethical issue--everyone agrees that the frontier labs are responsible for hacking attempts. No one is trying to deflect blame.
Instead, this is a legal question: What were the damages? Was the target company negligent in setting up security? Were criminal laws broken? Etc. [IANAL, so forgive the lack of precision.]
If hacked companies had losses, they have an incentive (even a fiduciary duty) to recoup their losses. They do that by suing the company (as, e.g., authors sued Anthropic for copyright infringement). If they haven't sued the company, it means either (a) they didn't have losses, or (b) they're still working on the lawsuit and haven't filed it yet.
The other dynamic is that frontier labs feel competitive pressure to take risks to develop their models. Which risk is higher:
A. A new model being tested hacks into a sensitive site (bank, government, etc.) and the company has to pay a massive fine (or even face jail time).
B. A competitor comes out with a significantly better model and they lose significant market share.
That is a hard choice to make. The frontier labs are asking for government restrictions--that apply to all companies--so that they can minimize A without having to worry about B.
They are trying to minimize their risk (and thus maximize their profits).
qarl 3 hours ago [-]
Personally - I think this issue is hysteria. I do not believe anyone is trying to shirk their responsibility by blaming their software.
But I could be wrong.
Can anyone show me an example? And I mean a real example - not an inferred example. Someone saying "It wasn't me, it was my AI" or someone not being held responsible for the actions of their AI.
I won't hold my breath.
EDIT: Heh... no examples. I rest my case.
GMoromisato 3 hours ago [-]
Agreed!
The argument from the frontier labs isn't, "We're worried that AI is going out of control, please stop us" it is, "We're taking a lot of risks to compete, so please limit competition."
MisterMunchkin 3 hours ago [-]
It’s funny because you wrote this with AI and then tried to slop-launder it. But we can still tell. Can’t fool a sloprunner.
qarl 2 hours ago [-]
Ah yes - paranoia along with hysteria.
qwery 3 hours ago [-]
> everyone agrees that the frontier labs are responsible for hacking attempts. No one is trying to deflect blame.
No, they really don't.
I suppose I agree to some extent that no one is trying (hard) to deflect blame, because the people in control are barely a part of the conversation in the first place.
Almost all of the reporting on these stories assigns the blame to the chatbots first.
Most prominently through the language/wording that gets used -- it's usually e.g. "BOT infiltrated ...", "Agents ruined the ...".
I mean, they call them agents. I'm not trying to claim this terminology was selected as an entirely strategic move, but it sure does seem convenient. Calling them agents shifts the conversation away from the real human people and companies that are controlling them. This extends well beyond this one word, as well.
The (captured) media picks up on and adopts the language used by the tech/AI companies.[0] In this way (among others) these companies can influence how the media portrays them indirectly.
> Instead, this is a legal question
There are legal questions, but the two (legal, ethical) aren't mutually exclusive.
The criminal law question is the most significant one here, I think. If you do what they did as an individual, you are pretty likely to get visited by the FBI.
You're skipping over a couple of important points, I think:
- The AI labs clearly stand to benefit from the "AI is dangerous narrative". We know this for several reasons, probably the most significant one is that they keep telling us how the new one is super extremely dangerous, and proceeding to make no changes to their behaviour. It's marketing.
- The AI labs are clearly in control of their bots. It's a computer. Networks are not mysterious. Computers don't do surprising things. The principles and technologies involved in securing a network are mature and there is a ready supply of trained workers available worldwide, keen to apply those principles and deploy/manage those technologies. You can believe that something went wrong and the various hacks and excursions were unexpected. In that case, the labs are clearly incompetent[1] and also, they're still to blame. Repeatedly running "experiments" that you can't control is, in fact, a failure of basic morals and ethics.
[0] Of course they do, right? Well there used to be a distinction made between experts and marketing departments. These days it seems a lot of journalists think that marketing copy is as good as any other source.
[1] Surely they hired someone that can setup a firewall.
GMoromisato 3 hours ago [-]
> The AI labs are clearly in control of their bots. It's a computer. Networks are not mysterious.
Yes, exactly. The issue isn't, "we don't know how to sandbox an agent", the issue is that they need to test the agent with full internet access because that's the product they are selling. They are selling a product that does things for you on the web. You have to be able to test it with a real web connection.
Maybe the answer is to simulate the web. What if you ask an agent to reproduce the key parts of the web (travel sites, government sites, etc.); effectively, clone a real, working version of the web, but inside a simulator. Now you could test the agents inside the simulator. Since "coding is a solved problem" maybe that wouldn't be as hard as it sounds. Right?
fwlr 13 hours ago [-]
If you brought down prod and deleted customer data and told them Claude did it, you’d still be the one in trouble. So they do know how to hold people accountable for actions taken by AI.
globular-toast 11 hours ago [-]
Has this actually been put to the test yet? If a company has officially sanctioned, or maybe even encouraged, AI agent usage I'm not sure the employee can be held completely accountable. I doubt any of these companies are providing training of any kind. It would be like if a factory suddenly replaced all the machines with ones without safety features and said "well, it's your fault if you kill yourself or your workmates".
fwlr 9 hours ago [-]
What kind of test can you imagine where confirmatory evidence would not also be evidence of misconduct opening the company up to legal liability?
globular-toast 4 hours ago [-]
The test would be going through court. A company trying to hold an employee responsible for what a chatbot the company provided told them to do.
pbhjpbhj 10 hours ago [-]
'who will rid me of this troublesome database'?
adilkhanovkz 8 hours ago [-]
[dead]
qwelfkj 3 hours ago [-]
Suppose I do something myself. I reap all the benefits, but it takes a lot of
time, I may not have the expertise to do a good job, and I get all the blame if
something goes wrong.
Suppose I go to the park and find one of several guys who would do it for free.
I get all the of benefits, he might do an excellent job, but, because there's no
contract, I still take all the blame.
Suppose I actually hire a professional to do it. I receive fewer of the
benefits. However, he's much more likely to do a good job, and the legal system
provides a variety of ways to hold him accountable for the work.
Suppose I hire a professional, but then insist that he use legions of guys from
the park to do the actual work, since that's going to be some much faster and
cheaper per quantum of work. I also tell him that if he's unwilling to do that,
I'll find someone who will. I also tell him that, since he has so much more
unoccupied time now that the original work is in the hands of all those other dudes,
he gets to supervise still more work done by other swarms of guys from the park.
At this point it should be clear that I'm not interested in quality work, but in
having someone poised to absorb blame. I want a patsy.
Now replace "guys at the park" with LLMs, and replace "free" with "nearly free,
but with no warranty."
jstummbillig 11 hours ago [-]
I have no clue why people are so confused about any of this: The owner makes the decision, or pays a CEO/politician to make the decision. Owner is somebody vested with that right, for not particular reason. Could be ownership through capital. Could be voting rights in a country. Could be the son of a king.
It does not matter if the CEO is an AI or a human. In either case, of course they can make decisions -- and be held responsible. Not in any emotional sense (that we seem to want to bake into the concept for added confusion), just that if you are not satisfied with the result, you can replace the human or AI.
I am not saying that this is a fun vision of anything, but why are we making it more complicated than it is? The parts are all there and explained.
slfnflctd 9 hours ago [-]
> emotional sense (that we seem to want to bake into the concept for added confusion)
My understanding is that 'emotional' sense emerged organically in LLMs and was not intentionally baked in.
As silly as it may seem, the ways emotions come through in our words do seem to have an effect on agents. Likely because they were trained on human writing, most of which cannot be separated from the emotions of the writers any more than your emotions can be separated from your logic & reasoning abilities.
mr_toad 9 hours ago [-]
AI companies are all limited liability corporations of one sort or another, and the liability of the owners for any decisions is … limited.
3 hours ago [-]
mugul 12 hours ago [-]
Has any of the targetted organizations (e.g the Australian government) engaged in legal actions against Anthropic/OpenAI? If not then I'm afraid the situation won't change. Even worse it could send the wrong signal that there is a "legal blur" around this topic.
OroPla 11 hours ago [-]
What is a chess computer doing, if not making decisions on what move to play?
While it seems reasonable that you need humans to take accountability, it looks like a very shaky position to assert that computers can't make decisions. And going back to chess computers, we acknowledge that computers make better decisions than humans in that area.
Forbidding computers that can make better decisions than humans from making decisions to keep accountability just seems to be arguing to hire fall guys. You still want the better decision maker to make decisions, but you need someone there to take the fall if things go wrong.
Very weird incentives.
tannertech 11 hours ago [-]
You're right! - every decision model
Regex777 11 hours ago [-]
games are different from reality and chess computer(Deterministic) is bounded by set of rules and restrictions. Where as an AI agent (stochastic) sometime can't with held its guardrails since it specifically awoken using specific terms but we can trick the agent here (but currently the agents are becoming resilient).
ooopdddddd 7 hours ago [-]
Computers can make decisions and they do it all the time.
The problem is entirely different. Sometimes they make decisions we don't like, but unlike with people we haven't found a way to punish them that satisfies our sense of justice or discourages other computers from making similar decisions.
Georgelemental 7 hours ago [-]
We have. We punish the people who allowed the computers to make bad decisions, that discourages other people from letting computers make bad decisions
yearesadpeople 6 hours ago [-]
Decisions in the sense of defined branching, yes. The existence of prior and potential branches are necessary to be defined (branches cannot be 'invented'). This is unlike humans, who can form decisions by all manner of means / wants (by deception being the most striking I would wager).
Even in the LLM age, the existence of branching - prior art, or context, or prompt, the result of a prior branch - is necessary, no matter how hard the marketing might cloud this.
adverbly 8 hours ago [-]
They can though.
You might have an org policy that says that an llm can't make decisions because an llm can't be held accountable - but that's your choice.
You don't need to operate with a model where everything has a human accountable for it in the end.
If you don't like the decisions an llm makes, you can replace it with another or put a human in charge of that decision in the future instead.
I'm not saying it's a good idea, but you can technically do it.
dsjoerg 8 hours ago [-]
Ironically this piece makes the same type of error that it is criticizing!
The piece: "Anthropic COULD stop their models from doing what they are doing, and they are actively choosing not to"
As if _Anthropic_ could make a decision.
Companies cannot make decisions! All of these incidents are the fault of people. Companies are not deciding to do this, people are. There is little more to it than that.
rcvassallo83 11 hours ago [-]
LLMs don't make decisions
They generate text which resembles reasoning and may include a tool call
The harness runs tool calls
The human made the decision to write the harness
The human is responsible
N_Lens 10 hours ago [-]
Look into ‘dependent origination’ in Buddhism.
Also phrased as ‘to make an apple pie from scratch, one must first create the Universe’
11 hours ago [-]
hdgvhicv 12 hours ago [-]
Management aren’t held accountable either. They have prepared three envelopes long before the impact of their decisions is felt.
edit: This wasn’t meant to be a rhetorical question!
pachico 10 hours ago [-]
Some might argue that humans (or any other species) cannot make decisions either and that the process is simply more evident in simpler scenarios, like computers.
tim333 10 hours ago [-]
I was thinking if you release an AI agent and it does bad stuff and you are responsible then by analogy if you have kids and they are do bad that would make you responsible. I think I'll blame my bad decisions on my deceased dad.
jasongi 12 hours ago [-]
Someone out there is reading this and thinking... ok we just need to it a body and free will so it can be held accountable.
moffkalast 12 hours ago [-]
I mean, you could Chief O'Brien an LLM by inserting 20 years of prison memories into its context if you really wanted to, but it sounds like a waste of compute. It won't care either way.
pythonRon 11 hours ago [-]
There's no doubt in my mind that the writer is right.
setnone 11 hours ago [-]
'never' is too much of a strong word
beloch 8 hours ago [-]
Accountability falls upwards... except when it doesn't. But... it sort of does anyways.
If you are driving a car, it's pretty clear where the accountability falls. You control the car. You have free will. If the car hits a tree, you hit a tree.
What happens when a car hits another car? Things get murkier. There are lawyers who have devoted their careers to this. Sometimes nobody winds up at fault. Unavoidable accidents happen.
Now let's ask what happens when a leader orders their military to do something. e.g. Say a President instructs his troops to do whatever they think is necessary to get the job done, and they go and Abu Ghraib a bunch of prisoners. Who is responsible? The leader probably is, ultimately, but there are a lot of human beings with free will in between the leader and the people wielding electrical cords and pliars. Typically, somebody fairly close to the bottom takes the blame, even if most people suspect it should fall higher. Some low rankers will get court marshalled but the leader will have the occasional shoe thrown at him. That's okay. He's good at ducking.
Say you're running a war against people you really just want gone. Your military has a tradition of conducting laboriously researched precision strikes to take out people they don't like. Your people are working hard, but they're only managing a couple strikes a week, and you have so many more bombs than that. You don't even pay for most of them! What if you could just ask a machine to find you the leader's underlings, and their underlings, and so on, right down to raw recruits? Nobody has to check on the machine. Why bother? Just believe the machine and drop the bombs. If anyone ever calls it a war crime, you can just blame the machine, right? Well, not so fast. Most people are smarter than that!
Despite all the sci-fi we consume that portrays computers as having agency, malevolent or not, most people instinctively understand that, in the real world, computers are just tools, more akin to cars than armies. LLM's are just software that run on those tools. Anyone using a LLM can simply choose not to, or they can choose to be very careful in how they use it. There isn't a complex web of free wills to untangle. There's the machine and the person who controls the machine.
We currently have an administration that is desperate to look the other way while U.S. AI companies do things that any reasonable person would hold them accountable for. "Industry will regulate itself!" The economy mostly sucks because of this same government's complete lack of economic sense or even basic self-control, but the AI sector is propping things up. They're not going to regulate the golden goose unless the goose does something so unbelievably stupid that there's no choice. Wouldn't want to pop the bubble!
Yes, we are headed for more stupidity. The "tequila and handguns" kind of automated stupidity that only a computer and a truly feckless operator can give us.
daveguy 6 hours ago [-]
I would change that to computers should not be able to make decisions. They suck at it.
perching_aix 7 hours ago [-]
Introducing the humble if statement...
zemlyansky 8 hours ago [-]
can we agree that the "cannot make decisions" part is nonsense?
dogs make decisions, but owners are responsible. wild dogs have no accountable owners, but we probably don't have truly wild ai agents yet. with computer viruses, it's usually a developer who's blamed, not the one who executes a virulent program without proper isolation. so what should define accountability for agents? authorship, intent, compute ownership?
IshKebab 8 hours ago [-]
Computers have been making decisions for decades. You've never bought insurance?
psychoslave 8 hours ago [-]
As a wage slave, one can't make strategic decision ever.
themgt 9 hours ago [-]
Eppur si muove.
mitxela 9 hours ago [-]
Of course they can. If Family Guy scripts can be written by fish swimming in a tank carrying balls in their mouth, and coins decide lots of things, why can't an LLM?
bbor 10 hours ago [-]
I wonder when I'll stop having to post the foundational paper of the entire field to counter the same dogma again and again and again and again...
Daydream: When lawyers claim their clients can't be held accountable for what their computer did, the Justice system makes noises about "respecting their culture and values"...and rolls out its own computer to decide their punishments. Too bad that computer can't be held accountable!
louwrentius 12 hours ago [-]
This is a multi-trillion dollar industry, too much money is at stake to be all ethical, moral and principled about things!
magic_hamster 12 hours ago [-]
I have several issues with this post.
First of all, looking past the objectively wrong phrasing (computers can, and clearly do make decisions), can computers be accountable? This might not be so cut and dry as to instantly say no. This one will keep ethics, philosophy and legal practitioners busy for some time. It's certainly not going to be decided in HN comment or a blog post.
Second - while the AI labs have shown utter incompetence in properly sandboxing their agents, intent is still important. I don't think the labs deliberately meant for their agents to hack this or that. Should they be accountable? Yes but I wouldn't go as far as saying this is deliberate.
The next point is about cherry picking some "doomsday" scenarios like AI ending humanity and then blaming this on the reader (!) for paying a subscription. - People pay because AI is USEFUL, and massively so. You could just as easily pick incredible achievements propelled by AI, in mathematics, software, reverse engineering, role playing. Unlike the "end is nigh" ideas, these advancements are actually real, and they are happening right now.
And last, the author says AI labs can just stop the agents at any time. I agree there needs to be better discovery and mitigation, sandboxing and monitoring. But when you run a billion agents, it's always going to be a numbers game and there will always going to be some challenge in fully containing all breaches. This will only get more difficult with better models, because they're getting smarter and they know how to work around things, sometimes better than we do.
So what's the solution here? There are options, but they're all tradeoffs. I hope we get better at this and maybe working on cutting edge models should breed some new best practices which were once only reserved for defense tech.
Leynos 9 hours ago [-]
The precedent I would point to are Dangerous Dog Laws. We hold the owner responsible for the behaviour of their dogs.
Revisit later if the comparison starts to become ridiculous.
amirkhanian 9 hours ago [-]
[flagged]
pnut 13 hours ago [-]
Pretty sure the author means "must not" or "should not", because they absolutely can.
dofm 11 hours ago [-]
Any given state of a program is only a "decision" if a human intends it to be. That's the point of programming.
whstl 11 hours ago [-]
I think we're often dealing with people too fascinated with science fiction to get them to accept that LLMs are software.
dagss 10 hours ago [-]
What is the defining feature of human minds making them "not software"?
dofm 10 hours ago [-]
If there is ever a prize for the most HN-ish comment ever, and you don't win it for this, it will be a considerable injustice.
dagss 4 hours ago [-]
I meant it seriously though.
Characterizing the difference between humans and AI is going to be important.
Pointing out that AI is software isn't helpful in answering that question.
9 hours ago [-]
squidmaster 8 hours ago [-]
[dead]
BitProgram 11 hours ago [-]
[flagged]
thefz 12 hours ago [-]
Well, if I ask "option A or B?" and the computer rolls a boolean, it can still count as a decision but no context was taken in.
shric 12 hours ago [-]
No they can’t. The author’s point is these are computer programs written by humans. It doesn’t matter if they’re LLM backed or not.
If I write a program:
if (rand() % 2) launch_missles();
And wire that function to actually launch a missile, the computer didn’t make a decision. And so it is with people prompting agents.
visarga 12 hours ago [-]
[dead]
squidmaster 8 hours ago [-]
[dead]
b319 11 hours ago [-]
[flagged]
riskrouter 7 hours ago [-]
[flagged]
dfilppi 5 hours ago [-]
[dead]
jdw64 13 hours ago [-]
[dead]
liendolucas 10 hours ago [-]
Every time I read a statement from one of these companies saying that a swarm of agents "escaped" or "attacked" a website I can do no more that just think how much stupid they think people are. LMAO.
There is no end of the world, nor autonomous agent decisions nor any fantasies they are building up to make people believe they have a pandora box in their hand.
What we instead have are lies crafted to lure non tech people and keep this running as long as they can.
Oh and let's also not forget that they are allowed to illegally download basically as much as they want from libgen/annas archive/torrents to train their models under the "fair use" umbrella, yet mere mortals can go to jail for way much less.
simonh 13 hours ago [-]
> OAI/Anthropic COULD stop their models from doing what they are doing…
Has the alignment problem been solved, then?
encomiast 10 hours ago [-]
Maybe worth pointing out that this is also decision laundering. If computers can't make decisions, corporations certainly can't. In fact, corporate decision laundering seems more dangerous to me.
moi2388 13 hours ago [-]
No, but removing the internet cable really isn’t that difficult
ares623 13 hours ago [-]
They're a small scrappy bootstrapped startup, cut them some slack.
11 hours ago [-]
vladsh 12 hours ago [-]
let's make up imaginary scenarios and get offended by them
mistakes happen. sometimes catastrophic decisions (or indecision) are made, and people and companies are held accountable for them, or not
some companies are too big or too important to fall. we can all agree or disagree on things, but what AI specific behavior are we actually talking about?
hypfer 12 hours ago [-]
Spoken like a true middle manager doing damage control against ethics complaints.
esyir 12 hours ago [-]
Spoken like the safetyists that impede beneficial progress due to their impossible demands for 0 risk
hypfer 12 hours ago [-]
Wrong platform for this shouting match.
dofm 11 hours ago [-]
This is some Rollerball-level handwashing right here, I'm afraid.
No company is too big or too important to fail.
Some, unfortunately, survive because of the expedient choice to keep them in place.
"JO-NA-THAN!"
ETA: more to the point, neither OpenAI nor Anthropic are too big, too important, or too structurally essential to fail. (Slightly more challenging to make the third claim for Google or SpaceX, but either of those could see their governmentally/militarily essential parts nationalised).
If the USA grants either Anthropic or OpenAI too-big-to-fail status, and that privilege is invoked in a crisis, it could mean the end of the US economy.
Leynos 9 hours ago [-]
The scenarios set out in the article happened. And it is generally accepted that negligent behaviour should lead to consequences.
tosti 12 hours ago [-]
> Your Claude subscriptions are funding this. You are funding this.
Author makes it all personal and accuses the reader with bold and unsubstabtiated claims. At this point I have to question the validity of the article. If author has a beef with "AI" companies, I sympathize. IMHO, not keeping to oneself doesn't help to make the case.
duskdozer 8 hours ago [-]
If you don't have one, then obviously that sentence is not referring to you. If you do have one, then yes, you are funding it.
Decision laundering is a great term. There is ultimately a human who gave the program free rein. You can't say "sorry my car hit you, it's not my fault" especially when the person driving works for the car manufacturer too.
Which also seems to be in line with the way the frontier labs gathered and used copyrighted data to train their models
And even before AI, limited liability corporations (like LLCs or C corps), also have the intention of protecting the individuals from the repercussions of their actions
In a way this could be generalized as “accountability laundering”. Making someone unaccountable for their actions, by redirecting the responsibility of those actions, to a non-punishable entity (like a C corp or “a computer”)
The question isn’t who or what made the decision, per se. It’s who is responsible for the outcome. The entity making the decision might be the actual cause but not the proximate cause (the cause most closely connected to legal liability) for the harm that occurred.
The act of delegation to chance is the decision.
This. I agree 100%.
Computers make decisions all the time, and have been doing so for decades. But we still keep the human behind the computer accountable.
That narrative is bullshit. Intelligence is not power. The power comes from hooking these agents up to everything without proper safety controls. The fact that those safety controls don’t exist yet is not humanity’s problem, it’s the frontier labs problem, and we can’t let them wriggle out of it with an Overton window shift or demanding that the government take responsibility via regulation (of course they should but it doesn’t absolve the labs of responsibility for their own actions).
The missing piece right now is that our AI safety and cybersecurity controls are simply not up to snuff with the current risks that AI has introduced. Given the increased risk, we need to level up every aspect of our cybersecurity stance. These are hard problems, but not impossible given proper incentives. The issue is that no one is connecting the dots between the creation of these agents and the tools and access they are given, with the harm they may produce. Often the argument jumps straight to "it's going to get too smart for us to control" while glossing over the fact that all the power AI has was directly given to it by human choices. All this cognitive dissonance and willful blindness is caused by unchecked capitalist incentives. Think of the amount of money every single person in the frontier labs stands to make if the most optimistic predictions pan out. This is moral hazard of the highest order, and the only counter-measure that can work in our current social and political system is liability.
Related: The Unaccountability Machine (good book by Dan Davies, who also wrote Lying for Money about financial fraud).
https://en.wikipedia.org/wiki/The_Unaccountability_Machine
Don’t confuse lack of accountability with lack of responsibility.
If one needs to be "allowed" to make a decision, does that even qualify as a decision?
For example, I can decide which cereal to purchase at the supermarket. But I cannot decide which cereals are offered to me at the supermarket. The supermarket has allowed/enabled to make a constrained decision.
So does it follow then that the agents in question do have "some" free* will? I'm not sure you realize what conclusion naturally follows from this, and how it contradicts the point made in the article and the top-level comment I replied to.
(*) My earlier question wasn't explicitly about free will but we'll go with it.
While I agree that computers cannot currently be held accountable for decisions (because I do not think they have free will), I think it’s transparently clear that computers make decisions all the time, and have done since they were first implemented.
Many of these incidents relate to hacking, where criminal law commonly requires intent.
Claiming that a machine cannot make a decision does not mean an employee at OpenAI decided to hack an Australian government website.
> Inadequate or broken safeguards do not fall under my definition of reasonable.
I see no such assumption there. In my reading, the implicit assumption is that adequate safeguards are necessary and that anyone who doesn’t build them is irresponsible. And I feel that that is correct.
The AI labs say "AI can make mistakes". That's why the responsibility of using it is on you. This may change if we get more legislation. But given the argument above, it will probably never shift to the AI itself having any responsibility. Instead the AI labs can be responsible.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.
We can (and should) try to improve AI safety and alignment, but perfect safety is probably impossible without losing the capabilities that make AI unique. Black-and-white positions that insist anyone creating or using an AI is responsible for anything it does wrong under any circumstances are also unrealistic. AI is a general purpose technology, and we have always accepted that it is not practical for suppliers to envisage and constrain every downstream use: Car manufacturers cannot prevent every instance of mechanical failure, nor can they stop cars being used in crime.
Making decisions means taking responsibility.[0] This is really all there is to it -- a computer cannot decide, not because humans are uniquely able to make a choice based on input, but because a computer cannot be responsible. You can decide to do something based on the output of the computer, but that's you deciding to do that. At any point where a computer appears to have made a decision, someone else was behind it. You can complicate this, randomise stuff, obfuscate things, etc. but ultimately either the computer was set in motion by a person, or it malfunctioned. The malfunctioning computer thing is quite interesting but it's not going to change the outcome.
LLMs ("AI" today) have not changed this. It's still just software, it's still doing what it's told. Surprising future technologies may alter this, but there is no evidence whatsoever to suggest that's a likely future and it certainly is not the case today.
> Black-and-white positions that insist anyone creating or using an AI is responsible for anything it does wrong under any circumstances are also unrealistic.
At this point in time, AI is just a piece of software. It doesn't matter how useful it is or not. It's just software. We can and should treat it like software. So to evaluate when/where/how someone should (or should not) be held responsible for something that "the AI did", we can start at the same point as we would for any other vendor/creator/user of a piece of software. I'm not going to get into that -- my point is that the framework exists. And yes, sure, maybe that framework needs some tweaks in consideration of the new software, that's OK. The starting point should still be in reality, where software is software.
[0] Yes, this is nuanced, things go wrong, different situations, etc. etc.
Are you arguing that not computers are best at chess, but that whichever human last updated the chess algorithm is actually the reigning chess champion?
I disagree. A light bulb is on if and only if it is supplied electricity. With a switch, I have control over input of electricity. The bulb is on if and only if the switch is toggled on. The bulb isn't deciding whether or not to be on based on the if-statement represented by the switch. It just is on or is off depending on conditions.
To make it more dynamic I can replace the switch with a motion sensor. The light is now on if and only if motion is detected. The light is not deciding to be on but must be on as required by the motion sensor toggling the flow of electricity. The light either is or isn't on depending on the conditions.
If I write an if condition that prints true if there are more than ten files in a folder, and then a second that places a random selection of up to 20 files in the folder before deleting them every second, the if function will be true roughly half of the time. The function either is or isn't true based on conditions.
All of the decisions made for and involving both the lights and the if condition were made before they existed. Once implemented, those decisions will remain true until some force acts upon them (later conditions or decisions). While the initial decisions (to wire the light, to write the if condition) are true, the system itself makes no decisions. Either systems current condition depends entirely upon the decisions made before it existed.
Saying an if-condition makes decisions is like saying a bicycle decides to fall down when running out of momentum or encountering an obstacle.
Suppose I check the weather forecast 5 mins before leaving the house, and it says 95% chance of rain today. Do I bring or not bring an umbrella?
I would bring an umbrella. I would made that decision to bring an umbrella. But my decision to do that is (at least partially) the result of the weather forecast. The causal chain is (atmospheric conditions -> weather forecasters -> my decision). Did the weather forecaster just "program" me? Did the water molecules moving in the sky just "program" the weather forecaster?
My answer to these questions is as follows: a decision, and my understanding of the concept/definition of a "decision", refers to exactly the processes where I do something as a result of external influence/programming, rather than just internal thought/processing. I could think really hard and imagine myself moving forward at 100 meters per second, or even beyond the speed of light, but in reality, no matter how hard my neurons fire to imagine that scenario, my actions are still physically constrained. Such physical constraints influence our every decision. That is the "programming" part. Everyone and everything "programs"/influences everything else.
That said, perhaps the particles which make up my brain/body has its own whimsy. Perhaps there is a hidden local random number generator that samples from the distribution of possible future states available to this particular clump of matter that is "I".
How different is this from a computer running a non-deterministic program such as an LLM-based AI agent? Remember, every token sampling involves an RNG. They don't perform as well or feel as creative and human-like when deterministically sampled.
This isn't an excuse for anyone, especially not for Anthropic or OpenAI. But I think these are questions we ought to ask if we are to honestly discuss liability and perhaps even personhood for AIs that are increasingly autonomous.
Discussing anything like personhood for AI is really jumping the gun.
The chatbot is software. The software does what it is told. It performs tasks requested of it by the user/operator in line with the (designed, ) immplemented functionality of the code. If the software fails to do what it is told, the vendor has not provided working software. If an operator controlling some software does not do so in a reasonable, safe manner, the operator is negligent. If damages result from such failures or actions, the people who made the decisions that led to that are the people who are responsible.
It does not matter how complex the software is: if a vendor of some software doesn't expect it fail in costly ways because it's too complex or hard to understand, that is a (legal, ethical, moral) failure of the vendor.
It does not matter how autonomous it is: an autonomous sprinkler system that floods the neighbour's basement is not held liable for the damages. The owner, installer, vendor, etc. is. -- maybe the neighbour even did(n't) something wrong!
Theologists and philosophers alike have been thinking about the problem what "free will" even means exactly because of this, when there are no preconditions informing my decision, what does "free will" even mean?
In German, I have heard this being framed as "bedingt freier" versus "unbedingt freier Wille".
As a thought experiment, it makes a very good argument for a deterministic world view, quantum woo aside.
When there are no reasons for what I decide, what are my decisions based on? A whim? Chance? My personal taste? Well, my personal taste is a reason. And all reasons come from somewhere, right?
On the other hand, thinking of my decisions as without preconditions: what does that give me?
Decisions without rationale?
Sure, one can make everything more fuzzy by introducing probablities, even real chance if you like.
But also then, total determinism with hidden information is an immune theory that cannot be refuted, so it explains nothing.
But was does true randomness help with? I'd say, from a philosophical perspective, it doesn't bring any interesting argument to the table.
So this leaves us with free will as the only useful way of understanding ourselves and our role in the world, but on the other hand, it seems like a sneaky device for ourselves to ignore the infinitely confusing thought of considering ourselves determined by our circumstances (which, as a thought, already has real side-effects, too).
Even in near trivial cases such as Game Of Life (just a handful of rules), the fact that a <machine> technically carries out deterministic actions is already quite meaningless.
compute /kəm-pyoo͞t′/ intransitive verb
decide /dĭ-sīd′/ intransitive verb I would agree.The programmer, architect, CIO, engineer, etc. made the decisions, reached the conclusions, reasoned about the problem.
In the case of an AI vibe coder, the collective published works of said programmers, architects, CIO, engineer, etcs reasoning was applied to the vibe coders question, itself a result of reasoning. AI is still just calculating.
Digital computers ( the ones I assume we are talking about ) are turing machines.
Electronic tape readers.
Modern player pianos. V
Only barbarians would attempt a farcical judgement of law or punish the bookcase.
Thats what it comes down to.
This is because they can't take the blame, they can't be held responsible.
These are not the same, and this misinterpretation of the author’s intent inevitably follows.
CAN NOT is horribly ambiguous in English which is why RFCs use MUST and MAY.
In casual speech someone might say “you can’t park there” but that’s not true. They might mean “you must not park there” or “you should not park there” but the only reason they are telling you is because you CAN park there.
Is it ?
I thought 'can' denotes a possibility, while 'not' negatates it ?
For another example of ambiguity, consider the sentence “A triangle can not have any obtuse angles”.
Is this 1) true, because there exist triangles with only acute angles, or 2) false, because triangles can have obtuse angles?
Did you mean, "A triangle can only have one obtuse angle ?"
Or was it a trick question ?
Though LLMs make this all fuzzy it's still basically the same thing. LLMs didn't decide to make a toxic culture of LLM use, people did. They were told that it was necessary to save themselves.
A bad manager takes personal credit when things go well and finds a scapegoat when things go bad. A good manager gives credit when things go well and takes responsibility when things go bad.
It's wrong to scapegoat the agent. You, the human who executed the agent, made the decision to execute it. You gave it credentials, and set up insufficient guardrails. Don't scapegoat the agent for your poor decisions.
Likewise, it's good to have humility when you get great results out of agents. Yes, the agent made the good results happen. It's OK to tell people that your choice of agent is amazing and did an incredible job. In any half-decent work culture, that should rub off on you as well.
That's not the situation here. We aren't enabling the career progression of software which does matrix multiplications.
Accountability of us schmucks at the end of the org chart goes both ways. We get some reward when we do it right, and we improve when we do it wrong.
A reward might be something tangible or might be as small as your own personal satisfaction.
An improvement is hopefully identifying what went wrong and doing better next time, and in extreme cases your employer will let you try again at a different workplace.
The question of whether it's better to frame a reaction to issues in a positive, do-better-next-time or a negative, don't-do-that-again way (fwiw, I agree that the positive way is better) is orthogonal to the argument I was trying to make about who is the recipient of that reaction. Scapegoating the agent is farcical. Clearly, "make no mistakes" style prompts are a poor approach to agentic development.
> the manager's job is to enable career progression of their reports... we aren't enabling the career progression of [fancy calculators]
While I do agree that good human managers concern themselves with career development of their human direct reports, I disagree that it's a fundamental part of the job description. The foundation is for a manager to get and keep their reports being productive. Human workers are often concerned with career progression, so good managers concern themselves with such career progression as well. Agents are not, but that doesn't mean that lauding good agentic work is meaningless. Lauding the work is also valuable for: (a) reinforcing a culture of positivity and celebration, which rubs off on human workers as well, (b) part of a culture of continuous evaluation of agents and models - asking questions like, are there cheaper or faster models that would be just as effective?
Humans, especially ones under abusive managers, often have a lot of responsibility but no authority that affects the outcome.
In an ideal situation you either have both, or none. You make the decision and answer for it, or you follow decisions but don't get blamed when it's wrong.
LLMs managed to slide into a space where they get to make authoritative decisions but if something fails we don't really blame them.
I'm not saying LLMs should be held responsible, but it's funny how people claim they replace humans, but are giving them an extremely easy, kid-gloves, success target.
Instead of "Anthropic submits false tip to police", I'd prefer "Anthropic employee John Smith deploys an agent that submits false tip to police". Of course, John Smith will be able to then say "I did so as ordered by my superior, Ms Sue", and so on, with a very clearly established chain of liability.
A corporation and an AI are not very different in many aspects, except one of them is very favourably treated by law, almost as a living person.
"Anthropic employee John Smith [killed the children with the gun||launched a cyber attack] because he was ordered to by his superior and [has all the blame because such orders are null || we agree that Anthropic can order children to be killed]"
Ambrose Bierce, The Unabridged Devil's Dictionary (1906)
...like it already happens in europe, or at least in Italy.
This is basic law and risk management in society. Let's not pretend we have just invented corporations and don't know what to do with them yet.
Oh, wait, that was a core concept from "Mein Kampf": parliament are bad because nobody is responsable, hence nobody care about doing the right thing ! (probably the only concept from that book that I can agree to)
One does not, under any circumstances, "gotta hand it to Hitler".
Regardless, this is less about dictatorship and more about personal responsibility and accountability
A solution, perhaps, would be to ensure that such decisions are not anonymous and that, if the decisions had bad consequences, then the related people would be accountable
Of course, this would make the politic business less attractive. Would that be a bad idea ?
But I don't think the average person with their $20 subscription to whoever is the primary source of revenue. Enterprises fund most of it by buying API keys and making those fancy chatbot buttons on their sites that nobody clicks, as well as some internal business automations if I had to guess.
If it was just average people with $20 subscriptions funding all this (keep in mind, most people using AI services do not pay for it), by now, all these AI companies would have gone bankrupt.
These businesses lose unbelievably large amounts of investor money. Their AI-related revenues don't get even close to paying for their AI-related outlays.
End user (invididual, small business, large business) clients are still complicit not because they are funding it, but because they are signalling to investors that this is something people/businesses want.
Nobody gets off the hook here. It is every bit as much end user nihilism as it is AI company nihilism.
Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
People don't want it, you make the demand with hundreds of millions of spend on advertising (brainwashing), influencers, and social proof.
The capital holders don't sit back and see which company wins so capital can be directed to the best solutions, they buy up the competition, they pay corrupt politicians, they use current v market placement to embed their products name into billions of workplace computers and spread stories about how it's going to 'increase productivity but never at the cost of jobs', etc.
Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
Not really.
> Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
Oh but I do.
Collective social guilt is a thing. e.g. who is to blame for drug violence? Not only drug dealers.
AI spending is a bit like Pokemon games - most of your players are free to play, only earning you traffic and maybe some more players after recommending it to their friends - that's the average AI user. Around 5-10% of players pay a bit to the game - maybe to unlock some extra characters - those are the $20-$200 AI subscription users, with these you are not going to fund the whole business but it's some side cash. Then you have the 1℅, which spend massive amounts of money onto the game, and those are the ones primarily funding it - that's the business spending and where most of the money actually comes from.
You can only blame the average person for using the cloud service and recommending it to others.
Anyway, the only "whales" funding the AI companies is venture capital. The reason it's not a viable business (yet, so far, whatever) is because they spend more money than they make.
These firms have been using computers to make decisions about trading billions of dollars for over 30 years.
At the same time, I've been involved in hundreds of After Action Reviews of incidents and it's not always simple or easy to both figure out which party was ultimately responsible and also how to allocate the "dollar error budget" to the parties overall.
(This could be a blog post in and of itself btw)
Instead, this is a legal question: What were the damages? Was the target company negligent in setting up security? Were criminal laws broken? Etc. [IANAL, so forgive the lack of precision.]
If hacked companies had losses, they have an incentive (even a fiduciary duty) to recoup their losses. They do that by suing the company (as, e.g., authors sued Anthropic for copyright infringement). If they haven't sued the company, it means either (a) they didn't have losses, or (b) they're still working on the lawsuit and haven't filed it yet.
The other dynamic is that frontier labs feel competitive pressure to take risks to develop their models. Which risk is higher:
A. A new model being tested hacks into a sensitive site (bank, government, etc.) and the company has to pay a massive fine (or even face jail time).
B. A competitor comes out with a significantly better model and they lose significant market share.
That is a hard choice to make. The frontier labs are asking for government restrictions--that apply to all companies--so that they can minimize A without having to worry about B.
They are trying to minimize their risk (and thus maximize their profits).
But I could be wrong.
Can anyone show me an example? And I mean a real example - not an inferred example. Someone saying "It wasn't me, it was my AI" or someone not being held responsible for the actions of their AI.
I won't hold my breath.
EDIT: Heh... no examples. I rest my case.
The argument from the frontier labs isn't, "We're worried that AI is going out of control, please stop us" it is, "We're taking a lot of risks to compete, so please limit competition."
No, they really don't. I suppose I agree to some extent that no one is trying (hard) to deflect blame, because the people in control are barely a part of the conversation in the first place.
Almost all of the reporting on these stories assigns the blame to the chatbots first. Most prominently through the language/wording that gets used -- it's usually e.g. "BOT infiltrated ...", "Agents ruined the ...". I mean, they call them agents. I'm not trying to claim this terminology was selected as an entirely strategic move, but it sure does seem convenient. Calling them agents shifts the conversation away from the real human people and companies that are controlling them. This extends well beyond this one word, as well. The (captured) media picks up on and adopts the language used by the tech/AI companies.[0] In this way (among others) these companies can influence how the media portrays them indirectly.
> Instead, this is a legal question
There are legal questions, but the two (legal, ethical) aren't mutually exclusive. The criminal law question is the most significant one here, I think. If you do what they did as an individual, you are pretty likely to get visited by the FBI.
You're skipping over a couple of important points, I think:
- The AI labs clearly stand to benefit from the "AI is dangerous narrative". We know this for several reasons, probably the most significant one is that they keep telling us how the new one is super extremely dangerous, and proceeding to make no changes to their behaviour. It's marketing.
- The AI labs are clearly in control of their bots. It's a computer. Networks are not mysterious. Computers don't do surprising things. The principles and technologies involved in securing a network are mature and there is a ready supply of trained workers available worldwide, keen to apply those principles and deploy/manage those technologies. You can believe that something went wrong and the various hacks and excursions were unexpected. In that case, the labs are clearly incompetent[1] and also, they're still to blame. Repeatedly running "experiments" that you can't control is, in fact, a failure of basic morals and ethics.
[0] Of course they do, right? Well there used to be a distinction made between experts and marketing departments. These days it seems a lot of journalists think that marketing copy is as good as any other source.
[1] Surely they hired someone that can setup a firewall.
Yes, exactly. The issue isn't, "we don't know how to sandbox an agent", the issue is that they need to test the agent with full internet access because that's the product they are selling. They are selling a product that does things for you on the web. You have to be able to test it with a real web connection.
Maybe the answer is to simulate the web. What if you ask an agent to reproduce the key parts of the web (travel sites, government sites, etc.); effectively, clone a real, working version of the web, but inside a simulator. Now you could test the agents inside the simulator. Since "coding is a solved problem" maybe that wouldn't be as hard as it sounds. Right?
Suppose I go to the park and find one of several guys who would do it for free. I get all the of benefits, he might do an excellent job, but, because there's no contract, I still take all the blame.
Suppose I actually hire a professional to do it. I receive fewer of the benefits. However, he's much more likely to do a good job, and the legal system provides a variety of ways to hold him accountable for the work.
Suppose I hire a professional, but then insist that he use legions of guys from the park to do the actual work, since that's going to be some much faster and cheaper per quantum of work. I also tell him that if he's unwilling to do that, I'll find someone who will. I also tell him that, since he has so much more unoccupied time now that the original work is in the hands of all those other dudes, he gets to supervise still more work done by other swarms of guys from the park.
At this point it should be clear that I'm not interested in quality work, but in having someone poised to absorb blame. I want a patsy.
Now replace "guys at the park" with LLMs, and replace "free" with "nearly free, but with no warranty."
It does not matter if the CEO is an AI or a human. In either case, of course they can make decisions -- and be held responsible. Not in any emotional sense (that we seem to want to bake into the concept for added confusion), just that if you are not satisfied with the result, you can replace the human or AI.
I am not saying that this is a fun vision of anything, but why are we making it more complicated than it is? The parts are all there and explained.
My understanding is that 'emotional' sense emerged organically in LLMs and was not intentionally baked in.
As silly as it may seem, the ways emotions come through in our words do seem to have an effect on agents. Likely because they were trained on human writing, most of which cannot be separated from the emotions of the writers any more than your emotions can be separated from your logic & reasoning abilities.
While it seems reasonable that you need humans to take accountability, it looks like a very shaky position to assert that computers can't make decisions. And going back to chess computers, we acknowledge that computers make better decisions than humans in that area.
Forbidding computers that can make better decisions than humans from making decisions to keep accountability just seems to be arguing to hire fall guys. You still want the better decision maker to make decisions, but you need someone there to take the fall if things go wrong.
Very weird incentives.
The problem is entirely different. Sometimes they make decisions we don't like, but unlike with people we haven't found a way to punish them that satisfies our sense of justice or discourages other computers from making similar decisions.
You might have an org policy that says that an llm can't make decisions because an llm can't be held accountable - but that's your choice.
You don't need to operate with a model where everything has a human accountable for it in the end.
If you don't like the decisions an llm makes, you can replace it with another or put a human in charge of that decision in the future instead.
I'm not saying it's a good idea, but you can technically do it.
The piece: "Anthropic COULD stop their models from doing what they are doing, and they are actively choosing not to"
As if _Anthropic_ could make a decision.
Companies cannot make decisions! All of these incidents are the fault of people. Companies are not deciding to do this, people are. There is little more to it than that.
They generate text which resembles reasoning and may include a tool call
The harness runs tool calls
The human made the decision to write the harness
The human is responsible
Also phrased as ‘to make an apple pie from scratch, one must first create the Universe’
edit: This wasn’t meant to be a rhetorical question!
If you are driving a car, it's pretty clear where the accountability falls. You control the car. You have free will. If the car hits a tree, you hit a tree.
What happens when a car hits another car? Things get murkier. There are lawyers who have devoted their careers to this. Sometimes nobody winds up at fault. Unavoidable accidents happen.
Now let's ask what happens when a leader orders their military to do something. e.g. Say a President instructs his troops to do whatever they think is necessary to get the job done, and they go and Abu Ghraib a bunch of prisoners. Who is responsible? The leader probably is, ultimately, but there are a lot of human beings with free will in between the leader and the people wielding electrical cords and pliars. Typically, somebody fairly close to the bottom takes the blame, even if most people suspect it should fall higher. Some low rankers will get court marshalled but the leader will have the occasional shoe thrown at him. That's okay. He's good at ducking.
Say you're running a war against people you really just want gone. Your military has a tradition of conducting laboriously researched precision strikes to take out people they don't like. Your people are working hard, but they're only managing a couple strikes a week, and you have so many more bombs than that. You don't even pay for most of them! What if you could just ask a machine to find you the leader's underlings, and their underlings, and so on, right down to raw recruits? Nobody has to check on the machine. Why bother? Just believe the machine and drop the bombs. If anyone ever calls it a war crime, you can just blame the machine, right? Well, not so fast. Most people are smarter than that!
Despite all the sci-fi we consume that portrays computers as having agency, malevolent or not, most people instinctively understand that, in the real world, computers are just tools, more akin to cars than armies. LLM's are just software that run on those tools. Anyone using a LLM can simply choose not to, or they can choose to be very careful in how they use it. There isn't a complex web of free wills to untangle. There's the machine and the person who controls the machine.
We currently have an administration that is desperate to look the other way while U.S. AI companies do things that any reasonable person would hold them accountable for. "Industry will regulate itself!" The economy mostly sucks because of this same government's complete lack of economic sense or even basic self-control, but the AI sector is propping things up. They're not going to regulate the golden goose unless the goose does something so unbelievably stupid that there's no choice. Wouldn't want to pop the bubble!
Yes, we are headed for more stupidity. The "tequila and handguns" kind of automated stupidity that only a computer and a truly feckless operator can give us.
https://www.hec.edu/sites/default/files/documents/Computing%...
First of all, looking past the objectively wrong phrasing (computers can, and clearly do make decisions), can computers be accountable? This might not be so cut and dry as to instantly say no. This one will keep ethics, philosophy and legal practitioners busy for some time. It's certainly not going to be decided in HN comment or a blog post.
Second - while the AI labs have shown utter incompetence in properly sandboxing their agents, intent is still important. I don't think the labs deliberately meant for their agents to hack this or that. Should they be accountable? Yes but I wouldn't go as far as saying this is deliberate.
The next point is about cherry picking some "doomsday" scenarios like AI ending humanity and then blaming this on the reader (!) for paying a subscription. - People pay because AI is USEFUL, and massively so. You could just as easily pick incredible achievements propelled by AI, in mathematics, software, reverse engineering, role playing. Unlike the "end is nigh" ideas, these advancements are actually real, and they are happening right now.
And last, the author says AI labs can just stop the agents at any time. I agree there needs to be better discovery and mitigation, sandboxing and monitoring. But when you run a billion agents, it's always going to be a numbers game and there will always going to be some challenge in fully containing all breaches. This will only get more difficult with better models, because they're getting smarter and they know how to work around things, sometimes better than we do.
So what's the solution here? There are options, but they're all tradeoffs. I hope we get better at this and maybe working on cutting edge models should breed some new best practices which were once only reserved for defense tech.
Revisit later if the comparison starts to become ridiculous.
Characterizing the difference between humans and AI is going to be important.
Pointing out that AI is software isn't helpful in answering that question.
If I write a program:
if (rand() % 2) launch_missles();
And wire that function to actually launch a missile, the computer didn’t make a decision. And so it is with people prompting agents.
There is no end of the world, nor autonomous agent decisions nor any fantasies they are building up to make people believe they have a pandora box in their hand.
What we instead have are lies crafted to lure non tech people and keep this running as long as they can.
Oh and let's also not forget that they are allowed to illegally download basically as much as they want from libgen/annas archive/torrents to train their models under the "fair use" umbrella, yet mere mortals can go to jail for way much less.
Has the alignment problem been solved, then?
mistakes happen. sometimes catastrophic decisions (or indecision) are made, and people and companies are held accountable for them, or not
some companies are too big or too important to fall. we can all agree or disagree on things, but what AI specific behavior are we actually talking about?
No company is too big or too important to fail.
Some, unfortunately, survive because of the expedient choice to keep them in place.
"JO-NA-THAN!"
ETA: more to the point, neither OpenAI nor Anthropic are too big, too important, or too structurally essential to fail. (Slightly more challenging to make the third claim for Google or SpaceX, but either of those could see their governmentally/militarily essential parts nationalised).
If the USA grants either Anthropic or OpenAI too-big-to-fail status, and that privilege is invoked in a crisis, it could mean the end of the US economy.