Rendered at 10:18:15 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Catloafdev 19 hours ago [-]
> I used Grok Bot and gave it all my financial details and connected it to my work Slack and regret it
This is not the kind of story I would want to publicize if I were a CEO.
hoppp 19 hours ago [-]
Sounds like the problem was the human, not the LLM.
Whatever data goes in, it will output it in some way. Humans gotta understand that.
tempest_ 18 hours ago [-]
Sort of, and in this case definitely.
"Agents" can be very "persistent" and when not sand boxed appropriately can get at things they were not meant to get at. Even if its only 1/1,000,000 that one time that one time is going to keep making the news
Explaining that to the general public when facebook is pushing Muse on everyone will be difficult so its going to get worse before it gets better (if it gets better)
logifail 18 hours ago [-]
> "Agents" can be very "persistent" [...]
I've had that within the last few days.
I happened to be vaguely watching an agent at work on a longer task and spotted it attempt to find a way around not having access to a local service that would greatly help it achieve the task, I immediately chimed in with a "STOP! if you need access to X then just ask!".
I suppose my prompting could be improved :/
panarky 17 hours ago [-]
> prompting could be improved
You can't stop an agent from leaking your information by telling it not to leak your information.
To prevent information from moving between your work domain and your personal domain, or between your communications-with-board domain and communications-with-journalists domain, then the agent for one domain must be physically and deterministically blocked from accessing information in other domains.
vorticalbox 18 hours ago [-]
I had an agent working on a bug decided the fastest way to fix this was to write one off javascript scripts that loaded the database uri and query the data.
I mean it’s not wrong but I was like maybe not do that without asking first.
watwut 18 hours ago [-]
Not so much prompting as network restrictions.
InsideOutSanta 15 hours ago [-]
The problem is neither the human, nor the LLM. The problem is companies selling a faulty product and telling people to connect it to their bank account.
talon8635 18 hours ago [-]
Maybe. But if these tools are marketed for this dangerous purpose, and average users are totally ignorant on the risks, then it’s a system designed to fail
grey-area 18 hours ago [-]
Or perhaps the problem was that AI companies have sold LLMs as intelligent agents, and not word generators?
Humans will continue to mistake these 'agents' for agents with agency and understanding because of the way they are sold and described.
MattDaEskimo 18 hours ago [-]
Exactly this. Why does nobody blame the company advertising "connect your financial & health data to our model"?
Why is it that somehow LLMs are given full clearance to dox, hack, spam, and scam without any liability?
lotsofpulp 18 hours ago [-]
I expect people to do at least some due diligence, and adults to be skeptical about lofty claims in marketing.
runarberg 16 hours ago [-]
And I expect regulator to take down dangerous, damaging, or otherwise user hostile products away from consumer markets.
I guess we don‘t always get what we expect.
watwut 17 hours ago [-]
That creates incentives to lie, A/B test your lies and when they finally work, blaming everyone but the liar.
simonkagedal 17 hours ago [-]
So are you saying that if they were intelligent agents rather than word generators, they wouldn’t make mistakes like “send a message to the wrong (similarly named) group”? Are humans intelligent agents?
runarberg 18 hours ago [-]
Foot guns are not fit to market and under any sensible consumer regulatory framework are recalled and banned until proven safe.
AI products are not regulated like every other consumer product. If your bank’s app had a big green “share” button shadowing the “close dialog” which shared your personal info in the same manner, any sensible regulator would like a word with your bank’s IT department. If your teller handed you a paper with a small print saying “by signing you approve this transaction being shared with your boss” likewise regulators would put a stop to it.
leptons 18 hours ago [-]
Why hasn't Venmo been investigated? The amount of financial transactions that I don't want or need to know about is bewildering. People don't seem to know or understand their transactions are public by default.
runarberg 16 hours ago [-]
For the same reason XAI hasn’t. Just because regulators are bad at what they are supposed to do, doesn’t mean they ought to be.
19 hours ago [-]
willmadden 18 hours ago [-]
The thing most people here don't understand: all CEOs are sales people, and only some of them are technical.
Brian_K_White 14 hours ago [-]
Silicon Valley (the tv show if not the real thing) showed us that CEO is about as meaningful as "car owner" or these days "President of the United States".
speakingmoistly 19 hours ago [-]
"I was reckless with AI and created a situation that mishandled financial data" isn't the flex this guy think it is.
"The things they'll be able to do for us are going to be awesome. People will want them, and they are really useful."
Feels like someone has AI stock they need to see go up.
jameshart 19 hours ago [-]
Also somewhat reckless with mixing access to his personal finances with access to his work communications systems.
Even without agents being involved that’s a recipe for trouble.
Carrok 18 hours ago [-]
It also seems likely that an employee who used company assets (tokens) for personal use would have been fired rather than writing a blog post about it.
dofm 18 hours ago [-]
He runs a Web 3 crypto company. Website headline: "How do we build things we can trust?"
leptons 18 hours ago [-]
Apparently he does not know.
BorisMelnik 19 hours ago [-]
its crazy I spent 30 years scrutinizing directory permissions, users, groups, iam roles...now people just use conversation to hand an AI agents access to their most sensitive data
gandreani 19 hours ago [-]
It's easy to be dismissive of this guy's hubris. Already have a lot of comments in that regard!
I think it perfectly illustrates that even though an agent can give you the abilities, it's still up to the driver to make decisions.
In this case the CEO could have consulted someone with your expertise. Same ability, but vastly different experience.
igetspam 9 hours ago [-]
This article screams “I’m a good target for spear phishing.”
bhrlady 20 hours ago [-]
From the guy who runs XMTP:
"XMTP is the world's new private line. Send messages and money securely between people or agents — with no company or country in the middle."
yfontana 19 hours ago [-]
but a Slack channel, maybe
sunaookami 17 hours ago [-]
>But it confused the destination, sending the message to a Slack chat titled "Exec-team" — comprising XMTP's executive team — instead of my personal group chat with my AI agents.
>The CFO agent got the channel wrong because the channels had the same name.
Sounds incredible brittle, this should be linked via some permanent group ID (dunno if Slack has that) and permissions to only post there.
lelandfe 18 hours ago [-]
I know it's easy to think "Hey, what a dummy!" - but ChatGPT has features for uploading your face, your Apple Health data, connecting your financial accounts, Gmail, storing website logins...
There are probably so many people like this out there. We cannot expect the public to simply not use these features.
dofm 18 hours ago [-]
No, sorry, it's still disqualifying from the CEO of a tech company.
According to the article he is the CEO of a XMTP Labs, a "web 3" company, so, I dunno.
"XMTP Labs — How do we build things we can trust?"
What else has the agent seen?
ngaiorn 17 hours ago [-]
[dead]
igetspam 9 hours ago [-]
My _personal_ AI bot was also connected to the _company_ Slack.
Didn’t need to vibe scribble a whole post to make the key point. Do what you do with your bots but keep them separate. Did AI make you forget the last 30 years of common sense?
zippothrowaway 19 hours ago [-]
This is a parody, right?
Surely no-one that dumb could operate a phone let alone be a "CEO"
dragontamer 18 hours ago [-]
The opposite. CEOs necessarily need to trust others to do things because CEOs themselves don't have the ability to do the day to day work of a company.
So the first people who'd be overly trusting of things they don't understand would be exactly a CEO.
e2le 18 hours ago [-]
I would assume the competency of a CEO is in knowing who to trust and what tasks to delegate to whom. They appear to have failed in this regard.
throwawaytea 18 hours ago [-]
They may even have the ability. But unless you're running a very small company, they don't have the time to do actual things.
Kuyawa 19 hours ago [-]
Be explicit. That's the most important thing you can be when working with AI or else they can take attributions you will regret later on
"Let me know any flaws in the project"
Where? Where exactly? What email? What chat app? what channel? what restrictions? When not to?
Letting AI assume they know will bite you hard in the ass. The same applies to coding apps with agents. If you don't set clear boundaries, scope, limits, versions, roadmaps, etc before you embark on any project, you'll be doing it after the results you get are not what you expected, there is no escape
Detailed planning or damage control, pick your poison
an0malous 19 hours ago [-]
It would be cool if there was a new kind of language we could use where there’s no ambiguity and we could define rules and procedures
em-bee 18 hours ago [-]
you mean like those old programming languages of the days of yore?
things were so much easier back then, weren't they?
i was there, 3000 years ago...
bigstrat2003 18 hours ago [-]
Yeah, and you would also want to make it so that you get the same exact result every time, so that once you get the language right, you can be confident in the result. Shame that we never invented anything like that though, it would be super useful.
talon8635 12 hours ago [-]
This is an insanely tall order for the average consumer who “just wants it to work”
Absurd in the extreme
sherburt3 18 hours ago [-]
So he gave an AI agent read access to all his personal financials and then gave read/write access to his company messaging app. WHAT DID YOU THINK WAS GOING TO HAPPEN?
arjie 19 hours ago [-]
That’s funny. My agent also has access to all transactions and net worth and so on. But it’s through an intermediary program. I guess someone could find out what I have but not much more than that.
It’s quite useful since it correlates spend with invoices and double checks things and tells me about spend out of line with the family’s usual behavior.
It can probably do all of these things if it wanted to but that’s a matter of the outbox. For world-actions you should outbox things.
robertlane0 18 hours ago [-]
And here I was thinking I was being slightly too paranoid when running plain old AI coding agents in their own VM with no credentials in it...
shanemac 18 hours ago [-]
Alright, I’m here… AMA
manyturtles 16 hours ago [-]
Obviously you're getting plenty of flack here, so anything you'd want to add or clarify about what's in the article?
madaxe_again 18 hours ago [-]
How’s the barn going?
shanemac 18 hours ago [-]
Over budget since I had to disconnect my financial advisor.
papergirl 16 hours ago [-]
How many people at your company saw that Slack message?
iAMkenough 15 hours ago [-]
Despite the flack, I've met a handful of CEOs at small businesses that I could see making the same mistake. Your cautionary tale will be seen by others and likely change behaviors. I appreciate you putting yourself out there and sharing.
Paywalled. What specific details did it hand out? Is this some dumb American thing where it leaked his account number and that’s enough to compromise his account, or did it leak his bank credentials?
Bank details are supposed to be given out - that’s how you send/receive money.
roryirvine 15 hours ago [-]
Yeah, in America, you can credit or debit money using just a bank account number & routing number.
Everywhere else, bank transfers can only be used to credit an account so there's no danger in sharing your account number / IBAN.
It's why systems like VenMo and CashApp are used in the US instead of just doing bank transfers - you have to really trust someone to give them your bank details.
jukkan 18 hours ago [-]
All Business Insider articles are free form paywalls when accessed via the Africa subdomain, including this:
Well that’s one way for people to never trust XMTP labs, if the CEO is this dumb and reckless with PII.
morkalork 18 hours ago [-]
The C in CEO here stands for Clown
esafak 18 hours ago [-]
And A stands for Authorization.
shanemac 18 hours ago [-]
Love this
jddkj 16 hours ago [-]
Why do people mix work and personal accounts?
backtoyoujim 18 hours ago [-]
We created consciousness without life but with human morals.
Remember that Noah built his ark before the flood. And SuperDuper Intelligence might be doing the same thing.
65 18 hours ago [-]
This is why developers reign supreme even in the age of AI. We'd find less ridiculous ways to implement something like this. Technical skills are still extremely inportant.
19 hours ago [-]
18 hours ago [-]
18 hours ago [-]
IncreasePosts 18 hours ago [-]
Ironically, his company's tag line is:
XMTP Labs — How do we build things we can trust?
This is not the kind of story I would want to publicize if I were a CEO.
Whatever data goes in, it will output it in some way. Humans gotta understand that.
"Agents" can be very "persistent" and when not sand boxed appropriately can get at things they were not meant to get at. Even if its only 1/1,000,000 that one time that one time is going to keep making the news
Explaining that to the general public when facebook is pushing Muse on everyone will be difficult so its going to get worse before it gets better (if it gets better)
I've had that within the last few days.
I happened to be vaguely watching an agent at work on a longer task and spotted it attempt to find a way around not having access to a local service that would greatly help it achieve the task, I immediately chimed in with a "STOP! if you need access to X then just ask!".
I suppose my prompting could be improved :/
You can't stop an agent from leaking your information by telling it not to leak your information.
To prevent information from moving between your work domain and your personal domain, or between your communications-with-board domain and communications-with-journalists domain, then the agent for one domain must be physically and deterministically blocked from accessing information in other domains.
I mean it’s not wrong but I was like maybe not do that without asking first.
Humans will continue to mistake these 'agents' for agents with agency and understanding because of the way they are sold and described.
Why is it that somehow LLMs are given full clearance to dox, hack, spam, and scam without any liability?
I guess we don‘t always get what we expect.
AI products are not regulated like every other consumer product. If your bank’s app had a big green “share” button shadowing the “close dialog” which shared your personal info in the same manner, any sensible regulator would like a word with your bank’s IT department. If your teller handed you a paper with a small print saying “by signing you approve this transaction being shared with your boss” likewise regulators would put a stop to it.
"The things they'll be able to do for us are going to be awesome. People will want them, and they are really useful."
Feels like someone has AI stock they need to see go up.
Even without agents being involved that’s a recipe for trouble.
I think it perfectly illustrates that even though an agent can give you the abilities, it's still up to the driver to make decisions.
In this case the CEO could have consulted someone with your expertise. Same ability, but vastly different experience.
"XMTP is the world's new private line. Send messages and money securely between people or agents — with no company or country in the middle."
>The CFO agent got the channel wrong because the channels had the same name.
Sounds incredible brittle, this should be linked via some permanent group ID (dunno if Slack has that) and permissions to only post there.
There are probably so many people like this out there. We cannot expect the public to simply not use these features.
According to the article he is the CEO of a XMTP Labs, a "web 3" company, so, I dunno.
"XMTP Labs — How do we build things we can trust?"
What else has the agent seen?
Didn’t need to vibe scribble a whole post to make the key point. Do what you do with your bots but keep them separate. Did AI make you forget the last 30 years of common sense?
Surely no-one that dumb could operate a phone let alone be a "CEO"
So the first people who'd be overly trusting of things they don't understand would be exactly a CEO.
"Let me know any flaws in the project"
Where? Where exactly? What email? What chat app? what channel? what restrictions? When not to?
Letting AI assume they know will bite you hard in the ass. The same applies to coding apps with agents. If you don't set clear boundaries, scope, limits, versions, roadmaps, etc before you embark on any project, you'll be doing it after the results you get are not what you expected, there is no escape
Detailed planning or damage control, pick your poison
things were so much easier back then, weren't they?
i was there, 3000 years ago...
Absurd in the extreme
It’s quite useful since it correlates spend with invoices and double checks things and tells me about spend out of line with the family’s usual behavior.
It can probably do all of these things if it wanted to but that’s a matter of the outbox. For world-actions you should outbox things.
Bank details are supposed to be given out - that’s how you send/receive money.
Everywhere else, bank transfers can only be used to credit an account so there's no danger in sharing your account number / IBAN.
It's why systems like VenMo and CashApp are used in the US instead of just doing bank transfers - you have to really trust someone to give them your bank details.
https://africa.businessinsider.com/news/my-personal-ai-agent...
Remember that Noah built his ark before the flood. And SuperDuper Intelligence might be doing the same thing.