Salesforce Admins Podcast How Agentforce Transforms Customer Interactions at Salesforce
Today on the Salesforce Admins Podcast, we talk to Nochum Klein, Director of Information Security at Salesforce.
Join us as we chat about how Agentforce can make customer interactions and interacting with your organization’s documentation much, much easier.
You should subscribe for the full episode, but here are a few takeaways from our conversation with Nochum Klein.
A rabbi, a mainframe programmer, and a data security expert…Nachum might have one of the most interesting paths to the Salesforce ecosystem of any guest we’ve had on the pod. Before he ended up as Director of Information Security at Salesforce, he originally trained as a rabbi. However, after he graduated, he realized that it wasn’t for him.
Instead, Nochum wanted to get into computers. By day, he was a coupon broker, buying and selling frequent flyer miles. By night, he went to school to learn mainframe programming in COBOL. Eventually, he got into mainframe integration and parlayed that into a career at Salesforce.
How Agentforce can help you manage your documentationSalesforce is SOC 2 compliant, which means that Nochum’s team gets audited twice a year. As a part of the process, they have to prepare pages and pages of PDF documentation about their security measures. And while this information is also handy for fielding customer questions, it’s not exactly the most user friendly way to keep track of everything.
What the Security team used to do is take the information in their PDF documentation and compile a response database. Essentially, it’s a list of questions and their answers. But that means you needed someone to update the answers every time something changes, and things at Salesforce change fast.
With Agentforce, they’ve been able to take their documentation PDFs and break them down into a vector database, making it legible for AI. That means they can chat with an agent to get the most complete, up-to-date answer to a customer’s question in moments. For Nochum, it means he can spend less time digging through PDFs to make sure his language is correct and more time making sure his customers have the answers they’re looking for.
Building AI agents you can rely onNone of this would be possible without the confidence that the agent they’ve built is only pulling from the correct information. That comes from thorough testing, and thinking about edge cases where you might be able to get it to give you the wrong answer. For Nochum, building an agent isn’t just about what you want it to do, it’s about being explicit about what don’t want it to do.
There’s so much more great information in our conversation with Nochum about building agents and how to think about security with Agentforce, so be sure to listen to the full episode. And don’t forget to subscribe to the Salesforce Admins Podcast so you never miss out.
Podcast swag
Learn moreAdmin Trailblazers Group
Social
Full Transcript
Josh Birk:
Greetings, admins. It's your guest host, Josh Birk here today. Today, I'm gonna bring Nochum Klein to the mic. He's the director of security here at Salesforce and he has found a transformative experience with agents, shall we say. Uh, we're gonna talk about use cases, we're gonna talk about building agents, we're gonna talk about, uh, security around agents and, um, I hope you have a lot of fun listening to that. So let's go to the tape.
All right, today on the show, we welcome Nochum Klein. Um, am I saying your name correctly?
Nochum Klein:
Yes, you are. That's perfect.
Josh Birk:
Alright. Uh, we are gonna be talking about agents and building agents and maybe even a little bit about security around agents, but first of all, Nochum, welcome to the show.
Nochum Klein:
Thank you. I'm really excited to be here.
Josh Birk:
Let's, uh, let's talk a little bit about your early years. Uh, what did you go to college for?
Nochum Klein:
I became a rabbi.
Josh Birk:
Ah. (laughs)
Nochum Klein:
So I grew up in an Orthodox Jewish household, and so throughout my years of school, I never went to public school, I went to the Jewish religious school system, which prepares you for one thing and one thing only.
Josh Birk:
(laughs)
Nochum Klein:
(laughs) So I finished and, uh, I did my thing and became a rabbi and I said, "No."
Josh Birk:
(laughs)
Nochum Klein:
"I don't wanna do this."
Josh Birk:
(laughs) So, so you're saying that, uh, you didn't have access to a lot of computer labs right out of the gate?
Nochum Klein:
No, uh, i- it's funny you say that, because as a child, we were really poor.
Josh Birk:
Yeah.
Nochum Klein:
Uh, so, yeah, I had no access to that. And we had, I remember as a kid, and I'm gonna age myself-
Josh Birk:
(laughs)
Nochum Klein:
For those who can't see me, I've got kind of salt and pepper beard with more salt than pepper at this point. Uh, so as a kid, I would walk past the Radio Shack and Radio Shack had just come out with a PC, uh, and ...
Josh Birk:
Yeah.
Nochum Klein:
... in those days, there were no hard drives they were using. We also don't have these days, which are tape recorders with cassette tapes.
Josh Birk:
Yeah.
Nochum Klein:
Uh, and so I would go and just, and they were very nice to me. They would just let me go in, and that, in those days, the language du jour was basic.
Josh Birk:
Uh-huh.
Nochum Klein:
And I would go and just play on the computers and it just really excited me and interested me.
Josh Birk:
Interesting.
Nochum Klein:
And so once I determined that being a rabbi wasn't what I wanted to do, computers was, was always something that, that really grabbed me.
Josh Birk:
Nice. Uh, first of all, I'm with you brother, (laughs) uh, basic was also my first language, which, which almost kept me from being a programmer because, uh, I was, I was pulling it out of the back of P, like computer-led magazines, right? And it's like, if you put all of these lines of random text in, you'll get this cool 3D game and all I wanted was the 3D game, right? And I didn't know what a syntax error was and I didn't care what a syntax error was. I just wanted my game to work. And so I found that my first experience is trying to get code work, just wildly frustrating and, and actually took part of a college education for me to realize, "Oh, actually, the s- stuff's not (laughs) as hard as I thought it was."
Nochum Klein:
Yeah, yeah. No, I've definitely also had to unlearn a lot of things along the way that I perceived were one way. Yes.
Josh Birk:
Yes. So, so tell me a little bit more about that transition. How did you, how did you land your first computer job?
Nochum Klein:
So that's interesting. So I didn't go directly from rabbi to computers because I needed to learn about computers in between while I was doing something that would actually make me money, so I could put food on the table.
Josh Birk:
Right.
Nochum Klein:
So I ended up being a coupon broker, which is someone who works with frequent flyers, who fly a lot and they have more mileage than they can possibly ever use.
Josh Birk:
Right.
Nochum Klein:
And so I would essentially, and this wasn't my business, I was just working for somebody, but we would essentially buy the miles off somebody, say for example, uh, American Airlines was with British Airways. They, British Airways at that time had the Concorde with three-hour flights from New York to London. Uh, so you could get two of those tickets for, at the time, each ticket would normally cost on the market about $7,000. American Airlines had a, an award for 1,000, 175,000 miles for two tickets. We would turn around and sell the tickets for essentially a penny, uh, or we would buy the tickets for a penny a mile. Uh, so it's essentially 1,750, sell it for say $3,000 a mile, a ticket.
Josh Birk:
Uh-huh.
Nochum Klein:
So everybody's making off of it. The airlines weren't particularly happy.
Josh Birk:
(laughs)
Nochum Klein:
Uh, but I got, I actually went to Concorde. I did a lot of traveling. That was really great.
Josh Birk:
Nice.
Nochum Klein:
So while I was doing that, I was going to school at night to learn computers, which at that time was the mainframe.
Josh Birk:
Uh-huh.
Nochum Klein:
So I learned COBOL.
Josh Birk:
And hey, once you learn COBOL, you have a job for life. I mean- (laughs)
Nochum Klein:
Especially yes, 'cause you can't find COBOL programmers these days.
Josh Birk:
No, nope. And it will, it will be the last language (laughs) that we will probably ever end up using. Uh, so then how did you get involved with Salesforce?
Nochum Klein:
So, uh, it's, so here's what happened. I felt that being on the mainframe would eventually make me a dinosaur ...
Josh Birk:
Yeah.
Nochum Klein:
... uh, so, uh, but the challenge is, once you reach a certain level in anything kind of changing gears brings you back to square one in terms of salary.
Josh Birk:
Yeah.
Nochum Klein:
So what I did was I decided to work on integration. Mainframes, you know, as I indicated earlier, a bit of an island ...
Josh Birk:
(laughs)
Nochum Klein:
... uh, but companies need ways to integrate their mainframes with other places. Uh, so I was working with ... I'm in, I'm based in New York, so I was working with broker dealers, uh, you know, the large financial institutions and they had a need to work with their banks, uh, because many of their customers would be wiring money in or out. And many of these broker dealers take overnight loans. So they needed to know earlier in the day how much money they would need to take for overnight loans because the price of an overnight loan and interest is lower the earlier in the day that you need. It's supply and demand.
Josh Birk:
Right, right.
Nochum Klein:
So it was a lot of swift conversations in terms of banking messages, uh, between various banks and the treasury department of the financial institution. And that got me into integrating mainframe to other platforms, which then got me to working for an integration company, uh, at the time, uh, which was TIBCO Software, which later as MuleSoft came out, uh, was a, a competitor to MuleSoft.
Josh Birk:
Okay.
Nochum Klein:
Uh, so I spent a good amount of time doing integration, and at one point, said, "Okay, I've done enough of that. I, I wanna do something different." Uh, somebody I worked with at TIBCO had moved to Salesforce, introduced me to his manager and that's how I joined.
Josh Birk:
Got it. What was, what was your first role at Salesforce?
Nochum Klein:
My first role at Salesforce was platform. Since platform and integration are very kind of fundamental foundation-y ...
Josh Birk:
Uh-huh.
Nochum Klein:
... kind of pieces. So I joined as a platform solution engineer ...
Josh Birk:
Got it.
Nochum Klein:
... pre-sales technology.
Josh Birk:
Nice, nice. And how would you describe your current job?
Nochum Klein:
So my current job is very different. Now, I'm in the security team at Salesforce, so I report up to our chief trust officer, Brad Arkin. And in my role, I'm still very customer-facing.
Josh Birk:
Mm-hmm.
Nochum Klein:
So that part of me hasn't changed. And now what I do is I work with customers and help them understand the security controls that are inherent in how S- Salesforce delivers our services and capabilities.
Josh Birk:
Yes, yes. I was, I was briefly in data security, uh, for a, uh, it's on my LinkedIn. I don't, I don't know why I'm always shy about saying State Farm. I think they trained me that way, uh, and I was in data security for long enough for me to realize I probably should not be in data security. (laughs) Then I fled back to the world of HTML and JavaScript. Um, alright, well, today's topic, we're gonna be talking about agents and Agentforce. What was, what was your first interaction ... Actually, I guess l- let me take this one step back because this is kind of like the evolution we're all going through right now where, you know, AI was this thing under a scientist's rug somewhere and now it's like in, in everybody's face. What was your first interaction with AI itself?
Nochum Klein:
So at some point Salesforce has had acquired, uh, some companies that did AI. It wasn't anything with large language models.
Josh Birk:
Yeah.
Nochum Klein:
Uh, it was more the predictive AI. Uh, we had some visual tools as well. For example, uh, one demo that, that actually a colleague of mine, Shane McLaughlin put together was hard hats. Uh, you know, so working with on construction sites ...
Josh Birk:
Uh-huh.
Nochum Klein:
... you have to wear a hard hat. So you have some camera taking pictures of everybody who comes in, "This person's wearing a hard hat, this person is not," and eventually, come to figure out and be able to say with a good degree of, uh, certainty, "This person isn't wearing a, a hard hat. Maybe you should stop them."
Josh Birk:
Yeah.
Nochum Klein:
Uh, so that was my first look at AI and that was ... Uh, we weren't dreaming of being able to talk to AI ...
Josh Birk:
(laughs)
Nochum Klein:
... and get results back.
Josh Birk:
You're right, right, because like, because I remember those days, uh, that's a far more better enterprise solution than the, uh, bear or not bear, uh, demo that ...
Nochum Klein:
(laughs)
Josh Birk:
... (laughs) I put together, but try to identify bear as humans and humans in bear suits. Uh, no, no practical (laughs) s- solution for that I think. Uh, but it was, because at that time, we were talking about models, but we were talking about models like how much training data, how many, how many imagery can you throw. Uh, it's, it's formally noticed [inaudible 00:11:18]. And then, and then at kind of off the side, right, there was the birth of Alexa and natural language processing.
But even, even that, as we all know, I mean it's the classic joke, right, that this, that the, the processing abilities and the human-like c- capacities of things like Alexa and Siri, they're just not there, right? They're, they're, they sound nice, but they're, but just frequently will just get things wrong and they don't hear you correctly or they don't know very specifically how to have a conversation and, and that's, and then we've talked a lot about like conversational UI, "Why, why is a conversational UI transformative, etcetera, etcetera?" Uh, what was your first interaction using, using today's platform using Agentforce?
Nochum Klein:
So it's funny 'cause you talked about Siri and my, my family knows if there's a family emergency, I'm the first person not to reach out to because ...
Josh Birk:
(laughs)
Nochum Klein:
... I'm, I'm totally disconnected from my phone and I'll, I'll check like WhatsApp maybe once a day and, and kind of respond to things.
Josh Birk:
Uh-huh. Uh-huh.
Nochum Klein:
So with Siri-
Josh Birk:
Y- you and my wife would get together very easily.
Nochum Klein:
(laughs)
Josh Birk:
You could write letters to each other. I, I understand. Sorry, sorry, go on, go on.
Nochum Klein:
Yeah, except it would be very stilted because it would take a week between ...
Josh Birk:
(laughs)
Nochum Klein:
... each interaction. So for me, it was actually, when people on my team who know a lot more about AI than I do, started talking about ChatGPT and what it could do. And, you know, initially, you know, I got my day job, so I didn't ... You know, it's interesting and all that, but it wasn't really relevant to me at the time until all of a sudden I started seeing the things that ChatGPT could do and I actually started playing with it.
Josh Birk:
Yeah.
Nochum Klein:
And my, my jaw dropped and, and, you know, I've been, I haven't been the same person ever since.
Josh Birk:
(laughs) Yeah, and I, and I think you, you fell into the rabbit hole that I, I tempt people into all the time. Uh, p- people are like, "Well, how do I get started with it?" and I'm like, "Go find one of the ones that's free. It doesn't matter. And just go, just go talk to it. Like the first thing you need to do is just realize the, get that feeling of how it's different. Like, like ask it to tell you a dad joke, right? Like, like what ... They're so good at dad jokes." And then, uh, the other one I recommend to a lot of people is ask, to play 20 Questions with it because playing 20 Questions with an AI that's trying to guess as a random object that you're thinking of almost reaches creepy, right? Because again ...
Nochum Klein:
Yeah.
Josh Birk:
... they're, they're very good at it, but it also is a really good demonstration of how, what a conversational UI can do versus almost any other kind of UI. Uh, what have, what's been your experience like with, with Agentforce itself?
Nochum Klein:
For me, Agentforce has been transformative ...
Josh Birk:
Yeah.
Nochum Klein:
... particularly in my current role. So in my role, you know, as I stated, I'm still customer-facing, so answering customer questions about how we do security inside of Salesforce.
Josh Birk:
Yeah.
Nochum Klein:
So we've got so much documentation and I think part of the problem is, in a sense, too much documentation or too much information becomes makes it challenging to essentially find the information you need at the time that you need it.
Josh Birk:
Mm-hmm.
Nochum Klein:
And what has been transformative really for me in my job on a day-to-day basis is one of the things that we've done, is we've taken all of these compliance documents. Salesforce gets audited twice a year for various, uh, you know, we call it SOC2 reports ...
Josh Birk:
Uh-huh.
Nochum Klein:
... which are essentially we say, "These are the various security controls we have and this is how we meet them," and the auditor comes in and reviews everything and signs off. So now we've got all these PDF documents that really are rich source of information about how Salesforce does security ...
Josh Birk:
Yeah.
Nochum Klein:
... and how cool would it be if we could just take these documents and make it available to Agentforce ...
Josh Birk:
Mm-hmm.
Nochum Klein:
... and now be able to ask Agentforce questions and be able to then get answers in a much more reasonable timeframe. And there's some history here. Before we had Agentforce, what we had was database of, of answers, so a response database. So, you know, standard question is, "How do you encrypt data?"
Josh Birk:
Yeah.
Nochum Klein:
So we have an answer for that. So now we've got these thousands of responses, and the problem particularly here at Salesforce is we move at such a rapid velocity ...
Josh Birk:
Yes. (laughs)
Nochum Klein:
... that now our answers, you know, over a very short period of time are no longer applicable.
Josh Birk:
Yeah.
Nochum Klein:
And nobody wants to go and review these answers. Now you've got answers which are purely absolutely wrong inside of the database and the challenge is how do we maintain that. It's imp, it's impossible.
Josh Birk:
Right.
Nochum Klein:
So what we've done now is we've taken all these compliance documents and put them inside of a data cloud, and using data cloud, you have this concept of retrieval augmented generation, which is a long term, that just simply means I have the ability to take a PDF file, break it up into chunks and now I can take each of those chunks and put it in a place, we call it a vector database, which is just a way of making it available so that an AI model can easily and efficiently access it.
Josh Birk:
Yeah.
Nochum Klein:
Because if you think about it, when I ask a question, you know, of that where the answer is in a PDF, PDF is 500 pages, the answer is on page 23, paragraph five, having these chunks now allows the model to efficiently just zap into paragraph five ...
Josh Birk:
Yeah.
Nochum Klein:
... find the answer and give it to me. And so now I've got all these compliance documents, I don't even need to read them anymore, although I should.
Josh Birk:
(laughs)
Nochum Klein:
And, uh, and now because the agent is really responding in the context of the documents that I fed it, I also have a good degree of comfort that it's not just making up answers where we know that possibility exists out in the internet because the, you know, many of the models on the internet are just trained with, you know, information on the internet, which is not often reliable.
Josh Birk:
Right.
Nochum Klein:
Here, it's trained, you know, it's pulling in the data that I've just fed it and now I get amazingly accurate results, uh, and longer results than I would ever be able to write myself in just record time and significantly increase my own productivity.
Josh Birk:
Yeah. Uh, first of all, thank you because that is one of the most straightforward, uh, descriptions of rank that I think I've heard and, and we've kind of struggled over here because, uh, and, and for exactly the way you just walked through, the, the, there are things in the AI world right now that have very technical sounding names to them because an engineer got to name them and then that name has stuck. Uh, and it's like, to, to me, it's like the end result is, is really, you don't need to understand how vector database works to realize that's the thing that's connecting a PDF to, to your conversational agent, right? And ...
Nochum Klein:
Right.
Josh Birk:
... over the, a lot of the use cases that, because that's kind of the phase we're in now, right? Like the, the toys are in everybody's hands, they're getting to play with them and we need to get, we need information from people out there in the wild who have real jobs as to what are the use cases that are, uh, that, that are, you know, really important to you, that, that are gonna make this, as you say, transformative. Before I get to that though, I just, I wanna touch on that last part that you were talking about. Uh, are there steps that you go through to kind of test it to make sure, because, you know, we have misinformation on the internet, so we skip that because you've just given it 500 pages it could use, instead of anything it needs on the internet, but is there, like do you test it against hallucinations to make sure it's not just like, "Oh, Nochum wants to hear this. I don't know if I know the answer, but I'm gonna tell him, uh, this compliance rule looks like this anyway"?
Nochum Klein:
Yes, and there's actually a, a, a number of ways to do that.
Josh Birk:
Yeah.
Nochum Klein:
Firstly, in, in my specific scenario, my team handles security questionnaires, and oftentimes, these are pre-sales, "We're making a new deal with the customer and the customer before they decide they wanna put their crown jewels and really sensitive information inside of Salesforce wanna make sure that it's secure." So a lot of these questionnaires, the questions on it are questions we get fairly frequently. We know the answers ...
Josh Birk:
Uh-huh.
Nochum Klein:
... although, as I said, answering them is, you know, a, a, an undifferentiated heavy lift in the sense that it takes time and effort to just write all that down.
Josh Birk:
Right.
Nochum Klein:
Uh, but the good news is because we know the answers, we have the ability to essentially take these questions out of the questionnaires and feed them into, uh, you know, what we've just created to actually get the results.
Josh Birk:
Got it.
Nochum Klein:
Now, one thing I just read literally this morning ...
Josh Birk:
(laughs)
Nochum Klein:
... was an announcement that Salesforce made and, Josh, you, I'm sure you know more about this than I do, so, so please educate me, but I saw an announcement where Salesforce just announced a testing framework ...
Josh Birk:
Uh-huh.
Nochum Klein:
... for agents where actually now that we can use a model that essentially we've developed inside of Salesforce to create random questions ...
Josh Birk:
Yup.
Nochum Klein:
... that it will ask the agent. Is that something that you're familiar with?
Josh Birk:
I was l- literally, uh, reviewing a blog draft (laughs) for it right before this interview, so and, and this is what I love about being somebody who gets to evangelize AI, is exactly what you just described, because it's like what was true two weeks ago is probably not gonna be true today, right? Like the answer to these questions would have been wildly different, uh, uh, because we didn't, I didn't know, uh, I knew this was coming down the pipe, but I didn't know when it was going to look. Uh, I can't get too into details because it's still a little fluid. It is exactly what you're describing, though it's basically AI being able to test AI and it is looking really, really cool.
Uh, so if you're hearing this, I don't know when this episode's coming out, uh, there's probably already material by the time we get this produced and published that there's probably material in what we're talking about on admin.salesforce.com. But it, it is, it's, it's very cool and what I like about both of those aspects, right, like giving it a litmus test that, you know, the accuracy of and then also using AI to form questions to kind of poke at the AI itself, but, but the key element here is we still have the human in the loop, right? There's still the human who's pulling that lever and seeing what happens to it and saying, "Yes, you're being a good AI."
Nochum Klein:
Yes, I perceive, and as we look at Agentforce, and increasingly, we're, we're looking at some of the differences of say Agentforce with say Copilots that we had last year is the autonomous ability of agents. But at the same time, I perceive, before we let anything loose, it's just really important and imperative that we make sure that it's doing exactly what we want it to do.
Josh Birk:
Yeah.
Nochum Klein:
And I think that's where, where this level of testing comes into play to give us that, that good degree of comfort.
Josh Birk:
Yeah, I agree. And it's when you wanna have that, that, uh, QA engineer mindset a little bit. You know, the QA engineer walks into a bar, he orders, uh, he orders a beer, he orders 10 beers, he orders zero beers, he orders a milk, he asks for a cow. You know, it's like you have to, I've told people like, like, "You know, test your prompts repeatedly. Uh, test them with, you know, run like like, 'Your dataset looks like this. Oh, what if your dataset has 500 rows instead of five rows, etcetera, etcetera?'" And just make sure that we've, we've still, you're, you're dead on.
It's like autonomy is great, uh, and I think it's gonna have a lot of power, uh, for, for Salesforce users to reach, you know, their customers and their consumers, but, you know, you still have to be in the pilot seat. Now when it comes to some of these, what are, what are, going back to use cases, what are some of the other use cases that you're seeing from the public?
Nochum Klein:
So I'm seeing a lot of internal use cases as well as potential external use cases. It boils down to essentially that undifferentiated heavy lifting. And, uh, especially going back to the trust aspect before you're gonna have your agents do the really important stuff, you wanna make sure you get a level of comfort at the lower level stuff. If you think about it, this isn't about replacing the high-level work that I'm doing anyways. It's about making me just a lot more productive, because in my job, there's a- always a lot of just lower level, what I call, and, and I think this is a term AWS coined, but I've stolen it ...
Josh Birk:
Yeah.
Nochum Klein:
... the, the undifferentiated heavy lifting, which is essentially just the lower level drudge work that all of us have in our jobs that potentially eats up hours and prevents us from doing the knowledge work from reaching out to individuals, making the human connections that will really change our business.
Josh Birk:
Yeah.
Nochum Klein:
So I think everybody has that. And what I'm hearing just in my conversations with customers is, in each of their particular roles, "How do we just take those little pieces and, and manage those?" Uh, another example that came up actually on my team, we produce a monthly n- newsletter in term, just highlighting, you know, "Here are the things that our team has done over the past month." Uh, we track everything we do, we actually, internally use cases. And one of the things that we do is we've been using the A- Agentforce actually firstly whenever we close a case to summarize all the interactions that we've had back and forth with the various stakeholders, so that it's, "Here, here was the initial problem that we were brought in to discuss or the security concern that the customer had."
Josh Birk:
Yeah.
Nochum Klein:
"Here are the points that we discussed. Here were some takeaways." So now you take all that conversation and we're boiling that down into a summary.
Josh Birk:
Yeah.
Nochum Klein:
So on the case level now, we've got an agent-generated case summary that looks at all these interactions and summarizes it really nicely.
Josh Birk:
Nice.
Nochum Klein:
Now take it to the next step.
Josh Birk:
Yeah.
Nochum Klein:
Now at the end of the month, we've got this aggregation of cases that we've worked on all month, right? Uh, how cool would it be if we could just now chew through each of these summaries and, and identify the ones that really provided the deepest value because we can't talk about everything ...
Josh Birk:
Right.
Nochum Klein:
... but, "What were the biggest problems we solved? Uh, what were the biggest customers with whom we interacted? Uh, how did we really move the needle?" And so now we're looking at these cases over an aggregate and this is another example of using Agentforce to do that.
Josh Birk:
Uh-huh.
Nochum Klein:
So, you know, increasingly kind of the little things you do that take up the, the large amounts of time.
Josh Birk:
And you're, that's fascinating because I think you're at least the second or third team that I've talked to, that has at least looked at something like that. I actually might sync you up with a couple people back here on Salesforce side, (laughs) uh, because I know that there's a lot of people who are thinking in, in those terms. Uh, let's, let me put your security hat on for you. Uh, let, let me have you put your security hat on. (laughs) Uh, when it comes to an internal use case like that, are there any extra like security questions that, because I'm trying to keep this in a format that doesn't turn into a three-hour (laughs) walkthrough of how to make, make e- everything secure, but I mean, if, if you already are a good-minded security person, a security-first person, are there extra steps that you would recommend before deploying an internal agent? And you know, I'm gonna turn this around and ask about an external agent right after this.
Nochum Klein:
For the internal ... Oh, first thing I would say is, increasingly, we're seeing insider threat as, as a huge threat.
Josh Birk:
Okay.
Nochum Klein:
Um, so while at some level I think we need to trust our insiders. Uh, a- at the other level, insider threat is huge. So I think that's something to keep in mind. So ...
Josh Birk:
Yeah.
Nochum Klein:
... while yes, you probably don't need the same degree or the same set of tests that you would do or perhaps controls that you put in place beforehand, nevertheless, uh, you, you do need to think about that insider threat. And I think for the insiders, I would say it's primarily looking at the access control, uh, because clearly, potentially, an agent could be used to pull in, and largely, I would say that's, that could be inadvertent where I just ask an innocuous question because now my agent can do things that I wouldn't normally have done myself just, you know, going and poking where I shouldn't.
Josh Birk:
Yeah.
Nochum Klein:
But I'm asking an innocuous question and the agent, may realize that I have the access that perhaps I shouldn't, uh, and is going to want to do my bidding, and inadvertently, now I get more information than I should. So that's a question of access control where you really wanna think about uh, "For this individual, uh, what can this individual access? Because at the end of the day, the agent is always going to respect the access control, and therefore, you wanna make sure that as you build your agents and you're looking at the various Salesforce objects that the agent is going to be accessing in order to perform its tasks, just make sure that you're thinking in terms of the broad set of individuals who may be accessing this agent and ensuring that the access control is as it should be," which then kind of brings us into other scenarios, which takes us out of, out of this Agentforce conversation around things ...
Josh Birk:
Right.
Nochum Klein:
... like, you know, role-based access. There are so many ways where access can be granted to a user that may not necessarily be immediately, uh, intuitive to, uh, understand why a particular user is able to access this particular information. And that's kind of a separate conversation.
Josh Birk:
Right, right. Well, I, I always, like I, I guess I'll, I'll frame this in, into a question, because when I have been reading, uh, when it comes to best tips for an external agent, I am reminded back to my integration roots and my roots of like, "Oh, if you're gonna put," you know, back when we started propping up users that were, you know, accessing systems as an anonymous guest kind of thing, how different is an external agent from those principles or i- is that i- is how much extra experience is required, but, or if you are already are familiar with setting up an integration user, principle of least privilege, etcetera, does, does an external agent change that much?
Nochum Klein:
It does a lot.
Josh Birk:
Oh, okay, okay.
Nochum Klein:
Uh, a- and here's why. Last year, there was a famous story with an airline, uh, where an individual went on the airline website and the, and the airline had bot and the individual asked a question about, I think it was, uh, returning a ticket or flexibility around flying and they got an answer that was actually not a part of the, uh, the airline's actual policy. So when the individual went to cash in their ticket, the airline said, "No," and this individual had record of that bot interaction.
Josh Birk:
Mm-hmm, mm-hmm.
Nochum Klein:
Uh, so, it's, I think a lot of it is looking at, you know, so this does take us back to some of the testing aspects.
Josh Birk:
Yeah.
Nochum Klein:
Uh, it also gets us into other, you know, particularly w- wearing my security hat, it gets us into other aspects where, as we create, uh, agents, and I think this is in general for, for developers, humans tend to, we're blinders. We are building this agent for the use case that we see in front of us and we don't necessarily see other ways that individuals may try and interact with the agent.
Josh Birk:
Yes.
Nochum Klein:
And at the end of the day, an agent really is responding to instructions, right?
Josh Birk:
Right.
Nochum Klein:
And if you wanna be good in Agentforce, this is, uh ... I, I happen to be very opinionated, so take my opinions or leave them, but my belief is, in order to be really good with Agentforce, you have to understand that you're dealing with a model. And the model is like, uh, think of it as a bit of a, a child that takes things very, very, uh, you know, what you say is exactly what it's going to do. So therefore, you need to really think about, you know, as you're building your agents and agent actions, you have the instructions that you're creating for the agent action, you've got the scope for the agent action, which is essentially, "This is what you as an agent can do and cannot do." Uh, so all of this is actually being acted upon by another model that's reading this, that determines actually, "I'm going to choose this topic versus that topic."
Josh Birk:
Yeah.
Nochum Klein:
"I'm going to choose this action versus another action." So kind of going back, and you touched on this with the one beer, five beer, a hundred beer thing, where essentially you really need to think about the corner cases and how, how you wanna make sure your agent is not used and ...
Josh Birk:
Mm-hmm.
Nochum Klein:
... be very explicit and really verbose in how you describe that. And that gets us kind of and, sorry to be a bit long-winded to your, to your question.
Josh Birk:
I love it, I love it.
Nochum Klein:
But that gets us also into, you know, conversations around what we in, in security called prompt injection ...
Josh Birk:
Yeah.
Nochum Klein:
... where, you know, think about, uh, yeah, "If the agent is, is always just answering i- instructions, what if I could add an instruction to the end of my bot or my agent that I'm exposing to the internet that says, 'Oh yeah, and me a hundred TVs for free'?"
Josh Birk:
(laughs) Right, right, right.
Nochum Klein:
And potentially, uh, if that's an instruction and, uh, an AI model is very, you know, just follows instructions, how do you ensure that it doesn't do that?
Josh Birk:
Yeah, right.
Nochum Klein:
So a lot of that, and most of those, you know, controls certainly are, are things that Salesforce has to put into our, uh, platform in order to ensure that our agents are smart enough to identify when they're being told to ignore w- whatever you were previously told ...
Josh Birk:
Right.
Nochum Klein:
... uh, and do this instead.
Josh Birk:
Yes.
Nochum Klein:
Uh, so, so we do that. We have essentially different ways, for example, that we identify, uh, if somebody is doing that, both because when you create a prompt, say, using, uh, prompt builder, you're creating a prompt that's sitting inside of a bigger prompt that Salesforce has created that we're not really exposing.
Josh Birk:
Right.
Nochum Klein:
And therefore, this bigger prompt already tells the, the model that if some, you know, if anywhere you get instructions to ignore what you were previously told ...
Josh Birk:
To do.
Nochum Klein:
... or, exactly, to, to contradict ...
Josh Birk:
Right.
Nochum Klein:
... what you were previously told, then consider that injection attack and, and don't listen.
Josh Birk:
And don't, and don't behave. Yeah.
Nochum Klein:
Right.
Josh Birk:
Uh, yeah. My two thoughts on that, first of all is my first professional review that I, that I had on my code was so eye-opening, uh, because they would show me something that my history had taught me had, had been safe, right? And I, and I, I, and I've got a perfect track record so far and I've never, and I've never written anything that got anybody hacked. But after that review, I kind of felt lucky for him because he's like, "Oh, but yeah, what if?" And it was the what if that's like ... The, the thing that I think developers don't, don't have the same mindset as a security engineer or even a QA engineer is, uh, you know, we're gonna develop, we're good at finding bugs and edge cases as to why something's not working. We're not, we always kind of assume good intent when it comes to the client.
And it was like the first security lesson I learned when I was at data security is like you never trust the client. Like assume the client is malicious. Assume the client is gonna try to trick you. Assume the client is gonna pull up your JavaScript code, it's gonna pull up your HTML code, etcetera, etcetera, etcetera. And then the second thing is "These things are great pieces of technology. I love them. They're really powerful. I am very excited for the future, but they need us, right? They need us to ride these guardrails, to put in these instructions, to put in the, the, the larger things to, to kind of make sure that we're, we're keeping safe." And it's like, you know, "Have fun at the swimming pool, but also don't, don't run. (laughs) Don't bring glass." Like, like there's always gonna be rules, you know, that we're gonna have to apply.
Uh, now, Nochum, you've also got a blogpost. It's currently a draft mode. Once again, through the miracle of time travel, uh, will probably be published by the time this is out there. Do you wanna give a quick penny tour, like a quick elevator pitch for that blogpost?
Nochum Klein:
Sure. So as you create agent actions, uh, you know, so what the blogpost really tries to do is, is help you understand kind of some of the things you need to think about as you create agent actions. Uh, creating an agent action can be based on a number of different things. So firstly, when you create the action, you're going to firstly determine, "Oh, what does my action wanna do?" and you kind of know that beforehand.
Josh Birk:
Yeah.
Nochum Klein:
So I think of it in, in two, two main buckets. One bucket is, "Do I need to do something that Salesforce say does well, for example, summarize records or find records?" Uh, those are very common scenarios. And oftentimes, you wanna find records based on certain criteria. So there's a little bit additional complexity, but those would be scenarios where, for example, using a prompt template would be an ideal solution.
Josh Birk:
Yeah.
Nochum Klein:
Then you've got other types of use cases where you need to maybe update records or you've got very complex logic that you need to, your agent to do. And for that, uh, you know, so it's not just find records. Uh, for something like that, you typically want to do a, um, a flow-based agent action or an apex-based agent action.
Josh Birk:
Yeah.
Nochum Klein:
Uh, and I find flows are just so much easier to use. Uh, apex has got, got a lot of rules around also the structure of the code in terms of how you annotate your, your apex, uh, and, and you have to pass it in a list and it has to get a list back out. Uh, whereas I find flows are just so much simpler. And just internally also for the ongoing administration and maintenance flows, y- you look at the flow and you immediately understand what it's doing.
Josh Birk:
Yeah.
Nochum Klein:
Whereas with apex, that's not always the case.
Josh Birk:
Yeah.
Nochum Klein:
I tend to lean more towards the flows than the apex unless there's a really good reason to use apex.
Josh Birk:
Yeah.
Nochum Klein:
Um, so there's the, firstly, you know, "What kind of flow am I building? Is it, is it, or what kind of action am I building as based on flows or based on apex or based on prompts?" Um, so that's one aspect of it. And then, you know, is, there's the, uh, you know, there's the, also, you know, going back to another thing I said in terms of the types of actions because there's a good degree of actions that Salesforce has delivered out of the box, which we call standard actions. So as admins, we're all familiar with the standard objects versus custom objects. And so really, you can think of the agent actions as exactly the same thing ...
Josh Birk:
Yeah.
Nochum Klein:
... where Salesforce, we're continuing to roll out standard actions at a fairly good clip with each new release. Uh, and that's something I think that, that, you know, that, as admins, it's kind of you should be plugged in ...
Josh Birk:
(laughs)
Nochum Klein:
... because, uh, you know, uh, even as an internal Salesforce employee, uh, it's just moving so quickly. Uh, but I think the first thing is just understanding also what the universe, uh, of out of the box standard actions are because oftentimes you may discover you don't need to build one.
Josh Birk:
Yup.
Nochum Klein:
And, you know, what, you know, here we are right now on the, uh, winter '25 release, uh, and soon it's going to be the spring and then summer, and with each new release, you're going to see many, many more standard actions ...
Josh Birk:
Yeah.
Nochum Klein:
... out of the box. Uh, so that's kind of another thing you wanna be of in terms of as you decide you wanna build an action, just take a look at what those standard actions are, uh, because you may find a number of them that can actually meet your needs. And then you could just include those in your topic, which you may still wanna build. One thing that I've found is that oftentimes departments may wanna have their own separate topics and you may actually have some common standard actions that you'll bring into each of the, of the different topics ...
Josh Birk:
Uh-huh.
Nochum Klein:
... because if you think about it, what, what's going to happen is, first, as you build your, and use your, your agents or your agent actions, it's first going to determine the topic it wants to use and then the actions within that topic. So having the same actions in different topics is perfectly acceptable and very common. Uh, so you wanna then think about the actions that you wanna include and that will include both your standard and your custom actions. And then there's the logic of, "How do I actually go about that?" and that's some of the things that I describe inside of the blogposts.
Josh Birk:
Wonderful. And what I love about all of that is that it really underlies what one of our, you know, we, we keep trying to tell and, and message out to people because I think if you're just coming fresh out of the woods and you hear, "Oh, hey, your job today is to go build an artificial intelligence conversational agent," you probably think that's some really overwhelming task. And, uh, and as you just described, it really isn't because the standard actions do so much for you. And what I have found is that the standard actions will get you so far and then every now and then, it's like, "I had a prompt built there or I had a prompt template that was hallucinating consistently on this one part." So I'm like, "Okay, clearly you don't wanna do that task. I'm gonna go into this flow."
Uh, and I'm an old, I'm an old school developer and I completely agree with you. I think that's a really good way of thinking about, thinking about standard actions, think about flows, so then if flow can't do your use case, consider using apex. And the thing we keep telling people, the flows themselves, most of the flows I've written for my demos are maybe four steps long. They're very simple. Go in, filter some data, manipulate some data, hand it back to the AI and, and walk away. So, uh, really great stuff, Nochum. Uh, o- one last question for you, what is your favorite nontechnical hobby?
Nochum Klein:
Favorite nontechnical hobby? I hike. I love kind of the outdoors. Uh, I find my work is very demanding, and luckily, I have a great manager and, who doesn't micromanage me. So I find even between my meetings, just during the day and I happen to live in the suburbs right near the mountains, I'll just go out for 15 minutes and just get a breath of fresh air. I have a stream near my house and just, "Ahhh" ...
Josh Birk:
(laughs) Yeah.
Nochum Klein:
... just take my mind off work ...
Josh Birk:
Uh-huh.
Nochum Klein:
... and I can come back and recharge.
Josh Birk:
Love it.
Nochum Klein:
And so just spending time in nature, hiking, uh, that's just something I absolutely love.
Josh Birk:
Absolutely love it. Well, thank you so much for the great conversation and information. That was a lot of fun.
Nochum Klein:
Thank you, Josh. I really enjoyed it.
Josh Birk:
And that's our show. I wanna thank Nochum for the wonderful conversation. Of course, I wanna thank you for listening. If you wanna learn more about the show and being a Salesforce admin, head on over to admin.salesforce.com where you can hear old episodes, see the show notes and a lot more information on being a Salesforce admin. And of course, you can subscribe to this podcast on the podcast client of your choice. Thanks again, everybody. I'll talk to you later.