Are Penetration Tests Dead?

If you're not sure a penetration test is worth the investment, you've come to the right place. Assurance IT invited VP of Breach, Detection and Response from GoSecure, Laurent Desaulniers, to chat about whether or not penetration tests are dead. There are many articles and experts in IT suggesting that they are not useful. In this workshop, Laurent reveals the following:

1. Misconceptions of pentests and why they're not right for every business  

2. Two reasons to conduct a penetration test  

3. The #1 Reason why people think Pentests are dead  

4. 10 mistakes people make with their pentest and how to get more value from conducting this test.  

5. Criteria of a good pentest  

6. Defining Vulnerability Assessments and why those aren't pentests  

7. When you should NOT conduct a pentest  

8. Tips on how to get even more value from a pentest  

‍

Transcript is below.

‍

‍

[00:00:00] Ernesto: So I'd like to thank everyone here- everyone for joining. The focus today, Are Penetration Tests Dead? Myself Ernesto Pellegrino, Chief Technology Officer at Assurance IT, helping organizations become cyber resilient.

[00:00:13] I'm joined today here with Laurent from GoSecure. Again, the focus is on pentest. How to get value if you do perform at a pen test and really just uncover those vulnerabilities in your, in your environment and what benefits you get putting in place actions or mediations in order to get better.

[00:00:30] Right? So some housekeeping notes. The call scheduled from one to 2:00 PM today. We do believe we can complete the call in about 45 minutes. We are recording the session. All participants will be muted. I will be the moderator for today's call. So if you have any questions throughout the presentation, we're going to try to keep this as interactive as possible.

[00:00:50] Please just add them in the chat. I will be moderating those questions and asking them either throughout the call or at the end of the call where we will reserve some time for some [00:01:00] question and answer period. So why pentests, right? So like I was mentioning to Laurent, you can't protect

[00:01:08] what you don't know, essentially. So, you know, pen tests are still and have been a tool that organizations could use really to uncover those vulnerabilities. And when we look at, you know, how, how we can protect, there are several approaches you can use to enhance the protection of your network and devices

[00:01:27] right. The Canadian Center of Cyber Security, they have identified 13 security controls that you can implement to effectively enhance your cyber security posture. Right? So today we'll look at how we can uncover these vulnerabilities and what our resolution, but best practices one leveraging a pen test.

[00:01:45] So I'll hand it off to Laurent, where he'll focus on how to, again, again, how to get the most volume from a penetration test and how to protect against those ever evolving threats of cyber attacks. So, Laurent I handoff. You're in [00:02:00] presenter mode. You're good

[00:02:00] to go.

[00:02:01] Laurent: All right. Thank you very much. You should all see my screen, so thank you once again for the invitation.

[00:02:06] Thanks. Thank you so much for to Assurance IT for inviting me here to present. My name is Laurent Desaulnier. I'm the vice president of the breach detection response services at GoSecure. And I have several other hobbies such as pickpocketing, lockpicking and magic. So anything deception based really. Ernesto was graceful enough to in this introduction to mention that we would like this to be as interactive as possible.

[00:02:30] So by all means ask questions. Don't wait until the end. Ernesto will be, is an amazing emcee. So we will take notes. There'll be a moderator. So by all means feel free to interrupt. Interject if you agree or disagree by all means you're more than welcome to to interact. So, as we, I mentioned in the introduction, it's been at least 20 years since I've been told that pen testing is dead.

[00:02:57] We can see here several slides, tweets, [00:03:00] or, or report by many people saying pen testing is dying. But then I'm wondering if pen testing is dying, how come we're so busy? How come everybody in the pentesting industry are hiring and hiring and hiring that there are more and more requirements such as PCI, such as NIST driving pen test, if it's dead.

[00:03:25] I asked myself the question, why is it? So first, well there's many definitions of what a pen test is and all of these groups, Offensive Security, SANS, EC-Council, ISACA, and many others. They all give different definition and they all give classes or trainings or certifications regarding pen testing.

[00:03:50] But they don't have all the same definition, which already starts to be worrisome. If you're thinking about how can somebody make a claim as the bold as [00:04:00] pentesting is dead if we don't have a common agreement on what it is. In this talk, I'll give to you what I understand to be a pen test, my vision, and what I think would lead the groundwork regarding what should a pentest be and how to derive

[00:04:15] value out of it. One other thing for many people, pen test is basically what we call the Nessus scan or a vulnerability scan, where you have an automated tool and you press next and you have this report saying, oh, these are the findings you have. And for me, that's not a pen test either. I'll get into why, but mostly because

[00:04:38] you have no intelligence out of it. Like if ultimately tools were any good, we would be out of jobs. If there are what we call a tech training and I'll talk a little bit more about this, but one thing that's important is the capacity for an, a tester to chain different flaws, like taking a one. Sometimes you might have two [00:05:00] small flaws that by themselves are not dangerous, but when chained together have a much bigger impact and being able to deliver that capacity and assess the risk and represent the risk to the client is super important.

[00:05:13] Automated tools don't have these this intelligence and thus are not able to provide full value of a pen test, but it's still quite common nowadays in 2022 that we have firms and companies that when do you have a pen test, this is type of thing they receive. Or sometimes it's what they expect. So there's a disconnect between what the understanding is versus what is delivered and that causes many problems.

[00:05:41] On the other side of the, of the coin, there is what we call tiger teaming or red teaming. So this is a screenshot from the true TV show Tiger Teams. And in it's a TV show, a reality show, where do you follow red teamers and pen testers? And like in [00:06:00] this one, there's stealing diamonds getting through the roof.

[00:06:02] They had smoke and they, you know, it felt like a heist movie. And for some people, they kind of expect these types of things where the expect the pen testers to drop from an helicopter on the roof of the building and break in using lock picks, and then start the fire alarm to be able to bypass.

[00:06:23] And you know, this expectation is also not really realistic for many reasons. One reason is the cost. If every test were to be like this, it would be much more expensive. So as you see, there's already not a formal definition, is tiger teaming or red teaming like you see here a pen test. Yeah, it is. But is it every pen tests?

[00:06:47] Certainly not. So here's my, my assumption. I think pen testing pen testing is still alive, but not everyone gets value out of it because there's no formal definition. Not everyone knows how [00:07:00] to scope it. Even if you scope it Wells, many, many things can go wrong. And I'll talk about this as well. So while pen testing is, is fully alive, not everyone gets value out of it.

[00:07:11] So by D by the end of this talk, my expectations or my hope is that you get better tools, how to make sure you scope the pen, the pen test properly and how to get more value out of it. So that should be all there is to this talk. why are people telling us pen testing is dead? Well, the first thing is scoping issues.

[00:07:36] So re I, I think some pen tests are bad, these scoped, but before we go, this Ernesto is, are there any questions already? I know it's just the intro, so

[00:07:44] not so far so good Laurent. We're looking good.

[00:07:48] All right. So in this section, I will show you actual excerpt of scoping requests we've had, and I'm sure you'll identify the problem really fast.

[00:07:59] Dear [00:08:00] client, please. I would like an external penetration test of our 2000 IPS and our web applications. Our budget is three days of work, including a full report. Well, if you think per IP, it's less than four minutes per IP to test the system. So if your goal is to do the minimal work, like for compliance reason, you might derive value of the checkbox of the compliance, but what will you learn?

[00:08:31] Like, will you learn anything from this? Will this test provide any value to you? The answer is no. Simply put real attackers don't have these time constraints. There's a saying among pentesters, if you burn money, at least you warn yourself in this case, it's even worse. You are actually just spending money for no reason.

[00:08:53] Cause your attack scope is way too large for the effort that you have. [00:09:00] Of course, if we see over scope attack surface, we also see underscope attack surface. Please, can you perform an external pen test on our perimeter? Please know that our exchange SharePoint servers or Azure VTO VPN in Berdahl main website, customer websites are out of scope.

[00:09:20] If the asked you, do you think an attacker will say, oh, I, yeah, I wish I could attack this SharePoint, portal. I know it's vulnerable, but I won't because the client wouldn't like this, that's not exactly how it works. Now, to be fair. It's absolutely valid to remove items from your, from your scope. Actually, you should do it.

[00:09:44] If you have servers that are vulnerable, that are mission critical, like I'm thinking, we did pentesting on a bank. They had a swift server. So for people who don't know what a swift server is, it's a server used to exchange money between banks and there's like $1.8 [00:10:00] billion every hour that goes through that server. They don't want to take the risk of us taking down that server.

[00:10:07] And for me, it's based on a logical decision. It's totally fine to exclude items from your scope. But if you remove every items from the scope, then of course you won't find anything because there's nothing to test. And once again, you might get a very good check, check box saying we didn't find anything.

[00:10:30] So again, for compliance, I understand why some people might do this, but in through what will you learn after you spent that money on a pen test? What valuable intelligence will you get? If you don't test. If, if your tester doesn't test anything. And that leads to lots of frustration for many clients because, and from testers as well, by the way, it goes as a pen tester

[00:10:58] if I see all of these vulnerable [00:11:00] systems that I'm not allowed to touch, I kind of feel that I'm doing my test for no reason, but mostly for the client perspective, if you don't scope it properly, you can, you can severely impact your results.

[00:11:14] Ernesto: Laurent Question. So do you see a lot of these demands coming from organizations, your, your customers organizations coming that are derived from an audit, or do you see most of these demands coming from organizations that are derived from getting better improving their security posture?

[00:11:37] Laurent: So it's about, it's about So I'm talking about the industry in general. In general, when you're dealing with companies that are a bit cheaper with less expertise, they will have client will focus on compliance because compliance tasks want to find as less as possible because the more you find, the more you have to [00:12:00] justify your auditors and so on, this is not what tip, what would typically do the role we, we take are usually the ones of where when it improve the posture.

[00:12:12] So I would say. If we looked at in Canada, United States, I would say, perhaps have the pen test or for compliance and roughly half the pen tests are for risk management. But ourselves we're really, really more focused on those used by risk management. And that's where we really have to focus on things like scope, because if it's misscoped, then it causes lots of issues.

[00:12:35] But in the industry where I would say about half and half, do I answer your question?

[00:12:39] Ernesto: Okay. 50% and a follow-up question and are customers focusing on internal internal pen tests or external

[00:12:48] pen tests?

[00:12:49] Laurent: That's a super good question. I have slides for this in a minute, but let me give you a little spoiler.

[00:12:55] I think if you had to do one test, I would do internal first. [00:13:00] And the reason I would start with the internal is because you want, once the, somebody from the external is inside you will have to have controls inside as well. So doing inside first, make sense, but what we call the M&M syndrome, do you know what the M&M candy, hard shell and, and soft inside, and most organizations are like this.

[00:13:23] So once you breach a perimeter inside, it's pretty easy. So I think the best solution is not to do an internal or external, there are other ways to manage your scope in a way that's I feel better things. What we call it, for example, purple teamming which I'm going to talk in a minute, but the short answer is if I had to do just one, I would do an internal.

[00:13:46] Perfect. So other questions?

[00:13:50] Nope. We're good for now.

[00:13:53] Alright vague RFPs, other things. This is an actual quote. Please perform an internal and external pen [00:14:00] tests in a controlled manner according to our internal guidelines that will include white box or black box testing, social engineering and red teaming approaches.

[00:14:07] The pen test should also include automated scanning from vulnerabilities and manual testing approaches. What is this test? I mean, I know from this call that since there's no number of IPS, no we didn't know what the internal guideline is. We don't know what, what, why red teaming or social engineering should be there?

[00:14:26] These are concepts. I'm going to talk in a minute. I know that in this case, the, he got RFPs for $5,000 and the $5 million for the same RFP. And try. I sometimes try to imagine the person who receives for the same RFP 5,000 and 5 million and trying to make sense out of this. Right. It's insane. And the reason why it's so difficult and when I'm a client and I see such a [00:15:00] broad range of proposals, I understand why somebody would doubt about the science of pen testing, because clearly the results are not repeatable or the process is not well understood.

[00:15:14] And one of the reason was the RFP, of course, but as an industry, since there's no formal definition, and in this case, it was vague. You would understand why the client would feel frustration and no matter which one he took, that might not answer his needs, because if you went for the $5,000 proposal, certainly that will not cover all, all his needs.

[00:15:38] And then one at 5 million while might, it might answer all his needs. It might go much, much further than what he really needed. For example. So, and Guzman pitfalls, let's assume that you did a Penta, you did a, a scope for pen test. You sculpted properly using the tools and the talk later. Now what [00:16:00] else could be wrong?

[00:16:01] Could go wrong. So let me show you things that happened in a pen test. So I'm going to have to switch a minute here. Let me know when you see my screen. Do you see my screen? All right. So what you're seeing here is a scene from the TV series. So basically what you're seeing right now is in that, in that TV series, there is a virus that the only solution is in the plane right now.

[00:16:34] And that's a sole copy. So in order to save the world, the hacker has to drive a Ferarri on an airplane runway while somebody else will be dropping a cable from the airplane and they will hack on the landing strip while the airplane is flying. It does not a typical engagement. So as pentesters, we [00:17:00] very rarely if ever drive Ferraris while hacking airplanes, but the problem we encounter with this is, and I'm going to just let it run for a minute.

[00:17:10] Sometimes you wonder if they heard about Bluetooth or wifi in these videos, but I guess it would be less anyway, so they're doing this

[00:17:25] Ernesto: it's I think it's an RG 45 cable. Yep. with a dongle

[00:17:42] Laurent: and the hacker saves the day, as you see here, the hacker saves the day. Everyone is happy and the guy drives a Ferrari and yeah, that's, that's what this TV series is about.[00:18:00]

[00:18:00] So why I'm telling you why I'm showing you this. It's not only for the entertainment value while it's still pretty funny. The reason why I'm showing you this is because many people, when they're exposed to pen testing and hacking is through movies, like your CIO's and CFO's when DC pen testers and hackers, it's always through movies.

[00:18:24] And this colors the interpretation of people. A lot. We've tested banks. And after an hour to client asked, are you done? And we barely had started, but the client in movies, it's always a guy in a hoodie after five minutes saying I am in. So they had the expectation and I'm not making I'm not laughing at clients here.

[00:18:49] What I'm saying is for most people, what they know about are from movies. So sometimes we need to educate people on managing the expectations [00:19:00] because otherwise we'll receive these types of requests. There is a legend about the hacker who whistled in a cereal whistle, and was able to hack things with this.

[00:19:13] And it's actually true. It's in early 1960s, but it's not something we can do anymore. So I can not whistle into a phone to hack a mainframe anymore, but these are the types of requests we've had. So of course this can lead to disappointment. I'm sure you understand that people have these expectations that don't actually happened a locked, oops.

[00:19:39] Other issue we've had, and this is a bit more sensitive is some people have asked, I've asked us things like can you hack my ex emails, or we have the union, they're doing things. Can you hack the union to let me know what they're working on? [00:20:00] Or can you hack a dis competitor because we want to know what they're working on.

[00:20:08] And once again, this is something that is technically possible, but in ethical hacking the word ethical is at least as important as hacking and

[00:20:19] clients don't necessarily mean bad when do they think that's the service we provide? The ask us, can you do these? And of course, well, we will say no, but I'm sure you understand that from their perspective, it's just a test and in movies these things are done all the time. So, but we have to set an ethical boundary.

[00:20:42] If the client doesn't set one. Other types of issues that happen in the in an engagement is the not really a surprise test. So this is a kind of funny story. Often times when people are doing tests for compliance, they want a clean test. So in order to [00:21:00] kind of fudge with the result, they will tell the people ahead.

[00:21:05] So for example, and this happens quite often, the client wants a phishing test, but they will tell a head their organization there will be a phishing phishing emails sent tomorrow at noon, and then we performed the phishing test. And of course we get great results, but let me ask you, what's the real value of that test? What did you prove?

[00:21:29] What did you, what was the value of this test. In the end you spent money, but what did you get out of it? Nothing. What you got was just a letter saying you guys did, did well, but they wouldn't do better in real life. What you have is a dream. What you have is just a letter. We've had clients who, during a pen test, had a system, they know was vulnerable.

[00:21:58] The took it [00:22:00] offline for the duration of the test and brought it back up after it. Let me ask again, what's the value out of this? What did you get out of this test? Did you just spend money? Because in the end, if you, if your system is only offline, when there's a pen tester, Then you're not getting much worth out of it, but furthermore, if you can afford to put it down while there's a pen test, maybe you can afford to put it down all the time and then perhaps it might be worth it

[00:22:32] Ernesto: it's a delicate topic, right?

[00:22:34] Because it does address typically a pentest does address vulnerabilities in your environment and it could raise flags throughout the organization. You know, that's why we believe, right? It's, it's very important to get buy-in from key stakeholders within the organization, right before we perform it on this.

[00:22:54] And really, as you mentioned, just focusing on the scope that we want to perform a targeted [00:23:00] scope. So we can, so we can derive value really from, from addressing those vulnerabilities and looking at how we can correct them. So in, in real life, really we can protect against those vulnerabilities and protect against the external threats that we do with.

[00:23:15] Laurent: Even more so one word are you that I don't think stealth is a good requirement for a pen test. When you're, we're doing pen tests and this is I'm still going ahead of my, my size, but when you do your pen test, your goal is to identify as many flaws as possible. And as little time when you're being stealthy, your goal is to be as slow as possible to go behind what we call the CRR, the, the, the detection level and those, those two objectives go against one another.

[00:23:42] So to be fast, to be cheap, you need to be fast to be stealthy to you need to be slow. So it's very difficult to do both of these things. Well, so oftentimes for a pen test I would really not recommend [00:24:00] having stealth as a Pre-requisites. There are other tests, just red teaming, where a stealth is mandatory, but from a pen testing, pure pen testing perspective, I think having a stout requirement is setting yourself.

[00:24:11] Wonder issue. We've encountered the overly controlled pen test. Testers are required to ask permission before running anything test clients don't know, but when we're doing a test, we're testing hundreds and hundreds and hundreds of techniques, most of them don't work. They're actually a very large percentage of the things we test.

[00:24:34] They don't work, but in order to know whether they would work or not, we have to test these. These are the checklist we go through, like I'm testing this has it worked? No, let's move on to the next one. So if we need to get approval for every single one, well, it means we're going to take much more time waiting for approval.

[00:24:54] Then we will be actually performing the test. And this comes from [00:25:00] a good intent. You want to control your changes. You want to be the person who make sure there's no impact. So this all makes sense. This requirement is from people who have not been exposed to pen tests, and they don't know how big of an ask this is.

[00:25:18] So, which is why I'm letting this sorts a problem. And finally, the other problems we encounter often is vulnerable exclusion. Please don't use this vulnerable tea. We know we're vulnerable. Thank you.

[00:25:32] Ernesto: Okay.

[00:25:33] Laurent: Once again, the problem is

[00:25:36] so we're trending vulnerabilities. So this vulnerability could be used to find something else later on. So if you stop, if you block something that would be like the first step of a chain, then you prevent testers from seeing what's behind that first step. But furthermore, if you're looking at it from a risk perspective, Hackers don't care.

[00:25:57] Like if at this point you're saying, don't use this [00:26:00] flaw, but you know, you're vulnerable. Well, at least have the test or put it in the report at least have some form of way to, for you to remediate or make it visible to your management. So you're able to fix it because it's not because the pen testers aren't allowed to test it, that you're not at risk test hackers will still use it.

[00:26:22] So these are the type of things. Now what I've explained the problems, do most problems come problems we've encountered before going to test types? Are there any questions or any other questions worries or anything

[00:26:34] Ernesto: so far so good in the chat Laurent.

[00:26:37] Laurent: Right. Great. So this is the fabled pyramid. It's an assessment of basically all the type of services.

[00:26:44] So we have the base, we have the middle layer and the top of there. So from us. The base there is what are called vulnerability assessment. Then we have what are called pen tests. And then the top layer is called. This are called what we call the strategic services. [00:27:00] So first what's a V vulnerability assessment.

[00:27:02] So I wonder if the assessment is an automated test where you look for flaws. It's fairly simple. There are lots of tools that are very good at it. Things like Nessus. Nexpose, there's lots of them, but first let me tell you, this is not a pen test. So if we could all repeat together, a vulnerability assessment is another pen test.

[00:27:23] If it's a one thing we should all get out of this, it all would have been worthwhile. Don't get me wrong. a vulnerability assessment is super useful, my opinion is that a vulnerable assessment should be done in house. I don't think there's a value of having a third party doing this. By having it in a house, you have a better control of inventory.

[00:27:43] You can do it on-demand. For me, this should be more in-house but nonetheless, you should. Is it a pen test? Now, if you don't have patching vulnerability management or [00:28:00] patching or vulnerability assessment capabilities, you don't need to do a pen test. There's no use of you doing your pen test if you don't patch. I'll give you this free pen test they're going to get in you didn't patch.

[00:28:14] So instead all that money you would spend on having a pen test to let you know this, invest it instead on having a whole number team management program I re purchasing licensing for these tools vehicles. If you don't manage your patching. There's no reason for you to do a pen test, simply put, because they're going to get in through missing patches is going to be a boring report.

[00:28:37] And they're going to tell you something that you know, already. So the first step would be to do a VA and repeat once again, from my understanding or the way I see it, it's much better of this capabilities in house, because it helps with inventory. You can do it on demand. There's lots of benefits of having it.

[00:28:55] In-house.

[00:28:57] Ernesto: So I have a question around here. The [00:29:00] question is, is a vulnerability assessment a pen test. No. Okay. Is it a good start for organized? So you're identifying it as a good start for organizations to get started for them to start and understand their environment, and then maybe take the results from their internal VA and, and, you know, have the information needed to really scope out a pen test going forward.

[00:29:24] Is that how they should use this

[00:29:27] Laurent: at least, but at least to know, to patch their assets or to identify what assets they should patch because otherwise the pen tests will be very, very easy. Yeah, exactly. So I think it's a first step and I repeat. I don't think you should use a third party. It will be way too expensive.

[00:29:46] Because when you think about it, a vulnerability assessment doesn't need any expertise. You enter your IPS, you press next, get a report. So hiring an expert that you're going to pay 300, $400 an hour to do this. I mean, of course we'll [00:30:00] take your money. There's no reason for having an expert to do this, where anybody would, I can move into copy paste should do it.

[00:30:09] Right. The value I get is all you respond out of this report and how you do management and having the know the internal knowledge, knowing the CIS admin, being able to be, have the political and knowledge of the organization to be able to navigate and make sure you push vulnerable management is much more important for these type of projects.

[00:30:32] And that requires internal skills. So that's why I'm saying it should be in house

[00:30:37] now, pen testing. So there's a plethora of types of pen tests. So we talked about internal, external, but there are web apps. They are wifi. There's code review. There's mainframe like there's so many types. Of of pen test. You can, you can do, but there are a few things that are re that you should focus on for it to [00:31:00] be a pen test.

[00:31:01] First, has to be manual, at least in some, in some forms. If you're only relying on a tool and not the expertise of a tester, you're missing a lot, also there should be attacked chaining. So at that training is showing that using exploit a, we were able to derive data first for something, and then we use flop B and so on because this chain of vulnerability is something that provides you value.

[00:31:30] And finally, and I mentioned already, I don't believe it has to be stealthy. I think that doing it start is sitting yourself for an error. And so these are the types of common type of pen tests. The most common, you'll see our web apps, internal and external, but there are so many others fishing. And if you have a specific need, that's super weird.

[00:31:53] Feel free to reach out to any pen testing firm. And sometimes they'll be able to help. We have clients who ask like for [00:32:00] SAP testing. So SAP is a ERP it's fairly large, and it's difficult to find expertise in testing for example, SAP. So nowadays, now we have a service. That's what we do exactly this because nobody else can test it.

[00:32:14] So, but so, but the most common are mobile web internal, external, phishing, these types of things. And I repeat, what makes it a pen test from my perspective is manual and attack chain. If you have these two things, then you're in a place where you have a setup for a pen test. Now, what I have, I've talked to you about pen testing and vulnerability assessment.

[00:32:37] Let's move to our layer that we called strategic services. So strategic services, basically what they are is we are trying to assess how good the blue team is. So the blue team has the defense side, the security operation center. So most organization, the invest a lot in SIM in EDR in suck. The, they do a lot of investment, but how do you know [00:33:00] how good, how good how good are they?

[00:33:03] Like you've made all this investment, but how do you rate them? How would you know if your investment is good? Well, in through you have a threat being the hackers, trying to assess. The the blue team and this time they are trying to be stealthy. They're trying to emulate real threats. The idea here is that this is a much long, much more long assessment.

[00:33:27] Typically we would see 60 days, 90 days, 120 days of work per assessment. So it's much, much bigger than your average pen test. One thing that for me is super important is the no shaming rules, because this thing gets pretty adversarial where the blue teams says the, the red team says the blue team didn't find this and so on, and this is meant to be positive.

[00:33:50] So, and I've seen horrible reports from both sides that shamed the other side, the idea is more to assess process and detection [00:34:00] capabilities. So this is what a red teaming is. Now, if there's a red team, There's a purple team. And what is purple teaming is collaborative pen test. So the idea is we work together with the blue team and we really try to improve detections and reaction.

[00:34:19] And we do it by focusing on one aspect at a time. So let's say one week has lateral movement. We do all of that, our own movement techniques. We know we tried to identify gaps between the technique and the detection. We fix it and we move on and move on and move on. And really that's from my belief that the future of pen testing. It's non-adversarial. The clients gets value out of it.

[00:34:42] There's some coaching. We the client by the end, his, his more tool to respond by themselves. They're more independent from my perspective. That's the future of pentesting.

[00:34:53] Ernesto: There's a, there's a lot more collaboration, right? Like you said, from the external team, right. Where the goal [00:35:00] is to get better, right. Versus exposing it and pointing fingers.

[00:35:04] Right. So if the goal is to get better, right. Collaboration is, is the right

[00:35:08] Laurent: way to go. That's correct. And furthermore when you're doing your red team, what you get by the end is a list of things to fix. When you're doing your purple team, what you have in the end is a list of things you have fixed. It's a very different message you're sending to your management at that point.

[00:35:25] Ernesto: Great, great point. Great point. Just doing a quick time check it's 1 43 15 minutes left. So I'll let you continue and we'll leave five, 10 minutes for some

[00:35:36] Laurent: questions you, yes, we're almost done, but do you have any, we have any other questions at this point regarding types of tests,

[00:35:43] Ernesto: we have a question in regards to what VA options are.

[00:35:47] Can you suggest any for

[00:35:50] Laurent: if you're doing this internally I would recommend you use auto authenticated VA. So there are several flavors ways to do flown up the assessment, but the best way [00:36:00] is to have a scanner that has an account to be able to log into the systems. So you have visibility into, what's visible on a network, but also on the systems, think about Adobe PDF reader from the outside.

[00:36:12] You cannot know whether or not the Adobe is up to date, but if you log in, if the tools logs in it works really well. I really like Nessus but I've heard great things about Nexspos and Qualis as well. So as for which one, I think they all pretty much are good in that regard. But authenticated is a way to go if you're doing it.

[00:36:32] In-house absolutely.

[00:36:33] Ernesto: Perfect. Thank you, Laurent. So as everyone knows this workshops being recorded, then we'll follow up with the recording and we'll follow up with the FAQ's with the recommended VA's.

[00:36:43] Laurent: All right. So the question now is what's the right test for you, cause we've talked about all these things, but which one should you pick?

[00:36:49] And what's the value? So many people think if he is less good than a pen test, which is as good their red team, but in truth, one test goes [00:37:00] into the other. I think that if you're not performing your VA, considering a pen test is a mistake. And if you're not doing your pen task, considering a red team is a mistake.

[00:37:10] But that being said, there's no shame in being at a vulnerability assessment step, as long as you understand you're at that step. And. Because we all need to start somewhere. The problem we encounter and more our clients aren't ready. And they want a red team? And it's it sets it's very difficult for them.

[00:37:30] And these are not good results when they show their management. So yeah,

[00:37:34] Ernesto: it gets put in a drawer somewhere. Right?

[00:37:36] Laurent: Exactly know how to derive value out of a test. So I've talked about all these things, but like how to get value out of it first have a goal. So why are you doing these tests? And I know it's obvious as a question, but when you think about it, if you don't know about this, then doing any, anything after it, if your goal is not set properly, everything else cannot be [00:38:00] right.

[00:38:00] So do you want to validate your patching levels? Well, then perhaps a view is all right. Would you like to emulate one threat? Would you like to assess the effectiveness of your, your blue team depending of what you want? It'll tell you already what type of test you want. Choose your threat. So this is our pyramid of pain.

[00:38:19] So that's how we assess threats. So we believe that automated scripts are the most common in the less sophisticated attackers. Then we have script kiddies, opportunistic attack here. So an opportunistic attacker, you know, you don't need to outrun the bear. You'd only need to outrun your neighbor. Have you sure?

[00:38:38] You've heard the saying like this well an opportunistic attacker, if you're good enough that the ACRA moves somewhere else, you're all set. And that's where we set the line for due diligence. My understanding of the risk is you need to at least protect yourselves against these three, these bottom three attackers.

[00:38:56] Now target the attackers is the same. [00:39:00] You would think you don't need to outrun the the bear, just a neighbor, but it's only true unless if you're covered in honey, then the bear is out for you. And then it's a much more difficult challenge. And then state actors like NSA, China, or whatever, these type of people, and depending of what threats you encounter

[00:39:18] well, then you have a quite different type of test, right? Having an inventory. You already mentioned it, but of course, if you don't have an inventory, how do you know you're covering everything? There's a pen test or seeing if your client has time to read the report, you've got the wrong client basically asked for presentation, it's it be much more effective?

[00:39:42] But nonetheless asked for a report because the report are the proof you've did it. You have written recommendations, you gets lots of value out of the report. But for me having a presentation where you can have key stakeholders and have technical people on board, you derive much more value than just [00:40:00] sharing the 300 page report.

[00:40:02] So if you have two, if you can ask for a presentation, you'll get value out of it. As long scope, avoid scoping issues and pitfalls. Reach out to your pen testing firms. If you don't know how to scope it pentesters organizations such Assurance IT or GoSecure they've scoped pen test before they can help you with making sure you have the right fit.

[00:40:26] Avoid cool. If you're in a meeting scoping for a pen test and you hear, oh, this is so cool. We're going to do this. Well, this is not a good business, valid reason. And we had clients that wanted to do this cause it was cool, but it's never how you get value out of it. Avoid blame. So this is a problem where when you're fixing this sometimes people will be identify like somebody opened an email and that email as it's, the [00:41:00] understanding would be we'll fire, the person who opened the email.

[00:41:03] But I think this is missing. You're seeing the tree and missing the forest here. The problem is not Larry open email. The problem is your organization didn't have controls in place that prevented the email from getting in there, the organization didn't do a proper security awareness. So by if you blame people, the problem will come back again and again and again.

[00:41:26] So when you're going to remediation, you really need to avoid. Assigning blame. This creates the most toxic security culture you, you face and that can lead to really big problem. So finally,

[00:41:39] Ernesto: I think that's the number one reason why the outcomes of the pentests fail, right? It's, it's avoiding blame and understanding that it's, it's, we're here to work together blue and red, right.

[00:41:50] Equals purple right work together and come to conclusions, come to solutions, you know, put forward some solutions in order to get a successful pentest, just my [00:42:00] input. But I believe that that's

[00:42:01] Laurent: proper. So now what's a future. Well, as I mentioned, I think for me to future is purple teaming. That's really where there's collaboration and that's where we have, we provide the most value for clients.

[00:42:14] Yeah, that's a

[00:42:17] Ernesto: questions. That's fantastic. Thank you, Laurent. That was that was fun fantastic. In regards to questions in the chat, I think it was interactive. So we ask questions as we went. I'll wait feel free to anyone in the chat or in the group session. Feel free to unmute themselves to ask the question.

[00:42:34] If not, I'll be waiting a couple of minutes here and seeing if there's any questions in the chat. I see. Giacomo type typing away here. So wait, first question. So again the question from Giacomo, what does a pen, a post pen test look like for an organization? It's

[00:42:52] Laurent: a really good question.

[00:42:53] So here's the, the happy path. So not all organizations take the happy path, but here's what I would [00:43:00] expect. So for most. Talk from the perspective of an internal pen test is the most common. We would perform a test then have at least one meeting with the technical people, a presentation where we can go over to fix and ask questions and so on.

[00:43:18] Then usually we would redo a test the following year taking into account with, with testing. So typically that's usually pretty light. It's our meeting. And after a test, the following year for things like web app or applications that are more mission critical, sometimes the clients ask for a retest.

[00:43:38] And one thing you should take into account is if your organization charged the same price for a retest, that they charge for the, the normal test, then you're paying too much because it retests. Usually you should be much, much, much faster. Then the right test because as I told you before, when we're doing a test where there's hundreds of things that we test, [00:44:00] that don't work, but we have to go through the other way around.

[00:44:03] We have a list of vulnerable to, to want to know whether or not they're still there. It should be much, much, much cheaper. I hope I answered your question.

[00:44:11] Ernesto: I think you hit the point, right? Laurent, so it's very important when you when you perform a bad test to follow up because you want to understand, did your efforts really lead to success, right?

[00:44:24] That it really close the gaps and where you able to protect once you did the initial one performed, but another pen test typically on a yearly basis, that's what you

[00:44:33] Laurent: recommend. Yes. But even more so. And I think I didn't put it in my slides, but thinking on it, we should is if you're doing all these things or you don't fix anything, you barely spend money for any reason.

[00:44:44] Yeah. And this is something we do encounter though, this something happens. I see. We have a

[00:44:49] Ernesto: cool. Go ahead. Yeah, no, it was definitely a door. If you do them for no reason and you keep on doing them, it'll definitely allow you to get that Ferrari. Yeah.

[00:44:58] Laurent: Yes, of [00:45:00] course. I suppose it is a great question by Anne Turski about is there indifference, if a similar in the cloud.

[00:45:05] So cloud testing is a bit different for two reason this is totally possible by the way. There's people have believed it's impossible. It doesn't the cloud. It is possible, but there are some caviats for example, depending on what technology you're assessing, some service providers don't want you to test.

[00:45:22] For example, I will take Amazon. So Amazon, you can have your, your own tenant. So basically these are your computers where they're in the cloud and these, we can test all we want. No problem. But when there are systems that are shared between clients, things like route 53, for example, that's a service by Amazon for managing DNS.

[00:45:43] While these ones there's an impact for many clients if we test it and we break something, so then it's much more difficult that we need to go into. We need to talk with Amazon and there's discussion and so on, but it's not impossible, but it's different, but I don't think it's [00:46:00] impossible. It's just a different flavor of a pen test.

[00:46:03] I would recommend you talk with your pen testing firm to go through to make sure you scope it properly, but it should not be a, a stop. It should just be, let's see what, what the provider wants and where are the limits.

[00:46:19] Ernesto: So, so I guess when, when a customer wants to initiate or have a conversation around which pen test would work for them I guess working with us, Laurent, you know, we've done these in the past, working with us to really scope it out and understand what the scope, identifying the scope and would be the key characteristic where the first start in, in, in getting in

[00:46:40] Laurent: getting, but it's not mandatory, but keep in mind that most pen testing organization are used to scoping these and they have experienced.

[00:46:47] So oftentimes oftentimes the the pen testing firms can provide value or suggestions, but just you worry about upsellers. Because the problem is not all pen testing companies are ethical, so just be [00:47:00] worried about. From saying you a red team when you need a VA. But involving them if you know what you want, but you're not sure which surface, for example.

[00:47:09] And I think it's a good moment of a discussion with a salesperson or an expert from the pen testing firm. I really hope I answered

[00:47:15] Ernesto: your question. I think you did a great points. Yeah, no, definitely. Right. So it really focus in on the scope and make sure that you work with a trusted provider when, when you're going forward, what would a pentest really to uncover those vulnerabilities?

[00:47:28] Because it is, it is a partnership, right? It's a value added service where if you do implement it correctly, or if you focus in on, on the longterm, it's, it's definitely something that you're working hand in hand with your provider and you establish a relationship because in the end you you're working together to resolve the common issue.

[00:47:46] Laurent would you like to

[00:47:47] Laurent: add anything? One more comment that I didn't see in writing, but I want to make sure is some pen testing firms are also into remediation and for me it should be a warning or a red flag [00:48:00] because you want independence. So your report, like, for example, The pen testing report should never see purchase dis firewall from this manufacturer.

[00:48:09] Independence is super important. So make sure when you're dealing with a firm you're dealing with pen testing firms or independent, or because otherwise you can set yourself up for a pretty expensive surprise.

[00:48:23] Ernesto: Good point. Good point. Well, I think that that's it guys. Well, thank you very much. Again, was a pleasure.

[00:48:30] Laurent was a pleasure. Thanks for taking the time today. Definitely a lot of value and feel free to reach out to the Assurance IT team or GoSecure team, to help you with your, your pen test going forward.

[00:48:41] Thank you everyone. Have a good afternoon.

[00:48:43] Laurent: Thanks again. Thank you. Good job.

[00:48:45] Ernesto: Thank you.

‍