Penetration test quality vs. recruitment quality?

What does it mean that in Citadelo a hacker is hiring a hacker? Dita has been an expert in recruiting ethical hackers in domestic and international markets for 7 years. We talked about hacker recruitment at Citadelo and how it affects the quality of the team and therefore the quality of the pentests performed.

Dita, why is it good to talk about the quality of recruitment in ethical hacking?

Let me illustrate this with the example of a penetration test of, for example, a mobile application. If a client outsources the penetration test of his application to two different vendors, in the same scope, he will most likely get two different results, which depend on the competence of the ethical hacker who performed the test. Both providers will deliver a completed penetration test, but the question will be the breadth, depth and criticality of their approach. How comprehensive and well-crafted a report, including recommendations, will they deliver and how will they be able to communicate it to the client.

Competencies, what should clients understand when speaking of them?

Penetration testing is not a discipline that everyone sees the same way. It is more of a science where the outcome is directly dependent on how experienced the ethical hacker is, what technical background he has, whether he has conceptual and critical thinking skills, how he keeps himself up to date on the current news and trends in the field of vulnerabilities found around the world, etc. this and much more determines whether he will be able to discover a hidden vulnerability for the client while exploring the tested territory or whether he will go looking in a different direction after some time. Which may or may not be a good thing. So if 2 or more hackers with the above mentioned competencies meet on a penetration test project, this is a prerequisite for a really good penetration test for the client.

In addition to technical skills, the ability to communicate well and skills management are also very important. On the basis of professional, timely and well-chosen communication, pentester and the client can iron out various shortcomings already in the preparatory phase of the penetration test so that its start and progress is as smooth as possible, which ultimately adds to the quality and is also cost-effective for the client.

So what is the link to recruitment?

The recruitment process and the quality of the team, and by extension the service provided, are linked vessels. A properly executed recruitment process lays the building blocks of a quality team, while a well assembled team conveys valuable information for proper recruitment execution. The result for the client is then a strong and expert team working on their project.

What are the main pillars that help you get these quality people out of the market for customers?

1. We build our recruitment processes based on the technical expertise required to meet the real needs of our customers. We translate this into technical tasks right through to internal interviews.
2. Each candidate will go through technical tasks in our Capture the Flag (CTF) in the range of 2-5 days.
3. Candidates who pass through the CTF progress to a technical interview where they are interviewed by up to 3 of our senior hackers on the progression and variations of tasks in the CTF, general technical knowledge, trends in ethical hacking, motivations for the job, and more. This is the point at which the hacker is selected by hackers.

Let’s stop at the CTF. What is the success rate of completion?

Around 7%. Last year, we made the CTF available to 98 applicants, and of those, 7 made it to the follow-up technical interview.

That’s a pretty rigorous assessment, what’s the reason?

The content of our CTF is a sequence of tasks that are based on frequently occurring vulnerabilities in the world and from real world testing. Compared to a penetration test, the CTF has:

1. a linear progression
2. fewer dead ends than a penetration test. These two certainties do not exist in a penetration test. Therefore, we have enriched our CTF with a task that requires the user to think critically without following testing methodologies, which effectively substitutes for conceptual thinking about the penetration test, which quite fundamentally affects who passes the CTF and how. The final step is uploading and sending your CV to the HR department.

So as an applicant, do I need to invest 2-5 days in completing the technical tasks before I am hired?

Yes, that is correct.

We know from the feedback from our candidates that the selection process itself is an experience and even if we don’t agree to work with them, they take away a lot of valuable insights and experiences that they then build on.

So do I understand correctly that clients can expect only seniors on their projects?

That would be a nice idea and we would love to fulfill it, but in terms of the price per penetration test, it would be unrealistic for the client and the capacity of the staffing market does not match it.

Therefore, we also work with med. or jr. ethical hackers who initially work on projects shadowing more experienced colleagues. We allocate them to projects independently only when they reach the necessary experience according to internal metrics. The length of this intermediate period is different for everyone. Of course, the selection process they have gone through also helps in speeding up this interim period.

What do you think raises the quality of a team in the post-hire period?

• The experience they gain from the variety of project types across market sectors.
• The number of pentests conducted within the year.
• Mutual sharing of experience on projects.
• Learning

A very important entry competency we look at in the selection process is people’s motivation and willingness to learn and develop, as expertise is one of Citadelo’s building blocks. We invest in the development of our hackers in various ways, whether it’s through our internal CTF platform, trainings, certifications or conferences.

With a lack of ethical hackers in the market, building an expert team like this must create a risk of being outbid by your competitors?

Yes, it does. Our competitors have our ethical hackers in their crosshairs and are constantly trying to convert them to themselves. Of course, turnover cannot be 100% prevented, for whatever reasons, but we still manage to keep a relatively strong seniority average. In some cases, it even happens that some of them come back to Citadelo after a while, and that is one of those moments when we have a confirmation that what we are doing is right.

Is there any way for customers to check the quality of the team in advance?

Yes, they certainly can. As a first step, if they don’t know the company, they can go to the company’s website, where they should have posted quality indicators such as references, case studies and CVEs, which are a kind of showcase of quality, or CVs of the team members. At Citadelo, for example, we attach detailed CVs of pre-selected ethical hackers to the bid so clients can see who can work on their project. Clients can also request a preview of the CV themselves.

What are CVE’s?

They are published and detailed newly discovered vulnerabilities in globally used software and applications. A newly discovered CVE by ethical hackers is a big catch for them to contribute to a database that the whole world can view, helping to protect the users of these SW. Once our ethical hacker discovers such a vulnerability, we follow our established disclosure policy, which results in the publication of vulnerabilities such as these in the CVE database.

Citadelo currently has an entry in the CVE database for 18 identified vulnerabilities. To give you an idea, you can check out these 3 from Andrej, or perhaps another listed on our blog. Every time we find a vulnerability like this, it’s a hacker holiday for us to celebrate.

Is there anything you’d like to say in conclusion?

I would like to say that although a well-executed recruitment process is not the only component of a quality team, it is the initial one. As Citadelo, we define ourselves in the marketplace by the fact that ethical hacking is our core business, so careful competency verification is key. Additionally, we have a wide range of customer types and need to be able to respond to their needs in the way they expect us to. Therefore, the recruitment process must be tailored to this, so that through it we can attract colleagues who will resonate with us and want to continuously develop together with us and our customers within this mission.

Whether you’re tackling penetration testing now or planning to in the future, feel free to get in touch with us to discuss what we can do for you.