Penetration testing, or "pen testing" for short, is attacking your own systems to identify weaknesses.
Penetration testing is important because it helps to identify vulnerabilities which may have been overlooked by admins or developers. Human intelligence and perception can spot issues which automatic scanning tools cannot. It can include social engineering techniques and synthesize several minor vulnerabilities into a larger attack. Pen testing encourages creativity and improvisation.
Penetration testing has become a standard part of a full security audit. It must be performed regularly by any business accepting credit cards in order to comply with the Payment Card Industry Data Security Standard (PCI DSS).
There are two types of penetration testing. "White box tests" are when the pen testers are provided details, such as the network topography, operating systems, software versions, and system configurations. "Black box tests" are when the pen testers are given no details about the system. They must instead make all determinations about the system, its software and configurations by inspection or guessing. There are advantages to each approach. White box tests allow pen testers to identify vulnerabilities faster because there is less guess-work and they have a map to guide them. Black box tests are the best approximation of real-world situations. Black box tests invites more creativity which could expose unexpected issues, but which could also overlook more obvious vulnerabilities.
Penetration testing is performed by a person or a team who plays the role of the hostile attacker. This is referred to as the "red team" (a term taken from military exercises). The red team could be drawn from a organization's developers, could be a full-time pen tester, or could be outsourced to a third-party penetration test company. The group responding to the threats posed by the red team is referred to as the "blue team" and should always be composed of the internal personnel who would respond to actual threats.
Penetration testing can be performed using any guidelines or criteria. The goal is simply to identify weaknesses. However, to get the most out of the exercise, a useful guide and formal descriptions of the process has been defined in the Penetration Testing Execution Standard.
The most important aspect of penetration tests is to start by establishing the terms of the engagement. This is especially true when testing live, production systems or when the testing team is not the owner of the system. The scope of the test should be clearly defined—how long will it last, what are the goals, what is fair game, what is off limits. The blue and red teams should discuss any security concerns they have before they begin. It should also be clearly established what procedures will be followed after a successful penetration. There may be legal requirements and compliance issues which must be considered. With external red teams, it may be worthwhile for the pre-engagement terms to be formalized as a signed contract and to require non-disclosure agreements.
The software used for penetration tests is basically the software used by attackers. It is a fine line between the two fields. Tools written for attackers become pen test tools. Tools written for pen testing become tools used by hackers. There are hundreds of tools which could be used for various types of attacks. There are also a few popular "suites" of tools which offer many capabilities in one software package.
There are also a number of projects which are deliberately insecure so that they can be used for penetration testing practice. (Or for attack practice, depending on your point of view.)
