Why annual pentests miss what matters

A penetration test report describes the systems as they were during the testing window. That is not a criticism of the testers — it is the definition of the engagement. The problem is what everyone does with the report afterwards, which is to treat it as a description of the present tense for the next twelve months.

What drifts, and how fast

Infrastructure changes constantly, and most of the changes that open exposure are not security work and are not reviewed as such:

• A service ships with a load balancer fronting it, and the security group is temporarily widened during rollout. The rollout finishes. The rule does not get narrowed.
• A dependency upgrade changes a default. A newer release starts binding an admin or metrics endpoint to 0.0.0.0 where the previous version bound to localhost.
• A DNS record is created for a staging environment and never removed after the environment is torn down, leaving a dangling record pointing at reclaimable infrastructure.
• Someone stands up a demo box for a customer call. It runs for eighteen months.
• A managed service gets a new public endpoint as part of a provider migration, and the change lands through a console rather than through the Terraform everyone reviews.

None of these are exotic. All of them are invisible to a report written in March, and all of them are reachable by anyone scanning the internet in April.

The asymmetry

The relevant comparison is not "how thorough is a pentest" against "how thorough is a scanner". A good human tester will find business-logic flaws, chained exploits and authorisation gaps that no automated tool will ever produce.

The comparison is frequency against change rate. An attacker running continuous internet-wide scans discovers a newly exposed service within hours. An annual assessment discovers it up to eleven months later. The depth of the annual test is irrelevant to that particular finding, because the finding did not exist when the test ran.

Both things are true at once: the pentest sees deeper, and the pentest sees less often. Treating them as competing purchases is the mistake.

What to run, and how often

A workable split:

1. Continuous or weekly automated scanning on high-change-rate assets. Anything fronting production traffic, anything in an environment where deploys happen more than occasionally.
2. Monthly on low-change assets. Marketing sites, legacy systems under change freeze, anything where a weekly scan would report the same result fifty-two times.
3. Alerting on delta, not on total. The number that matters operationally is "what is open now that was not open last week", which requires the scanner to deduplicate findings across runs rather than re-report everything each time.
4. The annual pentest, scoped to what humans are good at. Business logic, authorisation boundaries, chained exploits, social engineering if in scope — not re-enumerating your open ports, which you should already know.

The last point is worth pressing with your testing vendor. If a meaningful part of the report is a list of exposed services and outdated versions, you paid consultant rates for output a scanner produces continuously, and spent scarce human testing hours on the least human part of the job.

Conclusion

A point-in-time assessment can only ever tell you about the point in time it ran. That is fine, provided something else is watching the other fifty-one weeks. The failure is not the pentest — it is the assumption that a pentest is a substitute for continuous visibility.

CyberSpec runs scheduled scans on whatever cadence each asset deserves and deduplicates findings across runs, so the open-findings count reflects real exposure rather than climbing forever. It is not a replacement for an adversarial human engagement, and it is not marketed as one.