Security Testing & Evaluation: Measurement, Not Just Breaking

Prerequisites

• FND-K001: CIA Triad & the Modern Threat Landscape
• PEN-K001: Methodology, Authorisation & Rules of Engagement (recommended)

Why This Lesson Matters

Most people confuse penetration testing and security evaluation. They use the same tools. They test the same systems. But they answer fundamentally different questions. A penetration test asks: "Can an attacker break in?" A security evaluation asks: "Does this control perform as specified?" The difference shapes everything: scope, evidence, reporting, and what success looks like.

1. The STE Mindset: Measurement First

Analogy: A car safety test is not a crash. Engineers do not drive a car into a wall and report "it broke." They measure crumple zones, airbag deployment times, and seatbelt forces — against specific safety standards. Pass or fail is determined by measurement against criteria, not by the most dramatic outcome.

Security Testing & Evaluation works the same way. You are not trying to cause the most damage. You are measuring whether a control meets its stated requirement.

Penetration test question:
  "Can I get admin access to this web app?"
  → Success = admin access obtained

STE question:
  "Does the web app enforce role-based access control as specified
   in requirement SR-042?"
  → Success = measured outcome (pass/fail) documented with evidence

2. Assessment Type Taxonomy

The key STE distinction: Every STE finding must trace back to a control requirement. If there is no requirement to test against, it is not an STE finding — it may be a pentest finding instead.

3. The Control Requirement Model

Before any STE engagement begins, the evaluator maps each test to a control requirement:

Control requirement (from security policy or standard):
  SR-042: "The application shall enforce role-based access control.
           Users with role VIEWER shall not access admin functions."

Test case TC-042-01:
  Action:   Authenticate as a VIEWER-role user
  Stimulus: Attempt to access /admin/users endpoint
  Expected: HTTP 403 Forbidden
  Actual:   [fill in during testing]
  Result:   PASS / FAIL

Evidence:
  Screenshot of HTTP 403 response
  HTTP request/response pair
  User account details (role = VIEWER confirmed)

This traceability — requirement → test case → evidence → result — is what makes STE output usable for audits, accreditation, and regulatory compliance.

4. Risk-Based Test Planning

Not all controls need equal testing depth. A risk model prioritises:

Risk tier 1 (highest): Controls protecting the most sensitive assets
  → Deep testing: multiple test cases per control, negative and positive tests

Risk tier 2 (medium): Controls protecting important but less sensitive assets
  → Standard testing: primary positive and negative test cases

Risk tier 3 (low): Controls on non-sensitive systems
  → Spot testing: verify control exists and functions minimally

Positive test: Does the control allow what it should allow? (A firewall that blocks everything including legitimate traffic is not a passing control.)

Negative test: Does the control block what it should block? (A firewall that allows everything is obviously failing.)

Both are required for a complete STE.

5. Scoping an STE Engagement

STE scope is defined differently from pentest scope. Instead of "which systems can I attack," STE scope defines "which controls am I evaluating."

STE Scope Document:
  System under evaluation: Customer Portal v3.2
  Evaluation basis: Corporate Security Standard CSS-2026
  Controls in scope:
    CSS-A01: Authentication (MFA requirement)
    CSS-A02: Session management (timeout, regeneration)
    CSS-AC01: Role-based access control
    CSS-IN01: SQL injection prevention
    CSS-LOG01: Security event logging completeness
  Out of scope: Infrastructure, network layer, physical security
  Test environment: Staging (prod-equivalent data, no live customers)
  Credentials provided: yes — one account per role (admin, editor, viewer)

6. Common Mistakes

Mistake 1: Treating a failed scan as a failed control. A scanner finding is a candidate finding, not a confirmed evaluation result. Every scanner hit must be manually verified against the control requirement before it counts as a STE finding.

Mistake 2: No baseline before testing. Start every STE engagement by documenting what the system claims to do: architecture docs, policy statements, control descriptions. You cannot evaluate compliance without knowing what compliance looks like.

Mistake 3: Testing only negative cases. A control that blocks all access (including legitimate) technically passes a negative test but fails a positive test. Both sides must be verified.

7. Practice Exercises

1. A client asks: "Can you do a security evaluation of our login page?" Write a scope document with three test cases (SR number, action, stimulus, expected result) covering: MFA enforcement, session timeout, and account lockout.

2. A scanner reports CVE-2023-12345 on the web server. The control requirement CSS-IN01 says "the application shall prevent SQL injection." Does this CVE constitute a failure of CSS-IN01? Justify your answer.

3. Explain the difference between a negative test and a positive test using the example of an access control on /admin/users.

8. Lab

Assessment mode: quiz

7 questions: classify described activities as VA/pentest/STE/audit, write test cases from provided control requirements, and identify missing elements in a provided STE scope document.

9. Framework Alignment

10. Further Reading

• NIST SP 800-53A — Assessing Security and Privacy Controls — the authoritative STE methodology reference
• OWASP Testing Guide v4.2 — The practical complement to SP 800-53A for web systems
• Common Criteria (ISO/IEC 15408) — The international framework for IT security evaluation

["Distinguish STE from penetration testing and vulnerability assessment by articulating the different questions each answers and the different evidence each produces", "Write three STE test cases (requirement, action, stimulus, expected result) from a provided set of control requirements covering authentication and access control", "Apply risk-based test prioritisation to assign testing depth (deep/standard/spot) to five controls from a provided system with defined asset sensitivity levels"]

Prerequisites → Why this matters (car safety test analogy) → STE mindset (measurement first) → Assessment type taxonomy → Control requirement model (requirement → test case → evidence) → Risk-based test planning (positive and negative tests) → STE scope document → Common mistakes → Practice exercises → Quiz lab → Framework alignment → Further reading