Vendor Risk Management: Questionnaires to Scores

When Cl0p exploited CVE-2023-34362 in MOVEit Transfer, the victims were not just Progress Software customers --- they were customers of customers of customers. Over 2,600 organizations were compromised because their vendors used a managed file transfer tool that most downstream parties had never risk-assessed, never inventoried, and never knew existed. The breach cost the ecosystem north of $10 billion in aggregate, and the majority of affected organizations had no direct relationship with Progress Software at all.

That is vendor risk in its purest form: your security posture is partially controlled by people who do not work for you, whose tooling decisions you did not approve, and whose incidents become your incidents the moment data flows between you.

Why Vendor Risk Demands Attention Now

Three forces have converged to make vendor risk management the single highest-leverage investment in most security programs:

Supply chain attacks are accelerating. The xz backdoor (CVE-2024-3094) demonstrated that even open-source infrastructure maintainers --- the invisible plumbing of every enterprise stack --- can be socially engineered over years-long campaigns. SolarWinds (2020) was a state actor compromising a build system. MOVEit (2023) was a zero-day in commodity file transfer software. Change Healthcare (2024) was a single ransomware event that froze insurance claims for a third of the U.S. population and cost UnitedHealth Group $2.4 billion in the first year alone. The pattern is consistent: attackers target shared infrastructure because the blast radius per exploit is orders of magnitude larger than targeting individual organizations.

Regulatory frameworks now mandate flow-down. CMMC Level 2 requires prime contractors to verify that subcontractors handling CUI implement all 110 NIST 800-171 practices --- and that obligation flows recursively through the supply chain. DFARS 252.204-7012 imposes 72-hour incident reporting obligations that extend to subcontractors. The SEC's 2023 cybersecurity disclosure rules (Item 1.05 of Form 8-K) mean that a vendor's breach can trigger your material-event reporting obligation if it affects your operations.

Concentration risk is invisible until it detonates. Change Healthcare processed 15 billion healthcare transactions per year. When it went down, organizations discovered they had a single point of failure they had never modeled. Most vendor risk programs classify direct vendors; almost none map fourth-party dependencies or shared-infrastructure concentration.

Vendor Inventory and Classification

You cannot manage what you have not inventoried. The first step --- and the step most organizations do poorly --- is building a complete picture of every third party that touches your data, your systems, or your operations.

A functional vendor inventory answers four questions for each entry: What data does this vendor access? What systems does this vendor connect to? What would happen if this vendor disappeared tomorrow? Is this vendor within the authorization boundary of any compliance framework we operate under?

Classification follows naturally from those answers:

Tier	Criteria	Example	Assessment Cadence
Critical	Handles CUI/PHI/PII, direct system access, business cannot operate without them	Cloud hosting provider, EHR vendor, CUI processing subcontractor	Quarterly review, annual full assessment
High	Accesses sensitive data or environments, difficult to replace within 30 days	SIEM provider, payroll processor, identity provider	Semi-annual review
Moderate	Limited data access, replaceable within 90 days	SaaS productivity tools, marketing analytics	Annual questionnaire
Low	No data access, commodity service, trivially replaceable	Office supplies, facilities maintenance	Self-attestation at onboarding

The classification exercise is not a one-time event. Every new vendor onboarding, every contract renewal, and every material change in a vendor's service scope should trigger reclassification. A vendor that starts as "moderate" because they only process anonymized usage metrics can quietly become "critical" when someone grants them access to production logs containing PII.

For organizations pursuing multi-framework compliance, the vendor inventory feeds directly into authorization boundary documentation. NIST 800-53 SA-9 (External System Services) and FedRAMP both require explicit documentation of external services and the controls applied to them.

Assessment Methods: Questionnaires vs. Evidence

Here is a contrarian opinion that will save you thousands of hours: a 200-question vendor security questionnaire is security theater. Vendors lie on questionnaires --- or more charitably, the person filling out the spreadsheet does not know the actual answer and checks "yes" because saying "no" triggers a procurement review. The vendors who answer honestly are the same vendors who already have a SOC 2 Type II report you could have just read.

The hierarchy of vendor assessment evidence, ranked by reliability:

Independent audit reports (SOC 2 Type II, ISO 27001 certification, FedRAMP authorization package). These involve an independent assessor who tested controls over a sustained period. They are not perfect, but they are dramatically more reliable than self-attestation.
Technical evidence you can verify. Penetration test reports with identified findings and remediation timelines. Configuration exports. SBOM artifacts with provenance attestations. Vulnerability scan results. Evidence you can read, challenge, and correlate is worth more than a hundred "yes/no" answers.
Targeted questionnaires scoped to your actual risk. When you must use a questionnaire, keep it under 40 questions, tie every question to a specific control requirement that matters for your engagement, and require documentary evidence for any "yes" answer. A questionnaire that asks "Do you encrypt data in transit?" without requiring the TLS configuration or certificate chain is asking the vendor to check a box, not demonstrate a control.
Self-attestation. Appropriate only for low-tier vendors with no data access. If a vendor handles anything sensitive, self-attestation is not assessment --- it is hope.

The practical implication: for Critical and High-tier vendors, lead with audit report review and technical evidence requests. Use questionnaires only to fill gaps that the audit report does not cover (vendor-specific integration details, incident notification procedures, data residency confirmations). For Moderate-tier vendors, a scoped questionnaire plus a current SOC 2 bridge letter is typically sufficient. For Low-tier vendors, self-attestation at onboarding with a contractual clause requiring notification of material changes.

Risk Scoring That Actually Drives Decisions

A vendor risk score is useless if it does not connect to a decision. Too many organizations compute elaborate risk scores and then file them. A functional scoring model does three things: it prioritizes review cadence, it triggers escalation thresholds, and it informs contract negotiation.

Inputs to a practical vendor risk score:

Inherent risk (data sensitivity, system access, business criticality, regulatory scope) --- this is the "if everything goes wrong, how bad is it?" factor
Control effectiveness (derived from assessment evidence, not questionnaire answers)
External indicators (security rating changes, breach disclosures, dark web mentions, CVE exposure in their known technology stack)
Concentration risk (how many of your critical processes depend on this vendor or its fourth-party dependencies?)
Incident history (prior breaches, response quality, notification timeliness)

The score should map to concrete actions:

Score Range	Action
Critical risk (80-100)	Executive review, remediation plan required within 30 days, contingency planning initiated
High risk (60-79)	Increased monitoring cadence, contract addendum for specific controls, quarterly check-in
Moderate risk (30-59)	Standard monitoring, annual reassessment
Low risk (0-29)	Self-attestation cycle, re-score on material change

Track scores over time. A vendor whose score drifts upward over two consecutive quarters is telling you something. Either their security posture is degrading, or your engagement with them has expanded in ways that increase your exposure.

Continuous Monitoring: Beyond the Annual Assessment

Point-in-time assessments decay. A SOC 2 report covers a specific observation period; the week after that period ends, the vendor could decommission every control and you would not know until the next assessment cycle. Continuous monitoring closes this gap.

Effective vendor continuous monitoring combines:

External security ratings that track observable indicators (exposed services, certificate hygiene, DNS configuration, leaked credentials). These are imperfect proxies, but directionally useful as early warning signals.
Breach disclosure monitoring. Subscribe to vendor security advisories. Monitor SEC filings for material cybersecurity disclosures. Track industry ISACs for sector-specific threat intelligence.
Contract compliance verification. Are SLAs being met? Are quarterly attestations arriving on time? Has the vendor made infrastructure changes that affect your data residency requirements?
Technology dependency tracking. When a critical CVE drops (like CVE-2024-3094 in xz), can you determine within hours which of your vendors run affected software? This requires maintaining an asset inventory that extends into your vendor ecosystem --- at minimum, knowing the technology stack of your Critical-tier vendors.

The trigger model matters as much as the data collection. Define explicit thresholds that convert monitoring signals into actions: a security rating drop of more than 10 points triggers a vendor call within 48 hours. A disclosed breach affecting a Critical vendor triggers your incident response plan within 4 hours. A missed quarterly attestation triggers escalation to the vendor's account executive within one week.

Flow-Down and Shared Responsibility

In regulated industries --- defense, healthcare, financial services --- your compliance obligations do not stop at your organizational boundary. They flow down to every vendor that handles regulated data.

For defense contractors pursuing CMMC, this is explicit: if your subcontractor handles CUI, they need their own CMMC Level 2 certification. You cannot certify on their behalf, and you cannot waive the requirement. The flow-down obligation means your vendor risk program is not just protecting you from breaches --- it is ensuring your own certification is not invalidated by a non-compliant subcontractor.

For HIPAA-covered entities, Business Associate Agreements (BAAs) are the contractual mechanism, but a BAA without verification is a liability shield with holes. The BAA says the vendor will protect PHI; your vendor risk program verifies that they actually do.

For PCI DSS v4.0, Requirement 12.8 mandates maintaining a list of service providers, monitoring their PCI compliance status, and having a written agreement acknowledging their responsibility for cardholder data security.

The shared responsibility model requires clarity on two dimensions: who is responsible for each control (the RACI matrix), and what happens when responsibility is ambiguous. Most vendor breaches exploit exactly that ambiguity --- the space between "we thought they were handling patching" and "we thought they were handling patching."

Document the shared responsibility split explicitly in every Critical and High vendor contract. Map it to specific controls in your compliance framework. When a control is partially the vendor's responsibility, define the evidence they must provide to demonstrate their portion.

The Questionnaire Problem (And What Replaces It)

The fundamental problem with vendor security questionnaires is incentive misalignment. The vendor wants to win (or keep) your business. The person completing the questionnaire is usually in sales, legal, or a GRC function that is measured on deal velocity, not on the accuracy of security representations. There is no meaningful penalty for an inaccurate questionnaire response until a breach occurs --- at which point the contractual remedy is litigation, not prevention.

What actually works:

Evidence-based assessment. Replace "Do you have a vulnerability management program?" with "Provide your most recent vulnerability scan summary showing mean-time-to-remediate for Critical and High findings." The first question gets a checkbox. The second gets a document you can evaluate.

Continuous evidence streams. For Critical vendors, negotiate access to ongoing evidence: monthly vulnerability scan summaries, quarterly access reviews, annual penetration test executive summaries. This transforms assessment from a periodic event into a continuous relationship. It also makes the vendor's security investments visible --- vendors with mature programs are usually happy to share because it differentiates them.

Standardized frameworks with teeth. SOC 2 Type II reports, ISO 27001 certifications, and FedRAMP authorizations all involve independent assessors and sustained observation periods. They are not perfect (assessors vary in rigor, and scope limitations can hide gaps), but they represent a fundamentally higher bar than self-attestation.

Technology verification. For vendors with system access, implement technical controls that verify what the vendor claims. If they claim they only access your environment from hardened endpoints, enforce conditional access policies that verify it. If they claim encrypted connections, inspect the TLS configuration. Trust but verify --- except skip the trust part.

Key Takeaways

Vendor risk is supply chain risk. MOVEit, SolarWinds, Change Healthcare, and the xz backdoor all demonstrate that your weakest vendor defines your effective security posture.

Classification drives proportionality. Not every vendor needs a full assessment --- but every vendor that touches sensitive data or critical systems needs more than a checkbox questionnaire.

Evidence beats attestation. Audit reports, technical evidence, and continuous monitoring data are orders of magnitude more reliable than questionnaire responses.

Flow-down is now mandatory. CMMC, DFARS, HIPAA, and PCI DSS all require verifiable vendor compliance, not just contractual clauses.

Continuous monitoring closes the assessment decay gap. A SOC 2 report from 11 months ago tells you about 11 months ago, not today.

Concentration risk is the hidden killer. Map fourth-party dependencies before a single point of failure maps you.

Frequently Asked Questions

How often should we reassess vendors?

Cadence should match risk tier. Critical vendors warrant quarterly touchpoints (even if only reviewing continuous monitoring data) and a full reassessment annually. High-risk vendors need semi-annual review. Moderate vendors can operate on annual cycles. Any material change --- a breach disclosure, a significant score drop, a change in the data they access, or a change in their ownership --- should trigger an out-of-cycle reassessment regardless of tier.

What is the minimum viable vendor risk program for a small defense contractor?

Start with three things: a complete vendor inventory classified by CUI access, a requirement for SOC 2 or equivalent evidence from any vendor handling CUI, and contractual flow-down clauses that mirror your DFARS 252.204-7012 obligations. You can build from there, but those three elements satisfy the core NIST 800-171 requirements for external system services and give you a defensible position during a CMMC assessment.

How do we handle vendors who refuse to complete assessments?

A vendor that refuses to provide security evidence is telling you something important about their program maturity. For Critical and High-tier vendors, refusal should be a disqualification event or trigger executive-level escalation. For Moderate-tier vendors, you can accept the refusal if you implement compensating controls (network segmentation, data minimization, enhanced monitoring). Document the risk acceptance with an explicit owner. Never let a refusal simply go unaddressed --- silence is implicit risk acceptance without accountability.

Should we use external security rating services?

External ratings (SecurityScorecard, BitSight, etc.) are useful as one input to continuous monitoring but dangerous as a sole assessment method. They measure externally observable indicators, which correlate loosely with actual security posture. A vendor with an excellent external rating can still have catastrophic internal control gaps. Use ratings for trending and early warning; do not use them as a substitute for evidence-based assessment of Critical and High-tier vendors.

What is fourth-party risk and how do we address it?

Fourth-party risk is the risk introduced by your vendors' vendors. When Change Healthcare went down, organizations discovered that their claims processing vendor depended on Change Healthcare --- a fourth-party relationship they had never mapped. Address it by requiring Critical vendors to disclose their own critical dependencies, including concentration risk in your scoring model, and maintaining contingency plans that account for cascading vendor failures. You will never have perfect visibility into fourth-party risk, but you can identify the concentration points that would cause cascading failures.

How Advisedly Helps

Advisedly maps vendor risk across your entire compliance surface --- CMMC, NIST 800-171, SOC 2, ISO 27001, HIPAA, PCI DSS --- through a single vendor inventory that classifies, scores, and continuously monitors third-party relationships against 500+ supported frameworks. Rather than generating yet another questionnaire, the platform ingests audit reports, tracks evidence artifacts, correlates external monitoring signals, and surfaces the flow-down obligations specific to your regulatory environment so that vendor risk decisions connect directly to your authorization boundary documentation. Contact begin@advisedly.ai to build a vendor risk program that runs on evidence instead of spreadsheets.