{"id":25740,"date":"2022-11-10T00:53:09","date_gmt":"2022-11-10T00:53:09","guid":{"rendered":"https:\/\/www.esecurityplanet.com\/?p=25740"},"modified":"2022-11-10T22:09:55","modified_gmt":"2022-11-10T22:09:55","slug":"mitre-mssp-tests","status":"publish","type":"post","link":"https:\/\/www.esecurityplanet.com\/cloud\/mitre-mssp-tests\/","title":{"rendered":"MSSPs Fare Well in First MITRE Evaluations"},"content":{"rendered":"\n

If MITRE Engenuity’s new MSSP evaluations are any indication, managed security service providers<\/a> are a little like children from Lake Wobegon: They’re all above average.<\/p>\n\n\n\n

Of the 15 MSSPs that participated in MITRE’s first-ever security services testing, only three failed to report attack techniques in all 10 of the evaluation steps, and in two of those cases it was because the test didn’t successfully execute because of a web shell failure.<\/p>\n\n\n\n

While the sample is small – by some estimates there are roughly 10,000 MSSPs – it nonetheless should be reassuring to MSSP customers that the vendors charged with defending their networks have demonstrable cybersecurity expertise. As there are few measures of security effectiveness, and none better than MITRE, it would benefit information-starved security buyers<\/a> if more service providers participated in future rounds.<\/p>\n\n\n\n

Ashwin Radhakrishnan, general manager of MITRE Engenuity’s ATT&CK Evals, said in a statement that the organization decided to evaluate MSSPs because of their growing importance.<\/p>\n\n\n\n

\u201cMore than half of organizations use security service providers to protect their data and networks,\u201d Radhakrishnan said. \u201cWe wanted to research how they are employing threat-informed defense practices for their clients. We don\u2019t rank the vendors in our evaluations. Organizations, however, can use the Evals to determine which service providers may best address their cybersecurity gaps and fit their particular business needs.\u201d<\/p>\n\n\n\n

See the Best Managed Detection and Response (MDR) Services<\/strong><\/a> and the Top MSSPs<\/strong><\/a><\/p>\n\n\n\n

MSSP Tests Look At Reporting, Not Detection<\/h2>\n\n\n\n

MITRE is best-known for its endpoint security product evaluations<\/a>, but there are some important differences between the organization’s product and services evaluations.<\/p>\n\n\n\n

The MSSP evaluations examined how vendors performed under techniques that simulated attacks from the OilRig Iranian threat group, which was chosen because of its “evasion and persistence techniques, its complexity, and its relevancy to industry,” MITRE said<\/a>.<\/p>\n\n\n\n

The evaluation examined the MSSPs’ ability to report ATT&CK Techniques<\/a> across 74 techniques and 10 steps, from initial compromise through lateral movement, exfiltration and cleanup.<\/p>\n\n\n\n

An important emphasis in the new tests is on the word “report” rather than the detections measured in MITRE’s endpoint tests. MITRE purple teamers evaluated whether an ATT&CK Technique was reported<\/em> or not, rather than whether it was detected<\/em> by the service provider, MITRE said.<\/p>\n\n\n\n

“In many cases, the service provider may have detected the ATT&CK Technique under test but chose not to report it to MITRE Engenuity because they believe it is unnecessary information, or they believe it can be implied or assumed by other information provided to MITRE Engenuity,” MITRE said on the MSSP evaluation’s overview page<\/a>. “In order for an ATT&CK Technique to be considered Reported, the activity provided to MITRE Engenuity must contain sufficient context to explain the activity. Things like raw telemetry with no added analysis provided by the service provider were not considered Reported.”<\/p>\n\n\n\n

That means the data provided by the tests isn’t as clear as it is in the product evaluations. So while we’ve recorded below the number and percentage of techniques reported by the MSSPs, as always it’s important to dig into the data and find what’s relevant for your organization’s needs.<\/p>\n\n\n\n

In a blog<\/a> on interpreting the results, Radhakrishnan noted a number of important considerations, among them:<\/p>\n\n\n\n