Last week, we wrote about confusion over whether a claimed vulnerability in a WordPress plugin exists if it hasn’t been mentioned by a particular data source. That was in the context of a developer claiming there wasn’t a vulnerability in the plugin because it wasn’t mentioned by one of those, WPScan, despite being included in another, Patchstack. We also noted that Patchstack had not provided the information needed for anyone else to confirm their claim of a vulnerability.

Someone involved in yet another data source submitted a comment on that post, though it appears they didn’t pay attention to what the post said, to the detriment of those relying on it. Part of what they said in promoting their data source is they had this vulnerability in its data set. That isn’t surprising since on their website they admit to copying information from Patchstack. They didn’t address the inability to confirm the claimed vulnerability, which someone would want to before adding it to their data set.

In the source’s marketing, they don’t take responsibility for their dataset’s reliability, instead stating:

The information provided by the information database comes from different sources that have been reviewed by third parties.

If they haven’t reviewed the information, how would they know that it has been reviewed by someone else? They wouldn’t. We will get to how that causes them to include inaccurate data on plugin vulnerabilities in a moment, but they are not the only new source that is simply copying data from existing sources.

Recently, the security company PortSwigger’s news outlet covered another of those and the creator made this claim about it versus WPScan:

Miller has said on Reddit that WPHash is not intended to be a direct competitor to Automattic WordPress security scanner WPScan, whose vulnerability data and tooling he concedes is currently “more curated and more complete”.

That the creator thinks WPScan can be described with the words curated and complete doesn’t bode well for the quality of the data they are providing. To see that, we will take a look back at what has been going on with the data coming from various providers.

Patchstack and WPScan Provide Inaccurate Data

In April, we noted that the developer of the plugin Download Monitor had attempted to fix security issues in their plugins, but the security change they made was incomplete. We also noted that Patchstack had claimed that this was a vulnerability and had claimed five months before that, that the vulnerably had already been fixed. It wasn’t a vulnerability, but even it was, something clearly went wrong on their end to have said it was fixed five months before there was an attempt to do that. They claimed it was addressed in version 4.4.7 of the plugin:

That is far from the only time Patchstack has gotten basic details, like if there was even an attempt to fix a vulnerability, wrong. So it would be important for anyone to carefully check their claims before adding them to their own datasets. That isn’t what happens with WPScan.

WPScan added an entry for this vulnerability, without crediting Patchstack. Their entry confusingly claimed that the vulnerability affected versions below 4.4.7, but also was fixed in 4.4.5:

As there had yet to even be an attempt to fix this, both of those were wrong.

While WPScan claimed there was a vulnerability and it had been fixed, they also said they hadn’t verified it, which doesn’t make sense. If they haven’t verified it, how could they know those things? They couldn’t.

Both Patchstack and WPScan are selling access to their data, despite not doing the work that would justify that.

WPVulnerability

The commenter we mentioned earlier was promoting WPVulnerability. Considering they explicitly don’t verify their data, it isn’t surprising to see they also claimed that there had been a vulnerability, and it had been fixed when it hadn’t:

Confusingly they are saying that it affected version below 4.4.7, but also the max affected version is 4.4.7.

WPHash

With WPHash, it looks like they are copying information from WPScan, as they also confusingly claim it was fixed in version 4.4.5 while impacting version through 4.4.6:

We mentioned that this particular entry was inaccurate to the creator of that source in September, but they didn’t correct it.

The developer does at least have a disclaimer on the GitHub project for it:

This repo includes both user contributed and publicly available vulnerability data and as such varies in both accuracy and quality. The affected version numbers are unfortunately quite a pain to get right, if you stumble upon invalid data feel free to contribute!

They don’t have a similar warning on the website for the data source.

Also Missing Recent Vulnerability

A month ago we noted that both WPScan and Patchstack had failed to warn about a newly disclosed vulnerability in one of the most popular WordPress plugins, despite claiming that they will promptly warn about vulnerabilities. That was of more concern, since the developer originally hadn’t fully addressed the vulnerability.

As WPVulnerability and WPHash are copying their data from those sources, it seemed unlikely they would have added it either. But we checked and found it was also true of them.

History Repeats Itself

The way WPVulnerability markets itself, as being an “open and free API” and asking for donations from heavy users, is how WPScan used to market itself. WPScan then overtime put in place limits on free access. There isn’t anything to stop WPVulnerability from doing the same.

WPScan at least wasn’t wholesale copying other’s information while trying to get someone to pay them for doing that.

Don’t Pay for Unverified Data

If someone is going to spend money on this type of data, they should do it with a provider that is actually doing the needed verification work, instead of copying others, as it leads to the continued spread of inaccurate information and hardly leads to “improving the security of a site” as WPVulnerability markets itself as doing. All the providers mentioned here are failing to do that work.