-
-
Notifications
You must be signed in to change notification settings - Fork 309
Conversation
Implements ZDI importer pipeline (issue 1471) using the Zero Day Initiative RSS feeds at zerodayinitiative.com/rss/published/YEAR/. Fetches year-specific feeds from 2007 through the current year, parses advisory ID from the link URL, extracts CVE aliases from the description, and records the publication date. Deduplicates advisories across feeds using a seen-IDs set. Includes 7 unit tests covering normal parsing, missing CVE, missing link, invalid XML, and alias deduplication. Signed-off-by: newklei <magmacicada@proton.me>
Signed-off-by: newklei <magmacicada@proton.me>
a39c6a5 to
49660d4
Compare
Signed-off-by: newklei <magmacicada@proton.me>
@ziadhany
ziadhany
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@NucleiAv , This is a good start. Please see the comments there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i don't think we need this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are missing CVSS parsing. See:
https://www.zerodayinitiative.com/advisories/published/2025/
Also, have a look at the other importers to see how we can store the CVSS score.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we have a seed ids ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there might be a better way to do this. You could use the dateparser library. See the other importers for how we handle this parsing
>> import dateparser
>> dateparser.parse("Tue, 07 Jan 2025 00:00:00 -0600")
>> datetime.datetime(2025, 1, 7, 0, 0, tzinfo=<StaticTzInfo 'UTC-06:00'>)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have a function for this
see utils.py file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ziadhany
Fixed all points:
Removed repo_url
Removed seen_ids: agreed, the pipeline framework handles deduplication at the DB level
Replaced strptime + PUBDATE_FORMAT with dateparser.parse()
Switched to find_all_cve from utils.py
Added CVSS score parsing via CVSS_RE = re.compile(r"CVSS rating of (\d+\.?\d*)") - the RSS description contains e.g. "The ZDI has assigned a CVSS rating of 8.8." so this extracts the score and stores it as a VulnerabilitySeverity with GENERIC system
Also fixed two CI failures that showed up:
Black - zdi_importer.py and test_zdi_importer.py were not formatted to --line-length 100
isort - __init__.py had the zdi_importer import inserted out of alphabetical order (between gitlab and istio instead of after xen)
Signed-off-by: newklei <magmacicada@proton.me>
- Remove repo_url field (not needed by base class) - Remove seen_ids deduplication set - Replace strptime/PUBDATE_FORMAT with dateparser.parse() - Use find_all_cve from utils.py instead of local CVE_RE - Add CVSS score parsing from RSS description field - Update expected test fixture to include CVSS severity Signed-off-by: newklei <magmacicada@proton.me>
NucleiAv
commented
Mar 8, 2026
@ziadhany the workflows have succeeded... it is good to go
Let me know if anything else is needed!
Signed-off-by: newklei <magmacicada@proton.me>
940b36f to
03591b3
Compare
Uh oh!
There was an error while loading. Please reload this page.
Closes #1471
Collects ZDI security advisories from the public RSS feeds at zerodayinitiative.com/rss/published/YEAR/, one feed per year from 2007 to the current year. Each advisory gets its ZDI ID from the link URL, CVE aliases from the description text, publish date, and a reference URL. The RSS feed has no structured package or version data, so affected_packages is empty. New files are zdi_importer.py, test_zdi_importer.py (7 tests), zdi_rss_mock.xml, and expected_zdi_advisory_output1.json.