Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Proposal: Standardize User-Agent headers in Importers to prevent 403 blocking #2123

Kiran95021 started this conversation in General
Discussion options

Hi everyone! 👋

I am a new contributor looking into the importers module. While analyzing how we fetch data from upstream sources (like GitHub, GitLab, and NVD), I noticed that we don't seem to have a standardized User-Agent strategy.

The Problem

Many importers currently use the default library User-Agents (e.g., python-requests/x.x.x or aiohttp/x.x.x).
From my experience, many modern WAFs (Web Application Firewalls) and APIs block these default agents, resulting in silent failures or 403 Forbidden errors. This makes our data collection less reliable.

The Proposal

I would like to propose standardizing our network identity across the project.

1. Centralize the Configuration
Add a constant in vulnerablecode/settings.py:

# Example
VULNERABLECODE_USER_AGENT = "VulnerableCode/1.0 (+[https://github.com/aboutcode-org/vulnerablecode](https://github.com/aboutcode-org/vulnerablecode))"
You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant

AltStyle によって変換されたページ (->オリジナル) /