Wildcard Serving #259

As discussed in #227, wildcard serving is hurting search results for Codeberg.
I propose returning a 404 error or redirecting to codeberg.org/*.

We need a proper technical solution here: redirects do not work as browsers reject them with certificate error. Any idea?

@hw - While a redirect wouldn't work for browsers, it would work for crawlers, preventing duplicate search results. But in general, returning 404 Not Found would be the cleaner solution, as redirecting suggests the URL is valid.

crawlers are redirected to TLS too, and then fail the cert check just like a browser?

The search results from the other issue suggested differently, if I remember correctly. But either way, I think a 404 would be the better option 😉

Any updates on this @hw?

Not really

I don't understand the issue tbh. We only redirect to the main domain for a while now. Do I miss something or is this outdated?

Subdomains like mail.codeberg.org and smtp.codeberg.org still serve the Codeberg website rather than redirecting or returning an error. Although it doesn't appear to be a problem for search results anymore.

Huh? Wow.

mail.codeberg.org and smtp.codeberg.org are now answered by the pages server it seems.

Should pages react to *.codeberg.org in the first place?

Well, the setup is that everything haproxy does not know is forwarded to the Pages Server, because it might be a custom domain.

It's same like opening https://thispagedoesnotexist123456.codeberg.page. I also think it resolves the initial problem discussed here, because the Pages Server explicitly responds with 404 already.

If we want to fancy this up, we could add targets for known *.codeberg.org names that we want to get rid of and close the connection, or serve another 404 Page.

Oh, and I plan to get rid of the DNS wildcard for codeberg.org, too. This would also make this fancier in the future.