can a regex search engine for the web be done and if so, how?

Question

I think it would be a great idea if we had a regular expression search engine, but can it be done? It seems impossible since there's no feasible way of indexing such a huge amount of data properly and displaying results in the order of relevance from what I can think of.

I'm curious what the answer to this one would be.

Answer 1

It can be done, and it has been done: http://www.google.com/codesearch

I think it is useful here because this is in a specific domain where the users might be expected to understand regexes. I suspect in general the feature is not that useful to the vast majority of Google users so there is probably no commercial reason to do it.

• Valentin Brasso: This is great but the source code on codesearch is a very tiny fraction of the web pages indexed by Google. I suspect there are some limitations because the commercial reason argument isn't that strong (they do have a number of useful keywords like site, filetype, intitle, inurl etc.). I think the load on their servers would be too great for regex searches.
• Steve: If Google thought Regex would be useful to enough people they could scale. They scaled for instant answers, that was a huge effort. Search is worth billions of ,ドル if regex gave them an advantage they would do it, but 99% of users have no idea what a regex is, so it is simply not worth the effort.
• JeffO: Google doesn't shy away from throwing hardware at a problem.
• MetaEd: Note that Google Code Search is being retired.
• Vaibhav Garg: has been retired now!