Wanted to open up a discussion on search improvements (#1885), since search is a multi-objective optimization problem requiring opinionated decisions on how to weigh each objective. There are four main objectives: exact keyword matches, semantic matches, popularity (zaprank), and recency. (Do we know if people ever sort by comments or sats?)

Currently, searching by "recent" is broken because the "must" field requires a pretty precise keyword match, thus often returning no results. You can test this by changing the following lines in api/resolvers/search.js:

For example, if you change type to most_fields and add fuzziness: AUTO, searching by recent will no longer return null results while still prioritizing recency.

But before fixing that, it would be helpful to think through what SN's search is trying to accomplish and how search performance should be measured. There is a lot of info out there, and the specific details of the search engines of major companies is a closely guarded secret. Thus, there isn't a one-size-fits-all or industry-standard solution.

I have a couple of thoughts on how to proceed:

Learning-to-Rank

The obvious, but heavy-lift, solution would be to use ML on users' actual searches (what they searched and what they clicked on, and whether they re-did the search) to estimate optimal weights based on SN users' behavior. If user data is not available, synthetic data could be generated for learn-to-rank, but the results aren't likely to be as robust to the actual user experience.

The downside of this approach is that it's a lot of work. The juice may not be worth the squeeze.

Off-the-shelf solution plus trial and error

The only other approach I can really think of is to take some off-the-shelf approach suggested by others, like applying a gaussian decay function to created_at for recency searches, combined with a hybrid keyword/neural approach for query terms. Another suggested approach is reciprocal rank fusion, which is claimed to perform quite well and is simple to implement.

But even with off-the-shelf methods, some parameters will need to be chosen, like how much to weigh recency vs. relevance. However, without an obvious evaluation metric, it will be hard to know whether any changes to the search function resulted in improved searches. Here might just be where trial and error comes in: see whether a newly implemented search function results in positive or negative user feedback, as well as personal experimentation with the results.