Hello!

I was interested in creating a semantic search engine for the Codecademy docs. I wanted to start the discussion about doing this. At the moment the ChatGPT hype has inspired a lot of projects where a client can basically talk to their documentation. Although this is a great solution for talking to ChatGPT based on the context of a document, I wanted to see if we could use a specific aspect of this design to make the docs more accessible. This aspect is the vector search. Pinecone has a good article that goes over the basics of what a vector search is.

From my understanding the docs currently do not have a search feature. Although it's UI is currently organized in a tree-like structure. As a result when users are searching for a specific document, they have to start with a broad topic and then narrow down their search to find the article they are looking for. This might be fine with the size of the documentation, but as the documentation scales it might become harder for a user to find the exact material they are looking for. In addition, this approach of starting broad, and then narrowing down the search poses a risk of not finding relevant articles. This is a missed opportunity for the user.

With vector search, a user can input a query (such as a sentence), and the search engine can return articles that has content that matches the user's query. This approach is more scalable, and even more precise than the current search method. Mainly because a vector database can serve articles that are within a specific threshold of similarity. Now, when a user queries "algorithms" they may get articles in JavaScript, Machine Learning, and Python, while previously they may have just narrowed down on Machine Learning.

I'd love to build this for the community! Although I think some buy-in is needed from Codecademy. Let me know what you guys think! More information can be provided upon request.

cc: @yangc95 @aherman91 @caupolicandiaz @HishamT @KTom101 @SSwiniarski