Beginning Algorithm Design and Proving

Question

I know this is a general question, so you can address this by giving some examples that are specific to your field.

I want to start learning algorithm design and consequently prove its convergence. Most of the algorithms I am interested in will be for use in machine learning and its applications. Where do I begin? I am reasonably proficient in mathematics and machine learning, but have been the enduser of such algorithms. Where/what books do I start learning for 'design' if I want to make an algorithm and analyse it myself?

thanks

Answer 1

If you really want to study algorithms, start by looking at a few courses on algorithms at reputable universities, check what textbooks they offer, and start reading one of those textbooks. Or, pick an online algorithms course (perhaps a MOOC, through companies such as EdX, Coursera, Udacity, etc.) and take it.

I personally am fond of Algorithms by Dasgupta, Papadimitriou, and Vazirani, but others have other preferences.

If you search this site, I think you can find many other recommendations for other algorithms textbooks.

It's possible you might actually be more interested in learning about optimization and numerical methods / numerical analysis, rather than classical algorithms. You can use the same procedure to find a textbook or course on one of those subjects.

Be warned that it's unlikely you will be able to prove convergence guarantees for most modern machine learning algorithms (e.g., those that use neural networks). Modern machine learning is primarily an empirical science, rather than one that comes with provable guarantees.