Anant Raj and neural networks: an overparametrization perspective

Anant Raj is a laureate of the highly selective Marie Sklodowska-Curie Actions Individual Fellowship-Global Fellowship (MSCA-IF-GF). This programme supports the mobility of researchers across Europe, as a chance to boost their career. There is no nationality restriction to apply and all research areas are covered.

Under H2020, Anant applied to a Global Fellowship grant that offers the possibility to work as a researcher outside the European Union, with a compulsory return phase in the European Union. Two years at the University of Illinois Urbana-Champaign (UIUC) followed by one year at Inria Paris is the exciting programme that awaits Anant.

What is your research area? Why have you specialised in your current field of research? 

I work in machine learning and optimization. More specifically, I am interested in (Non)-Convex Optimization Theory, Neural Network Theory (Optimization and Generalization) and Statistical Learning Theory.

I explored a lot before settling in this area. I did research in the field of speech processing and computer vision before I started to work in this field. While exploring, I realised that I like to work on the mathematical problems which have good practical implications. Hence, I decided to specialise in this field. 

Can you tell us about your research path so far?

Well, that was quite a journey. I started doing research in my second year of undergraduate studies. I was then interested in speech signal processing. Along with that I completed some projects in computer vision on my own and decided to pursue computer vision research after I published my first paper in speech signal processing. Since then, I continued working on computer vision problems. In fact, I did my master’s thesis in computer vision. In the meantime, I visited Georgia tech and worked there on kernel methods which was more math oriented. When I finished my internship at Georgia tech, I was sure that I wanted to pursue a PhD in optimization theory and machine learning theory. I worked in other areas as well but I kept working on optimization theory and machine learning theory as my main research area in my PhD.

What is your research project about?

In recent times, overparametrized neural networks where the number of model parameters far exceeds the number of training samples available are the methods of choice for learning problems. Despite improved empirical performance of overparametrized neural networks, the theoretical understanding of these models is quite limited which hinders the progress of the field in the right direction.

During the Marie-Curie Global Fellowship, I plan to study the optimization problem arising while training overparametrized neural networks and the approximation guarantees of overparametrized neural networks.       

What are you expecting from your Marie-Curie grant?

I can already see it would be 3 amazing years of research life ahead working in the group of Francis Bach at Inria Paris and in the group of Maxim Raginsky at UIUC. Both of these research groups are doing amazing research works in the field of machine learning theory. 

Was it difficult to get your grant? Do you have a piece of advice for those who are thinking to apply to such a grant?

I think I got a bit lucky in that sense. My supervisor at Inria (Francis Bach) was very helpful in the entire process. Marie-Curie application takes time and it requires patience to apply for the grant. So, it is very important that you do not give up in the process. I remember I was not able to write anything in the proposal for almost a week. But slowly, it happened. First thing would be to find a supervisor for the fellowship duration. Once you find the advisor, my advice would be to think through the problem you want to work on and make a sketch in your mind first before writing anything in the proposal. Then, slowly it would happen. Start well before (2 months) the deadline. Once the application is submitted then do not worry about the final decision. It would come someday. 

What do you like the most about being a researcher?

I have enjoyed being a researcher so far. It gives you the opportunity to be in the company of very smart people. You will find that most of your colleagues are much smarter than you but that’s where the fun starts. Your learning curve is exponentially steep. You get to choose the problems which you like and dedicate the next few months or years on that problem while working on them, isn’t it amazing? 

What does Europe mean to you, especially as a young researcher?

I have spent 5 years of my time in Europe while being a PhD student. Being in Europe has contributed a lot in shaping my research. Due to easy immigration law, it was possible for me to visit various research labs having outstanding researchers in different countries of Europe and all these places have contributed in making me a better researcher as well as a better person. This would not have been possible anywhere else. I am very grateful for this.