Ok, so I have my project idea, what do I do next? Think about how you are going to judge it first.
When I was trying to figure out my project plan, I read the second chapter of Building Machine Learning Powered Applications. Which establishes 3 steps to start your project. They were:
Attempt to reproduce results from a similar open-source model.
Find a data set closer to your objectives and attempt to train the previous model on that dataset
Judge how the model works using the metrics you defined and start iterating
well, metrics? This book talks about how to choose them as well. There are two kinds of metrics: project and model metrics, the first one talks about how the application tackles its main problem, and the latter is more about the ml model itself.
In my case, the metrics I chose were:
how many recommendations are actually clicked when they appear in the app,
F-score
About the project metric, this book copes with a recommendation project as well. They define a similar metric for theirs, and I found it really useful for my project as well. Nevertheless, I'm still hesitating about it.
For my model metric. As I will use text sentiment analysis for recommending the first results, I found here that the most used evaluation metrics in this field are Precision, Recall, F-score, and Accuracy. Since F-score resembles the mixture of both the precision and recall score, I think this is the best approach for me at the moment.
I would go on by leveraging open source code next.