I often read "the performance of the system is satisfactory" or " when your model is satisfactory".
But what does it mean in the context of Machine Learning?
Are there any clear and/or generic criteria for Machine Learning model to be satisfactory for commercial use?
Is decision what model to choose or whether additional model adjustments or improvements are needed based on data scientist experience, customer satisfaction or benchmarking academic or market competition results?
The answer is "when it works well enough to perform the task that you have set it". It is a good idea to set your performance criteria in advance so that you can clearly identify the goal that you are trying to achieve and also so that you will know if the model is likely to be successful or not.
From what I have observed, the ability to scale an ML model is key.
Real time inference must be quick, and cause no delays from the provider side. Being able to deploy the model also carries enormous weight - that is, how easy would it be to build the data pipelines and how easy would it be to integrate it in a web application from the server perspective.
Apart from the obvious achievement of set metrics and performance criterion, speed and ease of deployment also carry a very important role. There have been scenarios of brilliant solutions being denied (from what I have seen) because they exceeded the limits set for time and compute in an application scenario.
