When we are working on an AI project, does the context (academia, industry or competition) make the process different?

Question

When we are working on an AI project, does the domain/context (academia, industry, or competition) make the process different?

For example, I see in the competition most participants even winners use the stacking model, but I have not found anyone implementing it in the industry. How about the cross-validation process, I think there is a slight difference in industry and academia.

So, does the context/domain of an AI project will make the process different? If so, what are the things I need to pay attention to when creating an AI project based on its domain?

Answer 1

I cannot comment about the process for AI for academia. I can compare AI for competitions and AI for business. To clarify whatever I say is about ML not any other AI techniques. The process might be different for other techniques. But most of things that I say are general enough that I am assuming should still apply.

The main difference that I saw while doing ML for a competition vs. for a business was that of focus.

When doing it for a competition for Kaggle the focus was mainly creating the model

• machine learning metrics are specified for you
• some data was given to you
• business problem was given to you

When doing it for business what is different

• given a business problem finding the parts that can actually benefit from ML. You have to define the ML problem in it and define how it actually benefits the business. This may involve significant discussions with business stakeholders, weighting the pros and cons of doing it versus doing something else, communicate the benefits to the business stakeholders, take them into confidence for the process to start
• find the right data for the problem from scratch, ensure it is collected by rest of the system or brought from 3rd parties
• define business metrics over and above machine learning metrics. At the end of day nobody really cares about whether ML model recall, accuracy is good or bad. What is important is the relevant business effect.
• make the model, deploy it and integrate it with the rest of the system. This is important because if your goal is just to making the model you would not care about the factors associated with actually using it i.e. latency of predictions, cost of machines needed to run it etc.
• A/B testing for the models, running multiple models in parallel, dynamically being able to adjust which models to use

Hope this gives some idea about the differences in AI for competitions and AI for business.

Answer 2

Not very sure about the AI in competitions, as I have not taken part in any competitive competitions. On comparing AI in Academia and Industry, the biggest difference is probably freedom.

In academia, considering a research project or so, a large number of experiments and trying new things are encouraged. New learnings are heeded to, and it usually involves rigorous literature survey and studies of previous works. Even if a model performed badly, if there were new learnings one could take from it, it wouldn't be deemed a failure. There is also a lot of data available that could be used for research purposes, and open-source projects used or learned from, are always thanked and appreciated.

In industry the scene is quite different. There is more of a focus on using pre-trained models or transfer learning. Quite frequently, open-source projects are just cloned, mildly developed, and deployed under the companies name without releasing the code - basically requiring bare minimum effort towards literature. More of a focus was given (In my case at least) on reading blog posts and readme's, over the papers themselves, in order to save time. And compute efficiency is key. In industry, the effort is more directed towards scaling these models, building the data pipelines, and satisfying the clients needs. Data is also another concern in industry, with it being common practice to outsource data collection and preparation to third parties (Usually other companies that specialize in this area).

The key difference, I would say, is the amount of freedom one has in academia, as compared to a strong sense of direction towards a singular goal in industry. AI in industry pretty much mostly is in the solutions-and-services sector (mostly), making it quite similar to software engineering, broadly speaking.

So, summarizing, the domain of the AI project makes a big difference, with the main difference being what part of the project most effort and focus is put into.