1
$\begingroup$

(Disclaimer: I don't know much about ML/AI, besides some basic ideas behind it all.)

It seems like ML/AI models can often be boiled down to statistics, where certain levers (weights) get fine-tuned based on the specific input of a large set of training data.

Clearly, ML/AI models don't only distinguish themselves in their training data alone, otherwise there would not be so many improvements happening in the field all the time. My question therefore is: What does distinguish different models of the same category?

If I have an AI that completes real-life pictures that have some missing parts, and an AI that completes a painting with missing parts, what key concepts separates the two?

If I have an AI detecting text in an image, and an AI detecting... trees in an image, what key concepts separates the two?

In other words, what is stopping me from "taking" an existing implementation of a certain AI category, and just feeding it my specific training set + rewards (i.e. judgement criteria for good vs bad output), in order to solve a specific task?

In yet again other words, if I wanted to use ML/AI to build a new model for a specific task, what concepts and topics would I need to pay extra attention to? (I guess you could say I'm trying to reverse engineer the learning process of the field here. I don't have the time to properly teach myself and become an "expert", but find it all very interesting and would still like to use some of the wonderful things people have done.)

Improve this question
$\endgroup$
5
  • $\begingroup$ I don't fully understand what you mean by this "ML/AI models don't only distinguish themselves in their training data alone". Are you saying that ML models are not only different in terms of the data they need to deal with (in terms of inputs and outputs)? This post is a bit problematic, because, even though you do not realise it, you're asking us to describe what machine learning is and which types of machine learning are there, from what I understand. $\endgroup$
    – nbro
    Dec 10, 2020 at 1:30
  • $\begingroup$ If you're looking for different types of machine learning technique, maybe you should start from this or this answers. I suggest that you read those answers carefully, then try to narrow the scope of this post. Right now, it's broad, because essentially you're asking us how to solve problem with machine learning. I suggest that, as an example, you focus on a specific problem that you would like to solve, then ask a more specific question. $\endgroup$
    – nbro
    Dec 10, 2020 at 1:34
  • $\begingroup$ Moreover, note that AI and ML are not the same thing. ML is a subfield of AI concerned with learning from data, as you will understand after having read the answers I linked you to, which you should really read before proceeding. $\endgroup$
    – nbro
    Dec 10, 2020 at 1:35
  • $\begingroup$ @nbro Thank you for your answer. I read the answers you linked, and think that I was already aware of those "generalities"; What I'm trying to ask, I guess, is, for a GIVEN model, in what steps of a complete algorithm do two different tasks differ. For example, take two image inpainting tasks, getting rid of wrinkles and removing an object. Both will use a similar model/approach, but what I want to know is what makes them different: Say, if I were given one of the two algorithms, what would I have to change to produce the other? $\endgroup$
    – Fly
    Dec 10, 2020 at 15:09
  • $\begingroup$ But there are so many possible differences between a model/solution and another. For example, the number of layers, the number of units, the optimizer, etc., etc., that you use in one case could be different than in another. Or maybe the model is completely different, the data is different. So, I think you should really focus on a more specific problem and ask a more specific question, because, right now, I don't think this question can be reasonably answered. $\endgroup$
    – nbro
    Dec 10, 2020 at 16:26

1 Answer 1

Reset to default
1
$\begingroup$

If I understand what you mean correctly then the answer is basically nothing. Fundamentlaly all ML algorithms are ding the same thing, which is to optimize some weights for a certain output (this is true even for non-parametric methods in an implict way, but lets not dive too deep here). The only differences are:

  1. Dataset the model is trained on
  2. Specific dimensionality of inputs and outputs to make them compatible with the input data and the output labels.
  3. Complexity that can be expressed by the model (the number of weights and their structure in layers in the case of Neural Networks).
  4. Changes to the optimization process (gradient clipping, regularization, ecc.)
  5. Structural changes to the architecture that embed assumptions about the specific problem setting (e.g. 2D-Convulutions embed assumptions for images, softmax activations embeds the assumption for classification with probabilities, hidden states embed the assumptions about memory and "fogetting", ecc...)

Therefore, if you have a "similar" problem setting where you can assume that 3, 4, 5 can be the same without problems, then you can just make the appropriate changes in 2 and change the dataset (1) to get a model that works on something different.

Of course, being able to tell how "similar" a problem is to another and what possible things could be different is quite tricky and requires a lot of knowledge about the domain of Machine Learning and how every algorithm works and why.

In essence, I'm saying that in principle you could take a model and train it for a new setting with very minor and simple changes that don't require a lot of knowledge. However, without the wider knowledge on the field of ML/AI you won't be able to tell if what you are doing is ok and how well can it work in general.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Ah, interesting, thank you for the answer. So I'm assuming that I can't really go around learning the details of each of these steps in order to judge where I can "order" my task into? Do you have a recommendation on what learning approach would be best? $\endgroup$
    – Fly
    Dec 13, 2020 at 22:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .