What is the “thing” which is trained in AI model training

Question

I am a newbie in the fantastic AI world, I have started my learning recently. After a while, my understanding is, we need to feed in tremendous data to train a or many models.

Once the training is complete, we could take out the trained models and "plug in" to any other programming languages to use to detect things.

So my questions are:

1. What are the trained models? are they algorithms or a collection of parameters in a file?

2. What do they look like? e.g. file extensions

3. Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??

Thank you!

Answer 1

This answer applies to Machine Learning (ML) part of AI, as that seems to be what you are asking about. Please bear in mind that AI is still a broad church, including many other techniques than ML. ML, including neural networks for deep learning, and Reinforcement Learning (RL) is only a subset of AI - some AI techniques are more focused on the algorithm than parameters.

1. What are the trained models? are they algorithms or a collection of parameters in a file?

In ML, the usual process is to feed data into parametric function (e.g. a neural network) and alter the parameters of it to "fit" the data. The main output of this is a collection of parameters and hyperparameters that describe the parametric function. So 90% of the time, when discussing the "trained model", it means the same thing as the collection of paramaters.

However, those parameters are of limited use without a library that can re-create the function from them. Parameters will be saved from a specific library and can be loaded back into that library easily. It is also possible for libraries to read or convert from models saved from other libraries, much like how different spreadsheet programs can read each others' files.

2. What do they look like? e.g. file extensions

This varies a lot, depending on which library was used. It is not possible to make a general statement. For example, Tensorflow can save variables to a "Checkpoint" file with .ckpt extension, but can get more sophisticated depending on how much of the model you want to export, and full models with whole structure will contain more than just the variables and have the .pb extension.

3. Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??

There are a few places where you can find selections of pre-trained models. One such place is Tensorflow's Model Zoo and you might be interested in Tensorflow detection model zoo.

Other frameworks may also provide example code. For instance Caffe also has a "Model Zoo" (searching Model Zoo is a good starting strategy).

If you are working at the level of collecting model parameters and want to run these models yourself, you will need to learn a bit about each library, what language is used to work with it, and maybe follow some tutorial about how to use it. A few models will be packaged up with working scripts to use from the command line, but many are not and may take sometime and effort to get working.

When you have a specific detection target, you may be disappointed to find models that don't quite match what you want. For image classifying, it is common if you have a specialist need to take a pre-existing general model that has been trained on large dataset for weeks, and then "fine tune" it with your own image dataset for your purpose. Most NN libraries will have tutorials and examples of this fine tuning process.

Answer 2

1. What are the trained models? are they algorithms or a collection of parameters in a file?

"Model" could refer to the algorithm with or without a set of trained parameters. If you specify "trained model", the focus is on the parameters, but the algorithm is implicitly part of that, since without the algorithm, the parameters are just an arbitrary set of numbers.

2. What do they look like? e.g. file extensions

That very much depends on both the algorithm you're using and the specific implementation. A few simple examples might help clarify matters. Let's suppose that the problem we're trying to learn is the exclusive or (XOR) function:

a | b | a XOR b
--+---+---------
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0

---

First, let's use a 2-layer neural net to learn it. We'll define our activation function to be a simple step function:

$ f(x) = \begin{cases} 1 & \text{if } x > 0.5 \\ 0 & \text{if } x \le 0.5 \end{cases} $

(This is actually a terrible activation function for real neural nets since it's non-differentiable, but it makes the example clearer.)

Our model is:

$h_0 = f(1\cdot a+1\cdot b + 0)\\ h_1 = f(0.5\cdot a + 0.5\cdot b + 0)\\ \,\;y = f(1\cdot h_0 - 1\cdot h_1 + 0)$

Each step of this essentially draws a hyperplane and evaluates to 1 if the input is on one side of the hyperplane and 0 otherwise. In this particular case, h_0 tells us if either a or b is true. h_1 tells us if they're both true, and y tells us if exactly one of them is true, which is the exact definition of the XOR function.

Our parameters are the coefficients and biases (the offset added at the end of each expression):

$ \begin{bmatrix} 1 & 1 & 0 \\ 0.5 & 0.5 & 0 \\ 1 & 1 & 0 \\ \end{bmatrix}$

They can be stored in a file in any way we want; all that matters is that the code that stores them and the code that reads them back agree on the format.

---

Now let's solve the same problem using a decision tree. For these, we traverse a tree, and at every node, ask a question about the input to decide which child to visit next. Ideally, each question will divide the space of possibilities exactly in half. Once we reach a leaf node, we know our answer.

In this diagram, we visit the right child iff the expression is true.

     a+b=2
    /     \
  a+b=0    0
 /     \     
0       1

In this case, the model and parameters are harder to separate. The only part of the model that isn't learned is "It's a tree". The expressions in each interior node, the structure of the tree, and the value of the leaf nodes are all learned parameters. As with the weights from the neural network, we can store these in any format we want to.

---

Both methods are learning the same problem, and actually find basically the same solution: a XOR b = (a OR b) AND NOT (a AND B). But the nature of the mathematical model we use depends on the method we choose, the parameters depend on what we train it on, the file format depends on the code we use to do it, and the line between model and parameter is fairly arbitrary; the math works out the same regardless of how we split it up. We could even write a program that tries different methods, and outputs a program that classifies inputs using the method that performed best. In this case, the model and parameters aren't separate at all.

3. Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??

I don't know of any pretrained models that specifically recognize birds, but I'm not in image-recognition, so that doesn't mean much. If you're not averse to training your own model (using existing code), I believe the ImageNet dataset includes birds. AlexNet and LeNet would probably be good starting points for the model. Most if not all of the the state of the art image recognition models are based on convolutional networks, so you'll need a decent GPU to run them.