I have very practical problem.

When I would like to solve some "harder" problems, I usually have multiple ideas how to solve it. My problem is, that I don't know how to verify my ideas in easy/fast way.

The "suggestions" that I get from the internet is : "Test them!"

And I know, that on AI project the most important think is testing,...


If a model take 1 week to train, then it is very time consuming and expensive (specially if you do it for a client)

So, what is the best way to test idea in faster way, so that you spend long testing time just for "best" ideas,..

  • $\begingroup$ Apologies. Closed as too-broad. (There are many Machine Learning techniques with different requirements. Please consider providing more information about a specific technique you're working with.) $\endgroup$ – DukeZhou Dec 2 '19 at 19:56

Historically, academic progress can be realized with practical experiments and with theories as well. A practical experiment within the computer science is equal to write software and try it out on a real computer. Theory building is done within the Gutenberg galaxy which is sometimes called the typographic man.[1] That's a scientist, who reads and writes academic papers.

Practical experiments come to a limit, because they are too costly, take to much time and don't result into new insights. The next adorable way in making progress, is located within the Gutenberg galaxy. It has to do with going to a library, ask at the reference desk for newly published conference proceedings and the general idea is to do everything except writing sourcecode or trying things out by themself.

Critics are arguing, that alone from reading books, it's not possible to figure out something new. But books can become a powerful tool because they explain which experiments were done by others, and they can give ideas which experiments would make sense.

Verify a model vs. annotate human intelligence

The advice “read a book and go into the library” will fasten up the experiment but it's nothing completely unknown for scientists. In most cases the problem is, which book is the right one and which other techniques are available to improve the process of innovation. In case of creating AI models there are two different ways of testing available. The first one is to test a model, if it's match to the reality. This technique is slow and the alternative is to observe a human who solves a task and converts his strategy into a model. The second one (Learning from demonstration) is much faster.[2] It doesn't verify if a model is the right one, but it builds the model according to a given experiment.

Learning from demonstration is also called plan recognition and behavior annotation. It's located within the domain of psychology, because a human operator who can be observed is needed in the first place. The model is created around the operator.

[1] Wikipedia Gutenberg Galaxy, https://en.wikipedia.org/wiki/The_Gutenberg_Galaxy

[2] Scholarpedia Learning by demonstration, http://www.scholarpedia.org/article/Robot_learning_by_demonstration


Not the answer you're looking for? Browse other questions tagged or ask your own question.