2
votes
$\begingroup$

During weeks and months of your work, many things may change, for example :

  • You may modify the loss function
  • Your training or validation datasets may change
  • You modify data augmentation

Which tools or processes do you use to track modifications you have made and how did they affected the model ?

$\endgroup$
0

2 Answers 2

Reset to default
3
votes
$\begingroup$

Maybe you are looking for a combination of a version control system (like git and Github) and a tool like comet.ml. In the past, I used comet.ml to keep track of different experiments performed with different hyper-parameters or different versions of the code. There are other alternatives to comet.ml, such as sacred, but they may also have different features and may not be as visually pleasing as comet.ml or even free. Personally, I liked comet.ml (even though, at the time, it still lacked some features). In any case, a VCS, like git, is widely used in software development (not just in AI projects) to keep track of different versions of the code, etc. You may also be interested in continuous integration (e.g. Travis CI) and code review (e.g. codacy) tools.

$\endgroup$
0
1
vote
$\begingroup$

Full disclosure: I work at Dessa, the company that developed this tech.

We built a machine learning experiment management tool, called Atlas. The main feature is experiment management, allowing you to run thousands of experiments concurrently. This might help with your problem above https://github.com/dessa-oss

$\endgroup$
1
  • $\begingroup$ Hi and welcome to AI SE! Thanks for sharing this info! Atlas looks promising and similar to comet.ml. What about prices? Is it free to use? Maybe you should edit your answer to describe these details. $\endgroup$
    – nbro
    Apr 1, 2020 at 20:52

Not the answer you're looking for? Browse other questions tagged .