6
$\begingroup$

If I replicated the results of a machine learning paper, is there a website that I can publish my code to that collects informal replication attempts?

https://paperswithcode.com/ allows anyone to publish implementations, but I am looking for something similar that allows anyone to publish implementations along with replicated experimental results. Does such a website exist?

$\endgroup$
  • $\begingroup$ Why isn't a code-hosting service, like Github, enough for your purposes? $\endgroup$ – nbro May 14 at 19:36
  • $\begingroup$ The effort of replicating a paper would be somewhat wasted if the results were not easily discoverable. A database for linking papers to replications might be better for this than Github. $\endgroup$ – Jared Jacobsen May 18 at 15:28
  • $\begingroup$ So, you mean something more sophisticated than Github that keeps also track of datasets? $\endgroup$ – nbro May 18 at 15:31
  • $\begingroup$ Right, and which makes it easy for others to discover your replication and to integrate it with the results of other replication attempts. $\endgroup$ – Jared Jacobsen May 18 at 16:37
1
$\begingroup$

I think you're looking for something like Comet.ml or IDSIA's sacred, but bear in mind that these tools can also be combined with code-hosting services such as Github.

| improve this answer | |
$\endgroup$
1
$\begingroup$

OpenReview (https://openreview.net/) may be what you're looking for. It's both a platform for ML conferences to organize their reviewing process as well as an arXiv-like preprint repository.

One major machine learning conference (ICLR) has been holding "reproducibility challenges," in which participants can submit replication reports of existing papers, but that probably only applies to papers submitted to that conference.

There is a general preprints section, which might be your best bet.

Disclaimer: I used to be on the OpenReview development team

| improve this answer | |
$\endgroup$
  • $\begingroup$ "I am looking for something similar that allows anyone to publish implementations along with replicated experimental results.". I am not sure how OpenReview provides this service of sharing code, so I am not sure why you think OR would be suited for the OP's needs. $\endgroup$ – nbro May 15 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.