I googled this, but other than some papers, I couldn't find any reverse engineering tool that was built using machine learning.

I'm not an expert in machine learning and deep learning, but it seems rational to think that considering we have billions of open source code lines out there, we can use them so our "machine" can learn how the assembly and executable of this code looks like and just study on them, mastering the art of reversing, and therefore building a tool this way that can reverse any given program with great accuracy.

Now is this doable or am I missing something here? Is there a tool built this way or on its way to coming out? What are your thoughts on this? Is a better reversing tool even needed or is there already a great reversing tool that can do the job with the best accuracy possible?

  • $\begingroup$ Can you describe precisely what you mean by 'reverse engineering' in this context? $\endgroup$ – DrMcCleod Dec 29 '18 at 10:07
  • $\begingroup$ @DrMcCleod Decompile the executable $\endgroup$ – John Pence Dec 29 '18 at 10:26
  • $\begingroup$ The idea is to restrict the results to academic papers only with the help of Google Scholar. It has found 7610 hits for “decompilation”. The question is not, if it's possible to get the sourcecode for a binary file, the question is do you prefer genetic algorithms, deeplearning, LSTM or expert systems? $\endgroup$ – Manuel Rodriguez Dec 29 '18 at 12:42

AI in general, including machine learning, promises significant improvements in reverse engineering tooling. Disassemblers are fairly dumb. If symbols are not stripped from the executable, a file formated much like the source assembly code can be created from an executable. Library files such as .a, .so, or .dll files can be restored to assembly as well. In the case of Java, byte code can be restored to virtual machine language.

Restoring higher level code in C or Java is more difficult. Higher structure in C++ or Scala create more difficulty. The problem is that more than one high level structure compiles to a single machine, byte code, or assembly level program. Compiling is close to a many to one relationship for any given compiler and optomization level. Decompiling is a one to many relationship, and the optimization only worstens the situation. The code in a pretend compiled language

    for int i = 0 to 10 by 2
      squares[i] = x[i]^2

may compile at a high level of optimization to the same code as this code in the same pretend language.

    int i = 0
    while (true)
      s = x[i]^2
      squares[i ++] = s
      if ((i ++) > 10) break

Just as AI can be used in the optimization (to find the smallest and fastest set of machine instructions to run the algorithm), AI can also be developed to assign higher level programming style to the decompilation of assembly, machine, or byte code. A few things would be needed.

  • A deep network, possibly leveraging a model that represents development and reverse engineering to integrate into network training and use
  • A set of compilers {gcc, g++, llvm, llvm++, vcc, v++, javac, scala, ...} to use to generate features (compiled programs) from labels (source code)
  • A generator of valid programs or an existing large collection of them (from GitHub, Bitbucket, or GitLab) to feed the compilers

In the case of decompiling files from which symbol information was stripped, AI may be able to assign sensible names to variables, constants, and functions from the use of the variables in the program and domain-specific vocabulary extracted from any text written about the programs.

This may be an area that is a bit beyond the leading edge of technology, but it is not infeasible.

| improve this answer | |

Currently decompilation is merely a process of producing a high-level language code from given binary (machine-, byte-code, etc.) that can complied back to the same code.

That process is more or less done for the languages that matter.

I don't think there is a pressing need to generate a better decompilation tools as I'm even not sure how that would help. Yes, you could apply ML to get more human-looking code with potentially variable names and comments present, but that's about it.

So in two words -- the current needs are fulfilled and further development in this area is not justified.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.