3
$\begingroup$

I'm trying to develop a kind of AI that will assist in debugging a large software system while we run test cases on it. The AI can get access to anything that the developers of the system can, namely log files, and execution data from trace points. It also has access to the structure of the system, and all of the source code.

The end goal of this AI is to be able to detect runtime errors during execution, and locate the source of these errors.

I was considering making use of a deep neural network, where the input would be the execution data and log output. Using this input it would be able to verify whether the current version of the system we are running is functional, or non-functional. The problems with this approach is that the system it would be evaluating would be constantly changing as it gets developed, so the only training material the NN would have is from the last stable version of the system (and even that could have some errors). Additionally, producing test cases for the system off of which we could train the NN would be very time consuming, and would defeat the purpose of using the NN in the first place.

I would like to know what AI design you think would be suitable for this task. Please let me know if you would like any other information relevant to the problem. As far as I can tell, nothing quite like this has been done before.

It's probably worth mentioning that my team has a some extremely powerful machines on which we can run the AI.

Improve this question
$\endgroup$
5
  • $\begingroup$ I dunno neural networks very well. For what you are trying to achieve, I guess you need to first freeze the state and do the execution. If your task is telling if the system is functional or non functional, it can be thought of as a classification task. You can start off looking at using DNN for classification. I know this is a broad overview, however, should give you some lead. $\endgroup$
    – okkhoy
    Commented May 11, 2017 at 10:30
  • $\begingroup$ Could you please try to re-phrase your question.ideally,we don't have types of AIs but specific fields where your concept idea could fall given the problem is solved ie machine learning,In this case a piece program once coded effectively, can learn from previous errors or data logs and keeps on advancing thus helping you at the end;debugging your code.You need to be kind specific.However,CybergrandChallenge one team did the same program and i think it's being worked on there! Ain't asleep!!. $\endgroup$
    – quintumnia
    Commented May 11, 2017 at 11:06
  • 1
    $\begingroup$ Let me comment on assist in debugging a large software system : I've worked on a subclass of that problem with great success using linear regression on code features. Rephrasing the question, or changing the requirements slightly could make it a lot easier. For example, if your specific objective was assign a probability to each function that a defect will be discovered in our next round of regression tests and you had data of which functions had defects from the last regression, then that is very doable! $\endgroup$
    – Anton Panchishin
    Commented May 11, 2017 at 15:31
  • 1
    $\begingroup$ you are not clear on what is it exactly that you want, you want an AI that can run your program for different inputs and log errors if they exists? $\endgroup$
    – Moher
    Commented Sep 17, 2018 at 18:38
  • $\begingroup$ Is the programming language you are using Turing Complete? $\endgroup$
    – MilkyWay90
    Commented Jun 5, 2019 at 16:44

4 Answers 4

Reset to default
1
$\begingroup$

I think you would find this link helpful. It demonstrates how to identify patterns in large arbitrary byte data.

https://devblogs.nvidia.com/malware-detection-neural-networks/

Improve this answer
$\endgroup$
2
$\begingroup$

You are trying to solve a variant of The Halting Problem, which is the problem of detecting whether a computer program is going to stop, or run forever.

The Halting Problem is incomputable, which means it is not possible to write a computer program that solves it. It is easy to see that your problem is also incomputable. If you could predict whether a program would generate an error, then for any program X that someone wants to solve the halting problem for, we could write a new program:

  1. run(X)
  2. Error("X finished running").

and use your algorithm to determine whether X would finish running or not. Since the halting problem is known to be incomputable, this means your problem must be incomputable too.

That's not to say all is lost though. Formal verification is a field that uses some AI techniques (mostly reasoning-based, but I think there's some machine learning now too) to try to solve this problem for some programs. It can't work for every program though.

Improve this answer
$\endgroup$
0
$\begingroup$

Debug and Validation of large software systems like video software stack If you take example as validation and debug of video software stack, It is very difficult for naked eye to identify failures on the display. In this case you can use DNN based image classifier to identify functional failure.

Improve this answer
$\endgroup$
0
$\begingroup$

If you consider also 3rd party options instead of building your own system, you may want to check this solution https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Anomaly_Detection.html

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .