2
$\begingroup$

I am having a difficult time explaining to my boss that what he is trying to achieve may not be possible or within reason. We have a database of 3 Million data points per computer across hundreds of machines and when any data point is updated, changed, or removed. Some of these data points are the number of times a computer has been logged in, the names of printers attach, folders on the root of the drive. Some of the data points we do care about, others like a printer being out of ink, we don't care about but the same method would return if the printer was offline which we do care about.

He wants to design an AI that would check these data points and return with true or false on whether the data point is significant when they are changed, removed or updated. We are storing the name of the method to retrieve the data, the current data, all previous data, and the time the change was made. I can not foresee a way to train the data efficiently as we currently don't know which methods retrieve significant data or which values are not significant within the method.

Is it possible to design such an AI without hours of supervisor learning?

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes... But... you have a lot of hours.. possibly days or weeks.. of work before you are at that point.

Your bigger problem is apparent from your concerns in the second paragraph. It doesn't sound as though your org has a solid grasp of the problem at the moment. For that reason, it seems that some first steps are in order.

Data Exploration

Begin by collecting all of the data points. I'd recommend that you perhaps begin with some statistical analysis of the data as a whole, including some basic visualization and generating a covariance matrix. From there, begin to look at using some clustering methods to identify possible patterns. Along the way, you will almost certainly go down the path of some dimensionality reduction, either via something like PCA or possibly identifying useless features.

Feature Selection

Based on the exploration work above you should now have a much better understanding of your data and relationships within it. Based on this, it's time to start thinking about how to generate a model that produces the desired output. Frankly, you may discover that something as simple as a random forest classification or even a clustering method such as DBSCAN can be used to initially train and then continuously fit your data, producing either a binary classification with the random forest or a yes/no cluster with the clustering technique.

Is More Required?

Of course, something more sophisticated might be required, but if it is you would know have a better handle on the problem and likely be able to intuitively generate a large dataset that could be used for a neural network.

Oh... And as a concluding thought... It might turn out that after all of this analysis the problem cannot be solved with the data that you have. At that point you have to go back and see if there are other data points that could be gathered.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .