2
$\begingroup$

I am a newbie in deep learning and wanted to know if the problem I have at hand is a suitable fit for deep learning algorithms. I have thousands of fragments each of about 1000 bytes size (i.e. numbers in the range of 0 to 255). There are two classes in the fragments:

  1. Some fragments have a high frequency of two particular byte values appearing next to one another: "0 and 100". This kind of pattern roughly appears once every 100 to 200 bytes.
  2. In the other class, the byte values are more randomly distributed.

We have the ability to produce as many numbers of instances of each class as needed for training purposes. However, I would like to differentiate with a machine learning algorithm without explicitly identifying the "0 and 100" pattern in the 1st class myself. Can deep learning help us solve this? If so, what kind of layers might be useful?

As a preliminary experiment, we tried to train a deep learning network made up of 2 hidden layers of TensorFlow's "Dense" layers (of size 512 and 256 nodes in each of the hidden layers). However, unfortunately, our accuracy was indicative of simply a random guess (i.e. 50% accuracy). We were wondering why the results were so bad. Do you think a Convolutional Neural Network will better solve this problem?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Your network is essentially memorizing data but not extracting features. You need to apply CNN.

That said the CNN architecture will need to be kind of unusual, each bit will need to be turned into representing a positional element.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Can you please clarify what you mean by "network is essentially memorizing data" -> we are currently getting a very low training error but test performance is bad. This result basically seems to imply what you said above. However, can you please explain why this is happening with the "dense" layers? Any pointers to outside resources on this phenomenon will also be greatly appreciated. $\endgroup$
    – Phani
    Commented Apr 20, 2020 at 20:22
  • $\begingroup$ just look up CNN or time series analysis. $\endgroup$
    – FourierFlux
    Commented Apr 20, 2020 at 20:26
1
$\begingroup$

I was writing a comment, but I'll write an answer instead. With a dense network your model has too many parameters an will overfit to the data. Even if it learns that "0 and 100 tend to appear together" at the input's positions 34 and 35, it wouldn't automatically learn that this pattern is common also an all other positions. CNNs don't have this problem.

I also suggest that you change the network's input, if it is now just a series of integers 0 - 255. I would either one-hot encode them, or pass the values through an embedding layer. Typically CNNs are used for image or sound signals where the values have a natural order, but data file bytes don't have such structure.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .