Recently my friend asked me a question: having two input matrices X and Y (each size NxD) where D >> N, and ground truth matrix Z of size DxD, what deep architecture shall I use to learn a deep model of this representation?

  • N ~ is in the order of tens
  • D ~ is in the order of tens of thousands

The problem is located in the domain of bioinformatics, however, this is more of an architectural problem. All matrices contain floats.

I tried first a simple model based on a CNN model in keras. I've stacked input X and Y into an Input Matrix of size (number of training examples, N, D, 2). Outputs are of size (number of training examples, D, D, 1)

  1. Conv2D layer
    • leaky ReLU
  2. Conv2D layer
    • leaky ReLU
  3. Dropout
  4. Flattening layer
  5. Dense (fully connected) of size D
    • leaky ReLU
    • droout
  6. Dense (fully connected) of size D**2 (D squared)
    • leaky ReLU
    • droout
  7. Reshaping output into (D,D,1) (for single training set)

However, this model is untrainable. It has over billion parameters for emulated data.

(Exactly 1,321,005,944 for my randomly emulated dataset)

Do you find this problem solvable? What other architectures I might try to solve this problem?


  • 1
    $\begingroup$ Is there any relation between X and Y through Z? Perhaps X1*Z = Y1? $\endgroup$ Feb 26 '18 at 23:06
  • 1
    $\begingroup$ There is no known relationship between X and Y. We only know that the output matrix Z is symmetrical, therefore we could predict the "upper/lower triangle" of it. $\endgroup$
    – vaxherra
    Feb 27 '18 at 17:36
  • 1
    $\begingroup$ Is there any other relationships between the matrices? If Z isnt a transformation is it part of a decomposition? Although the relationship between X and Y isnt know, is the goal to find this relationship? What should Z be representing? $\endgroup$ Feb 27 '18 at 18:54
  • 1
    $\begingroup$ Just to be clear, you have an output (and ground truth) of at least 10,000 x 10,000, or 100 million elements, and you would like to generate it to match an observed distribution over 20 x 10,000, or 200 thousand elements? How many training examples do you have? $\endgroup$ Feb 28 '18 at 7:49
  • 1
    $\begingroup$ Are there any simplifying assumptions that are reasonable to make in the problem domain? The big ones that may help would be sparsity (only some small percentage of inputs and outputs are populated, or different from some mean distribution), or smoothness (the 10,000 x 10,000 output could be approximated reasonably by e.g. a 1,000 x 1,000 output, and maybe has similar properties to a real-world image) $\endgroup$ Feb 28 '18 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.