Is there some special type of deep neural networks or their architecture that would work better with only binary input data particulary for classification purpose (also binary)? I mean anything that could help with this task, like activation functions, optimizers or maybe some pre-model data transformations?
EDIT: The inupt data would count from 5 000 to 20 000 features having values of 0 or 1. Values of 1s would be very rare. For example for a row of 10 000 features I guess there would be maybe 10-30 of 1s.
Data are generally flatten 2D representations of time series where 1 is appearance of a certain manually added 'feature'. To explain further: I conduct slicing window loop on the time series data that creates 50x50 2D plot-like frames, after that I run algorithm that marks with 1s appearances of certain configurations or characteristic points of that plot in separate channels. As a result I get 5 channels with 50x50 arrays where 1s are located where a certain characteristic point appeared on such graph and then it is all flattened to the row of 12 500 columns.
I am not asking to help me solve one particular problem with one particular dataset (like stock prices, industrial measurements etc.), but rather I am trying to develop new methodology for TS analysis in general that I will test on many datasets.