Questions tagged [network-design]
For questions about the design of neural networks. So, for example, when you don't know how many layers or which type of layer you need to use, you can use this tag.
19 questions
0
votes
0
answers
28
views
Hypernetwork too many parameters problem
I want to use a hypernetwork on an entire vision backbone (39m parameters).
The hypernetwork structure looks like:
512 -> 512 -> 512 -> 39m
Unfortunately, the last layer means the ...
- 155
0
votes
1
answer
98
views
Advanced / Complex Neural Network (System) Design
In addition to basic forward networks, many books cover other basic network designs like CNN's and RNN's. However, they don't really go any further than that, explaining things like common approaches ...
- 101
0
votes
1
answer
106
views
Batch Normalization as the Final Layer
In this community implementation of Inception-ResNet, Batch Normalization is used as the final layer.
This is unlikely to be a mistake, but also seems counterintuitive. The inceptionv2 paper suggests ...
- 155
0
votes
1
answer
165
views
Are there metrics for image complexity for informing neural network design?
BACKGROUND: I am trying to think of rational approaches to designing deep learning models for image classification. One thought is to quantify the complexity of image datasets and use that to inform ...
- 997
1
vote
1
answer
1k
views
How to output an integer/discrete number n with a single output neuron?
Say I have a game with 4 base actions [left, right, up, down] and then a value n, which determines how many times the chosen action is repeated.
For example, action = left, n = 3 -> go left 3 times....
- 47
0
votes
0
answers
146
views
How can Siamese Neural Networks accept a variable number of inputs?
Traditionally, Siamese Neural Networks have two inputs. With some tweaking, you can get them to accept any number of inputs. What I don't understand is how to get them to accept variable numbers of ...
- 101
0
votes
0
answers
37
views
Network design to learn multiple sequences of multiple categories
For learning a single sequence, LSTM only should suffice.
However, my situation is different here. I have a list of sequences to learn:
The sale volumes of 12 months, these are the sequences
And ...
- 1,318
0
votes
1
answer
1k
views
Number of LSTM layers needed to learn a certain number of sequences
Theoretically, number of units for a LSTM layer is the number of hidden states or the max length of sequences as per my practice.
For example, in Keras:
...
- 1,318
8
votes
1
answer
3k
views
Do all neurons in a layer have the same activation function?
I'm new to machine learning (so excuse my nomenclature), and not being a python developer, I decided to jump in at the deep (no pun intended) end writing my own framework in C++.
In my current design, ...
- 230
5
votes
2
answers
356
views
What kind of output should be used for predicting angles in DNNs?
I am building a model which predicts angles as output. What are the different kinds of outputs that can be used to predict angles?
For example,
output the angle in radians
cyclic nature of the ...
- 199
3
votes
2
answers
750
views
Which neural network should I use to approximate a specific but unknown function?
We have convolutional neural networks and recurrent neural networks for analyzing, respectively, images and sequential data.
Now, suppose I want to approximate the unknown function $f(x,y) = \sin(2\pi ...
- 51
5
votes
1
answer
4k
views
How to constraint the output value of a neural network?
I am training a deep neural network. There is a constraint on the output value of the neural network (e.g. the output has to be between 0 and 180). I think some possible solutions are using sigmoid, ...
- 51
2
votes
1
answer
1k
views
How would I go about creating a neural network that outputs a non-binary number?
I would like to create a neural network, which, given the training data (e.g. 58, 2) outputs a non-binary number (e.g 100). Perhaps I am not searching for the correct thing, but all the examples I ...
- 31
1
vote
0
answers
138
views
How would you design a neural network that generates the positions of comparators in a sorting network given a set of numbers?
How would you design a neural network that generates the positions of comparators in a sorting network given a set of numbers?
I've tried to modify some already implemented networks that given a set ...
- 119
2
votes
1
answer
689
views
How to design a neural network to predict the quadrant where a given point lies?
I've written a single perceptron that can predict whether a point is above or below a straight-line graph, given the correct training data and using a sign activation function.
Now, I'm trying to ...
- 205
28
votes
4
answers
12k
views
How could we build a neural network that is invariant to permutations of the inputs?
Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if
$$f(x_1 ... x_n) = f(\sigma(x_1 ... x_n))$$
for any permutation $\sigma$.
How ...
- 395
8
votes
2
answers
304
views
How to design a neural network that gets the author name of a piece of art as input?
I'm building a neural net to predict the value of a piece of art with a wide range of inputs (size, art medium, etc.) and I would like to include the author as an input as well (it is often a huge ...
- 81
10
votes
2
answers
1k
views
How to design a neural network when the number of inputs is variable?
I'm looking to design a neural network that can predict which runner wins in a sports game, where the number of runners varies between 2-10. In each case, specific data about the individual runners (...
- 211
5
votes
1
answer
371
views
How do I design a neural network that breaks a 5-letter word into its corresponding syllables?
I am going to design a neural network which will be able to break a 5-letter word into its corresponding syllables (hybrid syllables, I mean it will not strictly adhere to grammatical syllable rules ...
- 174