Questions tagged [architecture]

For questions related to the architecture of AI models, e.g. the architecture of neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
22 views

Do deep learning researchers generally visualize intermediate steps?

Many researchers in deep learning research come up with new architectures. The architectures are (just) combinations of few existing layers. Along with their mathematical intuition, in general, do ...
0
votes
1answer
36 views

Where does batch normalization layers present in a neural network?

Batch normalization is a procedure widely used to train neural networks. Mean and standard deviation are calculated in this step of training. Since we train neural network by dividing training data ...
2
votes
1answer
98 views

Can you explain me this CNN architecture?

I am starting to get my head around convolutional neural networks, and I have been working with the CIFAR-10 dataset and some research papers that used it. In one of these papers, they mention a ...
0
votes
0answers
14 views

Are there resources to learn what architectures are suited to which tasks?

I am very new to ANN, and haven't built the intuition yet. The problem I'm working on needs to process ~405 integers. The output should be a single number. Where can I find reading material about ...
1
vote
0answers
27 views

How to learn transition type in a 1-hour extended DJ Mix?

How would you design a model which learns the transitions in a given 1-hour DJ Mix? To be specific, the model should be able to learn transitions, specify the occurring time and the type (Crossfade, ...
3
votes
1answer
96 views

Which ANN structure to use?

Let $\mathcal{S}$ be the training input data set where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design a ANN so that the cost function below is minimized (the sum of square of ...
3
votes
1answer
100 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
1
vote
1answer
95 views

Should batch-normalization/dropout/activation-function layers be used after the last fully connected layer?

I am using the following architechture: 3*(fully connected -> batch normalization -> relu -> dropout) -> fully connected Should I add the ...
1
vote
2answers
47 views

Heavily mixing signal differentiation from Open Set of backgrounds via CNN

To whomever can help out, I appreciate it. I am currently attempting to detect a signal from background noise. The signal is pretty well known but the background has a lotttt of variability. I've ...
1
vote
0answers
29 views

Under what circumstances is a fully connected layer similar to PCA?

I am reading this paper on image retrieval where the goal is to train a network that produces highly discriminative descriptors (aka embeddings) for input images. If you are familiar with facial ...
0
votes
1answer
44 views

How to use a NN for seq2seq tasks?

I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and ...
0
votes
1answer
61 views

What type of ANN architecture to choose?

I have $N$ number of teachers each of which has an input feature vector ($25$ dimensional) consisting of positive numerical values for different quality of aspects (for example: lecturing ability, ...
2
votes
2answers
2k views

What is the exact output of the Inception ResNet V2's feature extraction layer?

I am working with the Inception ResNet V2 model, pre-trained with ImageNet, for face recognition. However, I'm so confused about what the exact output of the feature extraction layer (i.e. the layer ...
0
votes
0answers
31 views

How to train feedforward network to recognize images?

Context I'm trying to create network for digits recognition. All digits are the same font and size of 40x40. I know that I can use feedforward network or CNN. I'd like to use the first one. Issue I ...
2
votes
1answer
46 views

How to properly use Flatten layer?

Context I'm trying to create net that will be able to recognize printed-like digits. Something like MNIST, but only for standard printing font. Images are of the size 40x40 and I'd like to put them ...
2
votes
1answer
66 views

How can we get a differentiable neural network to count things?

Imagine I have images with apples in them. I want to train a neural network which can count the number of apples in each image. BUT, I don't want to use a detector, then count the number of bounding ...
4
votes
2answers
106 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
0
votes
0answers
12 views

Things to consider while adding custom function to generator output in GAN

I am training a GAN model (DCGAN) to generate 128x128 images. Now, I wish to add a function which will take the generator output, perform some pre-defined operations on it, and return the modified ...
3
votes
1answer
44 views

Are there deep neural networks that have inputs connected with deeper hidden layers?

Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)? If so could you give some ...
1
vote
0answers
40 views

Are there regularisation methods related only to architecture of the CNNs?

Are there any methods of regularisation of deep neural networks, particularly CNNs (or generally ANN but that will also work on CNNs) that are related only to the network's architecture and not the ...
2
votes
2answers
59 views

In classification, how does the number of classes affect the model size and amount of data needed to train?

When solving a classification problem with neural nets, be it text or images, how does the number of classes affect the model size and amount of data needed to train? Are there any soft or hard ...
4
votes
2answers
245 views

Are Neural Net architectures accidental discoveries?

Recently, I have been learning about new neural networks, which are used for specialised purposes, like speech recognition, image recognition, etc. The more I discover the more I get amazed by the ...
4
votes
1answer
76 views

What is a unified neural network model?

In many articles (for example, in the YOLO paper, this paper or this one), I see the term "unified" being used. I was wondering what the meaning of "unified" in this case is.
0
votes
0answers
94 views

Understanding Stable Baselines Custom Policies

I'm trying to understand the structure of the custom recurrent policy introduced in the documentation of the Stable Baselines: How exactly is the Lstm NN constructed? (check code below) From what I ...
1
vote
0answers
25 views

What exactly are deep learning primitives?

I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44). There doesn't seem to be a lot of articles in the ...
2
votes
3answers
126 views

Why does a neuron in a multi-layer network need several input connections?

For example, if I have the following architecture: Each neuron in the hidden layer has a connection from each one in the input layer. 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
1
vote
1answer
39 views

Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?

I'm looking at some baseline implementations of RL agents on the Pendulum environment. My guess was to use a relatively small neural net (~100 parameters). I'm comparing my solution with some ...
6
votes
0answers
88 views

How do neural network topologies affect GPU/TPU acceleration?

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
0
votes
0answers
21 views

The convolutional network architectures with enhanced invariance

It is well known, that CNN have advantage with respect to the Dense neural networks in the image classification and other pattern recognition tasks, because they have a translationall invariance built ...
6
votes
2answers
86 views

Why do very deep non resnet architectures perform worse compared to shallower ones for the same iteration? Shouldn't they just train slower?

My understanding of the vanishing gradient problem in deep networks is that as backprop progresses through the layers the gradients become small, and thus training progresses slower. I'm having a hard ...
1
vote
0answers
41 views

How can one be sure that a particular neural network architecture would work?

Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say "Price of a ticket" ...
0
votes
1answer
70 views

Is the size of a neural network directly linked with an increase in its inteligence?

Just came across this article on GPT-3, and that lead me to the question: In order to make a certain kind of neural network architecture smarter all one needs to do is to make it bigger? Also, if that ...
1
vote
0answers
44 views

Using U-NET for image semantic segmentation

I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy, but I'm stuck (and I have a terrible headache). So, I need your help. I'm going to segment MRI to find white ...
3
votes
1answer
126 views

What is the input to AlphaGo's neural network?

I have been reading an article on AlphaGo and one sentence confused me a little bit, because I'm not sure what it exactly means. The article says: AlphaGo Zero only uses the black and white stones ...
1
vote
2answers
477 views

How to teach a model-based reflex agent for doing some task using machine learning methods?

I would like to know how to teach an agent for performing prediction of the severity of disease and also for alerting patients using machine learning methods. I found the model-based reflex agent can ...
0
votes
2answers
111 views

RL: What should be the output of the NN for an agent trying to learn how to play a game?

Say the game is tic tac toe. I found two possible output layers: Vector of length 9: each float of the vector represents 1 action (one of the 9 boxes in Tic Tac Toe). The agent will play the ...
10
votes
4answers
2k views

Should neural nets be deeper the more complex the learning problem is?

I know it's not an exact science. But would you say that generally for more complicated tasks, deeper nets are required?
2
votes
0answers
32 views

Merge two different CNN models into one

I have 2 different models with each model doing a separate function and have been trained with different weights. Is there any way I can merge these two models to get a single model. If it can be ...
1
vote
1answer
51 views

Why are denser layers needed in computer vision neural nets?

Many neural net architectures for computer vision tasks use several convolutional layers and then several fully-connected (or dense) layers. While the reasons for using convolutional layers are clear ...
0
votes
1answer
94 views

What does a CNN classifier default to?

I've been messing around with an Open Set, Binary Classifier and am having trouble with it. I'm sure there are a lot of reasons for that trouble. One thing I am struggling with is, what does the ...
1
vote
0answers
19 views

Number of weights in historical to cutting edge deployment of deep networks [closed]

In cutting edge deployment of deep networks for different architectures (such as $CNN$, $QRNN$ etc) what is the historical trend of current limits of trainability possible computationally? By this I ...
1
vote
0answers
14 views

Can Grad CAM feature maps be used for Training?

I am trying to recreate the architecture of the following paper: https://arxiv.org/pdf/1807.03058.pdf Can someone help me in explaining how are the feature maps coming out of the output of GradCam ...
5
votes
1answer
706 views

How to create an AI to solve a word search?

This at first sounds ridiculous. Of course there is an easy way to write a program to solve a wordsearch. But what I would like to do is write a program that solves a word-search like a human. That ...
6
votes
1answer
203 views

Are there well-established ways of mixing different inputs (e.g. image and numbers)?

I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.?...
1
vote
1answer
105 views

How do the current input and the output of the previous time step get combined in an LSTM?

I am currently looking into LSTMs. I found this nice blog post, which is already very helpful, but still, there are things I don't understand, mostly because of the collapsed layers. The input $X_t$,...
4
votes
0answers
301 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
1
vote
0answers
32 views

Neural Network training on one example to try overfitting leads to strange predictions

tldr; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which ...
2
votes
1answer
83 views

Tweaking a CNN for large number of input channels

I am using a CNN for function approximation using geospatial data. The input of the function I am trying to approximate consists of all the spatial distances between N location on a grid and all the ...
1
vote
0answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
1
vote
0answers
28 views

AlphaZero value at root node not being affected by training

I have written my own AlphaZero implementation and started training it recently. Problem is, I am 99% sure there is a mistake and I do not know how to tackle this, since I cannot explain it. I am new ...