Questions tagged [architecture]
For questions related to the architecture of AI models, e.g. the architecture of neural networks.
37
questions with no upvoted or accepted answers
6
votes
0
answers
101
views
How do neural network topologies affect GPU/TPU acceleration?
I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip.
If, ...
- 161
4
votes
1
answer
141
views
Which neural network can I use to solve this constrained optimisation problem?
Let $\mathcal{S}$ be the training data set, where each input $u^i \in \mathcal{S}$ has $d$ features.
I want to design an ANN so that the cost function below is minimized (the sum of the square of ...
- 159
4
votes
1
answer
348
views
What's the difference between architectures and backbones?
In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using:
Feature Pyramid Networks (as the ...
2
votes
0
answers
27
views
Is there any way to force one input have more effect on model?
Now I am working on building a deep learning model for a regression problem. I used 50 inputs and try to add one new categorical input. The problem is that this one input is much more important than ...
- 21
2
votes
0
answers
281
views
Merge two different CNN models into one
I have 2 different models with each model doing a separate function and have been trained with different weights. Is there any way I can merge these two models to get a single model.
If it can be ...
2
votes
2
answers
61
views
Heavily mixing signal differentiation from Open Set of backgrounds via CNN
I am currently attempting to detect a signal from background noise. The signal is pretty well known but the background has a lot of variability. I've since come to know this problem as Open Set ...
2
votes
2
answers
2k
views
What is the exact output of the Inception ResNet V2's feature extraction layer?
I am working with the Inception ResNet V2 model, pre-trained with ImageNet, for face recognition.
However, I'm so confused about what the exact output of the feature extraction layer (i.e. the layer ...
- 21
2
votes
0
answers
47
views
Multi-field text input for LSTM
I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
- 71
1
vote
0
answers
13
views
What would be the reason for having a different network architecture for the actor vs. value function networks in PPO?
I was reading this link , and saw some creative architectures for PPO.
I know the "No Free Lunch Theorem" and all, but what would be the logic/reasoning for why you would choose to have a ...
- 292
1
vote
0
answers
21
views
How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?
Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
- 11
1
vote
0
answers
168
views
How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?
Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2.
Formally, my question is about their computational graph:...
- 19
1
vote
0
answers
30
views
How to learn transition type in a 1-hour extended DJ Mix?
How would you design a model which learns the transitions in a given 1-hour DJ Mix? To be specific, the model should be able to learn transitions, specify the occurring time and the type (Crossfade, ...
- 11
1
vote
0
answers
32
views
Under what circumstances is a fully connected layer similar to PCA?
I am reading this paper on image retrieval where the goal is to train a network that produces highly discriminative descriptors (aka embeddings) for input images. If you are familiar with facial ...
- 1,139
1
vote
0
answers
50
views
Are there regularisation methods related only to architecture of the CNNs?
Are there any methods of regularisation of deep neural networks, particularly CNNs (or generally ANN but that will also work on CNNs) that are related only to the network's architecture and not the ...
- 1,039
1
vote
0
answers
217
views
What exactly are deep learning primitives?
I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44).
There doesn't seem to be a lot of articles in the ...
- 11
1
vote
0
answers
42
views
How can one be sure that a particular neural network architecture would work?
Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say "Price of a ticket" ...
1
vote
2
answers
135
views
Should batch-normalization/dropout/activation-function layers be used after the last fully connected layer?
I am using the following architechture:
3*(fully connected -> batch normalization -> relu -> dropout) -> fully connected
Should I add the ...
- 599
1
vote
0
answers
50
views
Using U-NET for image semantic segmentation
I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy, but I'm stuck (and I have a terrible headache). So, I need your help.
I'm going to segment MRI to find white ...
- 493
1
vote
0
answers
16
views
Can Grad CAM feature maps be used for Training?
I am trying to recreate the architecture of the following paper: https://arxiv.org/pdf/1807.03058.pdf
Can someone help me in explaining how are the feature maps coming out of the output of GradCam ...
- 11
1
vote
0
answers
32
views
Neural Network training on one example to try overfitting leads to strange predictions
tldr; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which ...
- 163
1
vote
0
answers
46
views
Is there data available about successful neural network architectures?
I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
- 243
1
vote
0
answers
31
views
AlphaZero value at root node not being affected by training
I have written my own AlphaZero implementation and started training it recently.
Problem is, I am 99% sure there is a mistake and I do not know how to tackle this, since I cannot explain it. I am new ...
- 163
1
vote
0
answers
24
views
Architecture and Use of Different Algorithms for Health Goal Feedback
I wanted to get some opinions from the community for a certain problem that I will be approaching.
The problem is to provide feedback to a user based on a image of the upper male torso. The image ...
- 11
1
vote
0
answers
97
views
Algorithms that connect neurons to previous layers as well as next
Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes ...
- 348
0
votes
0
answers
10
views
How state is combined with action in crtitic networks?
Actor-critic networks are present in deep reinforcement learning algorithms.
Actor-network takes a state as input and gives action as output.
Critic-network takes state and action as input and gives a ...
- 3,201
0
votes
0
answers
9
views
Approach for predictive model being trained while running at the same time
I have only surface level knowledge in all things AI but am thinking about tackling a specific use case with it in the future.
I would like to predict user input, specifically a resident doing stuff ...
- 1
0
votes
0
answers
20
views
Where Can I Find Resources on Extracting Meaningful Content From Web Pages?
I am in the process of conducting a literature review for my thesis. Currently, I am struggling when it comes to developing a theoretical framework/methodology or to even correctly outline an approach ...
- 1
0
votes
0
answers
18
views
Transformers for regression on permutation of fixed size sequence?
Transformers have shown remarkable performance operating on sequences, but are equivariant to the order in the input sequence. Positional Encoding alleviates that problem, but how good is it?
In my ...
- 246
0
votes
0
answers
32
views
How to learn a neural network with equivalent constraints on the weights
Let $f(x)$ be an output of a neural network with input $x$.
My data is a pair $(x,y)$ and my loss function is a function of $f(x)$ and $f(y)$, i.e., $g(f(x),f(y))$.
What kind of architecture enables ...
0
votes
0
answers
31
views
Learning a quasi-convex function that passes the origin
I am trying to learn a function $y=f(x)$ through samples $(x_i,y_i)$, where $x$ is $n$-dimensional.
We know that the function $f$ is quasi-convex and passes through the origin, i.e., $f(0)=0$.
Is ...
0
votes
0
answers
19
views
Is it possible that a deep neural network, with some variations, can be used for multiple tasks?
I am asking this question on deep neural network architectures only. If you want to restrict the domain of tasks then you can choose computer vision for this question.
Suppose there is an architecture ...
- 3,201
0
votes
0
answers
47
views
Is there any deepfake detectors with multiple deep learning models in the classifier component?
I observed that the deepfake detectors are of two types as Deep learning-based (DL-based) and machine learning-based (Non-DL methods) models.
In those DL-based deepfake detectors, the model consists ...
0
votes
0
answers
38
views
How to select dimensions of kernel and stride for pooling?
Consider a tensor of size $512 \times 512$. I need to reduce it to $32 \times 32$.
There are several ways to do it. There are a lot of possibilities. Each possibility has its own kernel dimensions and ...
- 3,201
0
votes
0
answers
45
views
How to get Attention Maps from Attention Gates in Attention UNET?
Contex
I have Attention UNET for image segmentation. I use it for humans segmentation.
Question
Everything works fine. I want to get attention maps from my network, so I could see what my UNET is ...
- 132
0
votes
0
answers
38
views
Which Neural Network Topology to choose, are Transformers suitable?
I have a regression problem and I am not quite sure which architecture to choose. I never worked with transformers before, but I generally understand how they work and I think they might be suitable. ...
0
votes
0
answers
32
views
Do deep learning researchers generally visualize intermediate steps?
Many researchers in deep learning research come up with new CNN architectures.
The architectures are (just) combinations of a few existing layers.
Along with their mathematical intuition, in general, ...
- 3,201
0
votes
1
answer
56
views
How to use a NN for seq2seq tasks?
I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and ...
- 126