Questions tagged [architecture]

For questions related to the architecture of AI models, e.g. the architecture of neural networks.

37 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
0 answers
101 views

How do neural network topologies affect GPU/TPU acceleration?

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
user avatar
4 votes
1 answer
141 views

Which neural network can I use to solve this constrained optimisation problem?

Let $\mathcal{S}$ be the training data set, where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design an ANN so that the cost function below is minimized (the sum of the square of ...
user avatar
4 votes
1 answer
348 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
user avatar
2 votes
0 answers
27 views

Is there any way to force one input have more effect on model?

Now I am working on building a deep learning model for a regression problem. I used 50 inputs and try to add one new categorical input. The problem is that this one input is much more important than ...
user avatar
2 votes
0 answers
281 views

Merge two different CNN models into one

I have 2 different models with each model doing a separate function and have been trained with different weights. Is there any way I can merge these two models to get a single model. If it can be ...
user avatar
2 votes
2 answers
61 views

Heavily mixing signal differentiation from Open Set of backgrounds via CNN

I am currently attempting to detect a signal from background noise. The signal is pretty well known but the background has a lot of variability. I've since come to know this problem as Open Set ...
user avatar
2 votes
2 answers
2k views

What is the exact output of the Inception ResNet V2's feature extraction layer?

I am working with the Inception ResNet V2 model, pre-trained with ImageNet, for face recognition. However, I'm so confused about what the exact output of the feature extraction layer (i.e. the layer ...
user avatar
  • 21
2 votes
0 answers
47 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
user avatar
1 vote
0 answers
13 views

What would be the reason for having a different network architecture for the actor vs. value function networks in PPO?

I was reading this link , and saw some creative architectures for PPO. I know the "No Free Lunch Theorem" and all, but what would be the logic/reasoning for why you would choose to have a ...
user avatar
1 vote
0 answers
21 views

How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?

Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
user avatar
1 vote
0 answers
168 views

How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?

Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2. Formally, my question is about their computational graph:...
user avatar
1 vote
0 answers
30 views

How to learn transition type in a 1-hour extended DJ Mix?

How would you design a model which learns the transitions in a given 1-hour DJ Mix? To be specific, the model should be able to learn transitions, specify the occurring time and the type (Crossfade, ...
user avatar
1 vote
0 answers
32 views

Under what circumstances is a fully connected layer similar to PCA?

I am reading this paper on image retrieval where the goal is to train a network that produces highly discriminative descriptors (aka embeddings) for input images. If you are familiar with facial ...
user avatar
1 vote
0 answers
50 views

Are there regularisation methods related only to architecture of the CNNs?

Are there any methods of regularisation of deep neural networks, particularly CNNs (or generally ANN but that will also work on CNNs) that are related only to the network's architecture and not the ...
user avatar
  • 1,039
1 vote
0 answers
217 views

What exactly are deep learning primitives?

I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44). There doesn't seem to be a lot of articles in the ...
user avatar
  • 11
1 vote
0 answers
42 views

How can one be sure that a particular neural network architecture would work?

Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say "Price of a ticket" ...
user avatar
1 vote
2 answers
135 views

Should batch-normalization/dropout/activation-function layers be used after the last fully connected layer?

I am using the following architechture: 3*(fully connected -> batch normalization -> relu -> dropout) -> fully connected Should I add the ...
user avatar
1 vote
0 answers
50 views

Using U-NET for image semantic segmentation

I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy, but I'm stuck (and I have a terrible headache). So, I need your help. I'm going to segment MRI to find white ...
user avatar
1 vote
0 answers
16 views

Can Grad CAM feature maps be used for Training?

I am trying to recreate the architecture of the following paper: https://arxiv.org/pdf/1807.03058.pdf Can someone help me in explaining how are the feature maps coming out of the output of GradCam ...
user avatar
  • 11
1 vote
0 answers
32 views

Neural Network training on one example to try overfitting leads to strange predictions

tldr; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which ...
user avatar
  • 163
1 vote
0 answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
user avatar
  • 243
1 vote
0 answers
31 views

AlphaZero value at root node not being affected by training

I have written my own AlphaZero implementation and started training it recently. Problem is, I am 99% sure there is a mistake and I do not know how to tackle this, since I cannot explain it. I am new ...
user avatar
  • 163
1 vote
0 answers
24 views

Architecture and Use of Different Algorithms for Health Goal Feedback

I wanted to get some opinions from the community for a certain problem that I will be approaching. The problem is to provide feedback to a user based on a image of the upper male torso. The image ...
user avatar
  • 11
1 vote
0 answers
97 views

Algorithms that connect neurons to previous layers as well as next

Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes ...
user avatar
  • 348
0 votes
0 answers
10 views

How state is combined with action in crtitic networks?

Actor-critic networks are present in deep reinforcement learning algorithms. Actor-network takes a state as input and gives action as output. Critic-network takes state and action as input and gives a ...
user avatar
  • 3,201
0 votes
0 answers
9 views

Approach for predictive model being trained while running at the same time

I have only surface level knowledge in all things AI but am thinking about tackling a specific use case with it in the future. I would like to predict user input, specifically a resident doing stuff ...
user avatar
  • 1
0 votes
0 answers
20 views

Where Can I Find Resources on Extracting Meaningful Content From Web Pages?

I am in the process of conducting a literature review for my thesis. Currently, I am struggling when it comes to developing a theoretical framework/methodology or to even correctly outline an approach ...
user avatar
0 votes
0 answers
18 views

Transformers for regression on permutation of fixed size sequence?

Transformers have shown remarkable performance operating on sequences, but are equivariant to the order in the input sequence. Positional Encoding alleviates that problem, but how good is it? In my ...
user avatar
  • 246
0 votes
0 answers
32 views

How to learn a neural network with equivalent constraints on the weights

Let $f(x)$ be an output of a neural network with input $x$. My data is a pair $(x,y)$ and my loss function is a function of $f(x)$ and $f(y)$, i.e., $g(f(x),f(y))$. What kind of architecture enables ...
user avatar
0 votes
0 answers
31 views

Learning a quasi-convex function that passes the origin

I am trying to learn a function $y=f(x)$ through samples $(x_i,y_i)$, where $x$ is $n$-dimensional. We know that the function $f$ is quasi-convex and passes through the origin, i.e., $f(0)=0$. Is ...
user avatar
0 votes
0 answers
19 views

Is it possible that a deep neural network, with some variations, can be used for multiple tasks?

I am asking this question on deep neural network architectures only. If you want to restrict the domain of tasks then you can choose computer vision for this question. Suppose there is an architecture ...
user avatar
  • 3,201
0 votes
0 answers
47 views

Is there any deepfake detectors with multiple deep learning models in the classifier component?

I observed that the deepfake detectors are of two types as Deep learning-based (DL-based) and machine learning-based (Non-DL methods) models. In those DL-based deepfake detectors, the model consists ...
user avatar
0 votes
0 answers
38 views

How to select dimensions of kernel and stride for pooling?

Consider a tensor of size $512 \times 512$. I need to reduce it to $32 \times 32$. There are several ways to do it. There are a lot of possibilities. Each possibility has its own kernel dimensions and ...
user avatar
  • 3,201
0 votes
0 answers
45 views

How to get Attention Maps from Attention Gates in Attention UNET?

Contex I have Attention UNET for image segmentation. I use it for humans segmentation. Question Everything works fine. I want to get attention maps from my network, so I could see what my UNET is ...
user avatar
0 votes
0 answers
38 views

Which Neural Network Topology to choose, are Transformers suitable?

I have a regression problem and I am not quite sure which architecture to choose. I never worked with transformers before, but I generally understand how they work and I think they might be suitable. ...
user avatar
0 votes
0 answers
32 views

Do deep learning researchers generally visualize intermediate steps?

Many researchers in deep learning research come up with new CNN architectures. The architectures are (just) combinations of a few existing layers. Along with their mathematical intuition, in general, ...
user avatar
  • 3,201
0 votes
1 answer
56 views

How to use a NN for seq2seq tasks?

I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and ...
user avatar
  • 126