Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [architecture]

For questions related to the architecture of AI models, e.g. the architecture of neural networks.

Filter by
Sorted by
Tagged with
3 votes
0 answers
51 views

How random should an untrained generative AI output really be?

I am developing a particular implementation of VAE, and, how usually one does while implementing any architecture, I passed a random input to the model to test if everything worked fine (e.g. check ...
GPU'njoyer's user avatar
0 votes
0 answers
21 views

When to know if I am "on the right track" for a CNN architecture

Context Very new to CNNs and ML in general. I am building a simple binary image segmentation network for generating black and white image masks (white pixels = desired object; black pixels = all else)....
gladshire's user avatar
0 votes
0 answers
23 views

What kind of learning architecture can I use to solve a set of nonlinear thermodynamic equations?

I am in the business of solving equations based in thermodynamics. Technically speaking, I am trying to solve for the binodals for a ternary system. The binodal is a curve in a phase diagram. ...
bad_chemist's user avatar
1 vote
2 answers
452 views

Tips and tricks when training a very large language model?

Have never trained a (very) large language model, so I am wondering if the process is the same as training a (regular) language model, i.e. you prepare the data, set up the architecture, ...
information_interchange's user avatar
0 votes
0 answers
57 views

Machine learning network design for 2D point prediction labeling

I'm looking to develop a basic neural network for labeling a detected set of 2D points. I've generated a random set of 3D points within a certain radius, all of which lie on a flat plane (Y value is 0)...
colyton's user avatar
1 vote
1 answer
29 views

Practical differences between the different RL architectures

I have tried many different RL architectures: DQN, PPO, Policy optimization, and for my specific problem they all failed in their basic setup. Eventually I discovered that my problem had too sparse/...
Erik Storm's user avatar
0 votes
0 answers
23 views

How to change network that works on a subset of the input?

To clarify my question, I'll take an example: let's say you're trying to classify pictures, by determining whether there's a dog or not in the picture. Let's assume the pictures to be 1000x1000 pixels....
WINTERSDORFF Raphael's user avatar
0 votes
1 answer
82 views

Which models can be applied recursively?

I come from a math background, so I am not up-to-date with machine learning literature. For the purpose of learning dynamics, I would like to train a model to minimize the following loss: $$\mathcal{L}...
user572780's user avatar
0 votes
1 answer
37 views

Patterns binary classification - model doesn't overfit

I am working on a very basic binary classification problem. For each set of four float numbers $(x,y,z,w)$, I want to check if they fall or not into one category. I have written a model with 3 dense ...
apt45's user avatar
  • 123
0 votes
0 answers
98 views

How many Neurons are in a single Layer Transformer?

For example, in the base model described in the original paper, each layer has the following configuration: 6 layers in the encoder 6 layers in the decoder A model size (i.e., the dimension of the ...
ofou's user avatar
  • 161
0 votes
2 answers
173 views

BYOL: Why is there a prediction network in the online network but not in the target network?

In the BYOL paper, the following architecture is presented: Why is a prediction network added to the online network, which is not present in the target network? How are the online and target compared ...
Robin van Hoorn's user avatar
0 votes
1 answer
17 views

What architecture is used for deep quadruplet network for person re-identification

I am trying to implement the paper Beyond triplet loss: a deep quadruplet network for person re-identification. In the paper, they provide a figure (attached below) containing the network architecture,...
Varghese Kuruvilla's user avatar
1 vote
1 answer
68 views

Does Number of Fully connected neural networks changes in transformer architechture based on max length input size?

Considering the architecture of encoder and decoder in transformer as shown below: Does each input token after self attention mechanism (z1,z2,z3,...)is passed to it's specific separate Feed forward ...
Arjun Reddy's user avatar
1 vote
1 answer
71 views

Transfer Learning for Solar Energy Production Forecasting with LSTM: Generalized vs. Specialized Models

I am working on a solar energy production forecasting problem using LSTM multi-step models to predict 1/4/8h ahead of solar energy production for different solar installations. Our goal is to help ...
Guilherme Vieira's user avatar
0 votes
1 answer
102 views

Do different architectures really make a difference or is it just a matter of the training process?

I was wondering which influence different architectures for deep learning truly have on the performance. Of course, substantial changes in the paradigms we use when building neural networks (such as ...
convaldo's user avatar
  • 121
2 votes
0 answers
28 views

How can I learn about NN architecture?

I have a pretty good understanding of individual neural net layers (fully connected, convolution, pooling, activation, etc) but struggle to construct combinations of them to solve a given problem. I ...
cmauck10's user avatar
0 votes
0 answers
47 views

Is ANN architecture mesh topology exist?

I'm just wondering if there's ANN architecture that looks like mesh topology at context of computer networking. If exist or possible, is layer notion still applied?
Muhammad Ikhwan Perwira's user avatar
2 votes
1 answer
62 views

Propagating gradients through an "Item Selector" network

Consider the following problem: There are $N$ items and $S$ slots. Each item is a vector of length $D$. The goal is to train a neural network to select one item per slot in order to minimize the loss ...
Daniel's user avatar
  • 191
1 vote
1 answer
178 views

What would be the reason for having a different network architecture for the actor vs. value function networks in PPO?

I was reading this link , and saw some creative architectures for PPO. I know the "No Free Lunch Theorem" and all, but what would be the logic/reasoning for why you would choose to have a ...
Vladimir Belik's user avatar
1 vote
0 answers
27 views

How Can We Create Neural Networks with Different Depths and Widths But Same Number of Parameters?

Right now I am doing a research project investigating how the depth of a Neural Network affects its capacity to learn. In order to do this, I wanted to test different Networks with the same number of ...
Vincent 's user avatar
3 votes
1 answer
700 views

How do I design the network for Deep Q-Network?

I am playing with deep q-learning and I am thinking about what the proper architecture of a network used for deep q-learning is. I have a very simple environment, basically a 18x18 matrix, where 3 ...
Joysn's user avatar
  • 133
2 votes
0 answers
1k views

How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?

Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2. Formally, my question is about their computational graph:...
keyboardAnt's user avatar
0 votes
0 answers
58 views

How to learn a neural network with equivalent constraints on the weights

Let $f(x)$ be an output of a neural network with input $x$. My data is a pair $(x,y)$ and my loss function is a function of $f(x)$ and $f(y)$, i.e., $g(f(x),f(y))$. What kind of architecture enables ...
user15988's user avatar
0 votes
0 answers
22 views

Is it possible that a deep neural network, with some variations, can be used for multiple tasks?

I am asking this question on deep neural network architectures only. If you want to restrict the domain of tasks then you can choose computer vision for this question. Suppose there is an architecture ...
hanugm's user avatar
  • 3,890
2 votes
0 answers
33 views

Is there any way to force one input have more effect on model?

Now I am working on building a deep learning model for a regression problem. I used 50 inputs and try to add one new categorical input. The problem is that this one input is much more important than ...
taegyun kim's user avatar
0 votes
1 answer
338 views

Are the output dimensions of the first and second convolutional layer in YOLO paper correct?

I was reading the last version of the YOLO paper available in Arxiv, and I don't fully understand the output dimensions (I understand width and height, but not depth) of the first and second ...
ldemaeztu's user avatar
5 votes
2 answers
3k views

Why do Transformers have a sequence limit at inference time?

As far as I understand, Transformer's time complexity increases quadratically with respect to the sequence length. As a result, during training to make training feasible, a maximum sequence limit is ...
chessprogrammer's user avatar
2 votes
1 answer
347 views

CNN Architectures for local features vs global context

Kaparthy in his blog post said [this] hints at the kinds of architectures we’ll eventually explore. As an example - are very local features enough or do we need global context? I'd like to gain ...
Tom Huntington's user avatar
0 votes
1 answer
216 views

What is the difference between a vision transformer and image-based relational learning?

I am trying to figure out the difference between the architecture used in this and this paper. It looks like both used multi-headed self-attention and therefore should be the same in principle.
desert_ranger's user avatar
0 votes
0 answers
36 views

Do deep learning researchers generally visualize intermediate steps?

Many researchers in deep learning research come up with new CNN architectures. The architectures are (just) combinations of a few existing layers. Along with their mathematical intuition, in general, ...
hanugm's user avatar
  • 3,890
1 vote
1 answer
265 views

Where does batch normalization layers present in a neural network?

Batch normalization is a procedure widely used to train neural networks. Mean and standard deviation are calculated in this step of training. Since we train a neural network by dividing training data ...
hanugm's user avatar
  • 3,890
1 vote
0 answers
35 views

How to learn transition type in a 1-hour extended DJ Mix?

How would you design a model which learns the transitions in a given 1-hour DJ Mix? To be specific, the model should be able to learn transitions, specify the occurring time and the type (Crossfade, ...
Cactuser's user avatar
1 vote
0 answers
43 views

Under what circumstances is a fully connected layer similar to PCA?

I am reading this paper on image retrieval where the goal is to train a network that produces highly discriminative descriptors (aka embeddings) for input images. If you are familiar with facial ...
Alexander Soare's user avatar
0 votes
1 answer
126 views

What type of ANN architecture to choose?

I have $N$ number of teachers each of which has an input feature vector ($25$ dimensional) consisting of positive numerical values for different quality of aspects (for example: lecturing ability, ...
user3489173's user avatar
2 votes
1 answer
140 views

How to properly use Flatten layer?

Context I'm trying to create net that will be able to recognize printed-like digits. Something like MNIST, but only for standard printing font. Images are of the size 40x40 and I'd like to put them ...
MASTER OF CODE's user avatar
4 votes
2 answers
226 views

Which neural network can I use to solve this constrained optimisation problem?

Let $\mathcal{S}$ be the training data set, where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design an ANN so that the cost function below is minimized (the sum of the square of ...
user3489173's user avatar
2 votes
1 answer
891 views

How can we get a differentiable neural network to count things?

Imagine I have images with apples in them. I want to train a neural network which can count the number of apples in each image. BUT, I don't want to use a detector, then count the number of bounding ...
Alexander Soare's user avatar
1 vote
0 answers
60 views

Are there regularisation methods related only to architecture of the CNNs?

Are there any methods of regularisation of deep neural networks, particularly CNNs (or generally ANN but that will also work on CNNs) that are related only to the network's architecture and not the ...
GKozinski's user avatar
  • 1,260
2 votes
2 answers
3k views

In classification, how does the number of classes affect the model size and amount of data needed to train?

When solving a classification problem with neural nets, be it text or images, how does the number of classes affect the model size and amount of data needed to train? Are there any soft or hard ...
conscious_process's user avatar
4 votes
1 answer
494 views

What is a unified neural network model?

In many articles (for example, in the YOLO paper, this paper or this one), I see the term "unified" being used. I was wondering what the meaning of "unified" in this case is.
Reactionic's user avatar
5 votes
2 answers
951 views

What's the difference between architectures and backbones?

In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using: Feature Pyramid Networks (as the ...
codinggirl123's user avatar
2 votes
0 answers
898 views

What exactly are deep learning primitives?

I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44). There doesn't seem to be a lot of articles in the ...
Frank's user avatar
  • 21
3 votes
1 answer
74 views

Are there deep neural networks that have inputs connected with deeper hidden layers?

Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)? If so could you give some ...
GKozinski's user avatar
  • 1,260
0 votes
1 answer
86 views

How to use a NN for seq2seq tasks?

I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and ...
neel g's user avatar
  • 146
2 votes
3 answers
217 views

Why does a neuron in a multi-layer network need several input connections?

For example, if I have the following architecture: Each neuron in the hidden layer has a connection from each one in the input layer. 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
iwab's user avatar
  • 121
1 vote
0 answers
45 views

How can one be sure that a particular neural network architecture would work?

Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say "Price of a ticket" ...
Naveen Reddy Marthala's user avatar
1 vote
1 answer
44 views

Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?

I'm looking at some baseline implementations of RL agents on the Pendulum environment. My guess was to use a relatively small neural net (~100 parameters). I'm comparing my solution with some ...
Kris's user avatar
  • 171
0 votes
1 answer
91 views

Is the size of a neural network directly linked with an increase in its inteligence?

Just came across this article on GPT-3, and that lead me to the question: In order to make a certain kind of neural network architecture smarter all one needs to do is to make it bigger? Also, if that ...
Gonçalo Peres's user avatar
2 votes
0 answers
452 views

Merge two different CNN models into one

I have 2 different models with each model doing a separate function and have been trained with different weights. Is there any way I can merge these two models to get a single model. If it can be ...
Nautatava Navlakha's user avatar
2 votes
1 answer
182 views

Can you explain me this CNN architecture?

I am starting to get my head around convolutional neural networks, and I have been working with the CIFAR-10 dataset and some research papers that used it. In one of these papers, they mention a ...
SanMu's user avatar
  • 141