What's the deal with Deno? We talk with a major contributor to find out. Listen now.

Questions tagged [architecture]

For questions related to the architecture of AI models, e.g. the architecture of neural networks.

Filter by
Sorted by
Tagged with
1
vote
1answer
43 views

Should batch-normalization/dropout/activation-function layers be used after the last fully connected layer?

I am using the following architechture: 3*(fully connected -> batch normalization -> relu -> dropout) -> fully connected Should I add the ...
1
vote
2answers
35 views

Heavily mixing signal differentiation from Open Set of backgrounds via CNN

To whomever can help out, I appreciate it. I am currently attempting to detect a signal from background noise. The signal is pretty well known but the background has a lotttt of variability. I've ...
0
votes
1answer
25 views

How to use a NN for seq2seq tasks?

I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and ...
1
vote
3answers
111 views

Why does a neuron in a multi-layer network need several input connections?

For example, if I have the following architecture: Each neuron in the hidden layer has a connection from each one in the input layer. 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
1
vote
1answer
38 views

Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?

I'm looking at some baseline implementations of RL agents on the Pendulum environment. My guess was to use a relatively small neural net (~100 parameters). I'm comparing my solution with some ...
5
votes
0answers
75 views

How do neural network topologies affect GPU/TPU acceleration?

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
3
votes
1answer
63 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
0
votes
0answers
18 views

The convolutional network architectures with enhanced invariance

It is well known, that CNN have advantage with respect to the Dense neural networks in the image classification and other pattern recognition tasks, because they have a translationall invariance built ...
6
votes
2answers
84 views

Why do very deep non resnet architectures perform worse compared to shallower ones for the same iteration? Shouldn't they just train slower?

My understanding of the vanishing gradient problem in deep networks is that as backprop progresses through the layers the gradients become small, and thus training progresses slower. I'm having a hard ...
2
votes
1answer
67 views

Can you explain me this CNN architecture?

I am starting to get my head around convolutional neural networks, and I have been working with the CIFAR-10 dataset and some research papers that used it. In one of these papers, they mention a ...
1
vote
0answers
36 views

How can one be sure that a particular neural network architecture would work?

Traditionally, when working with tabular data, one can be sure(or at least know) that a model works because the included features could explain a target variable, say "Price of a ticket" ...
0
votes
1answer
62 views

Is the size of a neural network directly linked with an increase in its inteligence?

Just came across this article on GPT-3, and that lead me to the question: In order to make a certain kind of neural network architecture smarter all one needs to do is to make it bigger? Also, if that ...
0
votes
0answers
49 views

What are the differences between backbones, frontends, models and architectures in applied deep learning?

Context I'm trying to dive into deep learning for tasks on images, and trying to figure out how to reuse some well-known structures* that have been published, mainly on github. ( *Here, structure can ...
1
vote
0answers
29 views

Using U-NET for image semantic segmentation

I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy, but I'm stuck (and I have a terrible headache). So, I need your help. I'm going to segment MRI to find white ...
0
votes
0answers
24 views

Neural Network architecture for going from scalar input to time series outputs?

I have a problem where I know p features which are each scalar values and the output of 1 set of those features is a time history. Is there a specific neural network (NN) type architecture that can ...
3
votes
1answer
74 views

What is the input to AlphaGo's neural network?

I have been reading an article on AlphaGo and one sentence confused me a little bit, because I'm not sure what it exactly means. The article says: AlphaGo Zero only uses the black and white stones ...
1
vote
2answers
382 views

How to teach a model-based reflex agent for doing some task using machine learning methods?

I would like to know how to teach an agent for performing prediction of the severity of disease and also for alerting patients using machine learning methods. I found the model-based reflex agent can ...
0
votes
2answers
71 views

RL: What should be the output of the NN for an agent trying to learn how to play a game?

Say the game is tic tac toe. I found two possible output layers: Vector of length 9: each float of the vector represents 1 action (one of the 9 boxes in Tic Tac Toe). The agent will play the ...
8
votes
4answers
2k views

Should neural nets be deeper the more complex the learning problem is?

I know it's not an exact science. But would you say that generally for more complicated tasks, deeper nets are required?
2
votes
0answers
28 views

Merge two different CNN models into one

I have 2 different models with each model doing a separate function and have been trained with different weights. Is there any way I can merge these two models to get a single model. If it can be ...
1
vote
1answer
46 views

Why are denser layers needed in computer vision neural nets?

Many neural net architectures for computer vision tasks use several convolutional layers and then several fully-connected (or dense) layers. While the reasons for using convolutional layers are clear ...
0
votes
1answer
70 views

What does a CNN classifier default to?

I've been messing around with an Open Set, Binary Classifier and am having trouble with it. I'm sure there are a lot of reasons for that trouble. One thing I am struggling with is, what does the ...
1
vote
0answers
19 views

Number of weights in historical to cutting edge deployment of deep networks [closed]

In cutting edge deployment of deep networks for different architectures (such as $CNN$, $QRNN$ etc) what is the historical trend of current limits of trainability possible computationally? By this I ...
1
vote
0answers
12 views

Can Grad CAM feature maps be used for Training?

I am trying to recreate the architecture of the following paper: https://arxiv.org/pdf/1807.03058.pdf Can someone help me in explaining how are the feature maps coming out of the output of GradCam ...
5
votes
1answer
470 views

How to create an AI to solve a word search?

This at first sounds ridiculous. Of course there is an easy way to write a program to solve a wordsearch. But what I would like to do is write a program that solves a word-search like a human. That ...
6
votes
1answer
80 views

Are there well-established ways of mixing different inputs (e.g. image and numbers)?

I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.?...
1
vote
1answer
103 views

How do the current input and the output of the previous time step get combined in an LSTM?

I am currently looking into LSTMs. I found this nice blog post, which is already very helpful, but still, there are things I don't understand, mostly because of the collapsed layers. The input $X_t$,...
4
votes
0answers
152 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
1
vote
0answers
32 views

Neural Network training on one example to try overfitting leads to strange predictions

tldr; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which ...
2
votes
1answer
62 views

Tweaking a CNN for large number of input channels

I am using a CNN for function approximation using geospatial data. The input of the function I am trying to approximate consists of all the spatial distances between N location on a grid and all the ...
1
vote
0answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
1
vote
0answers
24 views

AlphaZero value at root node not being affected by training

I have written my own AlphaZero implementation and started training it recently. Problem is, I am 99% sure there is a mistake and I do not know how to tackle this, since I cannot explain it. I am new ...
10
votes
1answer
868 views

Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?

AlphaGo Zero contains several improvements compared to its predecessors. Architectural details of Alpha Go Zero can be seen in this cheat sheet. One of those improvements is using a single neural ...
2
votes
1answer
584 views

What do the numbers in this CNN architecture stand for?

So I've got a neural net model (ResNet-18) and made a diagram according to the literature (https://arxiv.org/abs/1512.03385). I think I understand most of the format of the convolutional layers: ...
1
vote
3answers
335 views

Are there ways to learn and practice Deep Learning without downloading and installing anything?

As per subject title, are there ways to try Deep Learning without downloading and installing anything? I'm just trying to have a feel of how this work, not really want to go through the download and ...
16
votes
2answers
328 views

Are Modular Neural Networks more effective than large, monolithic networks at any tasks?

Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network. In principle, the hierarchical organization ...
2
votes
6answers
2k views

What is the difference between AI architecture and AI model?

What is the difference between AI architecture and AI models? Are both of them the same? If not, please distinguish both of them and give example of each.
2
votes
2answers
184 views

How does Tara AI work?

Cisco and other companies are using Tara AI—a matching tool that connects IT projects with freelancers who have the exact skills required to complete them. Looking for an explanation of how Tara AI ...
1
vote
0answers
24 views

Architecture and Use of Different Algorithms for Health Goal Feedback

I wanted to get some opinions from the community for a certain problem that I will be approaching. The problem is to provide feedback to a user based on a image of the upper male torso. The image ...
2
votes
0answers
31 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
3
votes
1answer
137 views

Neural network architecture for comparison

When someone wants to compare 2 inputs, the most widespread idea is to use a Siamese architecture. Siamese architecture is a very high level idea, and can be customized based on the problem we are ...
2
votes
0answers
24 views

Is Hebbian learning the progenitor of AI?

Hebb's postulate attempts to explain associative learning via the processes of sampling (using sensors), emitting responses and receiving feedback. This is a form of control orientated architecture ...
1
vote
2answers
148 views

Are artificial networks based on the perceptron design inherently limiting?

At the time when the basic building blocks of machine learning (the perceptron layer and the convolution kernel) were invented, the model of the neuron in the brain taught at the university level was ...
3
votes
1answer
238 views

What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?

This concerns a set of finite, non-trivial, combinatorial games [M] in the form of an app. A sample game can be found here. Because this is a mass market product, we can't take up too much space, ...
0
votes
1answer
589 views

How can I create my own Google duplex?

I am trying to create my own variant of Google duplex however, it won't make calls but just have a real time conversation. My question is, where and how to start? How do I train my model with real ...
1
vote
0answers
74 views

Algorithms that connect neurons to previous layers as well as next

Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes ...
3
votes
2answers
222 views

Are there any learning algorithms as powerful as “deep” architectures?

This article suggests that deep learning is not designed to produce the universal algorithm and cannot be used to create such a complex systems. First of all it requires huge amounts of computing ...
3
votes
1answer
107 views

What linear rectifier is better?

What rectifier is better in general case of Convolutional Neural Network and how about empirical rules to use each type? ReLU PReLU RReLU ELU Leacky ReLU