Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

Why is the embedding of a task using Task2Vec not depend on the model?

I saw this in the Task2Vec paper: TASK2VEC depends solely on the task, and ignores interactions with the model which may however play an important role. To address this, we learn a joint task and ...
1 vote
0 answers
31 views

Training a neural network simultaneously with two different loss functions rather than considering the weighted sum

This is a follow up on the already asked question: Is the neural network 100% accurate on training data if epoch loss is minimized to 0? I want to train a neural network that works as an approximator ...
  • 111
1 vote
0 answers
12 views

Can the Jacobian of a Neural Network be Full Column Rank?

Let $\mathcal{X}$ be the input data space and $\mathcal{Y}$ be the output data space. $f: \mathcal{X} \to \mathcal{Y}$ is a function represented by some Neural Network. Is it possible to to check if ...
  • 21
0 votes
0 answers
30 views

Implementation of DQN

Good day I attempted to implement DQN from scratch to solve the cartpole problem, the Tested my neural network class on the XOR table and it worked so I'm assuming the issue isn't with the neural ...
0 votes
1 answer
23 views

What kind of neural network and GPU should I use to classify images into > 10 000 classes?

I am trying to developp an image classifier that would have more than 10 000 classes but I don't know what kind of neural network I should use ? Some Other questions arise from this one : How big ...
1 vote
1 answer
42 views

Can neural network be used to predict deltas between numbers?

I have a list of increasing numbers with no duplicates for example : 3,6,11 and so on. Difference or deltas between these numbers such as in above case : 3, 3, 5 are frequent and with greatest ...
  • 11
1 vote
0 answers
6 views

How do I use the N correctly in NEATs speciation delta function?

When implementing NEAT I'm having some issues with the speciation distance/delta function, specifically the term N (number of genes in biggest genome). Won't term $N$ in $δ=c1*E/N+c2*D/N+c3*W$ just ...
  • 11
0 votes
0 answers
19 views

Can Graph Neural network leverage only the topological structure?

Graph Neural Networks (GNNs) are a powerful tool that allow learning on graphs by leveraging both the topological structure and the feature information for each node. For the particular problem I am ...
1 vote
2 answers
53 views

Can I minimize a mysterious function by running a gradient descent on her neural net approximations?

So I have this function let call her $F:[0,1]^n \rightarrow \mathbb{R}$ and say $10 \le n \le 100$. I want to find some $x_0 \in [0,1]^n$ such that $F(x_0)$ is as small as possible. I don't think ...
0 votes
2 answers
36 views

What is the difference between Restricted Boltzmann Machine and Artificial Neural Network?

In the deep learning course I took at the university, the professor touched upon the subject of the Restricted Boltzmann Machine. What I understand from this subject is that this system works ...
1 vote
1 answer
52 views

Training Loss Value Increasing instead of Decreasing [closed]

I am developing my first feed-forward fully-connected ANN from scratch based on batch learning mode on a toy training set. I am using back-propagation for calculating the gradient of the loss ...
  • 11
0 votes
1 answer
25 views

Are neural networks a strict special case of a transformer?

Since transformers contain a neural network, are they a strict generalisation of standard feedforward neural networks? In what ways can transformers be interpreted as a generalisation and abstraction ...
0 votes
0 answers
26 views

Is the neural network 100% accurate on training data if epoch loss is minimized to 0?

This seems like a silly, trivial question, but I just want to confirm it in case I'm missing something. I'm trying to train a ReLU neural network, which is supposed to be a function that satisfies ...
  • 111
0 votes
0 answers
30 views

Combining loss function while training with another objective function

I would like to train a ReLU neural network minimizing an objective function that looks like this: $$L(W) + \eta + 1_{S}(\eta,W)$$ where $W$ is the set of weight matrices, $L(W)$ is a custom loss ...
  • 111
0 votes
0 answers
14 views

Is it overfitting?

hi i'm new in this field. I am trying to do a video classification project by using 3DCNN and I plotted the loss curves & accuracy curves. I have some questions. i'm using kfold Cross validation. ...
  • 1
1 vote
0 answers
12 views

Distributions over outputs for randomly initialized neural networks

Does anyone have any pointers to resources about the properties of randomly initialized neural networks (with no training)? I'm guessing this might depend on the network architecture and ...
  • 340
0 votes
1 answer
23 views

Is data augmentation beneficial even if the dataset is large/diverse enough?

I'm training a deep learning model to map binary images to grayscale values of the same shape. For the dataset, I can genearate one as large and diverse as I want it to be. My question is: let's say ...
0 votes
0 answers
11 views

Any learning method to connect an edgeless graph?

Given N nodes with no edges connecting them at all. Each node has certain n features. Is there a way to connect these nodes and form a connected graph. The idea is to then feed the outputted graph ...
  • 101
0 votes
1 answer
29 views

Why the x-input should be multiplied by weight in an artificial neuron?

So why weight should be multiplied with input? Yes I know, weight is intended for tuning the connection strength of input that will affect output so that's will be useful for learning (CMIIW). But why ...
-1 votes
1 answer
22 views

Can anyone help me how this code extracts features from the graph? [closed]

I have this code from DGCNN Neural Network but i don't understand how it extracts features. In particular i understand that we get the top knn point but i don't understand the idx_base. ...
0 votes
1 answer
33 views

Can MSE be used for NN categorical classification problems

I currently have a neural network that can manage to perform polynomial (single output) regression problems. I now want to upscale to classification problems (eg: image recognition). Can I do this ...
  • 13
9 votes
6 answers
4k views

If an event has a statistical probability of only 50%, is it possible to use a neural network to predict it with more than 50% accuracy?

For example using a neural network to predict a coin toss. Can a trained neural network to predict it with more than 50% accuracy?
  • 213
0 votes
0 answers
28 views

The result of back propagation for a neural network

I have created a neural network that feeds an image into a convolutional neural net, then feeds the flattened output of this network into an artificial neural network. I have a feeling that my ...
  • 1
0 votes
0 answers
9 views

is Flipout an upgrade of the local reparameterization trick or a completely different technique?

I was reading the Flipout paper and I am confused about 1 thing: when the author samples the perturbation matrix $\hat{\Delta W}$ does he do directly from the real variational distribution $q_{\...
  • 101
0 votes
1 answer
26 views

How do I train a model to classify if it's a Full Human Body in the picture?

recently I started a personal project that uses some Machine Learning techniques in the process, so I'm currently collecting human images with a web scraper. I know that I can use some pre-trained ...
1 vote
1 answer
25 views

Hyperparameter tuning methods for neural networks

I have a fully connected feedforward classifier neural network that uses the leaky ReLU activation function. I would like to apply a state-of-the-art hyperparameter tuning method to my methodology. ...
  • 111
0 votes
0 answers
19 views

How do GPT models pass information for each token prediction?

So, I'm trying to understand what is going on in the following picture (from this paper): Each decoder blocker in the above GPT model has attention heads (red) and MLPs (green). I know that we add ...
-1 votes
1 answer
32 views

Why does some AI professionals think deep learning is "intelligent"?

Human is intelligent, this is the ultimate definition (although not rigorous) of intelligence. Ordinary programs are not considered "intelligent", even they can do so many smart things with ...
  • 199
0 votes
0 answers
26 views

How do you display a neural network

I'm new to tensorflow and ML but am progressing slowly. I know how to look at the weights and biases but am still trying to figure out if there is an easy way to display a neural network in the ...
  • 141
0 votes
1 answer
20 views

Reach optimal values by not decreasing gradient

Is it possible to reach the optimal values ​​for the parameters by not applying the decreasing gradient in some layers?
  • 1
0 votes
0 answers
10 views

Predicting time spent to build a metal piece using RNN

My data consists in many metal pieces which are put together to make a final metal mould. To make each of this metal pieces, machinery recieves many operations to follow, like chopping, facing, etc... ...
  • 1
0 votes
0 answers
17 views

Predicting occurence within multiple time horizons

i recently found a paper (sorry cannot share) where it's about predicting occurence of an event within several future time horizons, e.g. 6h, 12h, 24h.... In order to address the problem that an ...
  • 101
0 votes
1 answer
36 views

Are there limitations on network output architecture and action mapping in reinforcement learning?

I'm easing my way into a toy reinforcement learning problem where my objects can move and take different actions on a simple grid, but I'm having trouble understanding what constraints might exist in ...
  • 77
0 votes
2 answers
52 views

use artificial intelligence in predicting the price of stocks

Is it possible to use artificial intelligence for example method like reinforcment learning, LSTM, ... in predicting the price of stocks or currencies like Bitcoin, etc.? And has the work been ...
  • 9
-1 votes
1 answer
31 views

Is there a way to freeze training for weights, but not biases in PyTorch? [closed]

I'm constructing a neural network where the weights of my first hidden layer (connected to the input) are all 1 (identity matrix), but the biases are variable. Is there a way to "freeze" any ...
  • 101
1 vote
0 answers
30 views

How to detect peak locations via Neural Networks?

As part of my masters thesis, I'm developing generative models for ECGs. Right now, I have a Denoising Diffusion Implicit model (DDIM), that transforms random noise into a valid ECG (2s long, or 1024 ...
0 votes
0 answers
14 views

Exact definition of WRN-d-k (Wide ResNet)

I am a little confused about the WRN-d-k notation from Wide Residual Networks. To quote the paper, In the rest of the paper we use the following notation: WRN-n-k denotes a residual network that has ...
  • 161
0 votes
1 answer
44 views

Machine Learning book for fundamentals - Simon Haykin vs. Christopher M. Bishop

Since I started studying Machine Learning, I was torn between two books in this area, and I could never decide which one is the best to follow. The first book is widely used and known: Pattern ...
0 votes
1 answer
32 views

Shuffling vs Non-shuffling train/test set yields drastically different results

I am currently working with a very deep NN (200mio. to 350mio. params). My data set is roughly of shape (2mio, 350), i.e. 2mio samples and 350 features. In fact, the features are time series. As input ...
0 votes
1 answer
29 views

Why is the derivative of activation function all positive?

All the activation functions I see have positive derivatives. Will negative ReLU work as well as its positive counterpart or will it lead to instability?
  • 101
0 votes
0 answers
7 views

Shuffle data inside learning sample in order independet transformer model

Does it make sense to create new samples with shuffled items "tokens" inside a learning sample for the order independent (no positional encoding) transformer model to improve model accuracy?
0 votes
0 answers
15 views

linear layer bert sentence embedding

I have a special situation where I need to embed a sentence with bert-sentence transformer and get a numeric value for it. This is not possible with this model, ...
0 votes
0 answers
6 views

How to obtain the graph in the video tutorial

I am watching this video tutorial https://www.youtube.com/watch?v=gmjzbpSVY1A&t=1002s . At 16:34, the author show the variation of the line when weight change from 0.97 to -1, the below graph is ...
  • 103
1 vote
0 answers
17 views

How to use NN to generate a model which produces given distributions?

For a non-Markovian random walk, each step can go up or down. And for the $n-th$ step, its step size $s(n)$ may depend on the path of walk, and the probability for going up or down may also depend on ...
  • 151
0 votes
0 answers
16 views

Difference between training algorithms

I'm using GPS Data for my Total electron Content (TEC) Prediction, for which I'm using Non-linear Autoregressive with External (Exogenous) Input (NARX) Model. My question is what's the difference ...
0 votes
0 answers
15 views

How state is combined with action in crtitic networks?

Actor-critic networks are present in deep reinforcement learning algorithms. Actor-network takes a state as input and gives action as output. Critic-network takes state and action as input and gives a ...
  • 3,251
-1 votes
1 answer
38 views

In a neural network's neuron that has no activation function, to calculate the delta for the neuron during back propagation do you use a derivative?

I have a neural network that is composed of an input layer, two hidden layers and an output layer. The topology is [151, 200, 100, 1] I am using ReLU activation function on the neurons that are in the ...
  • 1
1 vote
0 answers
15 views

What is the next step in top-down brain simulation after spiking neural networks?

This paper from Yamazaki et al. describes a 68 billion spiking neural network model of the cerebellum. The simulation was about 600 times slower than real time, and the cerebellum is perhaps one of ...
  • 41
0 votes
1 answer
19 views

U-Net Maxpooling vs Convolution

Hello I'm implementing a CycleGAN and most of the other implementations I've seen on the internet use Convolution with stride 2 instead of a Maxpoolinglayer for downsample. On to my question, why ...
  • 1
0 votes
0 answers
8 views

Simple dimension unmatch problem of a simple neural network

In this simple neural network: the derivative for the cost function J when assuming binary cross entropy loss would be If we assume that the dimension of X is 2x1, then wouldn't A1 be 2x1 and A2 be ...
  • 1

1
2 3 4 5
46