Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

Exact definition of WRN-d-k (Wide ResNet)

I am a little confused about the WRN-d-k notation from Wide Residual Networks. To quote the paper, In the rest of the paper we use the following notation: WRN-n-k denotes a residual network that has ...
user avatar
  • 161
0 votes
0 answers
14 views

How to feed an image sequence into a CNN while keeping input images independent? [closed]

I'm using a convolutional neural network (CNN) to preprocess my input for an LSTM. I have the following input dimensions: 128 x 10 x 3 x 32 x 32 (batch size, sequence length, color channel, height, ...
user avatar
0 votes
1 answer
25 views

Machine Learning book for fundamentals - Simon Haykin vs. Christopher M. Bishop

Since I started studying Machine Learning, I was torn between two books in this area, and I could never decide which one is the best to follow. The first book is widely used and known: Pattern ...
user avatar
0 votes
1 answer
25 views

Shuffling vs Non-shuffling train/test set yields drastically different results

I am currently working with a very deep NN (200mio. to 350mio. params). My data set is roughly of shape (2mio, 350), i.e. 2mio samples and 350 features. In fact, the features are time series. As input ...
user avatar
0 votes
1 answer
23 views

Why is the derivative of activation function all positive?

All the activation functions I see have positive derivatives. Will negative ReLU work as well as its positive counterpart or will it lead to instability?
user avatar
  • 101
0 votes
0 answers
16 views

multi-agent RL training schemes

I would like to do a project in multi-agent reinforcement learning. I have a quite simple self-made environment which is ready for RL training in single agent fashion. It includes an agent which can ...
user avatar
  • 31
0 votes
0 answers
6 views

Shuffle data inside learning sample in order independet transformer model

Does it make sense to create new samples with shuffled items "tokens" inside a learning sample for the order independent (no positional encoding) transformer model to improve model accuracy?
user avatar
0 votes
0 answers
6 views

linear layer bert sentence embedding

I have a special situation where I need to embed a sentence with bert-sentence transformer and get a numeric value for it. This is not possible with this model, ...
user avatar
0 votes
0 answers
6 views

How to obtain the graph in the video tutorial

I am watching this video tutorial https://www.youtube.com/watch?v=gmjzbpSVY1A&t=1002s . At 16:34, the author show the variation of the line when weight change from 0.97 to -1, the below graph is ...
user avatar
  • 103
1 vote
0 answers
14 views

How to use NN to generate a model which produces given distributions?

For a non-Markovian random walk, each step can go up or down. And for the $n-th$ step, its step size $s(n)$ may depend on the path of walk, and the probability for going up or down may also depend on ...
user avatar
  • 151
0 votes
0 answers
16 views

Difference between training algorithms

I'm using GPS Data for my Total electron Content (TEC) Prediction, for which I'm using Non-linear Autoregressive with External (Exogenous) Input (NARX) Model. My question is what's the difference ...
user avatar
0 votes
0 answers
10 views

How state is combined with action in crtitic networks?

Actor-critic networks are present in deep reinforcement learning algorithms. Actor-network takes a state as input and gives action as output. Critic-network takes state and action as input and gives a ...
user avatar
  • 3,201
0 votes
1 answer
28 views

In a neural network's neuron that has no activation function, to calculate the delta for the neuron during back propagation do you use a derivative?

I have a neural network that is composed of an input layer, two hidden layers and an output layer. The topology is [151, 200, 100, 1] I am using ReLU activation function on the neurons that are in the ...
user avatar
  • 3
1 vote
0 answers
14 views

What is the next step in top-down brain simulation after spiking neural networks?

This paper from Yamazaki et al. describes a 68 billion spiking neural network model of the cerebellum. The simulation was about 600 times slower than real time, and the cerebellum is perhaps one of ...
user avatar
  • 41
0 votes
1 answer
24 views

Do we feature scale one hot encoded variables in neural networks? [closed]

If I have a categorical variable in my neural network which I encode using one hot encoding, do I need to feature scale it along with other features before training the artifical neural network? or do ...
user avatar
0 votes
1 answer
13 views

U-Net Maxpooling vs Convolution

Hello I'm implementing a CycleGAN and most of the other implementations I've seen on the internet use Convolution with stride 2 instead of a Maxpoolinglayer for downsample. On to my question, why ...
user avatar
  • 1
0 votes
0 answers
7 views

Simple dimension unmatch problem of a simple neural network

In this simple neural network: the derivative for the cost function J when assuming binary cross entropy loss would be If we assume that the dimension of X is 2x1, then wouldn't A1 be 2x1 and A2 be ...
user avatar
  • 1
1 vote
1 answer
22 views

fondamental question about regularization techniques to solve overfitting problem in neural networks

I have a text classification neural network based on BERT that overfits. The accuracy on the training dataset is 95%, whereas it is 68% on the validation dataset. Using some classical regularization ...
user avatar
  • 113
0 votes
0 answers
7 views

Are there any general guidelines for the architecture of critic network based on actor network?

Suppose the actor-network looks like the following ...
user avatar
  • 3,201
1 vote
0 answers
18 views

Combining Different Inputs in a Neural Network for Numerical Integration

I am building a NN that numerically integrates a non-linear differential equation. Given a DE: $$ \frac{d}{dt}x(t) = f(x, p) $$ with solution $x \in \mathbb{R}^n$ and parameters $p \in \mathbb{R}^m$, ...
user avatar
0 votes
0 answers
9 views

Extracting behavior (switch On/Off) of an electric load from unlabeled time series data

Following are the details of my dataset: sampling frequency: 1 Hz No. of useful features: 10 The time series dataset is from household wherein I'm required to find ...
user avatar
0 votes
1 answer
18 views

Question regarding matlab computer vision application and color recongnition [closed]

I am thinking of choosing a computer vision project for my school project(detect crack on surface) and the duration I have is roughly 4 months. With no prior knowledge in neural network, is matlab ...
user avatar
  • 103
0 votes
1 answer
43 views

Entirely linear neural network learning non-linear function

I have a neural network that's trained on a sine wave. It uses a lookback of 20 to see what the last 20 predictions were and predict the next value. This network has only a single Linear layer (input ...
user avatar
  • 1,246
0 votes
0 answers
26 views

Why Is There The Term 1/m In Backpropagation

In backpropagation the gradients are used to update the weights using the formula $$w = w - \alpha \frac{dL}{dw}$$ and the loss gradient w.r.t. weights is $$\frac{dL}{dw} = \frac{dL}{dz} \frac{dz}{dw} ...
user avatar
0 votes
1 answer
32 views

How are Neural Networks protected from false training data?

Suppose the training data there exist an element of some data being misleading and some being right, how could the Neural network be trained so that it could filter the right data from the wrong one? ...
user avatar
0 votes
1 answer
14 views

How to handle anomaly detections with multiple different timeseries' from network traffic?

I would like to implement an anomaly detection algorithm on multiple timeseries' from different network users. Since each user has different behavior and network traffic usage, my question is how can ...
user avatar
  • 131
0 votes
1 answer
17 views

How does a sigmoid neuron act like a perceptron in this scenario?

I have been reading Michael Nielsen’s book online on his website at http://neuralnetworksanddeeplearning.com/chap1.html. I am struggling to understand the second exercise: When c approaches infinity, ...
user avatar
0 votes
0 answers
11 views

How to investigate annual time series data?

I have annual time series data from 2000 to 2020. The brand has introduced new marketing camping in 2010 and I want to investigate the impact of this policy, that's why I am trying to explore the ...
user avatar
0 votes
0 answers
33 views

Neural Networks in molds industry

I recently began an internship at a moldmaker, where I'm supposed to learn about NN and how to use them (as you can imagine, I don't know much). Each mold is composed of many pieces, and for each ...
user avatar
  • 1
0 votes
0 answers
33 views

How to make neural networks more robust(intuitive explanation)?

Low spectral Norm ensures tight Rademacher complexity and ensures low generalization error for neural nets. Can anyone explain me this in a intuitive manner along with Rademacher view point. ...
user avatar
  • 121
1 vote
0 answers
19 views

Understanding Rademacher Complexity Deeply

empirical Rademacher complexity is defined as, $$ \hat{R}_{m}(\mathcal{F}, S)=\frac{1}{m} \mathbb{E}_{\boldsymbol{\sigma}}\left[\sup _{f \in \mathcal{F}} \sum_{i=1}^{m} \sigma_{i} f\left(z_{i}\right)\...
user avatar
  • 111
0 votes
0 answers
34 views

Hot to calculate Maximum Normalized log Probability for Active Learning with BERT

I have encountered difficulties understanding the calculation of Maximum Normalized Log Probabilities acording to Shen et al.. With n being the sequence length, yi the label of word i. Xij is the ...
user avatar
0 votes
0 answers
22 views

Neural network and logical gates

I have a network witch consist of two fully connected layers (without bias) and a ReLu function in between. The network input is two binary numbers, and the output should be the a logical gate result: ...
user avatar
  • 101
0 votes
0 answers
10 views

How to propagate backwards in a neural network with an error term based on the average error over an episode of actions?

I am writing a neural network. I have an average error over an episode of actions to work with in order to update my weights. I know that in a 1 step neural network I take the most recent action of ...
user avatar
  • 3
0 votes
0 answers
19 views

is it possible to train the same neural network with different numbers of inputs and outputs?

is it possible to create an adaptative neural network that can change the number of its inputs and outputs without having to train it each time it changes? the neural netwrok has to take purchases and ...
user avatar
1 vote
1 answer
23 views

Rationalle behind SE3 Transformer?

I have just finished reading the SE3 transformer paper (SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks) by Fuchs et-al and although I'm sure I understand less than 50% of the ...
user avatar
0 votes
0 answers
16 views

What is a good neural network approach for this time-based data series

I’m trying to work out a neural network approach to a particular problem and would appreciate any advice. I have a machine that collects data over a period of time using 2 sensors. Data is collected ...
user avatar
0 votes
1 answer
53 views

How special tokens in BERT-Transformers work?

"[SEP] tokens are useful to differentiate the questions from answers through type_ids" Yes, but how is this helping model to understand that "I should look paragraph and generate ...
user avatar
  • 1
7 votes
2 answers
988 views

Deep Learning with Best-so-far instead of Where-you-are

It is my understanding that when training a Deep NN in Tensorflow/PyTorch/... we only keep the current state of the network in memory, except perhaps when we manually decide to save the current ...
user avatar
0 votes
0 answers
6 views

Is is mandatory to have same networks for pre-text and downstream tasks

I'm going to train a logo detection dataset by using MoCo network as a pre-text. Is it mandatory to use same network for downstream task? If yes, how to convert the pre-text task to a downstream task? ...
user avatar
0 votes
2 answers
70 views

Principles of designing a neural network

I have become more familiar with libraries such as tensorflow for a while now, and have become interested in utilizing neural networks for solving specific problems. The big question I have is, what ...
user avatar
0 votes
0 answers
20 views

Can we use NN for solving linear programming problem?

Problem: Solve some kind of Knapsack Problem which sounds like this: I have several clusters and I have to optimally redistribute the contents of each cluster between objects that have a capacity ...
user avatar
1 vote
2 answers
27 views

How to manually adjust output from model? [closed]

I wonder if it is possible to add manual inference to the output of a model? For example, I have a model called 'net', and the output value of 'net' is a vector called v = [v1, ... vn]. v is a binary ...
user avatar
  • 111
0 votes
1 answer
19 views

Should I include overlapping (input) Data in my training data

If I have time dependent data and want to predict the relative change for a future time. Should I separate the data so that the input times don't overlap? With an example: I have hourly temperature ...
user avatar
0 votes
1 answer
29 views

What is the distribution of autoencoder embeddings?

Is there any result on the distribution of autoencoder embeddings? For example, the following image (taken from this article) visualizes the latent space with t-SNE. As you can see, images from the ...
user avatar
  • 161
0 votes
1 answer
35 views

Autoencoders: Where does the encoder end and the decoder begin?

Consider a simple Autoencoder neural net: ...
user avatar
0 votes
0 answers
15 views

What is the logic behind using a trained classifier's gradients to synthesize controllable image?

In the controllable image synthesis, we are manipulating a noise vector z such that our generator ( in our GAN model ) creates images that the desired feature exists. For instance, take the feature of ...
user avatar
2 votes
1 answer
150 views

Why cannot linear activation functions be used to approximate any function?

In neural networks we use nonlinear activation functions such as sigmoid, ReLU, etc. Using a combination of these functions (with required scaling and shifting), we manage to estimate any nonlinear ...
user avatar
1 vote
1 answer
45 views

How do neural networks learn specific features throughout the layers?

I was reading about convolutional neural networks and I came across such an explanation: Consider detecting features in human face. The earlier layers of neural networks learn coarse features such as ...
user avatar
1 vote
1 answer
92 views

Where is memory stored in a chatbot like LaMDA?

I have a basic understanding of how neural networks work, and I have always thought that those chatbots work in a similar way (but I might be wrong): they take an input, shape it in a way that can be ...
user avatar

1
2 3 4 5
46