Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Changing order of backpropagation training set

I'm training a simple feedforward network to model the XOR truth table. So far, I can get the network to converge to the solution quite quickly if the training input is always the same. For instance: ...
0
votes
0answers
3 views

Is "width of a neural network" a wrong phrase?

Depth of the neural network is equal to the total number of layers in the neural network except input layer. so, neural network with more number of layers are called deep neural networks. Width, in ...
0
votes
0answers
11 views

Multiple GRU layers to improve a text generation

I am using the model in this colab https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb#scrollTo=AM2Uma_-yVIq for Shakespeare like text generation. ...
0
votes
1answer
30 views

Why is the validation accuracy lower in case of CNN?

I fed the same set of 1.4 million data to two different models: MLP CNN model In both cases, I used the same parameters and hyperparameters. The CNN is showing comparatively lower accuracy (80%) ...
0
votes
1answer
21 views

How to handle random order of inputs and get same output?

I am a beginner with DL. I did some tutorials and I know the basics of TensorFlow. But I have a problem understanding how to construct more advanced NNs. Let's say I have 6 inputs and a list of 500 ...
2
votes
1answer
42 views

What problem does the neural network really solve?

In the image below taken from a Youtube video, the author explains that the neural network can be used to fit a relational graph for a set of data points shown by the green line. And that this is ...
0
votes
0answers
19 views

NLP problem Phrase/Token labeling

Looking for suggestions on how to define the following NLP problem and different ways in which it can be modeled to leverage machine learning. I believe there are multiple ways to model this problem. ...
0
votes
0answers
27 views

What deep reinforcement learning algorithm should I use for my problem?

So here is a description of my problem: Essentially, I have a large amount of files filled with code for a number of different tasks. However, lets say these codes are inefficient, and should be ...
0
votes
0answers
19 views

How to model the inputs and outputs of the neural network for the Splinterlands card game?

I have recently just completed a course on deep learning and I feel like an intermediate, but I still don't know how to structure this problem. I'm looking to create a NN to play the card game ...
0
votes
0answers
30 views

Which neural network architecture to use to detect very close and very small blobs in high resolution fluorescence images?

Context I am developing a pipeline to automate the detection of small, almost circular, bright blobs (4px) (see first image below) on high-resolution fluorescence images (2048px) and later to assign ...
0
votes
1answer
27 views

Do authors generally use fully connected layer instead of affine transformation?

We generally encounter the following statement several times The input vector is first fed into a fully connected layer...... Since linear activation functions, such as identity function, can so ...
0
votes
1answer
29 views

Why identity function is generally treated as an activation function?

It is known that the primary purpose of activation functions, used in neural networks, is to introduce the non-linearity. Then how can the linear activation function, especially the identity function, ...
0
votes
0answers
30 views

Using parameter estimation for training a neural network

Assume that we have 4 layers in a neural network. $$z_1 = L_1(x, W_1)$$ $$z_2 = L_2(z_1, W_2)$$ $$z_3 = L_3(z_2, W_3)$$ $$y = L_1(z_3, W_4)$$ Where $x$ is the vector input, $y$ is the vector output ...
0
votes
0answers
24 views

What's the benefit for using a Kalman filter for training a neural network compared to other optimization algorithms?

I found a paper about using an Unscented Kalman Filter(UKF) for traning a neural network. The UKF filter is modified so it works for parameter estimation. Assume that we have a neural network model $\...
1
vote
0answers
22 views

Machine learning with raw data alone / or raw data with its statistics

My question is very general and it does not originate from a specific problem. Let's assume that, through experience, we have learned that some statistical property of a set of data is important in ...
0
votes
1answer
39 views

How do I prepare this 3D data for NN?

How do I prepare the info of 3D models to use with NN? For example, I have thousands of models with boxes similar to the ones in the image below. I can extract the vertices and their normals that make ...
0
votes
0answers
16 views

Discrepancy of backpropagation formula between Andrew Ngs ML Course and those derived by neuralnetworksanddeeplearning.com

I'm currently working through Week 5 of Andrew Ngs Machine Learning course on Coursera, which goes through the backprop algorithm for basic neural networks. Whilst trying to derive the formulae he ...
1
vote
0answers
49 views
+50

How to build neural network that detects changed signal firing pattern and is trained on positive patterns only?

Let's have a set of n devices firing signals. Devices are firing in the same cycles, but each device can fire in different phase of the cycle. More, the exact firing point can fluctuate, for example ...
1
vote
1answer
59 views

What does it mean when accuracy of regularized model is higher for training set than for validation set?

Accuracy of my regularized model is higher for training set than for validation set. The situation improves when regularization coeefficient is reduced: What does this really imply? From my ...
0
votes
0answers
16 views

Is my understanding of RNNs wrong?

I asked a similar question a few days back here, but since no one replied, I thought I should subdivide my question further. My understanding of RNNs is as follows, Suppose I have a standard MLP. To ...
0
votes
1answer
54 views

Is my single layer perceptron getting biased input some way or the other?

I was working a little bit on a school project my team and I decided to do for submission in the year-end. It's a small game which I call 'Quattro', and its rules are as follows: The game is played ...
0
votes
1answer
25 views

What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?

I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
0
votes
0answers
22 views

Modelling of output neuron for mixed features?

Dataset in artificial intelligence, in general, consists of some features (say $n$). Assume that $m$ among them are output features. I want to model this function using neural network. So, input to my ...
0
votes
0answers
38 views

What are some machine learning frameworks for supervised clustering?

I have a task where I need to take "data points" which consist of collections of items. Each item needs to be categorised according to predefined categories. That's the easy part - my ...
1
vote
0answers
26 views

Are there any benefits of adding attention to linear layers?

Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
1
vote
1answer
38 views

What does it mean by bottleneck and representational bottleneck in feedforward neural networks?

Consider the following paragraph from section 2: General Design Principles of the research paper titled Rethinking the Inception Architecture for Computer Vision ...
0
votes
0answers
35 views

Which model is more efficient and why?

Suppose, I have two NN models: CNN model Sequential NN model They are solving the same problem. The data points have the same number of features. In the case of #1, we used 0.6 million data points, ...
1
vote
1answer
30 views

How to make an output independent of input feature in neural networks?

Is there a way to make a certain output dimension of a neural network independent of a particular feature dimension? For example, I have a function $f_{\theta} : \mathcal{R}^{10} \rightarrow \mathcal{...
2
votes
1answer
27 views

What type of neural network do you need if you want to detect an action or dynamic pattern instead of a static pattern?

Let's say that you want to detect if a man is running, walking, or dancing instead of just detecting a man still. What type of neural networks will you use for this purpose?
0
votes
1answer
41 views

How do CNNs handle inputs of different sizes and shapes?

I am new to deep learning so feel free to correct me where I am wrong. Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over ...
0
votes
0answers
33 views

What's the best way to feed stories to a neural network?

I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because ...
2
votes
0answers
43 views

How does back propagation adjust the hidden layers' weights and biases?

I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm. In back propagation, I understand we want to go backwards from the ...
0
votes
1answer
54 views

Residual Blocks - why do they work?

I've learnt that idea that the residual block was invented to solve the vanishing gradient problem due to the deep layer to layer multiplication. I understand that for example if I have 10 layers, and ...
0
votes
0answers
6 views

What is an intuitive explanation for the weighted sum of inputs plus bias that cause a neuron to be activated when it sees some samples but not others

Im stuck on some of the intuition thats cause specific neurons to fire versus others. Take a feedforward MLP that is able to classify MNIST (and has been optimised). A silly example might be that in ...
0
votes
0answers
6 views

Are there any connectionist parametric models with non-neuron building blocks?

Parametric models allows learning by converging to the desired parameters, which are randomly initialized initially. Among the parametric models, especially in connectionist AI, neural networks are ...
3
votes
1answer
41 views

How can I interpret the way the neural network is producing an output for a given input?

I'm using a small neural network (2 hidden layers, 60 neurons apiece) for a rather complex binary classification problem. The network works well, but I'd like to know how it is using the inputs to ...
2
votes
1answer
97 views

Is there any way to train a neural network without using gradients?

The only algorithm I know for updation of weights of a neural network is based on gradients. The update equation can be roughly written as $$w \leftarrow w - \nabla_{w}L$$ where $\nabla_{w}L$ is the ...
0
votes
0answers
14 views

Time series forecasting with some challenges

I'm attempting to devise a strategy to make time series forecasts based on costs accumulated over time. My dataset contains about 7500 time-series sequences (call it an instance for now), each having ...
1
vote
2answers
17 views

How can equivariance to translation be a benefit of a CNN?

I just learnt about the properties of equivariance and invariance to translation and other transformations. Being invariant to translation is clearly an advantage, as even if the input gets shifted, ...
0
votes
1answer
40 views

What does all the formula and pictures mean?

https://www.nature.com/articles/s41467-020-17419-7 I am a medical school graduate and I really want to learn AI/ML for computer-aided diagnosis. I was building a symptom checker and I found the ...
2
votes
1answer
166 views

Can neural networks have continuous inputs and outputs, or do they have to be discrete?

In general, can ANNs have continuous inputs and outputs, or do they have to be discrete? So, basically, I would like to have a mapping of continuous inputs to continuous outputs. Is this possible? ...
1
vote
1answer
38 views

In the NEAT algorithm, what is the purpose of treating disjoint and excess genes differently?

In the NEAT algorithm, what is the purpose of treating disjoint and excess genes differently? They are treated so (or may be treated potentially) at least when calculating the distance between 2 ...
1
vote
0answers
34 views

Can NeuralHash be used as a loss for an Autoencoder?

I've recently read about NeuralHash, and immediately thought that it might be used as a loss for an autoencoder. However, it only seems to preserve "structure" from what I've read, not ...
0
votes
0answers
13 views

Graph neural network - what level (node or link or graph) prediction should be used for my problem?

I posted this on cross-validated but did not get a response. Trying my luck here. Sorry if this is not recommended. I have an undirected graph with nodes separated within a specified distance, say d, ...
1
vote
0answers
22 views

How to pass variable length data as feature to a neural network?

I am working on building a model to classify the type of touch the user makes(Long Press, Left Swipe, Right swipe and so on). I have data with features that characterise the user's touch, like ...
0
votes
0answers
13 views

Training a sequential model that can only evaluate after several hundred cycles

I'm attempting to build a neural network to play the card game, Lost Cities. A brief overview of the game: The game involves two players taking turns to play cards on expeditions. Expeditions incur a ...
1
vote
0answers
32 views

Is stability an attribute of model or training algorithm used or combination of both?

From this answer, stability is attributed to learning algorithm A stable learning algorithm is one for which the prediction does not change much when the training data is modified slightly. At some ...
2
votes
2answers
63 views

Does it make sense for a logistic regression model to perform better than a neural network on the Iris data set?

Per a review post, a simple Logistic Regression model on the Iris data set gets about 97% test accuracy on iris dataset whereas a neural network gets just 94%. The neural network model used in Keras ...
2
votes
0answers
34 views

How do neural networks deal with inputs of different sizes that are padded in order to have them of the same size?

I am trying to create an environment for RL where the size of my input (observation space) is not fixed. As a way around it, I thought about padding the size to a maximum value and then assigning &...
5
votes
2answers
1k views

Are calculus and differential geometry required for building neural networks?

I've been studying geometry and linear algebra for months with the goal to build neural networks. But now I'm reading that perceptrons require fitting curves, and curves are not expressed as linear ...

1
2 3 4 5
44