Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
1
vote
1answer
70 views

Regression For Elliptical Curve Public Key Generation Possible?

As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression. I was interested in trying a harder problem and ...
4
votes
1answer
55 views

Bayesian hyperparameter optimization, is it worth it?

In the Deep Learning book by Goodfellow et al., section 11.4.5 (p. 438), the following claims can be found: Currently, we cannot unambiguously recommend Bayesian hyperparameter optimization as an ...
0
votes
1answer
26 views

Can the attention mechanism improve the performance in the case of short sequences?

I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise. ...
1
vote
1answer
37 views

How to normalize for perceptual loss when training neural net from scratch?

Let's say we are training a new neural network from scratch. I calculate the mean and standard deviation of my dataset (assume I am training a fully convolutional neural net and my dataset is images) ...
0
votes
1answer
39 views

RMSprop equation - dividing by a matrix?

I've been trying to understand RMSprop for a long time, but there's something that keeps eluding me. $dW$ and $db$ are matrices (that's what I understand from the element-wise comment), so that must ...
1
vote
0answers
38 views

Stack of Planes as the Action Space Representation for AlphaZero (Chess)

I have a question regarding the action space of the policy network used in AlphaZero. From the paper: We represent the policy π(a|s) by a 8 × 8 × 73 stack of planes encoding a probability ...
0
votes
2answers
58 views

In the multi-head attention mechanism of the transformer, why do we need both $W_i^Q$ and ${W_i^K}^T$?

In the Attention is all you need paper, on the 4th page, we have equation 1, which describes the self-attention mechanism of the transformer architecture $$ \text { Attention }(Q, K, V)=\operatorname{...
2
votes
1answer
58 views

Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

(The math problem here just serves as an example, my question is on this type of problems in general). Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
0
votes
0answers
31 views

Is there any rule of thumb to determine the amount of data needed to train a CNN

I am training an AlexNet Convolutional Neural Network to classify images in a dataset. I want to know if there is any general rule for using data augmentation in training a neural network. How can I ...
0
votes
0answers
11 views

What are the best metrics for Multi-Object Tracking (MOT) evaluation and why?

I want to compare multiple computer vision Multi-Object Tracking (MOT) methods on my own dataset, so first I want to choose the best metrics for this task. I have carried out some research in ...
1
vote
0answers
52 views

How to handle long sequences with transformers?

I have a time series sequence with 10 million steps. In step $t$, I have a 400 dimensional feature vector $X_t$ and a scalar value $y_t$ which I want to predict during inference time and I know during ...
0
votes
0answers
7 views

Given a system state, generate a sequence of state changes that lead to it

These systems are discrete and their state changes are rule based. Example: Given a chess position, generate a series of moves that will lead to it (there may be many, one, or none, but I only need ...
2
votes
1answer
67 views

How are these equations of SGD with momentum equivalent?

I know this question may be so silly, but I can not prove it. In Stanford slide (page 17), they define the formula of SGD with momentum like this: $$ v_{t}=\rho v_{t-1}+\nabla f(x_{t-1}) \\ x_{t}=x_{...
0
votes
0answers
13 views

Are there any good self annotating methods in machine learning

Currently I want to train a deep learning model on finger detection (drawing bounding boxes around fingers), so I haven't found an annotated dataset, so I decided to make it myself. but the number of ...
2
votes
1answer
59 views

How robust are deep networks to class imbalance?

Before deep learning, I worked with machine learning problems where the data had a large class imbalance (30:1 or worse ratios). At that time, all the classifiers struggled, even after under-sampling ...
0
votes
1answer
23 views

What is the process of inventing deep neural network models? How researchers deal with long training times?

After reading this topic on GitHub how long time it takes to train YOLOV3 on coco dataset I was wondering how researchers deal with long training times while inventing new architectures. I imagine ...
1
vote
1answer
27 views

Why is Adam trapped in bad/suspicious local optima after the first few updates?

In the paper On the Variance of the Adaptive Learning Rate and Beyond, in section 2, the authors write To further analyze this phenomenon, we visualize the histogram of the absolute value of ...
0
votes
0answers
11 views

Deep Unsupervised clustering on big data with no prior knowledge

I have around 3M BW images and I would like to organize them in as few clusters as possible in way which is meaningful for the dataset without any prior knowledge for this data, as they come from ...
2
votes
2answers
110 views

How to calculate the distance between the camera and an object using Computer Vision?

I want to create a Deep Learning model that measures the distance between the camera and certain objects in an image. Is it possible? Please, let me know some resources related to this task.
2
votes
0answers
21 views

Why does the loss stops reducing after a point in this Transformer Model?

Context I was making a Transformer Model to convert English Sentences to German Sentences. But the loss stops reducing after some time. Code ...
0
votes
0answers
30 views

Time series prediction using LSTM and CNN-LSTM: which is better?

I am working on LSTM and CNN to solve the time series prediction problem. I have seen some tutorial examples of time series prediction using CNN-LSTM. But I don't know if it is better than what I ...
0
votes
0answers
17 views

How should I use deep learning to find the rotation of an object from its 2D image?

I have 6600 images and I am supposed to know the rotation of the object in each image. So, given an image, I want to regress to a single value. My attempt: I use Resnet-18 to extract a feature vector ...
2
votes
0answers
21 views

CNN for a DQN agent with a 2D matrix state and action as a 2D matrix

I have a custom environment, where the state is a 2D matrix of 11 rows (equals to number of users to satisfy) and 3 columns. Each column can take the value of either 0 or 1, and in each row, there can ...
3
votes
1answer
62 views

How to build a DQN agent with state and action being arrays?

I have a Reinforcement-Learning environment where the state is an array of 0s and 1s with length equals to the number of users the agent must satisfy (11 users). The agent must choose one of 12 ...
0
votes
0answers
37 views

How to edit a photo using deep learning?

I just took a course on deep learning where one part of the syllabus was image classification and object recognition using CNNs, but I wonder how deep learning can be applied to apply certain filters ...
4
votes
1answer
151 views

Is AlphaFold just making a good estimate of the protein structure?

In the news, DeepMind's AlphaFold is said to have solved the protein folding problem using neural networks, but isn't this a problem only optimised quantum computers can solve? To my limited ...
3
votes
0answers
30 views

DQN Agent with a 2D matrix as input in Keras

I have a Reinforcement Learning environment where the state is a 2D matrix with 0s and 1s (only one column with the value of 1 in each row). Example: ...
0
votes
1answer
39 views

What is the impact of the number of features on the prediction power of a neural network?

What is the impact of the number of features on the prediction power of an ANN model (in general)? Does an increase in the number of features mean a more powerful prediction model (for approximation ...
0
votes
0answers
17 views

How to perform data augmentation on multiple input classification task?

I would like to add some more samples to my dataset which consists of two parts: 1. image and 2. numerical data. For each image in the dataset there is its corresponding numerical data as well. If it ...
0
votes
0answers
17 views

Estimating depth/perspective of image

I'd like to find a method in which the depth or perspective of an image is estimated. I imagine this could perhaps be done based on how quickly classified objects, such as cars or humans, grow or ...
0
votes
0answers
49 views

What could be a good $\mathcal{R}$ dataset in the article “Old Photo Restoration via Deep Latent Space Translation”?

There are three domains in this article: Old Photo Restoration via Deep Latent Space Translation. The real old pictures noted by $\mathcal{R}$, the artificial old pictures noted by $\mathcal{X}$, and ...
1
vote
0answers
16 views

What is the intuition behind equations 10, 11 and 12 of the paper “Noise2Noise: Learning Image Restoration without Clean Data”?

Can anyone help me understand these functions described in the paper Noise2Noise: Learning Image Restoration without Clean Data I have read the portion A.4 in the appendix but need a more detailed and ...
1
vote
0answers
25 views

Why should variance(output) equal variance(input) in Xavier Initialisation?

In a lot of explanations online for Xavier Initialization, I see the following: With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a ...
1
vote
0answers
20 views

How significant is the decoder part of the capsule network?

Capsule Networks use an encoder-decoder structure, where the encoder part consists of the capsule layers (PrimiaryCaps and DigitCaps) and is also the part of the capsule network which performs the ...
2
votes
1answer
72 views

How can we derive a Convolution Neural Network from a more generic Graph Neural Network?

Convolution Neural Network (CNNs) operate over strict grid-like structures ($M \times N \times C$ images), whereas Graph Neural Networks (GNNs) can operate over all-flexible graphs, with an undefined ...
1
vote
2answers
55 views

Why is my Keras prediction always close to 100% for one image class?

I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either "good" or "bad"). I am ...
1
vote
1answer
40 views

What are the conceptual differences between regularisation and optimisation in deep neural nets?

I'm trying to wrap my mind around the concepts of regularisation and optimisation in neural nets, especially around their differences. In my current understanding, regularisation is intended to tackle ...
1
vote
0answers
18 views

Object Detection as a means of Anomaly Detection

Is it possible to train an Object Detector (e.g. SSD), to detect when something is not in the image. Imagine an assembly line that transports some objects. Each object needs to have 5 screws. If the ...
2
votes
0answers
124 views

How should I build this DQN agent?

I have a set of users that can be one of 3 types. They will randomly request a service from the UAV which is a drone used as a Base Station. The UAV (the agent) is tasked with allocating subchannels (...
0
votes
0answers
29 views

How to generate a matrix out of sparse data?

I have a system that takes 32 inputs (all of which are 1 or 0) and generates 32 outputs (all of which are complex numbers that lie roughly in the range of (0,2)). The response of this system to its ...
0
votes
1answer
28 views

Semantic segmentation failing in small instance detection

I performed semantic segmentation with U-net. My dataset consists of grayscale images of defects. After training the dataset for I got an metric accuracy of only 0.3 - 0.4 IOU. Eventhough it is merely ...
0
votes
0answers
23 views

Find the distance between two objects from a 45 degree tilted camera images taken from a drone with a specific elevation from the ground

I have a project where i'm supposed to find the distance and the height of a specific object in an image taken by a drone using one camera. I have looked into perspective transformation/correction but ...
2
votes
1answer
26 views

Does this diagram represent several LSTMs, or one through several timesteps?

I'm trying to read this paper describing Google's LSTM architecture for machine translation. It features this diagram on page 4: I'm interested in the encoder block, on the left. Apparently, the pink ...
1
vote
1answer
39 views

Why are shallow networks so prevalent in RL?

In deep learning, using more layers in a neural network adds the capacity to capture more features. In most RL papers, their experiments use a 2 layer neural network. Learning to Reset, Constrained ...
1
vote
0answers
24 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
2
votes
1answer
32 views

Predict next event based on previous events and discrete reward values

Suppose, I have several sequences that include a series of text (the length of sequence can be varied). Also, I have some related reward value. however, the value is not continuous like the text. It ...
0
votes
0answers
12 views

What Deep Learning Applications Might Require Super-Computers or “SuperPODs”

With the release of NVIDIA's DGX SuperPOD of A100 GPUs, supercomputers will/are becoming more and more common-place. What potential deep learning tasks/applications might become more accessible with ...
1
vote
0answers
26 views

How is input defined for a biaxial lstm network for generating music?

I am reading Composing Music With Recurrent Neural Networks by Daniel D. Johnson. But I am really confused about the input passed to this network. If we pass notes of music along the time axis, then ...
0
votes
1answer
14 views

In a face database containing multiple images per subject, how do we determine the face image which is most suited for face recognition?

Let us imagine a face database with several subjects, each subject having multiple face images. How do we determine which is the best face suitable for face recognition purposes?
0
votes
0answers
8 views

Best strategy for Classification of Science Subjects. Phy, Chem , Maths and Bio? BERT, Transformers, Attention+SLTM, Self-Attention+LSTM?

I am working on a project where I have to first classify the Subjects of the given question and then the respective Chapter and then the sub-topic. In a nutshell, I have to predict the Subject, Grade ...

1
2
3 4 5
29