Skip to main content

Questions tagged [deep-learning]

For questions related to deep learning, which refers to a subset of machine learning methods based on artificial neural networks (ANNs) with multiple hidden layers. The adjective deep thus refers to the number of layers of the ANNs. The expression deep learning was apparently introduced (although not in the context of machine learning or ANNs) in 1986 by Rina Dechter in the paper "Learning while searching in constraint-satisfaction-problems".

Filter by
Sorted by
Tagged with
1 vote
0 answers
8 views

QLora using peft from HF and custom class for binary classification

I am fine-tuning an mistral-7B LLM model for binary classification. I realize it may be an overkill; but we are running some experiments. So far, I have used HuggingFace libraries like peft and ...
kms's user avatar
  • 111
0 votes
0 answers
10 views

Best method for finding centroid of 3D object?

I'm somewhat new to machine learning and want to implement a manufacturing application that finds the centroid of an irregular object so that as little material as possible is removed during ...
dstoddard's user avatar
0 votes
0 answers
19 views

Analyzing a deep learning model by constructing a Matrix from input and output data

I am currently completing my bachelor’s thesis, and recently my supervisor suggested that I could strengthen my arguments by explaining why deep learning models perform so well in my case. To give you ...
glaand's user avatar
  • 101
0 votes
0 answers
17 views

Classifier-Free-Guidance with Transformers

I'm working on music generation using transformers. Using the decoder part for the audio tokens with text conditioning by the T5 encoder In Classifier-Free-Guidance, the text conditioning randomly ...
qmzp's user avatar
  • 1
0 votes
0 answers
10 views

Bi-LSTM accuracy not improving at all

I am fairly new to deep learning in general and I apologize if this is a rookie mistake. I am trying to replicate a Bi-LSTM model to perform a binary classification process on extracted EEG features (...
nawzI's user avatar
  • 1
0 votes
0 answers
16 views

How Fourier Neural Operator achieves zero-shot super-resolution?

I've been delving Fourier Neural Operator recently, but one aspect continues to perplex me: the "zero-shot" super-resolution. The confusion lies in the implementation. To my understanding, ...
physics_rocks's user avatar
-1 votes
0 answers
20 views

unsupported operand type(s) for *: 'float' and 'collections.OrderedDict' error when doing network interpolation [closed]

I am trying to implement network interpolation with this script: import torch from collections import OrderedDict alpha = 0.8 net_PSNR_path = '/content/drive/MyDrive/BasicSR/experiments/...
Cemre Aldoğan's user avatar
0 votes
1 answer
29 views

Google Colab GPUs [closed]

Are there other GPUs faster than A100 or V100 in google colab? I am training a GAN network (ESRGAN) and it takes time with even using A100 GPU. Are there better options to accelerate the training ...
Cemre Aldoğan's user avatar
0 votes
0 answers
36 views

Good and free resources to learn about AI ML as a beginner (if possible, then do not suggest books)

I am a beginner at Machine Learning and AI and am having a very tough time to find good and free resources on the internet, but sadly I couldn't find any. Can someone from same tech stack could ...
Hitesh Chandra's user avatar
0 votes
1 answer
72 views

Is reinforcement learning suitable for application automation?

I have basically automatised the use of an app through the use of OCR and computer vision. So basically when a word or an image is detected it will perform a certain action. When that action is ...
zaxunobi's user avatar
  • 101
0 votes
0 answers
22 views

What’s more efficient in multihead attention: multiply QKV by $W_i$ then split or linearly project QKV $h$ times into dimensions $d_k$?

I’m looking to bridge two implementations of multihead attention. Approach 1: Multiply and Split Each of the queries, keys, and values is multiplied by a separate square weight matrix of size (...
marcocamilo's user avatar
0 votes
0 answers
26 views

Why can't we use only keys to calculate self-attention?

I was reading about the self-attention mechanism and the paper suggests to have 3 things to be computed: Key, Query and Value. As far as I understood the reason for having Value is to allow ...
Erik Nouroyan's user avatar
0 votes
0 answers
9 views

Incredibly High CrossEntropyLoss in Sequence-to-Sequence Generation

I'm trying to do SMILES chemical representation prediction from a large dataset (Around 5M Samples) to teach it do predict another downstream task. The model's part responsible for generating the data ...
Vivek Joshy's user avatar
0 votes
1 answer
18 views

What is $z|y$ in Conditional Adversarial Nets?

I am currently going through Conditional Adversarial Nets (CGANs) and the modified objective function of the two-player minimax game is stated as follows: $$\min_G \max_D V(D, G)=\mathbb E_{x\sim p_{...
insipidintegrator's user avatar
0 votes
0 answers
28 views

Segment a spectrogram into a series of images by (constant) beats per minute to train a Deep Neural Network

I have a .csv file with information about a soundtrack and it is divided into beats (per minute), which are ordered by row. As in: the index corresponds to each beat, and the columns have info about ...
Johnathan Smitherton's user avatar
-1 votes
1 answer
58 views

Can anyone please explain the Recurrent Neural Network calculation shown in the picture?

As you can see, this is a recurrent neural network. I want to understand how the calculations are being made. Please, be as detailed as possible no matter how simple or self-explanatory the ...
Syed_Hamza_Akbar_Ali's user avatar
0 votes
1 answer
23 views

Determining optimal data size for generalization in transformer encoders, particularly for Time-Series signal data

I'm currently experimenting with training a model that employs a single transformer encoder on time-series signal data. Despite having a relatively small dataset of around 50 examples, each with a ...
Kulin Patel's user avatar
0 votes
0 answers
25 views

Neural Network with Incorrect Calculation Better than Correct One

I have designed my own neural network and discovered an error. During backpropagation, instead of inserting the Z-values into the derivative of the activation function, I inserted the A-values. The ...
Apro9991's user avatar
0 votes
0 answers
26 views

No matter how I change a loss function I get it equal to infinity

I am a bioinformatician, and at the moment I am working with a dataset containing ~12.3 million mutations for ~5500 individuals. The goal is to perform binary classification. I use this framework to ...
YKY's user avatar
  • 101
8 votes
1 answer
2k views

When to use Tanh?

When and why would you not use Tanh? I just replaced ReLU with Tanh and my model trains about 2x faster, reaching 90% acc within 500 steps. While using ReLU it reached 90% acc in >1000 training ...
vxnuaj's user avatar
  • 83
0 votes
0 answers
20 views

Preparation of multivariate time series data

I am doing a university project on index/stock price prediction. I plan to use a combined cnn-lstm model, and I have several different types of data: Open High Low Close Volume, values, fundamental ...
Ivan's user avatar
  • 1
0 votes
0 answers
7 views

Is it possible to apply transfer learning between Temporal Fusion Transformer and sequential architecture LSTM and GRU

If TFT is a pretrained model, is it possible to transfer the weights to sequential neural network models like LSTM,BILSTM and GRU.
Santhana Lakshmi's user avatar
1 vote
0 answers
21 views

Why completely two different algorithms are being used in Deep Q Learning?

I'm a new student in reinforcement learning. Recently, I've been studying about different algorithms of RL. But I'm quite surprized that there are some algorithms which are named as "same" ...
Jahid Chowdhury Choton's user avatar
0 votes
0 answers
11 views

Is it possible to have lower ECE but worse reliable curve?

I am new to the calibration concept for classification. I have tried temperature scaling on my model's results. However, after applying temperature scaling, the reliable curve got worse despite ...
Sara's user avatar
  • 1
2 votes
1 answer
55 views

How is a LLM able to override its prior knowledge through In-Context Learning?

I came across a Google's blog (https://research.google/blog/larger-language-models-do-in-context-learning-differently/) discussing large language models (LLMs) and how we can overried their prior ...
tidealwari91's user avatar
0 votes
0 answers
22 views

CNN model configuration: advice

Assume that a CNN model is to be developed to recognize commercial domestic planes flying in the sky. The training data should include images of flying domestic planes for true positives. Additionally,...
KM23's user avatar
  • 1
0 votes
0 answers
57 views

UNets with a pretrained network as the encoder portion of U-Net

UNets with a pretrained network (like VGG16 or InceptionV3 or ResNet, or …) as the encoder portion of U-Net are common. However I'm struggling to understand how the 1D encoded second-to-last layer is ...
FluidMechanics Potential Flows's user avatar
0 votes
0 answers
13 views

AI service to extract text and train model based on user input if extracted text is incorrect

I am working with highly unstructured legal documents like contracts,lease or rental documents and trying to extract data and incorporate feedback loop so that the human corrections can be fed in and ...
user82659's user avatar
0 votes
1 answer
34 views

How to get Complexity per Layer, Sequential Operations and Maximum Path Length in CNN architecture?

In the paper Attention is all you need, here is Table 1, can someone explain what architecture is referred to in the "Convolution" row and hence describe the other 3 columns in it? The other ...
Harry's user avatar
  • 11
0 votes
0 answers
8 views

How to do object detection for 1 object in the image, with 3 possible classes (in a custom dataset)

I am new to deep learning, I hope you will lead me because I have been stuck for a week. I am trying to build a model for identifying a single object in the image. So, I made my custom dataset, which ...
Lisa Mck's user avatar
0 votes
0 answers
32 views

How to train continuous probability distributions as output from a neural network?

I'm training time series models on numerical forecasting, and I'm seeing inherent difficulty in modeling the uncertainty of the values. Time series forecasting generally has a pattern of uncertainty ...
TheEnvironmentalist's user avatar
0 votes
0 answers
36 views

Realtime cuboid vs cylinder classification of a 2D mask / object from a 3D scene?

Most realtime SOTA segmentation/detection model can reliably segment an object from a 2D input, and I can get the contours/polylines describing its edges in realtime. By realtime, I'll consider ...
Filip Dimitrovski's user avatar
0 votes
0 answers
19 views

1D CNN with Single vs. Two Channels for Number Image Recognition

I am taking images of numbers as input, in a convolutional neural network and building a model to predict the number. In particular, I am building a one dimensional convolutional neural network with ...
Ling Guo's user avatar
  • 121
0 votes
0 answers
9 views

Can you iteratively freeze and unfreeze parts of a neural network for efficient training?

I know you can do efficient training by freezing parts of a NN, but is there any work done where part 1 of a NN is frozen and part 2 is trained, and then part 2 is frozen and part 1 is trained?
JobHunter69's user avatar
0 votes
0 answers
23 views

Why the architeture of Resnet18 is suitable to images classification?

I am studying convolutional networks and in particular I have focused on the ResNet18 network. I've been studying ResNet18 and understand the purpose of skip connections and residual network. However,...
Domme's user avatar
  • 1
1 vote
1 answer
109 views

why we use learnable positional encoding instead of Sinusoidal positional encoding

In the original paper of transformers they using positional encoding to capture the position of each word in the sentence and for calculate that it using sin and cos ,like shom in the image. In Bert ...
LAILA EL OUEDEGHYRY's user avatar
0 votes
0 answers
18 views

Train a network to find the most sharp image from a group of images

I'm new to deep learning, and I am trying to find a solution to the below scenario, any suggestions about the keywords would be greatly appreciated. I have a data set with groups of images. In each ...
Sherry's user avatar
  • 1
0 votes
0 answers
18 views

Why does SSIM in pytorch-mssim need the data range to be specified?

The SSIM metric (https://en.wikipedia.org/wiki/Structural_similarity_index_measure) formulas do not seem to depend on the range of the values the pixel have (from 0 to 1, from 0 to 255, or any other ...
FluidMechanics Potential Flows's user avatar
1 vote
0 answers
18 views

Compare two songs content using Audio Spectogram Transformer

I'm trying to establish a similarity metric between two songs. To do this I'm using the AST model on HuggingFace. This model basically works in a way very similar to a ViT but applied to spectograms ...
user491880's user avatar
0 votes
2 answers
127 views

How the Q,K,V be calculated in multi-head attention

I want to understand the transformer architecture, so I start with self attention and I understand their mechanism, but when I pass to the multi-head attention I find some difficulties like how ...
LAILA EL OUEDEGHYRY's user avatar
0 votes
0 answers
23 views

Policy gradient - future looking returns

In the policy gradient approach, one differentiates the expected reward $$ \mathbb{E}J=\sum P(\tau;\theta) R(\tau) $$ to obtain $$ \Sigma R(\tau) \nabla \log P(\tau;\theta) $$ (with some abuse of ...
Eli's user avatar
  • 1
0 votes
0 answers
23 views

How to Optimize model for faster Training

Below is the forward pass of my model. The input x is split about time-dimension (last-dim) which has indices till 250. Below is the code... ...
Sarvagya Porwal's user avatar
0 votes
0 answers
13 views

Optuna Hyperband Algorithm Not Following Expected Model Training Scheme

I have observed an issue while using the Hyperband algorithm in Optuna. According to the Hyperband algorithm, when min_resources = 5, max_resources = 20, and reduction_factor = 2, the search should ...
Tnb Marketplace's user avatar
0 votes
0 answers
23 views

Influence of Unused FFN on Model Accuracy in PyTorch

I am encountering a peculiar issue with my PyTorch model where the presence of an initialized but unused FeedForward Network (FFN) affects the model's accuracy. Specifically, when the FFN is ...
Riya's user avatar
  • 1
0 votes
0 answers
35 views

Yolov8 object detection model visualization in Netron

I trained a YOLOv8 nano detection model with imgsz=320 and 36 classes. When I conver it to ONNX, I get the following message: ...
Mary H's user avatar
  • 1
1 vote
0 answers
38 views

History of Neural Networks and Deep Learning

I'm interested in learning about the history of neural networks and deep learning. I've been reading about the field and am familiar with many of the developments since the 1950s. Is there a textbook,...
neuralode's user avatar
0 votes
1 answer
42 views

How do I input multi-channel Numpy array to U-net for semantic segmentation

I had lidar 3D point cloud data from semantckitti. I want to perform Semantic Segmentation on the data using U-Net. I converted the 3d point cloud data into 2D using spherical conversion and saved the ...
Leibniz 24's user avatar
0 votes
0 answers
22 views

What are some good resources to understand the code for 3D Gaussian Splatting?

I am looking for some good resources like videos or blogs (or other githubs!) that go through the code of Gaussian Splatting and explain the major components and how they are working. Haven't found ...
ChaoS Adm's user avatar
  • 101
0 votes
0 answers
32 views

Why doesn't my toy transformer model "grok"?

I'm working on reproducing the results by Neel Nanda on teaching a small transformer to perform modular addition: (operand_1+operand_2)%mod_value. The expectation ...
LawlessWalrus's user avatar
0 votes
0 answers
21 views

Is this right logic flow of Variational Autoencoder?

Object Function $$ maximize = \Pi_d^D P(X_d) = \Pi_d^D \sum_i^N P(X_d,Z_i) $$ D : # of data N : # of latent variable state $$ NLL = - \sum_d^D \log \sum_i^N P(X_d,Z_i) = - \sum_d^D \log \sum_i^N P(X_d,...
Shin Joong Hyun's user avatar

1
2 3 4 5
41