Skip to main content
8 votes
Accepted

While fine-tuning a decoder only LLM like LLaMA on chat dataset, what kind of padding should one use?

I got an answer to this question, probably a correct explanation. In decoder-only model architectures, the output of the model is a continuation of the model input. For example, input: I love apple [...
尹雅博's user avatar
4 votes
Accepted

Why in Multi-Head Attention implementation should we use $3$ linear layers for Q, K, V instead of $3 * h$ layers?

It is just an optimization technique. If you have a vector $x$ of size $d$ and you want to multiply with $n$ different matrices $W_i$ of shape $d \times d_k$, then you could simply stack these ...
pi-tau's user avatar
  • 815
2 votes

Why is the output of my graph neural network not permutation equivariant?

It seems that the graph filter layer in your GNN takes information from the immediate neighbors for every node and applies a ReLU nonlinearity. However, the architecture you've shown does not ...
michael williams's user avatar
2 votes

Super Resolution CNN generates black dots on output images

This is a while ago, but still this problem might occur to someone. I encountered the same problem and found that the reason was how the resulting tensors are transformed to images. It seams that ...
Ingmar Ludwig's user avatar
2 votes

I'm trying to understand the use model for different Python libraries

In short: Gym is complementary (but optional) to both tensorflow and pytorch. Stable-baselines (be sure to check version 3, which is based on Pytorch) supports gym natively (if I remember correctly.) ...
Luca Anzalone's user avatar
2 votes

I'm trying to understand the use model for different Python libraries

Your choices here are not really any different to choosing an open source library for any other purpose. Each library will have its own idiosyncratic parts, but usually these are minor things compared ...
Neil Slater's user avatar
  • 32.7k
2 votes
Accepted

Why Does the Model not Improve in PyTorch?

For whom this post and reply might be useful, If the model is examined carefully, the ReLU activation function at the end before the output neurons is what causes ...
Burak Karaosmanoğlu's user avatar
2 votes

Is it possible to reconstruct convolutional layers' input using transposed convolution?

Alexander has some great explanations above. After doing some more research myself I came up with some understandings as well. One interpretation of transposed convolution is that it can be seen as ...
Shawn Li's user avatar
  • 143
2 votes
Accepted

Is it possible to reconstruct convolutional layers' input using transposed convolution?

You're not able reconstruct convolutional layers' inputs using transposed convolutions (in most cases). The term invert is a bit confusing here -- I interpret this to mean inverting the space of ...
Alexander Wan's user avatar
2 votes

Filling replay buffer with expert trajectories for PPO/DQN

PPO is an on-policy algorithm so you must use trajectories generated by the current policy. DQN is an off-policy algorithm, so you could add these trajectories to the buffer, but you also need "...
pi-tau's user avatar
  • 815
2 votes

add a layer in deep learning model pytorch

From your code, I think you are attempting to apply an attention mechanism to your model using the torch framework. the issue you are facing seems to be due to the way you are applying the softmax ...
Keval's user avatar
  • 111
2 votes
Accepted

Is there any advantage to providing multi-dimensional input to torch modules?

Yes, for some applications, the spatial component of the tensor is indeed very important, but not for the examples you mentioned. The first point to clarify is that even though the ...
Cesar Ruiz's user avatar
2 votes

Neural network for specific numbers from a range (Q learning)

def action(self, state): if np.random.rand() > self.epsilon: return np.random.randint(0,4) return np.argmax(self.model.predict(state)) You do ...
foreverska's user avatar
  • 1,298
2 votes
Accepted

Neural network for specific numbers from a range (Q learning)

I am not able to understand how can I tell the neural network that it can take only one of the following values You don't have to. The neural network in Deep Q Learning (the DQN) is not configured to ...
Neil Slater's user avatar
  • 32.7k
2 votes
Accepted

How do I code so that the embedding output and input share the same weight matrices?

There are three weight matrices that can be shared: The weights of the embedding layer for the encoder -- that is the layer that embeds the source sequence before forwarding through the transformer ...
pi-tau's user avatar
  • 815
2 votes
Accepted

Why does the latent space in Stable Diffusion have a shape of 64x64x3?

There's not really a restriction on the shape for variational autoencoders. If you really wanted a 1D vector, you could just flatten the matrix and get a vector of size ...
Alexander Wan's user avatar
2 votes
Accepted

Batch wise Inference to speed up Muzero's MCTS

Batching: A Good Idea You're right, batching is a great way to speed up AlphaZero or MuZero self-play! Your proposed solution of running multiple games in parallel is the easiest way to achieve some ...
KarelPeeters's user avatar
1 vote
Accepted

Does 1-bit quantization (layers with boolean tensors) machine learning exist?

1-bit quantization does exist, at least at the inference stage: a common approach is to constrain weights (and sometimes activations) to be -1 or +1. I'd recommend this survey paper for a good ...
Alexander Wan's user avatar
1 vote
Accepted

Does ResNext split data or copy it?

I realized that groups in PyTorch works along the same dimension as channels, I thought it worked along the same dimension as the data. I believe the GitHub example does run the same data through each ...
eop3's user avatar
  • 11
1 vote

How to Create a 1D Embedding from Tensors of Varying Sizes?

If you're mapping from a higher dimension to a smaller dimension, you're almost always going to be losing data. The question is how to decide which data you want to keep. This all is highly domain ...
Alexander Wan's user avatar
1 vote
Accepted

3D Unet gives "output size is too small" error

Why are you using 3D convolution/pooling if you input a 2D image ? The expected input size should be BxCxDxHxW for a 3D convolution, not BxCxHxW . In your case, C=1 because you said you only have one ...
Lelouch's user avatar
  • 216
1 vote

Multiple Loss Functions For Proper Parameter Updates

For your use case, you want to update the model with a single .backward() call to a combined loss, similar to what you're doing in version 3. However, version 3 ...
Karl's user avatar
  • 206
1 vote

What is GNN Cheatsheet in PyG Docs

Regarding "what the purpose of this is as it shows a table and there are tick marks if checked.", it sounds like that the tick marks indicating convolution operation support. This means you ...
Cloud Cho's user avatar
  • 181
1 vote

How to transform a loss function into a score function?

I suspect you could be reinventing the wheel a little bit here, because some of the description of what this task is for appears to match to a Reinforcement Learning (RL) scenario. You may find if you ...
Neil Slater's user avatar
  • 32.7k
1 vote

While fine-tuning a decoder only LLM like LLaMA on chat dataset, what kind of padding should one use?

Decoder only text LLMs are autoregressive. Like any autoregressive function if you do not left pad there will be no prior token to your intial token for efficient gradient descent. In most text ...
Sam Marvasti's user avatar
1 vote

How to overcome symmetry in the solution space when learning a simple neural network?

learning may eventually stall due to the sum of the gradients over the network inputs converging to a tiny sum even though the individual gradients for each input to the network separately stay ...
Alberto's user avatar
  • 2,293
1 vote

LSTM text classifier shows unexpected cyclical pattern in loss

This weird pattern can be caused by a big learning rate. Check this: https://stackoverflow.com/a/49095437/13164928
ZappaBoy's user avatar
1 vote

How can an MLP be implemented with convolutional layers?

When I try to understand convolutional layer, I always look at the shape of filters. I try to stack matrices in my mind and visualize the convolution. If we consider that each point has 3 features (x, ...
Mihnea Aleman's user avatar
1 vote
Accepted

2D convolution with channels versus 3D convolution for layers of a map?

First of all, I don't think that the two approaches are the same as @lev1248 claims. When using a 3D convolution the 3d filters have depth equal to kernel_size and ...
pi-tau's user avatar
  • 815
1 vote

Super Resolution CNN generates black dots on output images

Great that a solution was found (clamp larger-than-one pixels' brightness before showing the image). But I suggest that you either add a sigmoid activation, or clamp the network's output directly from ...
NikoNyrh's user avatar
  • 787

Only top scored, non community-wiki answers of a minimum length are eligible