Skip to main content

Questions tagged [memory]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

How is the complexity of the chunked attention computation in "Self Attention Does Not Need O(n2) Memory" independent from the query chunks size?

In Self-attention Does Not Need O(n{2}) Memory the authors present a say to have a constant memory complexity attention algorithm that is sequential in nature and also present an implementation that ...
Daviiid's user avatar
  • 575
0 votes
1 answer
29 views

How to estimate Time vs Memory trade-off prior to modelling

It is often the case when the time vs memory trade-off is underestimated prior to using ML/DL for solving a particular task. Taking into account the type, size and format of the available data and ...
Deyan's user avatar
  • 1
1 vote
2 answers
55 views

What is the Essence of Memory of ANNs and Brain?

I am learning Computational Neuroscience. If I do reasoning in my mind, especially Classifying an object and Deducing to figure out its qualities, I have to memorize the category and the category's ...
Forest Liu's user avatar
1 vote
0 answers
82 views

Inference process and flow, and role of GPU, CPU, and RAM [closed]

This is a noob question. I load a HuggingFace transformer model into GPU and create a HuggingFace pipeline using that model. Then I run inference on the model using the pipeline. I would be glad to ...
ahron's user avatar
  • 181
2 votes
1 answer
185 views

How does the memory augmented neural network work, and how to make a simple implementation?

How does the memory augmented neural network (MANN) work? How can I make a simple MANN with a vanilla neural network especially without a recurrent network?
Eka's user avatar
  • 1,066
2 votes
0 answers
2k views

How compressed is Chat GPT data?

I originally thought the Chat GPT was a neural network model in which the input data was highly compressed such that the input data could not be got out again. But then I asked Chat GPT "Write ...
zooby's user avatar
  • 2,216
0 votes
1 answer
109 views

Is batch size of 1 a valid choice for a very deep neural network with high memory requirement?

I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch ...
A_C's user avatar
  • 111
0 votes
1 answer
132 views

What is the bit memory task?

I learned from this post about the so-called bit memory: They froze its self-attention and feed-forward layers and, in separate copies, fine-tuned peripheral layers on each on a wide range of tasks: ...
Lerner Zhang's user avatar
4 votes
3 answers
9k views

How to calculate the GPU memory need to run a deep learning network?

In general, how do I calculate the GPU memory need to run a deep learning network? I'm asking this question because my training for some network configuration is getting out of memory. If the ...
Adolfo Graciano Da Silva's user avatar
4 votes
0 answers
81 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
Eka's user avatar
  • 1,066
1 vote
0 answers
59 views

How much Physical memory does Alpha Zero's Neural Net require?

I appreciate that there are many ways to arrange the memory in a NN, and that the numerical representations may be with bytes to floats depending on an implementation. What is typical amount of ...
Gary Rondeau's user avatar
1 vote
2 answers
313 views

How to manage large amounts of image data for training?

Right now, I am trying to synthesize training images for a CNN and due to the nature of the application, there is a finite number of sample images to learn from. From other research, I expect to be ...
Mitrimo's user avatar
  • 11
2 votes
1 answer
85 views

Does a data structure that models the encoding specificity principle in memory exist?

I find myself wondering if there exists a data structure with the following properties: Stores information Conforms to the encoding specificity principle
donkey's user avatar
  • 145
2 votes
1 answer
91 views

How can a specific connectivity pattern be stored in an optimally compact representation?

I am interested in optimizing the memory capacity of an AGI. Given a specific complex input an AI can create a simplified model. This is a problem that can be solved using sparse coding [1]. However, ...
noumenal's user avatar
  • 137