Questions tagged [memory]
The memory tag has no usage guidance.
13
questions
0
votes
2
answers
42
views
What is the Essence of Memory of ANNs and Brain?
I am learning Computational Neuroscience. If I do reasoning in my mind, especially Classifying an object and Deducing to figure out its qualities, I have to memorize the category and the category's ...
1
vote
0
answers
68
views
Inference process and flow, and role of GPU, CPU, and RAM [closed]
This is a noob question.
I load a HuggingFace transformer model into GPU and create a HuggingFace pipeline using that model. Then I run inference on the model using the pipeline.
I would be glad to ...
- 161
0
votes
0
answers
158
views
Does OpenAI ChatGPT has any "dialogue state tracking" besides max tokens to include previous conversation history?
When I chat with chatGPT earlier, chatGPT itself mentioned "dialogue state tracking" technique that is utilized in addition to chat completion with previous conversation history(up to max ...
- 101
1
vote
1
answer
111
views
How does the memory augmented neural network work, and how to make a simple implementation?
How does the memory augmented neural network (MANN) work? How can I make a simple MANN with a vanilla neural network especially without a recurrent network?
- 1,036
2
votes
0
answers
2k
views
How compressed is Chat GPT data?
I originally thought the Chat GPT was a neural network model in which the input data was highly compressed such that the input data could not be got out again.
But then I asked Chat GPT "Write ...
- 2,186
0
votes
1
answer
73
views
Is batch size of 1 a valid choice for a very deep neural network with high memory requirement?
I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch ...
- 111
0
votes
1
answer
113
views
What is the bit memory task?
I learned from this post about the so-called bit memory:
They froze its self-attention and feed-forward layers and, in separate copies, fine-tuned peripheral layers on each on a wide range of tasks: ...
- 897
3
votes
3
answers
8k
views
How to calculate the GPU memory need to run a deep learning network?
In general, how do I calculate the GPU memory need to run a deep learning network?
I'm asking this question because my training for some network configuration is getting out of memory.
If the ...
3
votes
0
answers
78
views
How does the memory mechanism (reading and writing) work in a neural Turing machine?
In neural Turing machine (NTM), reading memory is represented as
\begin{align}
r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2}
\end{align}
and writing to memory is represented as
...
- 1,036
1
vote
0
answers
54
views
How much Physical memory does Alpha Zero's Neural Net require?
I appreciate that there are many ways to arrange the memory in a NN, and that the numerical representations may be with bytes to floats depending on an implementation. What is typical amount of ...
- 19
1
vote
2
answers
275
views
How to manage large amounts of image data for training?
Right now, I am trying to synthesize training images for a CNN and due to the nature of the application, there is a finite number of sample images to learn from.
From other research, I expect to be ...
- 11
2
votes
1
answer
84
views
Does a data structure that models the encoding specificity principle in memory exist?
I find myself wondering if there exists a data structure with the following properties:
Stores information
Conforms to the encoding specificity principle
- 145
1
vote
1
answer
81
views
How can a specific connectivity pattern be stored in an optimally compact representation?
I am interested in optimizing the memory capacity of an AGI. Given a specific complex input an AI can create a simplified model. This is a problem that can be solved using sparse coding [1]. However, ...
- 127