All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
37 views

What are the update equations for Double Expected Sarsa with an $\epsilon$-greedy target policy?

This is question 6.13 in Sutton-Barto,page 136. What are the update equations for Double Expected Sarsa with an $\epsilon$-greedy target policy? The answer is given as follows: Let $Q_1$ and $Q_2$ ...
DSPinfinity's user avatar
0 votes
2 answers
32 views

Can't the loss function be used as a fitness function?

Can't the loss function used in backpropagation be used as a fitness function in an evolutionary algorithm?
Display name's user avatar
0 votes
0 answers
10 views

Why people always use l2 loss in Neural Tangent Kernel and other neural network theory?

What if we use l1 loss? I attempt to use NTK to get the convergence rate of a NN. Here is the original l2 loss version: https://rajatvd.github.io/NTK/. When I relpace it to l1 loss, I find the ...
orbin Lee's user avatar
-1 votes
0 answers
17 views

How are experienced and learned situations being stored? [closed]

Memorized events or experienced situations can really vary. The fact is that computer memory is a shadow of human intelligence. Therefore, humans could not disclosure human-beings' brain completely, ...
cell's user avatar
  • 1
-1 votes
1 answer
74 views

Violation of Markov property

Consider the following cases: a-) In solving an episodic problem we observe that all trajectories from the start state to the goal state pass through a particular state exactly twice. b-) In solving ...
DSPinfinity's user avatar
0 votes
1 answer
43 views

Confusing statement in Sutton-Barto on trajectory sampling

Suuton-Barto, page 176: experiment to assess the effect empirically. To isolate the e↵ect of the update distribution, we used entirely one-step expected tabular updates, as defined by (8.1). In the ...
DSPinfinity's user avatar
0 votes
0 answers
32 views

Structure and parameters of modern AI-baed chess engines

I understand some recent chess engines (like alphazero or muzero) are based on neural networks. This question is not specific to chess, any other game (e.g. go) would do, but I keep chess for ...
Manu's user avatar
  • 101
3 votes
1 answer
186 views

Can we implement a memory in a REINFORCE algorithm for RL?

In Q-learning with function approximators such as Neural Networks, we typically implement a memory so that at the end of each episode we also train on past experiences. This is typically fine because ...
FluidMechanics Potential Flows's user avatar
1 vote
1 answer
58 views

Why do we need complex neural network designs?

If we have a sufficient number of columns (features) and a sufficient number of rows (length or volume) in a dataset that can describe a system without redundancy, we can even train a simple MLP ...
user366312's user avatar
0 votes
0 answers
30 views

Confusing statement in Sutton-Barto on expected versus sample updates

Sutton-Barto, page 174. b successor states are equally likely and in which the error in the initial estimate is 1. The values at the next states are assumed correct, so the expected update reduces ...
DSPinfinity's user avatar
-1 votes
1 answer
62 views

Can anyone please explain the Recurrent Neural Network calculation shown in the picture?

As you can see, this is a recurrent neural network. I want to understand how the calculations are being made. Please, be as detailed as possible no matter how simple or self-explanatory the ...
Syed_Hamza_Akbar_Ali's user avatar
0 votes
0 answers
15 views

What is the main way of using ns3-ai to develop DQN algorithms? How does python interact with NS3 in this case?

0 I am currently researching the topic of using ML algorithms for Rate Adaptation (RAA) in wireless networks, specifically 802.11ax. My research objective is to demonstrate the capability and ...
Husain's user avatar
  • 1
2 votes
1 answer
60 views

Are there LLMs that are allowed time to contemplate freely?

We can call it "time to think", or "free contemplation", or whatever. Maybe a term already exists in the field? What I mean is - with so many LLMs being "trained" on ...
Mentalist's user avatar
  • 121
0 votes
0 answers
16 views

How Does AI-Generated Content Differ From Human-Written Content?

How Can I Determine if a Piece of Content Has Been Generated by AI?
Micky E.'s user avatar
0 votes
0 answers
11 views

Is it Possible to fit a sklearn model using a excel spreadsheet

hi i am new to this community, but i came up with this curiosity as i am having an embedded board to train a sklearn i have nvidia jetson tx2 with 256 nvidia cuda cores so inorder to take less load in ...
Ashok's user avatar
  • 1
0 votes
0 answers
14 views

Reinforcement Learning Gymnasium ValueError

I am testing out reinforcement learning for the first time with gymnasium. I am following a youtube tutorial. I am getting the following error when I run the training loop: ValueError: setting an ...
Sheila's user avatar
  • 1
0 votes
1 answer
30 views

Sparse Cross Entropy

I've been attempting to mess around with Sparse Categorical Cross Entropy Loss for the MNIST dataset. I can't seem to figure out what might be wrong with my implementation, the loss seems to ...
vxnuaj's user avatar
  • 93
0 votes
0 answers
21 views

Q-Learning conditions for convergence and ergodicity

Q-learning is guaranteed to convergence if the learning rate satisfies the Robbins-Monro conditions and if every state-action pair is visited infinitely often. Regarding the latter, does it mean that ...
Simon's user avatar
  • 153
0 votes
1 answer
20 views

Using additional parameters in PDDL precondition and effects of actions

Consider the following example: ...
Karl 17302's user avatar
0 votes
1 answer
31 views

Determining optimal data size for generalization in transformer encoders, particularly for Time-Series signal data

I'm currently experimenting with training a model that employs a single transformer encoder on time-series signal data. Despite having a relatively small dataset of around 50 examples, each with a ...
Kulin Patel's user avatar
0 votes
0 answers
26 views

Neural Network with Incorrect Calculation Better than Correct One

I have designed my own neural network and discovered an error. During backpropagation, instead of inserting the Z-values into the derivative of the activation function, I inserted the A-values. The ...
Apro9991's user avatar
0 votes
0 answers
25 views

Minimum remaining values heuristic more powerful than degree heuristic

When selecting a new variable to assign a value to in CSP assignment backtracking search, common heuristics include: MRV heuristic: pick the unassigned variable with the fewest remaining values in ...
hegash's user avatar
  • 101
2 votes
1 answer
70 views

Why fine tuning does not work as well as RAG?

I cannot find a definite answer to this question. Suppose I want to build a QA (question answering) system on a set of personal documents. It looks that RAG (retrieval augmented generation) is the way ...
Thomas's user avatar
  • 193
0 votes
0 answers
28 views

No matter how I change a loss function I get it equal to infinity

I am a bioinformatician, and at the moment I am working with a dataset containing ~12.3 million mutations for ~5500 individuals. The goal is to perform binary classification. I use this framework to ...
YKY's user avatar
  • 101
0 votes
0 answers
62 views

Any reasons LSTM does not pick up any patterns?

I'm trying to teach an LSTM to predict the next values in 3 related series. (Financial data) Unfortunately, it looks like I made some basic mistake and this network never gets past just returning ...
viraptor's user avatar
  • 111
0 votes
0 answers
35 views

Optimizing a nonlinear objective function in Deep Reinforcement Learning

I'm working on a reinforcement learning problem where the environment returns a reward pair $(r_{t+1}^{(a)}, r_{t+1}^{(b)})$. The goal is to maximize the following nonlinear objective function. $$ E[\...
Alex's user avatar
  • 1
2 votes
1 answer
66 views

What should be Relationship between embedding dimension and context length?

What should we keep hidden dimension/embedding dimension (d_model as per attention is all you need paper), greater, equal, or smaller to the context length (n)? Is there any such relationship between ...
Aamod Thakur's user avatar
0 votes
0 answers
10 views

Calculating prediction confidence from a sequence of token-level confidences

I am working with OCSR (optical chemical structure recognition) models, and they output a sequence of token-level confidences. I am looking for a method of summarising these token-level confidences ...
finlay morrison's user avatar
0 votes
0 answers
30 views

Is a single hour enough for Google AutoML training 1000 images with single objects in it?

I am a newbie AI practitioner. Please don't be too harsh. I'm currently looking at the pricing for AutoML for Google Vision Vertex AI. The pricing is in per hour. Let's say I have 1000 image files. ...
Abel Callejo's user avatar
0 votes
0 answers
30 views

can Vision transformers be used to retain the relevant features (drop unrelated features from the clutter in image) and map to the specific query

Background, I have good understanding of ML 101 (supervised, unsupervised, tensorflow etc), however just getting into transformers & gen-AI. I have recently started looking into Transformers/ViT ...
cyborgt8's user avatar
0 votes
2 answers
70 views

Why is dynamic programming an example of planning?

Sutton-Barto, page 160, towards bottom: Why is dynamic programming an example of planning? There is no simulation in dynamic programming.
DSPinfinity's user avatar
0 votes
0 answers
12 views

Is this a typo in n-step tree backup section in Sutton-Barto?

Sutton-Barto, page 153. Should not it be $t<T-n$ in Eq.16? The reason is we have $t<T-1$ and $t<T-2$ for the 1 and 2 step returns, respectively.
DSPinfinity's user avatar
9 votes
1 answer
2k views

When to use Tanh?

When and why would you not use Tanh? I just replaced ReLU with Tanh and my model trains about 2x faster, reaching 90% acc within 500 steps. While using ReLU it reached 90% acc in >1000 training ...
vxnuaj's user avatar
  • 93
1 vote
1 answer
21 views

Why is importance sampling ratio in n-step TD multiplying error rather than return n-step return?

Why is importance sampling ratio in n-step TD is multiplying error rather than return? In Monte Carlo methods for state values, importance sampling ratio was simply a multiplier for the return.
DSPinfinity's user avatar
0 votes
0 answers
22 views

Preparation of multivariate time series data

I am doing a university project on index/stock price prediction. I plan to use a combined cnn-lstm model, and I have several different types of data: Open High Low Close Volume, values, fundamental ...
Ivan's user avatar
  • 1
0 votes
0 answers
19 views

How to design a custom openai gym environment to carry out 5G resource slicing?

PROBLEM AT HAND: I have a resource (Bandwidth) of B Hz. I have to distribute the bandwidth B to users as per their requirements. For instance, voice calls would require some amount of bandwidth while ...
ANWESA ROY's user avatar
0 votes
0 answers
12 views

Simple CNN without pooling layers, differentiability

I need to be able to produce a convolutional neural network where I am able to extract the gradients of the output pixels with respect to the input pixels. I am not really sure if I can do this using ...
James Li's user avatar
0 votes
0 answers
15 views

DDPG Not Learning

I'm looking for some help related to my DDPG implementation. My code runs and correctly learns the pendulum gymnasium environment however trying it on any other environment has not worked. I've left ...
jhunter533's user avatar
0 votes
0 answers
18 views

What machine learning algorithms are used in demand forecasting?

What are the most commonly used machine learning algorithms used in demand forecasting?
Mika's user avatar
  • 341
0 votes
0 answers
10 views

Chernoff bound for sampling flipped bits and arriving at true average

The Basic Setting Let there be an $n$-bit string generated by $n$ successive trials of random variable $X$, name the string $x_1,..,x_n$, with $x_i \in \{0,1\}$. Define sample mean as $\bar{X} = \...
xiaolingxiao's user avatar
0 votes
0 answers
28 views

Shape mismatch error in a multi-input model with Keras API

My intention is to have a classification model where different types of input could have individual hidden layers before being concatenated downstream. I get the error (below) stating that my input ...
Art's user avatar
  • 1
1 vote
0 answers
20 views

Can a transformer detect same object with different sizes?

Suppose a vision transformer has trained to detect this cat picture Next we show it another picture of a zoomed-in cat (taken from the same image) and asked it to identify the picture The linearized ...
James's user avatar
  • 157
0 votes
0 answers
17 views

why CAM does not use ReLU, unlike Grad-CAM?

I learned that Grad-CAM uses ReLU becuase it is only interested in positive incluence on the class of interest. Then I think ReLU can also be used to CAM. But why they don't use ReLU?
COTHE's user avatar
  • 11
0 votes
0 answers
11 views

Find the pixel sensitivity of a Convolutional Neural Network

So assume I have a simple convolutional neural network with 2 or 3 convolutional layers, a pooling layer, and whatever else is typical. The network takes in an image and tries to produce an image that ...
R S's user avatar
  • 1
0 votes
2 answers
26 views

Confusing points in Dyna-Q in Sutton-Barto about model, simulated experience and model predictions

Sutton-Barto, page 164: In the pseudocode algorithm for Dyna-Q in the box below, Model(s, a) denotes the contents of the model (predicted next state and reward) for state–action pair (s, a). Direct ...
DSPinfinity's user avatar
0 votes
0 answers
8 views

Is it possible to apply transfer learning between Temporal Fusion Transformer and sequential architecture LSTM and GRU

If TFT is a pretrained model, is it possible to transfer the weights to sequential neural network models like LSTM,BILSTM and GRU.
Santhana Lakshmi's user avatar
0 votes
1 answer
19 views

Unclear points in Dyna Maze example in Sutton-Barto

Sutton-Barto, page 164: The main part of Figure 8.2 shows average learning curves from an experiment in which Dyna-Q agents were applied to the maze task. The initial action values were zero, the ...
DSPinfinity's user avatar
0 votes
1 answer
17 views

Confusing point point in Dyna-Q

Sutton-Barto, page 164: In the (f) loop, $S,A, S', R$ are from real experience (Model(S,A)=(R, S') where R and S' are also real expereince). This experience is used in direct RL part in (d). Why is ...
DSPinfinity's user avatar
0 votes
1 answer
20 views

Unclear arrow in general Dyna architecture

Sutton-Barto, page 163: Figure 8.1: The general Dyna Architecture. Real experience, passing back and forth between the environment and the policy, affects policy and value functions in much the same ...
DSPinfinity's user avatar
0 votes
2 answers
57 views

Does topology have a place in ai

Basic knowledge of topology How it relates to ai Does ai need to understand or integrate topology Can ai provide useful answers to hard topological problems
Raphael Pallais's user avatar

15 30 50 per page