# Tag Info

4

The choice of the batch size to be a power of 2 is not due the quality of predictions . The larger the batch_size is - the better is the estimate of the gradient, but a noise can be beneficial to escape local minima. However, there won't be much difference in optimization procedure for batch_size=61 and batch_size=64, since the amount of stochasticity would ...

2

Do they mean the strides that are related to the CNN, pooling, etc., or are they referring to any other stride information? The stride referred to by the quote "only plays with the size and stride information at the tensor level" is referring to internal storage of tensors. Luckily in most normal conversations about AI logic you do not care about ...

2

Let me try to explain here. Usually, we calculate the variance by subtracting the mean term and then square it. But here mean (first-moment m_t) is fluctuating like anything at each time "t" and is getting calculated with the influence of past mean as well, also with the influence of beta_1. So when the 2nd-moment term v_t is getting calculated ...

2

How about dividing the problem? You can first train a classification model that predicts the type of function (linear or exponential). Then you can use your seperately trained nn depending on the classification output. P.S. I'm not sure why you would use a neural network for this problem. Fitting a linear/exponential function seems to be a relatively simple ...

2

Since sport commentaries are a fairly restricted domain, and the language does not vary much, I would go for a canned text approach. Analyse what kind of events you get, and what variables you're dealing with. Then write some template sentences with placeholders for the variables. The more you write for the same data, the more varied your text will be. You ...

1

I have two suggestions that you can look into. Based on my own work in RL, I believe the first one will require less work to implement. If the observability of the environment is not an issue, then you could give the agent a relative measure (distance to the goal) as part of the observation to provide it with knowledge of how far away it is. You can also ...

1

It could be because there is simply not enough data for the late game. To make the model give more importance to the later stages of the game you can try to tweak the loss function such that it penalizes more for when there are fewer pieces on the board. ( This might give an idea on that: https://medium.com/visionwizard/understanding-focal-loss-a-quick-read-...

1

For deep learning models, embedding vectors have become the standard way of encoding text features almost immediately after their introduction. The reason for this is that neural networks work with data encoded with continuous values ranging from 0 to 1 (or sometimes from -1 to 1). Bag of Words and TF-IDF can be modified to produce values in this range, but ...

1

Recall is the fraction of the relevant documents that are successfully retrieved. \begin{aligned}{\text{Recall}}&={\frac {tp}{tp+fn}}\,\end{aligned} Labels for a Class is equal to total examples which are actually belonging to the class: P = FN + TP Hence (FN + TP)* Recall = TP Precision is the fraction of retrieved documents that are relevant to the ...

1

The expression "number of actions" is being used in the same way in both cases. In fact, the letter $m$ is used in both cases. The number of actions (in the state $s$) is the number of possible actions that you can take in the state $s$. So, here, $m$ does not refer to the dimensionality of an action, but to the size of the action set for a state. ...

1

As DKDK said, Indeed one could fit both linear and exponential function and see which one has smaller residual, without using any complex AI. But OTOH this could be a great toy-problem for learning about neural networks. You could have a network with these parts: A network with a final sigmoid activation, which predicts whether the function is linear or not....

1

The main reason to use powers of 2 is in the way existing hardware and software are made, there isn't any purely mathematical reason. CPUs, GPUs, memories, and internal buses all use a size that's the power of 2 since that's the most efficient way to address them.

1

The classification head works as follows. After the stack of BiFPN we have a feature map of size B x C x H x W. For EfficientDet H and W are 1/8 of the input image size. Then for each pixel in this feature map one applies one convolution to get the bounding boxes. The model predicts n_anchors - rescaled and shifted versions of reference boxes. The number of ...

1

Masks in Recurrent Neural Networks are used to transform variable-length inputs to one general length. Therefore we use padding and masking together. Padding: Usually we create a vector for every sentence in the dataset initialized with 0s and the length of the longest sentence in the dataset. Then we fill the mask with 1s for every position the sentence has ...

1

The difference between the two approaches (feature extraction vs fine-tuning) is well explained here: Fine Tuning vs Joint Training vs Feature Extraction Also, this paper evaluate the performance one can hope to achieve with 2 sequence models (ELMo and BERT) with each approach: To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks

1

This is the case as the loss doesn't have to monotonically decrease when it's updated in the negative direction. For example: Let $L(\theta) = \theta^2$ and $\theta_0= 3$ Let the subscript n in $\theta_n$ denote the iteration number. Then $\nabla_{\theta}L(\theta_0) = 2*\theta = 2*3 = 6$ For the loss to decrease in this case $\epsilon < 1$ needs to hold ...

1

There is already an approach similar to the one you describe: federated learning (FL), where local nodes (e.g. mobile, edge devices but also companies of different sizes) keep the training data locally, so each node might have different (unbalanced and non-i.i.d.) datasets, and models, which then need to be aggregated. One possible definition of federated ...

Only top voted, non community-wiki answers of a minimum length are eligible