Questions tagged [metric]

For questions related to the concept of metric, which is a function that defines the distance between pairs of elements in a set.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Why MAPE (mean absolute percentage error) is so large?

I trained a model with my data in the range (0, 50000) and I used the mean absolute percentage error (MAPE) as my metric. Now, the model gave me MAPE > 10 million, but on my test data I have the ...
1
vote
1answer
52 views

How is F1 score calculated in question answering system?

I have an NLP model for answer-extraction. So basically I have a paragraph and a question as input and my model extracts the span of the paragraph that corresponds to the answer to the question. I ...
2
votes
0answers
50 views

What is the reason for mode collapse in GAN as opposed to WGAN?

In this article I am reading: $D_{KL}$ gives us inifity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta=0$. Only Wasserstein metric provides ...
1
vote
1answer
48 views

Is it possible that every class has a higher recall than precision for multi-class classification?

I am a student learning machine learning recently, and one thing is keep confusing me, I tried multiple sources and failed to find the related answer. As following table shows (this is from some paper)...
2
votes
1answer
43 views

How should we interpret all the different metrics in reinforcement learning?

I'm trying to train some deep RL agents using policy gradient methods like AC and PPO. While training, I have a ton of different metrics being monitored. I understand that the ultimate goal is to ...
0
votes
0answers
34 views

Which loss function and evaluation metric should I use for a multiple output prediction problem?

I was running into a situation with a data set like this I have 4 events and and they might happen together in pairs. I want to use 3 features to predict the coupling between event. I am building a ...
1
vote
0answers
37 views

What's the meaning of the Jaccard decay and the Jaccard recall? [closed]

I know the meaning of the Jaccard index, but when it comes to Jaccard decay and Jaccard recall I cannot see the difference or the meaning of it.
3
votes
1answer
32 views

What is meant by the expected BLEU cost when training with BLEU and SIMILE?

Recently I was reading a paper based on a new evaluation metric SIMILE. In a section, validation loss comparison had been made for SIMILE and BLEU. The plot showed the expected BLEU cost when training ...
0
votes
0answers
12 views

How to measure the distance betw. instances with individual subsets of attributes depending on the istance?

in my case I have a data set with 15 examples, and each has 100 attributes. I want to use these 15 examples to classifiy unseen examples based on the similarity/distance between them. The ...
3
votes
1answer
73 views

Why is there more than one way of calculating the accuracy?

Some sources consider the true negatives (TN) when computing the accuracy, while some don't. Source 1: https://medium.com/greyatom/performance-metrics-for-classification-problems-in-machine-learning-...
1
vote
0answers
26 views

Steps to train and re-train a good model

I'm still a bit new to deep learning. What I'm still struggling, is what is the best practice in re-training a good model over time? I've trained a deep model for my binary classification problem (...
2
votes
0answers
112 views

Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?

In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on ...
4
votes
0answers
146 views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
3
votes
1answer
67 views

Using True Positive as a Cost Function

I wanted to use True Positive (and True Negative) in my cost function to make to modify the ROC shape of my classifier. Someone told me and I read that it is not differentiable and therefore not ...
2
votes
1answer
51 views

Metrics for evaluating models that output probabilities

I'm aware of metrics like accuracy (correct predictions / total predictions) for models that classify things. However, I'm working on a model that outputs the probability of a datapoint belonging to ...
2
votes
0answers
127 views

When to use which metric in machine learning?

In machine learning, there are several metrics to assess the quality of the models: accuracy, precision, recall, f measure, ROC (AUC), etc. There are cases when certain metrics are more appropriate ...