Questions tagged [performance]
For question about methods of evaluating the performance (e.g. the accuracy) of an artificial intelligence algorithm or model.
42
questions
0
votes
0answers
22 views
Identifying if a model is over or under-fitting via graphs
I am working on a Neural Network and have plotted the performance of my model. However the plots seem not to fit the "trends" (which help you identify the issue with your model) presented in ...
0
votes
0answers
33 views
What is the difference between learning agents and other kinds of agents, and more specifically, between performance standard and performance measure?
I am reading AI: A Modern Approach. In the 2nd chapter when introducing different agent types, i.e., reflex, utility-based, goal-based, and learning agents, I understood that all types of agents, ...
2
votes
1answer
33 views
How would the performance of federated learning compare to the performance of centralized machine learning when the data is i.i.d.?
How would the performance of federated learning (FL) compare to the performance of centralized machine learning (ML), when the data is independent and identically distributed (i.i.d.)?
Moreover, what ...
4
votes
1answer
133 views
How to evaluate an RL algorithm when used in a game?
I'm planning to create a web-based RL board game, and I wondered how I would evaluate the performance of the RL agent. How would I be able to say, "Version X performed better than version Y, as ...
0
votes
1answer
33 views
Can the attention mechanism improve the performance in the case of short sequences?
I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise.
...
1
vote
0answers
60 views
Is the performance of a neural network, which was trained with encrypted data and weights, affected if the weights are decrypted?
Suppose that a neural network is trained with encrypted (for example, with homomorphic encryption and, more precisely, with the Paillier partial scheme) data. Moreover, suppose that it is also trained ...
1
vote
0answers
35 views
How should I change the hyper-parameters of the C51 algorithm, in order to obtain higher reward?
I have a scenario where, in an ideal situation, the greedy approach is the best, but when non-idealities are introduced which can be learned, DQN starts doing better. So, after checking what DQN ...
1
vote
0answers
23 views
Would performance of atomic models matter in ensemble methods?
Suppose I have two fitted ensemble models $F_1 := (f_1, f_2, f_3, \cdots f_n)$ and $G_1 := (g_1, g_2, g_3, \cdots g_n)$.
And they were using the same ensemble methods (boosting or bagging).
And I am ...
2
votes
1answer
109 views
How do I check that the combination of these models is good?
I've selected more than 10 discriminative (classification) models, each wrapped with a BaggingClassifier object, optimized with a ...
2
votes
1answer
430 views
Can ConvNets be used for real-time object recognition from video feed?
Convolutional neural network are leading type of feed-forward artificial neural network for image recognition. Can they be used for real-time image recognition for videos (frame by frame), or it takes ...
0
votes
1answer
33 views
How is the performance of a model affected by adding a ReLU to fully connected layers?
How significant is adding a ReLU to fully connected (FC) layers? Is it necessary, or how is the performance of a model affected by adding ReLU to FC layers?
11
votes
4answers
2k views
What is the “dropout” technique?
What purpose does the "dropout" method serve and how does it improve the overall performance of the neural network?
2
votes
0answers
24 views
What is human-level performance for semantic segmentation?
I see so many papers claim to have an algorithm that beats 'human-level performance' for semantic segmentation tasks, but I can't find any papers reporting on what the human-level performance actually ...
0
votes
0answers
75 views
How to calculate the performance measure of a vacuum cleaner agent?
A simple vacuum cleaner agent is having two sensors. One sensor senses the current location of the agent (floor A or floor B). The other sensor senses the status of the current location (Dirty or ...
0
votes
0answers
18 views
Calculating processing time of a deep learning model
My model deals with videos, and I want to calculate how fast it can process frames as in frames per second or processing time for 1 frame.
I have made a single function to get predictions, it takes in ...
1
vote
0answers
23 views
How is the performance of a CNN trained with monochrome images on image recognition tasks degraded?
For CNN image recognition tasks, like object recognition/face recognition/object segmentation/posture recognition, are there experiment results about how much will the performance be degraded with ...
0
votes
1answer
106 views
Which kind of data does sigmoid kernel performance well?
While I was playing with some hyperparameters, I came to a wired situation. My dataset is IRIS dataset to be specific. SVM algorithm has some hyperparameters that we can tune, such as Kernels, and C ...
5
votes
2answers
140 views
How do I improve accuracy and know when to stop training?
I am training a modified VGG-16 to classify crowd density (empty, low, moderate, high). 2 dropout layers were added at the end on the network each one after one of the last 2 FC layers.
network ...
1
vote
0answers
37 views
How many training runs are needed to obtain a credible value for performance?
I'm trying to optimize a neural network. For that, I'm changing parameters like the batch size, learning rate, weight initialization, etc.
A neural network is not a deterministic algorithm, so, in ...
2
votes
4answers
160 views
Is it normal to have the root mean squared error greater on the test dataset than on the training dataset?
I am new to deep learning.
I am training a model and I am getting a root mean squared error (RMSE) greater on the test dataset than on the training dataset.
What could be the reason behind this? Is ...
1
vote
1answer
709 views
How can I merge outputs of two separate layers so that the overall performance improves?
I am training a combined model (fine-tuned VGG16 for images and shallow FCN for numerical data) to do a binary classification. However, the overall AUC score is not what I expected it to be.
Image-...
2
votes
1answer
147 views
How can we compare the intelligence of AI systems?
One way of ranking human intelligence is based on IQ tests. But how can we compare the intelligence of AI systems?
For example, is there a test that tells me that a spam filter system is more ...
1
vote
1answer
32 views
How to estimate the accuracy upper limit of any CNN model over a computer vision classification task
We are given a computer vision classification task, that is, a task that asks us to predict the category of an image over $n$ predefined classes (the so-called closed set classification problem).
...
2
votes
0answers
30 views
Can the addition of low-quality images to the training dataset increase the network performance?
I already trained a deep neural network called YOLO (You Only Look Once) with high-quality images (1920 by 1080 pixels) for a detection task. The result for mAP and IOU were 93% and 89% respectively.
...
1
vote
0answers
37 views
Sample size for the evaluation of Deep Learning Models
I'm evaluating the performance and accuracy in detecting objects for my data set using three deep learning algorithms. In total there are 24,085 images. I measure the performance in terms of time ...
1
vote
0answers
23 views
What are the properties of a model that is well suited for for high performance real-time inference
What are general best practices or considerations in designing a model that is optimized for real-time inference?
1
vote
1answer
539 views
Why is the effective branching factor used for measuring performance of a heuristic function?
For search algorithms with heuristic functions, performance of heuristic functions are measured by effective branching factor ${b^*}$ which involves total nodes expanded ${N}$ and depth of the ...
1
vote
0answers
17 views
Quantization techniques and new GPU architectures [closed]
Quantization means using low resolution formats for some variables some of the time: binary (e.g. BinaryConnect), ternary, etc.
The Turing architecture recently introduced by Nvidia is much faster ...
1
vote
1answer
60 views
What will change when workstations will have ARM Machine Learning Processors onboard?
lately we read that many manufacturers are forcing ARM architectures to be used on future workstations. One of ARM's recent announcements is a machine learning processor. What will change in terms of ...
0
votes
0answers
75 views
What to do when an image classifier does good for a class but bad for another?
So I wrote a convolutional neural network for a binary image classification. I have around 5300 images for each class which I thought would be enough to at least give me a good accuracy on the ...
5
votes
1answer
48 views
How can we conclude that an optimization algorithm is better than another one
When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
1
vote
0answers
21 views
Improving Recall of a Certain Class
Let's say that we have a test data set with $20,000$ observations for which we want to make a binary prediction for. When we apply our best trained model to this data set (e.g. logistic regression ...
5
votes
2answers
1k views
Does the quality of training images affect the accuracy of the neural network?
I just got into AI few months ago. I noticed most of the images in training datasets are usually low quality( almost pixelated).
Does the quality of training images affect the accuracy of the neural ...
2
votes
0answers
17 views
In addition to matrix algebra, can GPU's also handle the various Kernel functions for Neural Networks?
I've read a number of articles on how GPUs can speed up matrix algebra calculations, but I'm wondering how calculations are performed when one uses various kernel functions in a neural network.
If ...
2
votes
0answers
40 views
Performance measure on windowed time series data
I have time series data where I use a sliding window to detect anomalies in those windows. A sliding window is an interval of the dataset that steps one datapoint for each iteration. Datapoints are ...
1
vote
0answers
31 views
Why does precision-recall curve become more stable when neural net begins to overfit?
I am training a convLSTM with a dropout layer (with prob 0.5).
If I train over more than 5 epochs I notice that the network starts to overfit: my validation set loss becomes stationary while the ...
7
votes
1answer
464 views
Why isn't the ElliotSig activation function widely used?
The Softsign (a.k.a. ElliotSig) activation function is really simple:
$$ f(x) = \frac{x}{1+|x|} $$
It is bounded $[-1,1]$, has a first derivative, it is monotonic, and it is computationally ...
6
votes
4answers
487 views
What does deep learning offer with respect to standard machine learning?
I've been reading a lot about DL. I can understand to an extent how it works, in theory at least, and how it's technically different from conventional ML. But what I'm looking for is more of a "...
3
votes
1answer
129 views
What are the aspects that most impact on the inference time for neural networks in embedded systems?
I work with neural networks for real-time image processing on embedded softwares and I tested different architectures (Googlenet, Mobilenet, Resnet, custom networks...) and different hardware ...
4
votes
1answer
100 views
Relative compute time for each type of layer in a neural network
Hello,
I would like to know whether this picture from the paper: Distributed Training of Deep Neural Networks: Theoretical and Practical Limits of Parallel Scalability valid?
Questions:
1) Does ...
1
vote
0answers
32 views
How to visualize/interpret text prediction model results?
I am using LSTM model to predict the next xml markup from an input seed.
I have trained my model on 1500 xml files. Each xml file is generated randomly. I am wondering if there is a way to visualize ...
7
votes
2answers
530 views
Why is the generation of deep style images so slow and resource-hungry?
Consider these neural style algorithms which produce some art work:
Neural Doodle
neural-style
Why is generating such images so slow and why does it take huge amounts of memory? Isn't there any ...
