New answers tagged deep-learning
0
It's hard to answer this question without knowing what your goal is, but if your data is extensive, high quality, especially if it is labelled, and no similar dataset is publicly available, then publishing it freely with some kind of challenge could be very helpful if that's an option. Many organisations have the opposite problem: available computing ...
0
You can take tips from how protien folding is done
0
So after doing a bit of research, I finally found out why the model is not working at all when I change the LSTM to Bi-LSTM.
The task of the learning is Next Word Prediction for each cell of LSTM. When you have a Uni-directional LSTM, this is inherently a tough task for the model to learn good representations that can help it generate the next word with ...
0
Thanks for the extra details. There area good answers already, but I'll give just a bit more information since your requirements are a little more specific now.
Since you mentioned Research Engineer only, I'm going to assume you are not really interested in a plain engineering role.
I can say for a specific Research Engineer role I am aware of at a world ...
4
Many people without a formal/solid background in statistics (e.g. without knowing exactly what the central limit theorem (CLT) states) are doing research on machine learning, which is a very big and fundamental subfield of AI that has a big overlap with statistics, or using machine learning to solve problems.
So, in my view, you don't need to learn ...
2
I work in NLP, and use very little statistics. Actually, almost nothing I do can be classed as 'serious' statistics.
So yes, AI is a wide area, and in my company there is a group that does machine learning, so they probably use a lot more of it than I do. Previously I worked in conversational AI. Again, very little to no statistics at all.
I would contest ...
-2
“When you're fundraising, it's AI. Statistics is the field of mathematics which deals with the understanding and interpretation of data. ... Machine learning is nothing more than a class of computational algorithms (hence its emergence from computer science).
Artificial intelligence (AI) is intrinsically data-driven. It calls for the application of ...
0
Typically, in transfer learning, you have two stages/steps (as you realized)
pre-train some base model $M_\text{base}$ (i.e. the feature extraction part, where this pre-trained model is supposed to learn representations of the data, which can later be exploited to solve another task) on some "general" dataset $A$; note that you may not necessarily ...
3
In my experience researchers typically base their architectures on previously identified successful architectures and principles. That is to say published methods that have been successful in practice on similar tasks to the current one. This can be followed back to to very early networks like the Perceptron which took a lot of inspiration from existing ...
0
In my experience, it's not a matter of performance benefits; Variational Auto-Encoder GANs are much more useful if you want to have "knobs" to turn to influence the generated output. Since you have a latent layer that represents possibly the mean and the distribution of the data, you can tune to different "positions" in that latent space ...
0
I can say "Stable Learning" of a supervised machine learner is as follows:
A stable learning algorithm is one for which the prediction does not change much when the training data is modified slightly.
You can follow this link to know more in details about how can we measure the stability in the context of computational learning theory.
3
This is very difficult to tell with the information provided, but the phenomenon is something that I have encountered many times before. Sometimes this is not a bad thing, here are some possible considerations/explanations:
Data from the training set could be identical or leaking in to the validation set.
Using a high dropout rate can cause this as well as ...
1
There is not one answer to this question, but one could argue that transformers heavily rely on
transforming each input into latent subspaces of queries, keys and values in order to generate attention score
a pool of transformations of the attention vectors (multi-head) according to which models can capture richer interpretations as different sections of ...
2
This is akin to asking "Why do we need more than one instance of sine to represent any repeating function" or "why can't we represent any polynomial with an equivalent polynomial of just the first degree?" There are many, many problems... I'd even want to say most... that will require more than one layer to solve because the higher ...
0
Why do not simply perform some bilinear or bicubic interpolation?
Tensorflow and PyTorch deep learning frameworks have dedicated function to do this - https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Resize and https://www.tensorflow.org/api_docs/python/tf/image/resize .
0
You can rescale images to same size based on the classification model you are using (preferably 300x300). Also for preprocessing you can try some morphological operations and some brightness removal techniques from OpenCV-Python. One more factor that could have affected your accuracy might be the number of images, if you are having less number of images you ...
1
I do not understand why you say that your model is overfitting. An overfit occurs when the validation loss start increasing after diminishing. Here it seems that your model has reaches its potential and cannot improve anymore. What I would recommend here is to make your model bigger: add filters, increase the depth. Also consider trying transfer learning; it ...
0
If you reinitialize your model weights before training it on a new subset, you erase everything it learned before; is it what you want to ?
If not, saving your model after each subset and loading it before training on the next subset isn't a good practice neither because your model will see a lot of times a subset of samples and then move on the next ones ...
0
Answers in the comments are decent, particularly DuttA.
DuttA gives these, approximately
ease of derivative
Don't have to worry about ~0 in denominator causing huge gradient
But to me the most important is mathematical convenience, someone
might easily make the mistake of RMSE is just equal the difference
y−y′ instead of root of mean square of y−y′. The ...
1
I understand the confusion and I wanted to refer to this (older post) because the metric really is unclear in the context of the SDNE paper.
Perhaps I can try to explain it for future readers, in hopes that this makes sense. All this is my own interpretation, of course.
SDNE is an autoencoder setup that outputs both node embeddings ($y_i$ vector for focal ...
1
They use the same techniques, but study different problems.
Transfer learning always does not imply that the novel classes have very-few samples (as few as 1 per class). Few-shot learning does.
The goal of transfer learning is to obtain transferrable features that can be used for a wide variety of downstream discriminative tasks. One example is using an ...
1
You can save weights during training by passing checkpoint callback to model.fit() method.
# Instantiate your model here
model = create_model()
# Set model configurations here
model.compile(loss=..., optimizer=..., metrics=...)
# Set checkpoint path
checkpoint_path = "model_weights.ckpt"
# Create a callback that saves the model's weights
...
0
Data augmentations is usually done on the fly during training, meaning before each you apply the random augmentation for the entire dataset, because of the randomness there will be different transformation of the same image in each epoch.
Shuffle the dataset before batching in each epoch, so that each epoch will not have minibatch of same images, which will ...
Top 50 recent answers are included
