Skip to main content
4 votes

How to go about classifying 1000 classes?

If you are asking for arbitrary ML task dealing with 1000+ classes the most straigtforward thing that comes to mind is the ImageNet - https://en.wikipedia.org/wiki/...
spiridon_the_sun_rotator's user avatar
4 votes
Accepted

Top Frequent occurrence word effect in Model Efficiency?

As far as I know, there are few aspects that would probably improve the model score: Normalization Lemmatization Stopwords removal (as you asked here) Based on your question, "is ...
Coderio's user avatar
  • 166
4 votes

Top Frequent occurrence word effect in Model Efficiency?

Based on my experience, I did 2 tasks that is proven to improve the accuracy/score of my model. Normalization removing characters and symbols in a text lowercase folding Stopwords removal (as what ...
Angela Marpaung's user avatar
4 votes
Accepted

Export trained model offline to be used by an application

Is it possible that I can train my data online (using any cloud provider) using a text classification model, and then export the model already trained (in the form of a script, I don't know how the ...
Neil Slater's user avatar
  • 32.1k
3 votes

Is it possible to derive meaning from text by providing multiple ways of saying the same thing to a neural network?

No, it will not derive patterns of meaning, as the network has no understanding of language. What will happen, is that it picks up surface features (usually letter sequences) which are common between ...
Oliver Mason's user avatar
  • 5,387
3 votes
Accepted

How to use LSTM to generate a paragraph

Take the sentence that was generated by your LSTM and feed it back into the LSTM as input. Then the LSTM will generate the next sentence. So the LSTM is using it's previous output as it's input. That'...
jgleoj23's user avatar
  • 182
2 votes

What is the most accurate pretrained sentiment analysis model by 2019?

For best results, I'd recommend Google Cloud Machine Learning. It has [Natural Language Processing API] (https://cloud.google.com/natural-language/docs/basics) with Sentiment, Entity, and Entity-...
Arnav Gupta's user avatar
2 votes
Accepted

Is a dataset of roughly 700 sentences of an average length of 15 words enough for text classification?

It depends on the number of classes; we are getting good results with about 40 training examples per class. A good way to get an idea about this is to run a test with an increasing set of training ...
Oliver Mason's user avatar
  • 5,387
2 votes

Does summing up word vectors destroy their meaning?

Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original ...
Clement's user avatar
  • 1,745
2 votes

Distinguishing text with opposite meanings in SVM (False Information Detection)

Going step by step: Preprocessing Preprocessing is a big deal in NLP, out there you'll find many tutorials describing the classic steps but few explanations about why and when you should actually ...
Edoardo Guerriero's user avatar
2 votes

How to use LSTM to generate a paragraph

As you know, an LSTM language model takes in the past word and tries to predict the new one and continue over a loop. A sentence is divided into tokens and depending on different method, the tokens ...
Clement's user avatar
  • 1,745
2 votes

Is there an AI that can extract proper nouns from free text?

This is a hard problem, unless you have a list of proper nouns you want to recognise. If John Allen is in this list, then you can easily use a longest match to prefer it over John or Allen. The same ...
Oliver Mason's user avatar
  • 5,387
2 votes
Accepted

How to predict the rating of a text review and improve it?

Why is it better to treat the rating prediction of a text review as a regression problem rather than a classification one? Is it because the ratings (1,2,3,4,5) are ordinal variables? Well, the main ...
Green Falcon's user avatar
2 votes

How do I use ResNet for text processing?

ResNet as a name is defined as a CNN with a specific architecture, but the more general concept of Residual Networks are not necessarily CNNs, but networks that use skip connections. You could make a ...
Dr. Snoopy's user avatar
  • 1,355
2 votes
Accepted

How does a Machine Learning model predict this classification problem?

when I don't give new sentence features (F1, F2) to my model and don't specify any procedure to calculate the features for new inputs, how the model can predict the sentiment of my new sentences? It ...
Luca Anzalone's user avatar
2 votes

Noob crafting a simple "Zero-Shot Classifier" Using an API . How can I avoid passing the categories every single request?

There's been a couple ideas thrown around wrt the task of "prompt compression". For example, the gpttrim algorithm, applies some basic NLP preprocessing to reduce unnecessary tokens (e.g., ...
Alexander Wan's user avatar
1 vote

Using a pre-trained model to generate labels to data to then train a model on

If using BART is already giving you good results, why do you need a new model? Not a rhetorical question. You might have good reasons for that. Training a model with less parameters optimized only on ...
Edoardo Guerriero's user avatar
1 vote
Accepted

Which algorithm can be used for extracting text patterns in tabular data?

You should first segregate the rejected samples. You can use then use string matching or something more complex (like creating embeddings and then, taking L2 distance between them) between the ...
Abhishek Verma's user avatar
1 vote

Which pre-processing steps are necessary for Deep Learning models to solve a document classification problem?

It depends on the type of model you use and the task, you are attempting to solve. Almost all the preprocessing steps that you mention remove some information from the text. If you think the removed ...
Jindřich's user avatar
  • 391
1 vote

Get the name of a merchant from records

The problem: You are facing a Natural Language problem called Named Entity Recognition (that's the key word you are looking for). But before you dive deep into it, have in mind it's best suited for ...
Andre Goulart's user avatar
1 vote

Training and Evaluating BERT and XLNET

You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’...
G__'s user avatar
  • 266
1 vote
Accepted

Can I use one-hot vectors for text classification?

One hot encoding is a good strategy to apply with categorical variables that assume few possible values. The problem with text data is that you easily end up with corpora with a really large ...
Edoardo Guerriero's user avatar
1 vote
Accepted

How do RNN's for sentiment classification deal with different sentence lengths?

One of the essential pre-processing we do on the corpus involves treating the variable-length sentences to a fixed length. There are various ways in which we can do this: Truncate This involves ...
Saurav Maheshkar's user avatar
1 vote
Accepted

Text classification of non-equal length texts, should I pad left or right?

For any model that does not take a time series approach like an RNN does, the padding shouldn't make a difference. I prefer padding right simply because there also might be text you need to cut-off. ...
N. Kiefer's user avatar
  • 321
1 vote

NLP Identifying important key words in a corpus

I am sure there are complex methods to extract keywords, but the standard one which should serve as a strong baseline is the RAKE graph algorithm https://pypi.org/project/rake-nltk/. It should work ...
saiRegrefree's user avatar
1 vote

Top Frequent occurrence word effect in Model Efficiency?

Based on my project, there is how i clean and doing some preparation on the data. Delete specific charaters ('\r', '\n', '"',) Change into the lowercase Delete some symbols Lemmatization (change ...
Muhammad Khaifa Gifari's user avatar
1 vote

Top Frequent occurrence word effect in Model Efficiency?

The technical term for these words is "stop words". Have a look at Information Retrieval and indexing (eg TF/IDF) to make up your mind whether you want to remove them or not.
Oliver Mason's user avatar
  • 5,387
1 vote

Can artificial intelligence classify textual records?

AI can categorize documents very accurately. It is not a new application but in the last year the accuracy of the underlying algorithms such as text classifiers, and language models in general, has ...
Brian O'Donnell's user avatar
1 vote

When is it time to switch to deep neural networks from simple networks in text classification problems?

Source: https://blog.easysol.net/building-ai-applications/ When data is too big, complex and nonlinear it's time to try deep learning. It's always good to try to add some layers to see it can ...
Lerner Zhang's user avatar
1 vote

How does the weight update formula for logistic regression work?

I think i found out how that works, so i made a short article about it . https://medium.com/@kourloskostas/python-spam-filter-86b21d7d1564 I hope it helps!
kostas's user avatar
  • 31

Only top scored, non community-wiki answers of a minimum length are eligible