Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [text-classification]

For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

How can I predict ordered classes from unstructured text documents?

I have a number of unstructured text documents, each of which is a list of responsibilities. I plan to use some embedding to vectorise them (SBERT?). My target variables (I have multiple) represent ...
gmds's user avatar
  • 101
0 votes
0 answers
20 views

What kind of neural network can extract brand, subbrand, sku, percents, weight, type, taste as separate words from product titles?

I have a task to make product titles atomizer. For example: "Cheese BellaVitano Soft 30% 450g" -> Product category: dairy products Product type: cheese Product subtype: soft Brand: ...
Kosmo零's user avatar
  • 101
0 votes
0 answers
23 views

Validating bundles of XMLs with inconsistent structure

The problem: We have a large number of XML bundles, and each bundle needs to meet certain criteria to be considered valid; specifically, certain values (or types of values) should belong to certain ...
SpaghettiM4ster's user avatar
2 votes
1 answer
223 views

Export trained model offline to be used by an application

I'm trying to create a text-based game using AI. I was playing around with text classification AutoML from Vertex AI just to learn AI, and then pick the best solution for my use case. Is it possible ...
Danilo Ruziska's user avatar
0 votes
0 answers
17 views

Zero-shot out of distribution text classification

I'm building out a pipeline that would allow me to filter out text based on whether or not the text belongs to any of the classes I've defined. I feel like one (albeit naive) approach would simply be ...
thevoid's user avatar
  • 111
0 votes
0 answers
33 views

Multiple text inputs to neural network

How do I best input multiple text fields in a neural network for classification? For example, each data point has a title and an abstract: ...
raywib's user avatar
  • 1
0 votes
1 answer
99 views

How can I use Artificial Intelligence to compare to paragraphs to see if they share the same meaning?

I am looking for a way to compare two paragraphs and see if both share the same meaning. For example, these two paragraphs have the same meaning, but they are worded differently. Paragraph 1: The ...
Jay's user avatar
  • 101
1 vote
0 answers
96 views

NLP for classifying a YES or NO response to a question

I'm currently working on a project that requires some feature extraction. The data I have is text and comes from an interview. The interviewer asks a question, the client responds, and the interviewer ...
altheconda's user avatar
0 votes
1 answer
81 views

Noob crafting a simple "Zero-Shot Classifier" Using an API . How can I avoid passing the categories every single request? [closed]

I have a collection of 700 categories, all potential classifications for articles. My current need is to create a system that can dynamically categorize short texts or articles according to these 700 ...
Advanced's user avatar
  • 109
0 votes
0 answers
30 views

Difference between using transformer for multi-class classification and clustering using last hidden layer

My data is a collection of URLs where I am interested in categorizing them into multiple groups. At the moment, I am using a pre-trained transformer model and fine-tuning it according to my data with ...
Mathee's user avatar
  • 101
0 votes
0 answers
17 views

My text classifier behaves like regex

I'm trying to train binary classifier that classifies ask to ask programming questions, programming questions that say "I'm getting an error about x/I have problem about x" but don't say the ...
whatamnotsaying's user avatar
0 votes
1 answer
72 views

How does a Machine Learning model predict this classification problem?

Let’s imagine we want to create a simple Sentiment Analysis model using Machine Learning not Deep Learning algorithms, so we need to have a set of handcrafted features for this classification problem. ...
Z Bokaee's user avatar
0 votes
1 answer
31 views

Do different ngrams share embedding in Fasttext?

As per Section 3.2 in the original paper on Fasttext, the authors state: In order to bound the memory requirements of our model, we use a hashing function that maps n-grams to integers in 1 to K ...
Fijoy Vadakkumpadan's user avatar
0 votes
0 answers
205 views

Increasing "output_sequence_length" in TextVectorization layer worsens model's performance

When exploring the Twitter Sentiment Analysis dataset on Kaggle, I came up with a model that looks like this: ...
Tran Khanh's user avatar
1 vote
0 answers
18 views

Giving Specified Data a Larger Value/Weight in a Model

I'm in the process of creating a model to classify an occupational code based on a job title & description. I have a large sample of labelled data to achieve this. The government has a resource ...
Ryley Keegan's user avatar
0 votes
1 answer
34 views

Pre-Trained Model for Occupational Coding

I've recently embarked on a task to classify an occupation code, given a job title & description. I have come across clustering, a method of grouping data into clusters of which were not ...
Ryley Keegan's user avatar
0 votes
1 answer
422 views

Multilabel text classification with highly imbalanced training data

I'm trying to train a multilabel text classification model using BERT. Each piece of text can belong to 0 or more of a total of 485 classes. My model consists of a dropout layer and a linear layer ...
Fijoy Vadakkumpadan's user avatar
0 votes
1 answer
171 views

How do I use ResNet for text processing?

I need to implement a deep neural network [residual neural network (ResNet)] that takes some text as an input [length M x N] and then processes it. Now as far as my understanding goes, ResNet is used ...
Python's user avatar
  • 1
-1 votes
2 answers
160 views

How do I generate video classification labels using video descriptions/titles?

I've been scanning the internet for ways to generate baseball-based labels for youtube baseball videos using text collected from a YT video's description, title, and top 50 titles, but so far, I have ...
Ian Steffy's user avatar
1 vote
1 answer
153 views

How to predict the rating of a text review and improve it?

Why is it better to treat the rating prediction of a text review as a regression problem rather than a classification one? Is it because the ratings (1,2,3,4,5) are ordinal variables? What kind of ...
ebrahimi's user avatar
  • 205
3 votes
1 answer
346 views

Using a pre-trained model to generate labels to data to then train a model on

I'm trying to set up a pipeline for my ML models to automatically re-train themselves whenever concept drift occurs to recalibrate to the new output distributions. However, I can't get ground-truth ...
Sanger Steel's user avatar
0 votes
1 answer
56 views

Distinguishing text with opposite meanings in SVM (False Information Detection)

I am currently working on a Binary Text Classification Model (False Information Detection) using Support Vector Machine and used TF-IDF as text vectorizer in Python. I have already tried training the ...
alexand88r's user avatar
1 vote
1 answer
283 views

Which pre-processing steps are necessary for Deep Learning models to solve a document classification problem?

I have created a data set with 30.000 text documents (each text file is rather small with respect to its length), which are labelled with 0 and 1. Using this data set, I want to train machine learning ...
MiFischer22's user avatar
0 votes
0 answers
29 views

Which AI algorithm to use for identifying API for a specific use from a list of APIs?

We have a legacy code solution in C#. We have to change the code so that it fetches internal data via APIs and not via DB calls. E.g. if the current code GETS Payment object from DB, we have to ...
Jatin's user avatar
  • 1
0 votes
1 answer
788 views

Get the name of a merchant from records

I have a bunch of bank transaction records from which I want to extract merchants' names. In a few subsets of these records, the structure of the string is the same within the subset with only the ...
Clock Slave's user avatar
0 votes
1 answer
187 views

Training and Evaluating BERT and XLNET [closed]

I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice. In the project, I have been given a data set with (...
MiFischer22's user avatar
0 votes
0 answers
53 views

How to calculate cosine similarity for classification when you have say 10000 samples belonging to two classes have a bunch of samples

Does anyone have experience with using Cosine Similarity for text classification? I see a number of articles on how to find cosine similarity between documents using Doc2Vec, Gensim, etc. I have a ...
Sanny28's user avatar
1 vote
1 answer
2k views

Is there an AI that can extract proper nouns from free text?

I have some free text (think: blog articles, interview transcripts, chat comments), and would like to explore the text data by analysing the proper nouns it contains. I know of many ways to simply ...
stevec's user avatar
  • 111
0 votes
1 answer
178 views

Advantages of CNN vs. LSTM for sequence data like text or log-files

When do you tend to use CNN rather than LSTM (or the other way round) in classification or generation tasks of sequential data like text or log-data? What are the reasons for the decision and what ...
moooo112's user avatar
0 votes
1 answer
92 views

Which algorithm can be used for extracting text patterns in tabular data?

I am working with tabular data that is similar to the below: Name Phone Number ISO3 Country Amount Email ... ... Outcome Possible Reason Leona Sunfurry (555)-555-5555 United States 58.96 leo_sun@...
Petar Luketina's user avatar
0 votes
1 answer
169 views

What approach to use for selecting one of the category according to short category text?

I need some tool to classify articles based on short category text which consists of two or three words separated by '-'. The RSS/XML tag content is for example: Foreign - News Football - Foreign I ...
Banik's user avatar
  • 105
1 vote
1 answer
203 views

Multi class text classification when having only one sample for classes

I have a dataset of texts, each text was identified with an ID number. I would like to do a prediction by finding the best match ID number for upcoming new texts. To use multi text classification, I ...
Fara's user avatar
  • 11
2 votes
1 answer
1k views

How to go about classifying 1000 classes?

I am trying to find research paper with theory(preferably implementation) that is about classifying 1000 (or more) classes. I have heard of an implementation, that initially clustering needs to be ...
Naveen Reddy Marthala's user avatar
1 vote
0 answers
637 views

Is using a LSTM, CNN or any other neural network model on top of a Transformer(using hidden states) overkill?

I have recently come across transformers, I am new to Deep Learning. I have seen a paper using CNN and BiLSTM on top of a transformer, the paper uses a transformer(XLM-R) for sentiment analysis in ...
learner's user avatar
  • 111
1 vote
0 answers
26 views

Bechmark models for Text Classification / Sentiment Classification

I am currently working on a novel application in NLP where I try to classify empathic and non-empathic texts. I would like to compare the performance of my model to some benchmark models. As I am ...
Jakub's user avatar
  • 11
2 votes
0 answers
132 views

NLP Bible verse division problem: Whats the best model/method?

I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
rwreed's user avatar
  • 121
1 vote
1 answer
679 views

Can I use one-hot vectors for text classification?

For an upcoming project I'm trying to write a text classifier for the IMDb sentiment analysis dataset. This needs to vectorize words using an embedding layer and then reduce the dimensions of the ...
JumpJehovah's user avatar
2 votes
1 answer
108 views

How do RNN's for sentiment classification deal with different sentence lengths?

I have been doing a course which teaches you about Deep Neural Networks, during one of the exercises I was made to make an RNN for sentiment classification which I did, but I did not understand how an ...
jr123456jr987654321's user avatar
2 votes
2 answers
425 views

Is it possible that every class has a higher recall than precision for multi-class classification?

I am a student learning machine learning recently, and one thing is keep confusing me, I tried multiple sources and failed to find the related answer. As following table shows (this is from some paper)...
Cheleeger Ken's user avatar
0 votes
1 answer
92 views

Text classification of non-equal length texts, should I pad left or right?

Text classification of equal length texts works without padding, but in reality, practically, texts never have the same length. For example, spam filtering on blog article: ...
Dan D.'s user avatar
  • 1,283
0 votes
1 answer
1k views

NLP Identifying important key words in a corpus

I am intrigued with the idea of Zettelkasten but unsatisfied with the current implementations. It seems to me that a machine learning and NLP approach could be productive by helpfully identifying “...
Charlie Martin's user avatar
1 vote
0 answers
41 views

My LSTM text classification model seems not learn anything in early epochs

I am trying to use LSTM to do text classification and monitor the training process with tensorboard. But it seems that this model doesn't learn anything in early epochs. Is it normal for LSTM networks?...
Charles's user avatar
  • 11
1 vote
0 answers
150 views

Applications of polar decomposition in Machine Learning

Assume there exists a new and very efficient algorithm for calculating the polar decomposition of a matrix $A=UP$, where $U$ is a unitary matrix and $P$ is a positive-semidefinite Hermitian matrix. ...
Samuel's user avatar
  • 11
1 vote
0 answers
58 views

How many spectrogram frames per input character does text-to-speech (TTS) system Tacotron-2 generate?

I've been reading on Tacotron-2, a text-to-speech system, that generates speech just-like humans (indistinguishable from humans) using the GitHub https://github.com/Rayhane-mamah/Tacotron-2. I'm very ...
Joe Black's user avatar
  • 181
1 vote
0 answers
55 views

Creating Text Features using word2vec

My task is to classify some texts. I have used word2vec to represent text words and I pass them to an LSTM as input. Taking into account that texts do not contain the same number of words, is it a ...
ddaedalus's user avatar
  • 919
3 votes
4 answers
109 views

Top Frequent occurrence word effect in Model Efficiency?

Assume that I have a Dataframe with the text column. Problem: Classification / Prediction ...
Pluviophile's user avatar
  • 1,273
1 vote
1 answer
59 views

Can artificial intelligence classify textual records?

I am a records manager and I am being asked if I recommend Office 365. I'm having a hard time making a recommendation because I am missing an essential piece of information: can Office 365 replace ...
oymonk's user avatar
  • 111
2 votes
0 answers
73 views

Are bayesian neural networks suited for text (or document) classification?

I've tried to do my research on Bayesian neural networks online, but I find most of them are used for image classification. This is probably due to the nature of Bayesian neural networks, which may be ...
Nicole's user avatar
  • 21
1 vote
1 answer
120 views

Is it possible to derive meaning from text by providing multiple ways of saying the same thing to a neural network?

Let's say I feed a neural network with multiple string sentences that mean roughly the same thing but are formulated differently. Will the neural network be able to derive patterns of meaning in the ...
BreezeWind's user avatar
2 votes
0 answers
33 views

Language Learning feedback with AI

Is there a program under development that uses AI technology, like Siri, to "hold hands" so to speak with a language learner and coach them on accent, colloqiual expressions, or to let them guide the ...
Tristan Beckwith's user avatar