Questions tagged [named-entity-recognition]

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

Is Training Separate NER Models for Specific Labels a Viable Strategy for Improved Accuracy?

In my project, I'm utilizing Named Entity Recognition (NER) to identify crucial variables in prompts categorized into specific domains. However, incorporating numerous labels in the training data has ...
Raghul Azhagaiah's user avatar
1 vote
0 answers
176 views

Can I use Generative AI to reliably extract data (vs. Extractive AI)?

It seems that nowadays lots of people building tools to extract data from different sources (e.g. PDF reports) with the help of models like GPT-4, LLama or Falcon. I was wondering, if this really ...
T-Dog's user avatar
  • 11
0 votes
1 answer
35 views

How to label missing/default values for a named entity recognition dataset

I am building the training dataset for a named entity recognition model, with 2 tags: Name and Category and I am using a pre-...
mrang's user avatar
  • 3
0 votes
0 answers
48 views

How to create dataset to extract information and classify intent using BERT?

Given a message: "Hey I am XYZ person (description about oneself), and I was thinking to launch a youtube video, wanted to get in touch with someone with similar experience", the model ...
thecalendar's user avatar
0 votes
0 answers
124 views

Convert unstructured text to XML

I am working with a large dataset (example bellow) of unstructured and structured documents that contain a variety of text elements, including paragraphs, tables, and lists. Currently, the process of ...
Nasreddine's user avatar
1 vote
1 answer
101 views

How to extract product codes within unstructured data

I am working on a project where I have a dataset consisting of unstructured data from multiple ERP systems. Each dataset (extracted from an ERP) has different columns, and unfortunately, there is no ...
delucca's user avatar
  • 11
0 votes
1 answer
36 views

Is there a way to find a list of associated words to a named entity?

Is there a way to find a list of associated words to a named entity? For instance, let the named entity be FIFA. Now FIFA is a Football Organization and hence related to the term football and all the ...
Vivek Hotti's user avatar
0 votes
1 answer
43 views

Confused about NER evaluation

I'm reading from the book Speech and Language Processing by Dan Jurafsky and James H. Martin and I've stuck for a while trying to understand what the authors mean by the following The fact that named ...
Essam's user avatar
  • 101
0 votes
3 answers
792 views

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of ...
Unicorn07's user avatar
2 votes
1 answer
3k views

Does BERT freeze the entire model body when it does fine-tuning?

Recently, I came across the BERT model. I did some research and tried some implementations. I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace. ...
Joon's user avatar
  • 51
0 votes
1 answer
54 views

Extracting keywords from messages

I'm starting a project where I want to extract keywords from given messages. The keywords are for example something like: "hard disk", "watch" or other technical components. I'm ...
gurke's user avatar
  • 1
2 votes
1 answer
408 views

Which AI techniques are there that combine multiple models to make sense of data at different stages?

I have been working to design a system that uses multiple machine learning models to make sense of data that is dynamically webscraped. Each AI would handle a specific task, for example: An AI model ...
Bipbupbop's user avatar
2 votes
0 answers
83 views

Extracting "hidden" costs from financial statements using NLP

I'm designing a NLP model to extract various kinds of "hidden" expenses from 10-K and 10-Q financial statements. I've come up with about 7 different expense categories (restructuring costs, ...
hohner's user avatar
  • 21
0 votes
1 answer
44 views

How to train a sequence labeling model with annotations from three annotators?

I have a dataset of movie reviews annotated by 3 persons. The following example contains one sentence with corresponding annotations from 3 different persons. ...
Ahmad's user avatar
  • 103
4 votes
1 answer
5k views

How to understand 'losses' in Spacy's custom NER training engine?

From the tid-bits, I understand of neural networks (NN), the Loss function is the difference between predicted output and expected output of the NN. I am following this tutorial, the losses are ...
The White Cloud's user avatar
0 votes
1 answer
428 views

Should we use a pre-trained model or a blank model for custom entity training of NER in spacy?

Further to my last question, I am training a custom entity of FOODITEM to be recognized by Spacy's Name Entity Recognition engine. I am following tutorials online, following is the advise given in ...
The White Cloud's user avatar
1 vote
0 answers
175 views

How can I use this Reformer to extract entities from a new sentence?

I have been looking at the NER example with Trax in this notebook. However, the notebook only gives an example for training the model. I can't find any examples of how to use this model to extract ...
Alan Buxton's user avatar
1 vote
0 answers
54 views

Determining if an entity in free text is 'present' or 'absent'; what is this called in NLP?

I'm processing a semi-structured scientific document and trying to extract some specific concepts. I've actually made quite good progress without machine-learning so far, but I got to a block of true ...
vector07's user avatar
  • 111
-1 votes
1 answer
445 views

Do we have to use the IOB format on labels in the NER dataset? If so, why?

Do we have to use the IOB format on labels in the NER dataset (such as B-PERSON, I-PERSON, etc.) instead of using the usual format (PERSON, ORGANIZATION, etc.)? If so, why? How will it affect the ...
Rachmansyah Adhi Widhianto's user avatar
2 votes
1 answer
265 views

Is it possible to create a named entity recognition system without using POS tagging in the corpus?

Is it possible to create a named entity recognition system without using POS tagging in the corpus?
Rachmansyah Adhi Widhianto's user avatar
2 votes
2 answers
386 views

What are the main ideas behind NER?

Named entity recognition (NER), also known as entity chunking/extraction, is a popular technique used in information extraction to identify and segment the named entities and classify or categorize ...
Pluviophile's user avatar
  • 1,263
1 vote
0 answers
30 views

Are there any good resources (preferably books) about techniques used for entity extraction?

Given some natural language sentences like I would like to talk to Mr. Smith I would like to extract entities, like the person "Smith". I know that frameworks, which are capable of doing ...
BudBrot's user avatar
  • 61