I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text?
Just for reference, I am using the library spacy-neuralcoref in Python that is based on Deep Reinforcement Learning for Mention-Ranking Coreference Models by Kevin Clark and Christopher D. Manning, EMNLP 2016.
Why am I asking about splitting the text?
I am applying coreference to chapters of books (roughly 30 pages of text). All the examples I have seen show situations of coreference applied to small pieces of texts. I applied to a chapter and I found strange results. However, this is not a clear justification for that since the state of art in coreference is about 60%. Am I right?
I didn't check all databases that people use to test coreference but the ones I took a look (like MUC 3 and MUC 4 Data Sets), if I understand well, they were composed by a collection of a small number of paragraphs.
A test Example:
GUATEMALA CITY, 4 FEB 90 (ACAN-EFE) -- [TEXT] THE GUATEMALA ARMY DENIED TODAY THAT GUERRILLAS ATTACKED THE "SANTO TOMAS" PRESIDENTIAL FARM, LOCATED ON THE PACIFIC SIDE, WHERE PRESIDENT CEREZO HAS BEEN STAYING SINCE 2 FEBRUARY.
A REPORT PUBLISHED BY THE "CERIGUA" NEWS AGENCY -- MOUTHPIECE OF THE GUATEMALAN NATIONAL REVOLUTIONARY UNITY (URNG) -- WHOSE MAIN OFFICES ARE IN MEXICO, SAYS THAT A GUERRILLA COLUMN ATTACKED THE FARM 2 DAYS AGO.
HOWEVER, ARMED FORCES SPOKESMAN COLONEL LUIS ARTURO ISAACS SAID THAT THE ATTACK, WHICH RESULTED IN THE DEATH OF A CIVILIAN WHO WAS PASSING BY AT THE TIME OF THE SKIRMISH, WAS NOT AGAINST THE FARM, AND THAT PRESIDENT CEREZO IS SAFE AND SOUND.
HE ADDED THAT ON 3 FEBRUARY PRESIDENT CEREZO MET WITH THE DIPLOMATIC CORPS ACCREDITED IN GUATEMALA.
THE GOVERNMENT ALSO ISSUED A COMMUNIQUE DESCRIBING THE REBEL REPORT AS "FALSE AND INCORRECT," AND STRESSING THAT THE PRESIDENT WAS NEVER IN DANGER.
COL ISAACS SAID THAT THE GUERRILLAS ATTACKED THE "LA EMINENCIA" FARM LOCATED NEAR THE "SANTO TOMAS" FARM, WHERE THEY BURNED THE FACILITIES AND STOLE FOOD.
A MILITARY PATROL CLASHED WITH A REBEL COLUMN AND INFLICTED THREE CASUALTIES, WHICH WERE TAKEN AWAY BY THE GUERRILLAS WHO FLED TO THE MOUNTAINS, ISAACS NOTED.
HE ALSO REPORTED THAT GUERRILLAS KILLED A PEASANT IN THE CITY OF FLORES, IN THE NORTHERN EL PETEN DEPARTMENT, AND BURNED A TANK TRUCK.