I'm given pdf documents containing complex information about a financial product and my task is to write a disclaimer: a two page document with a very precise structure explaining this financial product, the risks associated to it etc, infos that are present in the pdf but hard to understand for common people. Do you know if it would be possible for an AI to read this complex document and write the disclaimer ? More precisely, could it be able to extract information from documents whose content and layout change and write a comprehensive summary in a defined setting?
2 Answers
I think it is a quite reasonable task.
- To extract text from a PDF there are many libraries.
- If you also have images or other plots could be transformed in an embedding using a detector followed by encoder (or any alternative solutions).
I'd pass the text (could also be plain text, the model should infer the structure from the word sequence contained in each section, if present and given enough text) to a text summarization model, which would output the summarized sentence.
Alternatively you should detect the text regions with a detector, then parse the text contained in each region and encode it, and successively pass it to the summarization model. There are many metrics to evaluate the performance, such as BLEU, METEOR, ROUGE ...
In my opinion, I think Bing is best suited for this task, as you can open it in a sidebar so it will be able to read the pdf file. To add on, it is powered by GPT-4, which is basically the most powerful language model as of now, so this task is definitely achievable with the help of this tool.