I have hundreds of .txt files. I need to get into each one of them and remove certain paragraphs that start with specific words but as a whole, are not exactly the same every time. Is there an automatic way that can help me clean these parts out? If yes, what is it? If not, is it easy/quick to create my own AI tool for this job? Assuming that I need to get this done very soon, does it take a lot of time to learn how to create an AI tool to get the job done for me?

Thanks in Advance!

  • 1
    $\begingroup$ What it means "remove certain paragraphs that have specific start words but they are not exactly the same each time" ? $\endgroup$ – pasaba por aqui Mar 10 '18 at 14:48
  • $\begingroup$ I think some regex logic and use of sed command can help you out. $\endgroup$ – Ugnes Mar 11 '18 at 3:18
  • $\begingroup$ @pasabaporaqui I modified that part..it means paragraphs starting with same sentences but changing off some where in the middle. $\endgroup$ – DuttaA Mar 12 '18 at 12:17
  • $\begingroup$ Yes, but never mind. thanks ! $\endgroup$ – user105139 Mar 12 '18 at 18:45

I think theres definitely a way to do what you want, but I'm not sure AI will work magic for you.

As far as I can tell there's a few things you need.

1: A program that finds strings and removes paragraphs with the offending strings indside them.
For this, some python code:

path = '/some/path/to/file'
for filename in os.listdir(path):
    with open(filename) as f:
        paragraphs = f.readlines()
        for para in paragraphs:
            for word in wordlist:
                if strip(para).startswith(word):
                    print para
                    #create and write the modified file etc

This is how you can find the offending paragraphs in a file.

2: You need that wordlist mentioned in the above program. If there are only a few words just define wordlist = {"words", "I", "don't", "like","to","start","paragraphs"}. If it's more vauge than that, I'd go to a thesaurus. If generating your wordlist is harder then we might be talking about machine learning.

If you absolutely need it right, and you're "only" doing 100s, it'll take a while to program an ai, then to verify it's doing things correctly, but I may very well have misunderstood your problem. If I did please let me know.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.