0
$\begingroup$

We have data in text format as sentences. The goal is to detect rules which exist in this set of sentences.

I have a limited set of contextless sentences that fit a pattern and want to find the pattern. I might not have sentences that don't fit the pattern.

What should be an approach to do that?

$\endgroup$
  • 2
    $\begingroup$ This I really to vague to answer. What’s the context? Where’s the data coming from? What kind of rules do you want to find? The answers for “I have a bunch of books and want to infer topics” and “I have a set of contextless sentences that fit a pattern and another set of contextless sentences that don’t fit that pattern and want to find the pattern” are extremely different. $\endgroup$ – Stella Biderman Sep 15 '18 at 20:35
1
$\begingroup$

If you don't have non-examples of your pattern and don't have some kind of heuristic guide, unfortunately the answer is that you can't. "All sentences" will always be 100% compatible with your examples, and you'll never be able to collect evidence that disconfirms (or even decreases the likelihood) of that hypothesis. Even if you rule out that hypothesis by fiat, the hypothesis that the only non-accepted sentence is

The answers for “I have a bunch of books and want to infer topics” and “I have a set of contextless sentences that fit a pattern and another set of contextless sentences that don’t fit that pattern and want to find the pattern” are extremely different.

will be impossible to rule out

With a good enough heuristic guide, you might be able to do something, but that would require knowing statistical facts about examples and non-examples and then sampling from the world IID, and just happening to not get any non-examples. That doesn't seem like the situation you're in. If you really have nothing to quantify the non-examples in any way, there's nothing that can be done.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.