Now I know this might break some StackExchange rules and I am definitely open for taking the thread down if it does! I am trying to build an AI that can write it's own book and I have no idea where to start or what are the appropriate algorithms and approaches to go with. How should I start and what do I exactly need for such a project?
There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time.
Many networks have been tried for text generation, with notable examples such as Markov chain. However RNN have been proven to work the best and is dominating the field of language modelling (text generation).
How text generation works
A neural network that generates text is commonly called a language model. It is trained on large amount of text with labels being the next token. The text generation process uses several random token as the starting phrase and then the network predicts the rest. However the network does not just predicts the most probable word, instead it randomly chooses one of the few most probable token, hence the generating part.
Why RNN work best on language modelling
RNN have a perception of time built into the architecture of teh network. LSTM, a popular RNN variant used, is composed of "memory units" that "remembers" past text, thus the "time" part. RNN process input according to the sequence of time, so the network can naturally understand time, thus the superior performance compared to other networks.
Architecture of language model
A language model consists of the encoder and the decoder. The encoder compresses word one-hot representation to a smaller sized vector representation. The smaller sized representation is then passed through the decoder, which maps the encoding to the words one hot vectors again.
State of the art results for language modelling
Language modelling is an actively researched field in the AI community, and recently the model GPT-2 have achieved a breakthrough in language modelling accuracy, producing almost human like text with a special component added, the attention layer. Attention basically maps the "memory states" of the encoder and feed it as input to the decoder. The data teh model is trained on is also very large, with over 20GB of web scraped data from sites like Reddit.
Limits of language modelling
One limit of language modelling is the size of generated text. As GPU don't have unlimited memory, language model usually limits the input token size to a specific number, padding or trimming to this number. The number is usually 500-1000, which includes a paragraph or two, but not an entire book. You can only generate short paragraphs and essay with language modelling. For long text it is much harder.
Resources to help you get started
GPT-2 open AI blog: https://openai.com/blog/better-language-models/ GPT-2 online interactive site for text generation: https://talktotransformer.com/ How to train and fine tune GPT-2 in python: https://minimaxir.com/2019/09/howto-gpt2/
Hope I can help you
Recurrent Neural Networks (RNNs) have been applied to generate text. In this blog post you will find a couple of interesting text examples (the author also has made his code available on github), e.g. their Shakespeare-like texts generated by an RNN:
PANDARUS: Alas, I think he shall be come approached and the day When little srain would be attain'd into being never fed, And who is but a chain and subjects of his death, I should not sleep.
Second Senator: They are away this miseries, produced upon my soul, Breaking and strongly should be buried, when I perish The earth and thoughts of many states.
DUKE VINCENTIO: Well, your wit is in the care of side and that.
Second Lord: They would be ruled after this chamber, and my fair nues begun out of the fact, to be conveyed, Whose noble souls I'll have the heart of the wars.
Clown: Come, sir, I will make did behold your worship.
VIOLA: I'll drink it.
As you can see the RNN is able to somewhat mimic the "flow" of the texts it has been trained on but some sentences (like at the very end) do not make much contextual sense.