Is doing a master's in natural language processing combined with machine learning worth it? What are the opportunities that can be opened by having such a master's?
The combination of NLP and ML is an interesting one. The most interesting work is not the mainstream systems employing feed forward, convolution kernel, and LSTM networks with word-to-vector input normalization or straight audio samples as input data.
The kinds that are most interesting are the kinds that more closely mimic the systems in which natural language developed, which are those of the language networks in the brain, including auditory, visual, and motor pathways and the kinds of cognitive abilities that support conversations beyond chat with no underlying comprehension or belief.
That some balanced rise and fall of beliefs and semantic associations are part of real conversation has been known for a few decades. The AI research directions along these lines include semantic nets, recursive artificial networks (an advanced type of RNN just entering the scene), and fuzzy belief models. Some of the current perspectives centered on letter, word, and grammar are obsolete from a linguistics point of view. Contemporary linguistics do not assume words are the elementary objects and formal grammar rules describe legal combinations of them. Smaller elements such as word roots, prefixes, suffixes, conjugations, phonetics, adjacency, and the dynamic aspects of language are more accurate modeling elements of real talk and writing.
Looking up some of these terms in an academic article search engine can pull up some of this research over the last two or three decades so that a historical background is clearer. Many of those articles have proposed systems, mathematics to describe the perspective, and sometimes algorithms and results of testing the modality. After spending a few days looking around, it will be easier to pick a master's thesis or a group of them to run past those at a university.
Applications for deeper forms of natural language capability beyond idiot talk chatbots is so profoundly needed, there will be little need to investigate the career options for most of these lines of research, provided the feasibility of developing components that fit into products and services is high for that line of research.
Not just any NLP+ML will qualify as an excellent choice for a masters thesis. It would be wise to focus on long term needs that won't easily go away. Some research directions are likely to be obsolete in the time it takes to finish a masters. Others will be so well covered by thousands of others interested in the highly publicized combination of NLP+ML, that there is no real advantage in choosing that direction.
Reaching further than simply NPL+ML is the better bet, but it will take significant ability and high motivation. The largest gap that is likely to remain a gap for several years is the depth of comprehension on the input side of machine based natural language. The shallow and domain ignorant comprehension of chatbots is a well known stumbling block to enterprise and consumer use.
Speech comprehension is by far the most valuable challenge before us as researchers and the most highly usable type of machine based natural language handling for the 2030s.
These are the current trends indicating the long term value of the above.
- Tacotron 2 and other approaches are well on the way to authentic sounding speech synthesis, so that educational direction will be in less demand.
- The trend in communications was transitioning, in the first decade of this century, from voice to text. That trend is reversing now, and the trend is very likely to continue to be from text back to voice in the third decade of this century. There are neurological and genetic reasons why that is the case. Voice communication is more deeply embedded in humanity. It appeared much earlier in human history and develops much sooner in children, and those things cannot be changed without changing human DNA.
- Machine translation is limited by the level of comprehension achievable from natural language input.
Eliminating voice synthesis and text generation from semantic structure, which is difficult to do until the semantic structures to use for that research have been collected into data sets, you have speech comprehension left.
In summary, machine listening provides value far beyond machine hearing, just as human listening provides value far beyond human hearing.
To the advantage of this comprehension research there are these veins of knowledge.
- The mechanics of the ear and first level of neural processing is well mapped. It involves the cochlear geometry which can be simulated using standard windowing techniques with an FFT followed by the RMS spectrum. The time series of spectra can then be mapped to phonetics.
- Cognitive science is developing quickly and on scientific basis more deeply than during the age of psychology detached from data. Now there are is genetic research, bioinformatics, neural imaging supporting the understanding of cognition and semantic structures that cognition involves.
- A more mature field of linguistics has changed the understanding of language from formal grammatical structure (with parts of speech and allowed sentence structures) to the natural speech of real people, which involves word roots, prefix and suffix conventions, conjugation patterns, local language evolution, borrowing of foreign words, colloquialism, and use of accent and intonation.
The right person, seeking an M.S. and highly motivated to bring machine based natural language handling to greater depths, the following thesis features would be of great value in the coming decade.
- Input 1: Multi-voice discussion
- Input 2: A directed graph representing a semantic net of associations, compositions, and generalizations
- Output: An updated directed graph indicating the decoration of Input 2 with the semantics contained in the voices of the discussion in Input 1.
Do anything like that, and the opportunities will mushroom faster than cell phones did.
It's not difficult to find work along the lines of the above, but it is in its infancy in some ways, which is why its a great opportunity to get on board early. The ways in which machine listening and comprehension can enhance human experience with computers and further the root field of cybernetics is too numerous to list. Here are a small few of the many research papers in this general direction.
- Learning continuous semantic representations of symbolic expressions, M. Allamanis, P. Chanthirasegaran, P. Kohli, C. Sutton, 2017
- Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure, Dieuwke Hupkes, Sara Veldhoen, Willem Zuidema, 2017
- On the difficulty of a distributional semantics of spoken language, Grzegorz Chrupała, Lieke Gelderloos, Akos Kadar, Afra Alishahi, 2018
Is doing a M.S. in the combination of Natural Language Processing and Machine Learning worth it?
I think it's not about on what topic you'll take, but it depends on how you will do it, where you will study, and other factors. I'll try to give you both perspective from industrial and academical:
Pros (take MS in NLP):
- If you feel didn't get the basic in your undergrad degree, MS degree will refresh your basic knowledge of AI or NLP.
- If you do research seriously, it will give you a nice portfolio that you have done an advance research on NLP. But if you just do research only to get your degree with minimum quality, I don't think it can give you any additional value.
- You'll have a chance to know more researchers, lecturers, that may will give you a chance to get another job / research project in the future (I got job once from my ex-lecturers)
- Many companies give you a chance to do Master while working part-time / full-time as long you can manage your time.
- If you want to continue become a researcher, then doing MS is required so you can continue your study to PhD. Then with PhD degree you should get a deeper knowledge of NLP (as you've done many research on NLP).
- I saw many companies that focus on AI/NLP only recruit people with PhD degree for data scientist/NLP researcher position.
Cons (don't take MS in NLP):
- There are many other companies that don't see your academic degree. I know some friends that become senior programmer without any university degree. As long you can pass their test as NLP engineer, then you will get the job.
- In this modern era, You can learn a lot via online course, read published paper, etc., but you'll need a lot of motivation for this.
- In my country (Indonesia), sometimes some NLP projects are simple that you don't need advance methods (LSTM, RNN, Seq2seq, etc.) So you can learn by yourself from the internet.
- You can invest your time for studying advance topic NLP rather than re-learn your basic knowledge on your semesters in Master's program
What are the opportunities that can be opened by having such a Master's?
In my country, I think having Master's or not is not really different except you might have deeper skills that you can show off to the interviewer. But if you got PhD you'll have more opportunities to do research on NLP as researcher either in industry or university.
- It depends on how you will do it: if you commit to take MS degree seriously, learn a lot in the program, and do advance research on NLP, then I'll recommend you to take MS
- It depends on where you will study: if the master's program not really focus teaching NLP, you may waste some of your times to learn something that doesn't intersect at all with NLP area.
- It depends on what you want to be: if you want to become NLP researcher (both in industry or university) I'll recommend you to take Master, then try to get your PhD so yo can get deeper knowldege on NLP.
If you want to do NLP, using machine learning is a very, very good idea. Our natural languages have structures full of messy inconsistencies (I must constantly apologize to my children for the vagaries of English orthography) which can only be recovered empirically from the data and encoded as highly complex heuristics (which is exactly what machine learning is for). You may have noticed that machine translation (whether on Facebook, Google or Twitter) is now really, really effective, and that voice recognition with Google Voice Search or Alex actually works. These applications were quite infeasible when we tried to hand-write code to solve these problems.
Are there other uses for NLP-based machine learning? Absolutely! Large organizations have databases chock-full of free text fields. These might be notes in medical records, transcriptions of customer service interactions, patent filings or a million or things. Tradition database operations can't do much with this text beyond really low-quality search. This is why companies like the giant Salesforce now have world-class research teams applying machine learning to NLP - because it is really the only way to enable machines to really get to grips with our messy human language.