I want to develop a virtual assistant like Google Assistant, Alexa, etc., for mobile phones.

What I should be researching apart from M/C Learning and Python?


First of all, making app for mobile is not normally written in Python, but if you want it, that is possible [1].

Other non ai question is that the counting of speech recognition is so heavy, that at least Alexa etc make it on external server. Wake word recognition is done in machine because of real time needs [3]. Ofcourse your app can trigger from mobile touch event too so speech recognition implementation on mobile can be avoided.

Using server needs knowledge about communication and web apis as well as server maintenance in cloud.

Speech recognition can be made with open source tools, but the dictionaries with enough samples may not yet exist or they're coming [2]. Also Google offers speech api, that needs mainly programming knowledge to attach speech recognition.

For questions you also need to generate answers. To form that you may reverse the flow in speech generation [4]. All the logic to find useful answers needs some more effort depending how intelligent your assistant will be. That maybe is the most challenging part where direct answer do not exist.

Of course, if you could attach to ready assistant's public api, it would be magnificent. Actually, Alexa api at least is open [5]!! Playing with it directly from mobile makes app soon ready but if you want to learn more algorihms, some pointers are on my above other points.


[1] https://www.quora.com/Can-I-make-an-Android-app-with-Python

[2] https://research.mozilla.org/machine-learning/

[3] https://iot.stackexchange.com/questions/736/why-do-most-smart-assistants-offer-little-if-any-customisation-of-the-wake-wor

[4] How could you generate sentences from lists of facts

[5] https://github.com/alexa/alexa-avs-sample-app

  • $\begingroup$ Problem turned in my hands as a programming challenge. $\endgroup$
    – mico
    Jan 12 '18 at 20:42
  • $\begingroup$ About speech recognition, post of interest: ai.stackexchange.com/questions/4964/… $\endgroup$
    – mico
    Jan 12 '18 at 20:45
  • $\begingroup$ I noticed right away with iPhone that the speech recognition wasn't available when not connected. Still seems to be the case. Good links! $\endgroup$
    – DukeZhou
    Jan 12 '18 at 21:33

people at google and microsoft have worked so hard (for years) to make their assistants powerful enough , and it is a collective work of brilliant scientists and engineers , geoff hinton(prominent neural network researcher) once mentioned about how "gaussian mixture models" which is then the most widely used algorithm for speech-recognition purposes was replaced by neural networks , they probably have way more dataset and computing resources than you can ever imagine. so it would be pretty tough for you to compete with them . but if you really want to learn , nothing is stopping you, but you should know a lot more than basic "machine learning" and "python". i would suggest you to read papers on speech recognition and question answering systems , and try to implement them on your own and replicate the results mentioned in the papers to get a basic idea.


Not the answer you're looking for? Browse other questions tagged or ask your own question.