1
$\begingroup$

I've got a Lego Mindstorms EV3 with EV3DEV and EV3-Python installed. I wanted to try out artificial intelligence with the robot, and my first project is going to be to get the robot to try and recognize some images (using convolutions) and do an action related to the image it has seen. However, I can't find a way to use Tensorflow (or any AI module for that matter) on an EV3. Does anyone know how to incorporate Tensorflow or any other modules into the EV3? Help would be gladly appreciated.

$\endgroup$
0
$\begingroup$

There is a YouTube video LEGO EV3 Raspberry Pi Tensorflow Sorting Machine by ebswift that should help you although you will need a Raspberry Pi. From the abstract:

This is a sorting machine based on the EV3 45544 education kit sorting machine. The colour sorting camera is substituted with a Raspberry Pi with the v2 camera. The EV3 is controlled over wifi via RPyC and the object recognition work is done via Tensorflow.

A viewer asked: Can you share links about how to train a model on the PC and move the trained model to the Pi?

ebswifft replied: Glad you like it :). Training the model on the PC is following the guide for Tensorflow for Poets https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/. All you do is install Tensorflow on the Raspberry Pi, clone the github repository onto it and copy the model you trained from the codelabs article onto the Raspberry Pi to run the classifier as per step 5 of the codelabs article. Onboard image capture on the Raspberry Pi is just done using picamera. I used a tensorflow version that a user compiled for the Raspberry Pi from here: https://github.com/samjabrahams/tensorflow-on-raspberry-pi/issues/92. I might do up more of a general step-by-step sometime on my site, I'll report back if I can get that up and running.

There is also BrickClassifi3r:

This Lego Mindstorms EV3 robot uses a neural network to recognize a cube, a cylinder or a small cube put on a conveyor belt. See the video how it works. Each object on the conveyor belt is scanned by an IR-sensor every 40ms for about a second. The resulting data are 24 distance values representing one of the three objects. This data is fed into the neural network on the robot to classify the object within 180ms. In a test with 300 objects it reaches 95,6% accuracy. The neural network has been trained before by a machine learning algorithm with TensorFlow on a PC using a set of 375 training examples - 125 examples for each object.

The ev3-myo project uses LEGO Mindstorms EV3, TRACK3R, a Myo armband and TensorFlow for gesture recognition.

$\endgroup$
  • $\begingroup$ The first one looks great, but I don't have the source code for it. The second one also looks appealing, however, I don't understand how he's connecting the neural network to the robot, as the tensorflow recognitions aren't visible in the .rtf files. Also, this doesn't really how to implement the code. Any ideas on how to do that? $\endgroup$ – shreyshrey May 15 at 1:00
  • $\begingroup$ I added some more information for the first option. Also check out the ev3-mvo project in my answer. $\endgroup$ – Brian O'Donnell May 15 at 1:15
  • $\begingroup$ If you haven't used TensorFlow for object detection I suggest you start there before diving into the EV3. $\endgroup$ – Brian O'Donnell May 15 at 1:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.