Programming Lego Mindstorms with Python

by

Stoyan Shopov (@hrastche)

from

Welcome to Progamming Lego Mindstorms robots with Python.

My name is Stoyan and I am a solution architect from Innodev.

This is a story about how dreams come true. Ever since I remember I've wanted a robot of my own. I don't know how this idea got planted but I used to make robots out of toilet paper rolls, cigarette boxes I found on the street or yogurt containers. When I was a bit older I even joined the model maker club.

Then life happened, the dream faded and it wasn't until my 7-year old son Hugo started showing interest in Lego that I thought it was time to revisit the dream.

So I did some research on robotics kits. I was looking for:

  • an all in one kit, meaning I don't have to buy anything else to get going;
  • something open ended, meaning that there is room for imagination to build many models
  • a vibrant community with helpful resources
  • and of course it has to be programmable in Python.

So early this year I got myself an expensive birthday present on behalf of the family and it was Lego Mindstorms.

At this stage I would like to make the disclaimer that I am not sponsored by Lego in any way and check by a show of hands, who is already living the dream and playing with Mindstorms? Great!

So I acted surprised when I unwrapped my present. We opened the box and this is what we found:

Technic parts, controller brick, motors, sensors, cables and one set of build instructions.

It may not look like much but it's amazing what you can build with the 601 pieces you get out of the box.

The other good news is that you can mix it up with any of your existing Technic parts to make even bigger creations.

And if you want more there are sites which would 3D print your custom Technic parts or sites which sell classic Lego to Technic connectors.

All these parts actually have a name, so here they are in a word cloud based on total count per part.

Then we went online and downloaded PDFs with more models to build.

These are some of the models you can find on the official web page.

There are many others contributed by the community.

The models are a bit more difficult than standard Lego. The instructions range from 50 to 300 steps and it can take sometimes 6-8 hours for my son to build a model and he is pretty good at this.

The ingenuity in them is facinating, especially how the motors drive the contraptions. It's a somewhat different way of thinking compared to writing software, but after awhile you get used to it and start seeing the patterns.

For our very first model we chose to build the EV3STORM which is a rolling, skating, talking, shooting robot. The one on the top left. The kids called it Betty.

Once we built Betty, we had to give her the gift of life by programming her. Officially to do this we had to download the Lego Mindstorms Porgammer iPad app, connect to the EV3 brick via Bluetooth, select the right program (or mission as they call it) and load it up.

This shows the graphical programming language used to program Mindstorms. It's a LabVIEW dialect: you drag components from the toolbox to the canvas, connect and configure them, then save and upload.

These are the tools available.

There are actions to control the motors, steering, display, sound and light.

There are flow control constructs such as start, wait, loops, switch and interrupt.

Of course you can also control the sensors.

As well as perform data and logic operations such as round, random, compare, write text, assign variables, etc.

Finally there are advanced operations for file access, messaging, bluetooth access, access raw sensor values, start, stop and comment.

So, why are we talking about LabVIEW at a Python conference?

There are two reasons for this.

  1. These same interfaces are available through Python.

  2. It's good to understand LabVIEW so that you can read these programs and translate them into Python. In fact the .ev3 project files are just zip files and if you unzip them you'll find all the sounds, images and even an XML file which stores the logic.

.EV3 files

  • Lego Mindstorms LabVIEW projects have a .ev3 extension.

  • .ev3 files can be unzipped with 7-zip to a folder.

  • Within the zip are:

    • .x3a files

    • .laz is also a .zip file which contains assets such as images

    • .rgf

    • .rsf

    • .ev3p is an XML file which captures the program's logic

    • .lvprojx is an XML file

So how do we get started with Python? For this you need ev3dev: the Debian Linux-based OS which can run on Lego Mindstorms compatible platforms.
The EV3 brick runs Linux and has micro SD card slot for dual boot.

Getting started with ev3dev

  1. Buy an micro SD card (<32GB)
  2. Buy a wireless adapter (e.g. Edimax N150 USB Nano)
  3. Download the latest ev3dev image file
  4. Flash the SD card with Etcher (or equivalent)
  5. Boot ev3dev
  6. Set up a network connection
  7. Connect to the EV3 via SSH (e.g. Bitvise SSH Client or Pythonista with StaSh)
  8. Write some code [Python | JavaScript | Java | Go | C | C++ | Ruby | Perl]

How to run Python scripts on EV3

  • Via SSH
    • Username: robot
    • Password: maker
  • Via Brickman
    • Files must start with the shebang: #!/usr/bin/env python3
    • chmod +x file to make it executable
    • Need Unix-style line endings otherwise the script just exits
      • Run sed -i 's/\r//g' file to fix the line endings
      • Alternatively, write a .sh file to call the script: python3 file
  • Via RPyC in Jupyter notebooks

How to stop Python scripts on EV3

  • If running from an SSH prompt press Ctrl + C
  • If running via Brickman press and hold the backspace button
  • Worst case: take the batteries out

So what can you do with Python?

These are the things you can control.

Let's look at them one at a time!

EV3 brick

LCD screen monochrome 178x128 pixels Buttons (up, down, left, right, enter, backspace)
LEDs left and right with colors (red, green, amber, orange, yellow) Speaker
1 micro SD card slot 1 USB port
8 EV3 cable ports for motors and sensors

In [ ]:
import sys
import time
import rpyc

host = '192.168.198.70' #host name or IP address of the EV3

try:
    conn = rpyc.classic.connect(host) 
    ev3 = conn.modules['ev3dev.ev3'] #import ev3dev.ev3 remotely
except:
    print(sys.exc_info()[1])
finally:
    from ev3dev import ev3 #RPyC failed so we must be running on the EV3
    
ev3.Sound.beep() #success!

In [ ]:
#WORKING WITH SOUND:

ev3.Sound.set_volume(100) #set volume to 100%

ev3.Sound.beep().wait() #wait for the finish before continuing with the program

ev3.Sound.tone(1000, 2000).wait() # play a single 1000Hz tone for 2 seconds

#musical note to frequecy lookup: http://pages.mtu.edu/~suits/notefreqs.html
G =(392, 500, 50) #( frequency in Hz, duration in ms, delay in ms)
E = (329.63, 2000, 50)
F = (349.23, 500, 50)
D = (293.66, 2000, 50)
ev3.Sound.tone([G, G, G, E, F, F, F, D]).wait() #Bethoven Symphony #5

ev3.Sound.play_song((
    ('G4', 'e3'), ('G4', 'e3'), ('G4', 'e3'), ('E4', 'h'),
    ('F4', 'e3'), ('F4', 'e3'), ('F4', 'e3'), ('D4', 'h')))

#use pydub to convert mp3 to wav
ev3.Sound.play('sounds/r2d2.wav').wait() 

ev3.Sound.speak("Luke, I am your father")

In [ ]:
# A long time ago in a galaxy far, far away
ev3.Sound.play_song((
    ('D4', 'e3'),      # intro anacrouse
    ('D4', 'e3'),
    ('D4', 'e3'),
    ('G4', 'h'),       # meas 1
    ('D5', 'h'),
    ('C5', 'e3'),      # meas 2
    ('B4', 'e3'),
    ('A4', 'e3'),
    ('G5', 'h'),
    ('D5', 'q'),
    ('C5', 'e3'),      # meas 3
    ('B4', 'e3'),
    ('A4', 'e3'),
    ('G5', 'h'),
    ('D5', 'q'),
    ('C5', 'e3'),      # meas 4
    ('B4', 'e3'),
    ('C5', 'e3'),
    ('A4', 'h.')
))

In [ ]:
#WORKING WITH LEDs:

#configure the settings for the left LEDs
ev3.Leds.set(ev3.Leds.LEFT, brightness_pct=0.5, trigger='timer') #check ev3.Leds.triggers for options
ev3.Leds.set(ev3.Leds.LEFT, delay_on=3000, delay_off=500)

#possible colors are RED, GREEN, AMBER, ORGANGE, YELLOW
ev3.Leds.set_color(ev3.Leds.LEFT, ev3.Leds.RED)

#set the right LEDs to green at 100% brightness
ev3.Leds.set_color(ev3.Leds.RIGHT, ev3.Leds.GREEN, pct=100) 
    
ev3.Leds.all_off()

In [ ]:
#WORKING WITH BUTTONS:

btn = ev3.Button()

say = lambda sentence: ev3.Sound.speak(sentence)

def left(state):
    say('Left button {0}'.format('pressed' if state else 'released'))
    
def right(state):
    say('Right button {0}'.format('pressed' if state else 'released'))
    
def up(state):
    say('Up button {0}'.format('pressed' if state else 'released'))
    
def down(state):
    say('Down button {0}'.format('pressed' if state else 'released'))
    
def enter(state):
    say('Enter button {0}'.format('pressed' if state else 'released'))
    
def backspace(state):
    say('Backspace button {0}'.format('pressed' if state else 'released'))

In [ ]:
#WORKING WITH BUTTONS (continued): 

btn.on_left = left
btn.on_right = right
btn.on_up = up
btn.on_down = down
btn.on_enter = enter
btn.on_backspace = backspace

while True:
    #Check for currenly pressed buttons. 
    #If the new state differs from the old state, 
    #call the appropriate button event handlers.
    btn.process() 
    
    #exit if both the left and right buttons are pressed simultaneously
    if btn.check_buttons(buttons=['left','right']):
        break
    time.sleep(0.1)

In [ ]:
#WORKING WITH SCREEN:

#run change foreground virtual terminal (chvt) command to get exclusive use of the screen
!sudo chvt 6

from PIL import Image, ImageDraw, ImageFont

screen = ev3.Screen()
screen.clear()

#(0, 0) is top left and (177, 127) is bottom right coordinates
screen.draw.text((60,40), 'Help me Obi Wan Kenobi!', font=ImageFont.load('ncenB24'))
screen.update()

time.sleep(5)

logo = Image.open('/home/robot/images/tree.bmp')
screen.image.paste(logo, (0,0))
screen.update()

time.sleep(5)

In [ ]:
#WORKING WITH SCREEN (continued):

screen.clear()

width, height = screen.shape
#screen.draw returns PIL.ImageDraw instance
screen.draw.line((0, 0, width, height), fill='black') #diagonal from top left to bottom right
screen.draw.line((0, height, width, 0), fill='black') #diagonal from bottom left to top right
screen.update()

time.sleep(5)

screen.draw.rectangle((0, 0, 177, 127), fill='black')
screen.update()

time.sleep(5)

#there are also: arc, ellipse, pieslice, point, polygon, ... all the PIL/Pillow goodness

#release screen
!sudo chvt 1

Touch sensor


In [ ]:
#WORKING WITH TOUCH SENSOR:

ts = ev3.TouchSensor()
assert ts.connected

while not ts.is_pressed:
    time.sleep(0.5)

Color sensor


In [ ]:
#WORKING TO COLOR SENSOR:

ts = ev3.TouchSensor()

cl = ev3.ColorSensor()
cl.mode = 'COL-COLOR' #returns an integer [0,7]
#cl.mode='COL-REFLECT' #measures reflected light intensity and returns an integer [0,100]
#cl.mode='COL-AMBIENT' #measures ambient light intensity and returns an integer [0,100]
#cl.mode='RGB-RAW' #returns RGB tuple ([0,1020], [0,1020], [0,1020])

colors = ('unknown black blue green yellow red white brown'.split())

while not ts.value():
    ev3.Sound.speak(colors[cl.value()]).wait()
    time.sleep(1)

Infrared sensor


In [ ]:
#WORKING WITH INFRARED SENSOR:

ir = ev3.InfraredSensor()
ir.mode = 'IR-PROX'
#A measurement of the distance between the sensor and the remote, as a percentage. 
#100% is approximately 70cm

lm = ev3.LargeMotor('outB')
rm = ev3.LargeMotor('outC')

lm.run_forever(speed_sp=360)
rm.run_forever(speed_sp=360)

while True:
    if ir.proximity < 10: #or ir.value()
        lm.stop(stop_action='brake')
        rm.stop(stop_action='brake')
        break
    time.sleep(0.1)

Remote control


In [ ]:
#WORKING WITH REMOTE CONTROL:

remote1 = ev3.RemoteControl(channel=1) 
remote2 = ev3.RemoteControl(channel=2)

say = lambda sentence: ev3.Sound.speak(sentence)

remote1.on_red_up = lambda x: say("Red up pressed")
remote1.on_red_down = lambda x: say("Red down pressed")
remote1.on_blue_up = lambda x: say("Blue up pressed")
remote1.on_blue_down = lambda x: say("Blue up pressed")

remote2.on_red_up = sys.exit
remote2.on_red_down = sys.exit
remote2.on_blue_up = sys.exit
remote2.on_blue_down = sys.exit

while True:
    remote1.process()
    remote2.process()
    time.sleep(0.01)

There are also remote1.beacon event and ev3.BeaconSeaker class related to the IR and remote control

Large motors


In [ ]:
# WORKING WITH LARGE MOTORS:

m = ev3.LargeMotor('outB')

print(m.count_per_m) #number of tacho counts in one meter of travel of the motor
print(m.count_per_rot) #number of tacho counts in one rotation of the motor
#guideline: 1 tacho count = 1 degree, so speed of 360 = 1 revolution per second

#speed_sp is in tacho counts, recommended range is [-1000, 1000]
#stop_action in [break, hold, coast]
m.run_timed(time_sp=5000, speed_sp=250, stop_action="hold")

m.wait_while('running') #wait for the motor to stop moving

m.run_timed(time_sp=5000, speed_sp=-250, stop_action="coast") #go back to initial position

motors reference

STOP_ACTION_BRAKE = 'brake' Power will be removed from the motor and a passive electrical load will be placed on the motor. This is usually done by shorting the motor terminals together. This load will absorb the energy from the rotation of the motors and cause the motor to stop more quickly than coasting.

STOP_ACTION_COAST = 'coast' Power will be removed from the motor and it will freely coast to a stop.

STOP_ACTION_HOLD = 'hold' Does not remove power from the motor. Instead it actively try to hold the motor at the current position. If an external force tries to turn the motor, the motor will push back to maintain its position.

Medium motor


In [ ]:
#WORKING WITH MEDIUM MOTOR:

mm = ev3.MediumMotor()

#shoot a ball in the specified direction
#speed_sp recommended range is [-1400, 1400]

mm.run_to_rel_pos(speed_sp=900, position_sp=-1080)

while 'running' in mm.state:
    time.sleep(0.1)

The medium motor has the same attributes and methods as the large motors. It is less powerful but faster and more precise, thus its speed_sp range is higher [-1400, 1400]

Putting it all together: EV3D4


In [ ]:
import sys
import time
import threading
import signal 

import rpyc

host = '192.168.1.4' 

conn = rpyc.classic.connect(host) 
ev3 = conn.modules['ev3dev.ev3'] 

ev3.Sound.beep()

#The 'done' event will be used to signal the threads to stop:
done = threading.Event()

In [ ]:
def move(done):
    lm = ev3.LargeMotor('outB'); assert lm.connected
    
    rm = ev3.LargeMotor('outC'); assert rm.connected
    
    cl = ev3.ColorSensor(); assert cl.connected
    cl.mode='COL-AMBIENT'
    
    speed = -250 #cl.value() is too low

    lm.run_forever(speed_sp=speed)
    rm.run_forever(speed_sp=speed)

    while not done.is_set():
        time.sleep(1)  
    
    #stop both motors
    lm.stop(stop_action='brake')
    rm.stop(stop_action='brake')
    lm.wait_while('running')
    rm.wait_while('running')
    
    #run around in a circle
    done.clear()
    lm.run_forever(speed_sp=speed)
    
    while not done.is_set():
        time.sleep(1)
        
    lm.stop(stop_action='brake')
    lm.wait_while('running')

In [ ]:
def feel(done):
    ir = ev3.InfraredSensor(); assert ir.connected
    ts = ev3.TouchSensor(); assert ts.connected

    screen = ev3.Screen()
    sound = ev3.Sound()

    screen.draw.text((60,40), 'Going for a walk')
    screen.update()

    while ir.proximity > 30:
        if done.is_set(): 
            break
        time.sleep(0.1)

    done.set() #this will set it running in a circle
    
    ev3.Leds.set_color(ev3.Leds.LEFT, ev3.Leds.RED)
    ev3.Leds.set_color(ev3.Leds.RIGHT, ev3.Leds.RED)
    
    screen.clear()
    screen.draw.text((60,20), 'There is something is front of me')
    screen.update()
    
    while not ts.is_pressed:
        sound.speak("Where should I go next?").wait()
        time.sleep(0.5)
    
    done.set() #will stop the circle dance

In [ ]:
# We also need to catch SIGINT (keyboard interrup) and SIGTERM (termination
# signal from brickman) and exit gracefully:
def signal_handler(signal, frame):
    done.set()

signal.signal(signal.SIGINT,  signal_handler)
signal.signal(signal.SIGTERM, signal_handler)

# Now that we have the worker functions defined, lets run those in separate
# threads.
move_thread = threading.Thread(target=move, args=(done,))
feel_thread = threading.Thread(target=feel, args=(done,))

move_thread.start()
feel_thread.start()

# The main thread will wait for the 'back' button to be pressed.  When that
# happens, it will signal the worker threads to stop and wait for their completion.
btn = ev3.Button()
while not btn.backspace and not done.is_set():
    time.sleep(1)

done.set()
move_thread.join()
feel_thread.join()  

ev3.Sound.speak('Farewell and good bye!').wait()
ev3.Leds.all_off()

Cool NXT/EV3 Mindstorms videos on YouTube

in no particular order

https://www.youtube.com/watch?v=HZ_0qk7Yd7A pixy camera for lego http://charmedlabs.com/default/pixy-lego/ http://www.tribotix.com/Products/CharmedLabs/CharmedLabs.htm vision command lego cam +vision recognition sudoku solver https://www.youtube.com/watch?v=Mp8Y2yjV4fU https://www.youtube.com/watch?v=gA-zX3Tcjh0 vision dragster https://www.youtube.com/watch?v=7v-anhPAAVw helicopter https://www.youtube.com/watch?v=Oq4vm5Wq2-Y shooter https://www.youtube.com/watch?v=l7O1dVTMHyA space elevator https://www.youtube.com/watch?v=g2xDFeCnaOY paper plane folding machine https://www.youtube.com/watch?v=brQMq8Vz4QA egg decorating robot https://www.youtube.com/watch?v=aQru5c6GwFs http://jkbrickworks.com/ev3-egg-decorator/ rock paper scisors https://www.youtube.com/watch?v=5Rmd5M5ATcU 3d printer lego house builder https://www.youtube.com/watch?v=Sjix76QjtMU automated toilet https://www.youtube.com/watch?v=62YnuPKkdws simon sings music game https://www.youtube.com/watch?v=DFx8u8hV0-Y https://www.youtube.com/watch?v=dHmgaLgFRGM printer spirograph https://www.youtube.com/watch?v=1Ihjh_F7jn0 http://jkbrickworks.com/ robotilus.com forklift slots bot ai dragster tic tac toe card shuffler toilet paper challenge https://www.youtube.com/watch?v=urdHZAZ0brM rally car https://www.youtube.com/watch?v=uYygy5tTT0s autonomous car https://www.youtube.com/watch?v=zEou8mtcxhU pencil sharpener https://www.youtube.com/watch?v=-ozR2q1LxN0 logalas pringles vending machine https://www.youtube.com/channel/UCtfEIPhzdWCz7MEiiOMLS0w clever car https://www.youtube.com/watch?v=kAvduVtT748 outlander https://www.youtube.com/watch?v=znHbTL6Oxhg truck https://www.youtube.com/watch?v=2s9C-2m7m_w bike https://www.youtube.com/watch?v=2YtpPu-SI5M dancing robot https://www.youtube.com/watch?v=8H6ldqkI3uM http://arthursacek.com/lego-danc3r elephant https://www.youtube.com/watch?v=oYJCaeYTWnw trex https://www.youtube.com/watch?v=rgTDLO8i8cQ drover dog https://www.youtube.com/watch?v=Pgnpvx7JEcA m&m color sorter https://www.youtube.com/watch?v=B2QUI1gIUS4 fave 15 https://www.youtube.com/watch?v=HJ3XLFsd4zI&list=PLinR9IaPy4sr5amP8orLME9QyY-dFcztX weaving machine https://www.youtube.com/watch?v=IPIJsdvDjsc loom https://www.youtube.com/watch?v=iqlxO5RYyGU loom https://www.youtube.com/watch?v=xaj2M2KtH-w piano https://www.youtube.com/watch?v=TDvcTa3abRM https://www.youtube.com/watch?v=TlmEpdyKwpI https://www.youtube.com/watch?v=9m3qFaF2t7g https://www.youtube.com/watch?v=5smytm90z80 warehouse loading https://www.youtube.com/watch?v=GToA2tOVyHg ai vs human car park https://www.youtube.com/watch?v=KseycJFgi1U cleaner https://www.youtube.com/watch?v=mGbdfx1rP4c ldraw.org milling machine https://www.youtube.com/watch?v=oF0pMILT7_Y https://www.youtube.com/watch?v=pX1cO2XhMrg lathe https://www.youtube.com/watch?v=E4k7Nkr7sRg http://arthursacek.com/ chess playing https://www.youtube.com/watch?v=y3gJ2ERa0_8 https://www.youtube.com/watch?v=-rR3omAdOM0 $30,000 making the 100,000-piece chess set. sign language https://www.youtube.com/watch?v=aTk7guHpHLk cradle rock https://www.youtube.com/watch?v=-ZIckCvTnek shirt folder https://www.youtube.com/watch?v=gEmwU_t0qsE

Thanks!