Python 多线程

1 Python 全局解释器锁(GIL)

对Python虚拟机的访问由全局解释器锁(Global interpreter lock)来控制,正是这个锁能够保证同一时刻只有一个线程在运行,在多线程的环境中,Python虚拟机按照如下的方式进行。

  1. 设置GIL
  2. 切换到一个线程去运行
  3. 运行
    • 指定数量的字节码的指令,或者
    • 线程主动让出控制(可以调用time.sleep(0))
  4. 把线程设置为睡眠状态
  5. 解锁GIL
  6. 再次重复以上所有步骤

2 非多线程


In [5]:
from time import sleep, ctime
def loop0():
    print 'start loop 0 at: ',ctime()
    sleep(4)
    print 'loop 0 done at: ', ctime()
def loop1():
    print 'start loop 1 at: ', ctime()
    sleep(2)
    print 'loop 1 done at: ', ctime()
def main():
    print 'starting at: ', ctime()
    loop0()
    loop1()
    print 'all Done at:', ctime()
main()


starting at:  Wed Feb  1 15:45:34 2017
start loop 0 at:  Wed Feb  1 15:45:34 2017
loop 0 done at:  Wed Feb  1 15:45:38 2017
start loop 1 at:  Wed Feb  1 15:45:38 2017
loop 1 done at:  Wed Feb  1 15:45:40 2017
all Done at: Wed Feb  1 15:45:40 2017

3 Thread 模块

  1. start_new_thread(function, args, kwargs=None)
    产生一个新的线程,用指定参数和可选参数来调用该函数
  2. allocate_lock()
    分配一个LockType类型的对象
  3. exit()
    让该线程退出
    ## 3.1 多线程1

In [4]:
import thread 
from time import ctime, sleep
def loop0():
    print 'start loop 0 at: ',ctime()
    sleep(4)
    print 'loop 0 done at: ', ctime()
def loop1():
    print 'start loop 1 at: ', ctime()
    sleep(2)
    print 'loop 1 done at: ', ctime()
def main():
    print 'start at: ', ctime()
    thread.start_new_thread(loop0, ())
    thread.start_new_thread(loop1, ())
    sleep(6)
    print 'all Done at: ', ctime()
main()


start at:  Wed Feb  1 15:45:26 2017
start loop 1 at:  Wed Feb  1 15:45:26 2017start loop 0 at: 
Wed Feb  1 15:45:26 2017
loop 1 done at:  Wed Feb  1 15:45:28 2017
loop 0 done at:  Wed Feb  1 15:45:30 2017
all Done at:  Wed Feb  1 15:45:32 2017

3.2 多线程2

使用锁


In [9]:
import thread
from time import ctime, sleep
loops = [4, 2]
def loop(nloop, nsec, lock):
    print 'start loop', nloop, 'at', ctime()
    sleep(nsec)
    print 'loop', nloop, 'done at:', ctime()
    lock.release()
def main():
    print 'starting at: ', ctime()
    locks = []
    nloops = range(len(loops))
    for i in nloops:
        lock= thread.allocate_lock()
        lock.acquire()
        locks.append(lock)
    for i in nloops:
        thread.start_new_thread(loop,(i, loops[i], locks[i]))
    for i in nloops:
        while locks[i].locked():
            pass
    print 'all Done at: ', ctime()
main()


starting at:  Wed Feb  1 15:55:39 2017
start loop 1 atstart loop  Wed Feb  1 15:55:39 2017
0 at Wed Feb  1 15:55:39 2017
loop 1 done at: Wed Feb  1 15:55:41 2017
loop 0 done at: Wed Feb  1 15:55:43 2017
all Done at:  Wed Feb  1 15:55:43 2017

4 Threading 模块

  1. Thread 表示线程执行对象
  2. Lock
    锁对象
  3. Condition
    条件变量对象能让一个线程能够停下来,等待其他线程满足了某个条件
  4. Event
    通用的事件变量,多个线程等待某个事情的发生
  5. Timer
    等待一定时间后才能发生。

4.1 Thread 类使用

使用方法

  1. 创建一个Thread实例,传给它一个函数
  2. 创建一个Thread实例,传给它一个可调用的类对象
  3. 从Thread派生出一个子类,创建一个这个子类的实例(推荐)

4.1.1 传函数


In [10]:
import threading
from time import ctime, sleep
loops = [4, 2]
def loop(nloop, nsec):
    print 'start loop', nloop, 'at: ', ctime()
    sleep(nsec)
    print 'loop', nloop, 'done at: ', ctime()
def main():
    print 'starting at: ', ctime()
    threads = []
    nloops = range(len(loops))
    for i in nloops:
        t = threading.Thread(target=loop, args=(i, loops[i]))
        threads.append(t)
    for i in nloops:
        threads[i].start()
    for i in nloops:
        threads[i].join()
    print 'all Done at: ', ctime()
main()


starting at:  Wed Feb  1 16:16:45 2017
start loop 0 at:  Wed Feb  1 16:16:45 2017
start loop 1 at:  Wed Feb  1 16:16:45 2017
loop 1 done at:  Wed Feb  1 16:16:47 2017
loop 0 done at:  Wed Feb  1 16:16:49 2017
all Done at:  Wed Feb  1 16:16:49 2017

4.1.2 传递对象


In [14]:
import threading
from time import ctime, sleep
loops = [4, 2]
class ThreadFunc(object):
    def __init__(self, func, args, name=''):
        self.name = name
        self.func = func
        self.args = args
    def __call__(self):
        apply(self.func, self.args)
def loop(nloop, nsec):
    print 'start loop', nloop, 'at: ', ctime()
    sleep(nsec)
    print 'loop', nloop, 'done at: ',ctime()
def main():
    print 'starting at: ', ctime()
    threads = []
    nloops = range(len(loops))
    for i in nloops:
        t = threading.Thread(target=ThreadFunc(loop, (i, loops[i]), loop.__name__))
        threads.append(t)
    for i in nloops:
        threads[i].start()
    for i in nloops:
        threads[i].join()
main()


starting at:  Wed Feb  1 16:37:20 2017
start loop 0 at:  Wed Feb  1 16:37:20 2017
start loop 1 at:  Wed Feb  1 16:37:20 2017
loop 1 done at:  Wed Feb  1 16:37:22 2017
loop 0 done at:  Wed Feb  1 16:37:24 2017

4.1.3 派生类


In [21]:
import threading
from time import ctime, sleep
class MyThread(threading.Thread):
    def __init__(self, func, args, name =''):
        threading.Thread.__init__(self)
        self.name = name
        self.func = func
        self.args = args
    def getResult(self):
        return self.result
    def run(self):
        print 'starting', self.name, 'at: ', ctime()
        self.result = apply(self.func, self.args)
        print self.name, 'finished at: ', ctime()
def fib(x):
    sleep(0.005)
    if x < 2: return x
    return fib(x-1) + fib(x-2)
def fac(x):
    sleep(0.1)
    if x < 2: return x
    return fac(x-1) * x

def sum(x):
    sleep(0.1)
    if x<2: return x
    return x+sum(x-1)
funcs = [fib, fac, sum]
n = 12 
def main():
    nfuncs = range(len(funcs))
    print '*** SINGLE THREAD'
    for i in nfuncs:
        print 'starting', funcs[i].__name__, 'at: ', ctime()
        print funcs[i](n)
        print funcs[i].__name__, 'finished at: ', ctime()
    print '\n *** MULTI THREAD'
    threads = []
    for i in nfuncs:
        t = MyThread(funcs[i], (n,), funcs[i].__name__)
        threads.append(t)
    for i in nfuncs:
        threads[i].start()
    for i in nfuncs:
        threads[i].join()
        print threads[i].getResult()
    print 'all done'
main()


*** SINGLE THREAD
starting fib at:  Wed Feb  1 16:59:09 2017
144
fib finished at:  Wed Feb  1 16:59:11 2017
starting fac at:  Wed Feb  1 16:59:11 2017
479001600
fac finished at:  Wed Feb  1 16:59:13 2017
starting sum at:  Wed Feb  1 16:59:13 2017
78
sum finished at:  Wed Feb  1 16:59:14 2017

 *** MULTI THREAD
starting fib at:  Wed Feb  1 16:59:14 2017
starting fac at:  Wed Feb  1 16:59:14 2017
starting sum at:  Wed Feb  1 16:59:14 2017
facsum  finished at: finished at:   Wed Feb  1 16:59:15 2017Wed Feb  1 16:59:15 2017

fib finished at:  Wed Feb  1 16:59:17 2017
144
479001600
78
all done

4 Queue 模块

使用Queue结构来共享线程之间的数据

函数 描述
queue(size) 构造函数
qsize() 返回队列的大小
empty() 是否为空
full() 是否满
put(item, block=0) 把item放入到队列中,如果block不为0,函数阻塞到队列中有空间为止
get(block=0) 从队列中获取对象,如果block中不为0,函数阻塞只队列中有对象

In [24]:
from random import randint
from time import sleep
from Queue import Queue
import threading
class MyThread(threading.Thread):
    def __init__(self, func, args, name =''):
        threading.Thread.__init__(self)
        self.name = name
        self.func = func
        self.args = args
    def getResult(self):
        return self.result
    def run(self):
        print 'starting', self.name, 'at: ', ctime()
        self.result = apply(self.func, self.args)
        print self.name, 'finished at: ', ctime()
def writeQ(queue):
    print 'producting object for Q...',queue.put('xxx', 1)
    print 'size now', queue.qsize()
def readQ(queue):
    val = queue.get(1)
    print 'consumed object from Q...', queue.qsize()
def writer(queue, loops):
    for i in range(loops):
        writeQ(queue)
        sleep(randint(1, 3))
def reader(queue, loops):
    for i in range(loops):
        readQ(queue)
        sleep(randint(2, 5))
funcs = [writer, reader]
nfuncs = range(len(funcs))
def main():
    nloops = randint(2,5)
    q = Queue(32)
    threads = []
    for i in nfuncs:
        t= MyThread(funcs[i], (q, nloops), funcs[i].__name__)
        threads.append(t)
    for i in nfuncs:
        threads[i].start()
    for i in nfuncs:
        threads[i].join()
    print 'all Done'
main()


starting writer at:  Wed Feb  1 18:39:33 2017
producting object for Q... None
size now 1
starting reader at:  Wed Feb  1 18:39:33 2017
consumed object from Q... 0
producting object for Q... None
size nowconsumed object from Q...  00

producting object for Q... None
size now 1
producting object for Q... None
size now 2
consumed object from Q... 1
writer finished at:  Wed Feb  1 18:39:42 2017
consumed object from Q... 0
reader finished at:  Wed Feb  1 18:39:48 2017
all Done

In [26]:
import Tkinter
top = Tkinter.Tk()
hello = Tkinter.Label(top, text = 'hello world!')
hello.pack()
quit = Tkinter.Button(top, text='quit', command=top.quit, bg='red', fg='white')
quit.pack(fill = Tkinter.X, expand=1)
Tkinter.mainloop()

In [ ]: