YARN on Stampede

This is an advanced example intended to show the capabilities. To execute this example a Stampede account is required.

1. Setting up the enviroment

Before running the following example you will need to set a passwordless ssh connection between your tutorial account and your account on TACC Stampede.

First, login through ssh to the edge node with the tutorial account and set up a public key:

ssh <user>@sc15-aws.radical-cybertools.org
ssh-keygen -t rsa

Hit Return until you are in the terminal command prompt again.

Then execute the following command

cat .ssh/id_rsa.pub | ssh <tacc_username>@stampede.tacc.utexas.edu 'cat >> .ssh/authorized_keys'

Verify that everything is set by doing and you can connect without a password request:

ssh <tacc_username>@stampede.tacc.utexas.edu

2. Radical Pilot Setup



In [1]:

    
import os,sys
import radical.pilot as rp
import ast

os.environ["RADICAL_PILOT_DBURL"]="mongodb://ec2-54-221-194-147.compute-1.amazonaws.com:24242/sc15tut"
os.environ["RADICAL_PILOT_VERBOSE"]="DEBUG"

def print_details(detail_object):
    if type(detail_object)==str:
        detail_object = ast.literal_eval(detail_object)
    for i in detail_object:
        detail_object[i]=str(detail_object[i])
    return pd.DataFrame(detail_object.values(), 
             index=detail_object.keys(), 
             columns=["Value"])

3. Create a Radical Pilot Session



In [2]:

    
session = rp.Session()
print "Session id %s"%session.uid
c = rp.Context('ssh')
c.user_id = ""
session.add_context(c)









    



Session id rp.session.ip-10-99-194-113.ec2.internal.iparask.016749.0001

4. Create Pilot and Unit Managers



In [4]:

    
print "Initializing Pilot Manager ..."
pmgr = rp.PilotManager(session=session)
print "Initializing Unit Manager ..."
umgr = rp.UnitManager (session=session,
                           scheduler=rp.SCHED_DIRECT_SUBMISSION)









    



Initializing Pilot Manager ...
Initializing Unit Manager ...

5. Submit the pilot to the Pilot and Unit Managers



In [5]:

    
pdesc = rp.ComputePilotDescription ()
pdesc.resource = "yarn.stampede"  # NOTE: This is a "label", not a hostname
pdesc.runtime  = 60 # minutes
pdesc.cores    = 16
pdesc.cleanup  = False
pdesc.project  = '' #Include the Allocation here
pdesc.queue    = 'development' #You can select a different queue if you want.
# submit the pilot.
print "Submitting Compute Pilot to Pilot Manager ..."
pilot = pmgr.submit_pilots(pdesc)
print "Registering Compute Pilot with Unit Manager ..."
umgr.add_pilots(pilot)









    



Submitting Compute Pilot to Pilot Manager ...
Registering Compute Pilot with Unit Manager ...

6. Submit Compute Units

First create the description of the compute units which define the task to be executed



In [6]:

    
NUMBER_JOBS  = 16
cudesc_list = []
for i in range(NUMBER_JOBS):
    cudesc = rp.ComputeUnitDescription()
    cudesc.environment = {'CU_NO': i}
    cudesc.executable  = "/bin/echo"
    cudesc.arguments   = ['I am CU number $CU_NO']
    cudesc.cores       = 1
    cudesc_list.append(cudesc)

Submit the created Compute Units to the Unit Manager.



In [7]:

    
print "Submit Compute Units to Unit Manager ..."
cu_set = umgr.submit_units (cudesc_list)
print "Waiting for CUs to complete ..."
umgr.wait_units()
print "All CUs completed successfully!"









    



Submit Compute Units to Unit Manager ...
Waiting for CUs to complete ...
All CUs completed successfully!

Printing the output of a Compute Unit



In [8]:

    
for unit in cu_set:
    print "* CU %s, state %s, exit code: %s, stdout: %s" \
     % (unit.uid, unit.state, unit.exit_code, unit.stdout)









    



* CU unit.000000, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2466
2015-11-10 16:27:25 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2466
2015-11-10 16:27:25 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:25 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2467
2015-11-10 16:27:25 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2467
2015-11-10 16:27:25 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:25 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:25 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:25 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 0

* CU unit.000001, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2330
2015-11-10 16:26:32 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2330
2015-11-10 16:26:32 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:32 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2331
2015-11-10 16:26:32 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2331
2015-11-10 16:26:32 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:32 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:32 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:32 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 1

* CU unit.000002, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #873
2015-11-10 16:25:57 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #873
2015-11-10 16:25:57 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:25:57 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #874
2015-11-10 16:25:57 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #874
2015-11-10 16:25:57 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:25:57 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:25:57 INFO  Client:240 - Application completed successfully
2015-11-10 16:25:57 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 2

* CU unit.000003, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2523
2015-11-10 16:27:17 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2523
2015-11-10 16:27:17 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:17 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2524
2015-11-10 16:27:17 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2524
2015-11-10 16:27:17 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:27:17 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:17 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:17 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 3

* CU unit.000004, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2555
2015-11-10 16:27:41 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2555
2015-11-10 16:27:41 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:41 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2556
2015-11-10 16:27:41 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2556
2015-11-10 16:27:41 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:41 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:41 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:41 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 4

* CU unit.000005, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2240
2015-11-10 16:26:24 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2240
2015-11-10 16:26:24 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:26:24 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2241
2015-11-10 16:26:24 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2241
2015-11-10 16:26:24 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:24 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:24 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:24 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 5

* CU unit.000006, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2541
2015-11-10 16:27:10 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2541
2015-11-10 16:27:10 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:10 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2542
2015-11-10 16:27:10 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2542
2015-11-10 16:27:10 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:27:10 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:10 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:10 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 6

* CU unit.000007, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2623
2015-11-10 16:27:34 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2623
2015-11-10 16:27:34 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:27:34 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2624
2015-11-10 16:27:34 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2624
2015-11-10 16:27:34 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:34 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:34 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:34 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 7

* CU unit.000008, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2433
2015-11-10 16:27:48 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2433
2015-11-10 16:27:48 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:48 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2434
2015-11-10 16:27:48 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2434
2015-11-10 16:27:48 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:48 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:48 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:48 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 8

* CU unit.000009, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
pede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #1209
2015-11-10 16:26:04 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #1209
2015-11-10 16:26:04 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:04 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #1210
2015-11-10 16:26:04 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #1210
2015-11-10 16:26:04 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:04 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:04 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:04 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 9

* CU unit.000010, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
ede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2490
2015-11-10 16:26:17 DEBUG Client:1089 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2490
2015-11-10 16:26:17 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:17 DEBUG Client:1032 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2491
2015-11-10 16:26:17 DEBUG Client:1089 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2491
2015-11-10 16:26:17 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:17 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:17 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:17 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@3c1df663
I am CU number 10

* CU unit.000011, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
ede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2437
2015-11-10 16:26:40 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2437
2015-11-10 16:26:40 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:40 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2438
2015-11-10 16:26:40 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2438
2015-11-10 16:26:40 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:26:40 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:40 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:40 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 11

* CU unit.000012, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
ede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2447
2015-11-10 16:26:47 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2447
2015-11-10 16:26:47 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:26:47 DEBUG Client:1032 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2448
2015-11-10 16:26:47 DEBUG Client:1089 - IPC Client (1726925370) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2448
2015-11-10 16:26:47 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:47 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:47 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:47 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@53e2835b
I am CU number 12

* CU unit.000013, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
ede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #1834
2015-11-10 16:26:10 DEBUG Client:1089 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #1834
2015-11-10 16:26:10 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:26:10 DEBUG Client:1032 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #1835
2015-11-10 16:26:10 DEBUG Client:1089 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #1835
2015-11-10 16:26:10 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:10 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:10 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:10 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@3c1df663
I am CU number 13

* CU unit.000014, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
ampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2539
2015-11-10 16:26:55 DEBUG Client:1089 - IPC Client (983803009) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2539
2015-11-10 16:26:55 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:55 DEBUG Client:1032 - IPC Client (983803009) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2540
2015-11-10 16:26:55 DEBUG Client:1089 - IPC Client (983803009) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2540
2015-11-10 16:26:55 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:26:55 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:26:55 INFO  Client:240 - Application completed successfully
2015-11-10 16:26:55 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@531afc6a
I am CU number 14

* CU unit.000015, state Done, exit code: 0, stdout: [... CONTENT SHORTENED ...]
ede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2467
2015-11-10 16:27:03 DEBUG Client:1089 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2467
2015-11-10 16:27:03 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 1ms
2015-11-10 16:27:03 DEBUG Client:1032 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 sending #2468
2015-11-10 16:27:03 DEBUG Client:1089 - IPC Client (1193204385) connection to c557-402.stampede.tacc.utexas.edu/129.114.90.162:8032 from tg824689 got value #2468
2015-11-10 16:27:03 DEBUG ProtobufRpcEngine:250 - Call: getApplicationReport took 0ms
2015-11-10 16:27:03 INFO  Client:792 - Application has completed successfully. Breaking monitoring loop
2015-11-10 16:27:03 INFO  Client:240 - Application completed successfully
2015-11-10 16:27:03 DEBUG Client:97 - stopping client from cache: org.apache.hadoop.ipc.Client@3c1df663
I am CU number 15

7. Always clean up the session



In [9]:

    
session.close ()
del session