Optimize Trained Model

Types of Optimizations Applied for Inference

Remove training-only operations (checkpoint saving, drop out)
Strip out unreachable nodes
Remove debug operations (CheckNumerics)
Fold batch normalization Ops into the pre-calculated weights (super cool)
Fuse adjacent operators

Graph Transform Tool

https://petewarden.com/2016/12/30/rewriting-tensorflow-graphs-with-the-gtt/

https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms

Optimize Models

Summarize Graph Utility



In [ ]:

    
%%bash 

which summarize_graph



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/unoptimized_cpu.pb

Strip Unused Nodes



In [ ]:

    
%%bash

transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_cpu.pb \
--out_graph=/root/models/optimize_me/strip_unused_optimized_cpu.pb \
--inputs='x_observed,y_observed,weights,bias' \
--outputs='add' \
--transforms='
strip_unused_nodes(type=float, shape="1,299,299,3")'



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/strip_unused_optimized_cpu.pb

Remove Nodes



In [ ]:

    
%%bash

transform_graph \
--in_graph=/root/models/optimize_me/strip_unused_optimized_cpu.pb \
--out_graph=/root/models/optimize_me/remove_nodes_optimized_cpu.pb \
--inputs='x_observed,y_observed,weights,bias' \
--outputs='add' \
--transforms='
remove_nodes(op=Identity, op=CheckNumerics) 
fold_constants(ignore_errors=true) 
fold_batch_norms 
fold_old_batch_norms'



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/remove_nodes_optimized_cpu.pb

Fold Constants



In [ ]:

    
%%bash

transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_cpu.pb \
--out_graph=/root/models/optimize_me/fold_constants_optimized_cpu.pb \
--inputs='x_observed,y_observed,weights,bias' \
--outputs='add' \
--transforms='
fold_constants(ignore_errors=true)'



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/fold_constants_optimized_cpu.pb

Quantize Weights



In [ ]:

    
%%bash

transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_cpu.pb \
--out_graph=/root/models/optimize_me/quantized_optimized_cpu.pb \
--inputs='x_observed,y_observed,weights,bias' \
--outputs='add' \
--transforms='quantize_weights'



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/quantized_optimized_cpu.pb

Perform All Optimizations at Once



In [ ]:

    
%%bash

transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_cpu.pb \
--out_graph=/root/models/optimize_me/fully_optimized_cpu.pb \
--inputs='x_observed,y_observed' \
--outputs='add' \
--transforms='
strip_unused_nodes(type=float, shape="1,299,299,3") 
remove_nodes(op=Identity, op=CheckNumerics) 
fold_constants(ignore_errors=true) 
fold_batch_norms 
fold_old_batch_norms
quantize_weights
obfuscate_names'



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/fully_optimized_cpu.pb

Obfuscate Node Names



In [ ]:

    
%%bash

transform_graph \
--in_graph=/root/models/optimize_me/unoptimized_cpu.pb \
--out_graph=/root/models/optimize_me/obfuscate_names_cpu.pb \
--inputs='x_observed,y_observed,weights,bias' \
--outputs='add' \
--transforms='
obfuscate_names'



In [ ]:

    
%%bash

ls -l /root/models/optimize_me/



In [ ]:

    
%%bash

summarize_graph --in_graph=/root/models/optimize_me/obfuscate_names_cpu.pb



In [ ]: