SystemML Scala tutorial

This tutorial includes simple example to run DML script and display output.

Install SystemML jar file and configure kernel

Please visit http://systemml.apache.org/install-systemml.html site to know "How to configure Toree(Scala) Kernel".

This notebook is supported with SystemML 0.14.0 and above.



In [ ]:

    
import org.apache.sysml.api.mlcontext.MLContext
import org.apache.sysml.api.mlcontext.ScriptFactory.dml
import org.apache.spark.sql.SparkSession

val sparkSession = SparkSession.builder().master("local").appName("Tutorial").getOrCreate()
val ml = new MLContext(sparkSession)

print ("Spark Version: " + sc.version)
print ("\nSystemML Version: " + ml.version())
print ("\nBuild Time: " + ml.buildTime())

Run the script



In [ ]:

    
val sumScript = """
X = rand(rows=100, cols=10)
sumX = sum(X)
outMatrix = matrix(sumX, rows=1, cols=1)
write(outMatrix, " ", format="csv")
"""

val script = dml(sumScript).out("outMatrix")
val out = ml.execute(script)
val outMatrix = out.getDataFrame("outMatrix")

Display the output



In [ ]:

    
outMatrix.show