Load and save a GeoTiff

In this notebook the user can load a GeoTiff, extract a Tile and its metadata information to create a new GeoTiff, i.e., a clone. The clone is then upload to HDFS using a System command.

Dependencies



In [37]:

    
import sys.process._
import geotrellis.proj4.CRS
import geotrellis.raster.io.geotiff.writer.GeoTiffWriter
import geotrellis.raster.io.geotiff.{SinglebandGeoTiff, _}
import geotrellis.raster.{CellType, DoubleArrayTile}
import geotrellis.spark.io.hadoop._
import geotrellis.vector.{Extent, ProjectedExtent}
import org.apache.spark.mllib.linalg.Vector
import org.apache.spark.rdd.RDD
import org.apache.spark.{SparkConf, SparkContext}

//Spire is a numeric library for Scala which is intended to be generic, fast, and precise.
import spire.syntax.cfor._

Read a GeoTiff file



In [40]:

    
var projected_extent = new ProjectedExtent(new Extent(0,0,0,0), CRS.fromName("EPSG:3857"))
var num_cols_rows :(Int, Int) = (0, 0)
//var cellT :CellType = CellType.fromName("uint8raw")
var grid0: RDD[(Long, Double)] = sc.emptyRDD
  
//Single band GeoTiff
val filepath = "hdfs:///user/hadoop/spring-index/LastFreeze/1980.tif"

//Since it is a single GeoTiff, it will be a RDD with a tile.
val tiles_RDD = sc.hadoopGeoTiffRDD(filepath).values
val grids_RDD = tiles_RDD.map(m => m.toArrayDouble())

val extents_withIndex = sc.hadoopGeoTiffRDD(filepath).keys.zipWithIndex().map{case (e,v) => (v,e)}
projected_extent = (extents_withIndex.filter(m => m._1 == 0).values.collect())(0)

val tiles_withIndex = tiles_RDD.zipWithIndex().map{case (e,v) => (v,e)}
val tile0 = (tiles_withIndex.filter(m => m._1==0).values.collect())(0)
num_cols_rows = (tile0.cols,tile0.rows)
        
val cellT = tile0.cellType

val bands_withIndex = grids_RDD.zipWithIndex().map { case (e, v) => (v, e) }
grid0 = bands_withIndex.filter(m => m._1 == 0).values.flatMap( m => m).zipWithIndex.map{case (v,i) => (i,v)}









    





projected_extent = ProjectedExtent(Extent(-126.30312894720473, 14.29219617034159, -56.162671563152486, 49.25462702827337),geotrellis.proj4.CRS$$anon$3@41d0d1b7)
num_cols_rows = (7808,3892)
grid0 = MapPartitionsRDD[113] at map at <console>:123
filepath = hdfs:///user/hadoop/spring-index/LastFreeze/1980.tif
tiles_RDD = MapPartitionsRDD[94] at values at <console>:110
grids_RDD = MapPartitionsRDD[95] at map at <console>:111
extents_withIndex = MapPartitionsRDD[100] at map at <console>:113







    





projected_extent: geotrellis.vector.ProjectedExten...







    Out[40]:





MapPartitionsRDD[100] at map at <console>:113

Clone the GeoTiff file



In [42]:

    
val clone_tile = DoubleArrayTile(grid0.values.collect(), num_cols_rows._1, num_cols_rows._2)

val cloned = geotrellis.raster.ArrayTile.empty(cellT, num_cols_rows._1, num_cols_rows._2)
cfor(0)(_ < num_cols_rows._1, _ + 1) { col =>
    cfor(0)(_ < num_cols_rows._2, _ + 1) { row =>
        val v = clone_tile.getDouble(col, row)
        cloned.setDouble(col, row, v)
    }
}

val geoTif = new SinglebandGeoTiff(cloned, projected_extent.extent, projected_extent.crs, Tags.empty, GeoTiffOptions.DEFAULT)









    





clone_tile = DoubleConstantNoDataArrayTile([D@718e69bf,7808,3892)
cloned = UByteRawArrayTile([B@62ba2e40,7808,3892)
geoTif = SinglebandGeoTiff(UByteRawArrayTile([B@62ba2e40,7808,3892),Extent(-126.30312894720473, 14.29219617034159, -56.162671563152486, 49.25462702827337),geotrellis.proj4.CRS$$anon$3@41d0d1b7,Tags(Map(),List()),GeoTiffOptions(geotrellis.raster.io.geotiff.Striped@e91c2e7,geotrellis.raster.io.geotiff.compression.NoCompression$@6f3deffe,1,None))







    Out[42]:





SinglebandGeoTiff(UByteRawArrayTile([B@62ba2e40,7808,3892),Extent(-126.30312894720473, 14.29219617034159, -56.162671563152486, 49.25462702827337),geotrellis.proj4.CRS$$anon$3@41d0d1b7,Tags(Map(),List()),GeoTiffOptions(geotrellis.raster.io.geotiff.Striped@e91c2e7,geotrellis.raster.io.geotiff.compression.NoCompression$@6f3deffe,1,None))

Save a GeoTiff to /tmp



In [43]:

    
val output = "/user/pheno/spring-index/LastFreeze/1980_clone.tif"
val tmp_output = "/tmp/1980_clone.tif"
GeoTiffWriter.write(geoTif, tmp_output)









    





output = /user/pheno/spring-index/LastFreeze/1980_clone.tif
tmp_output = /tmp/1980_clone.tif







    Out[43]:





/tmp/1980_clone.tif

Upload a GeoTiff to HDFS



In [45]:

    
val cmd = "hadoop dfs -copyFromLocal -f " + tmp_output + " " + output
Process(cmd)!









    



DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.







    





cmd = hadoop dfs -copyFromLocal -f /tmp/1980_clone.tif /user/pheno/spring-index/LastFreeze/1980_clone.tif







    





warning: there was one feature warning; re-run with -feature for details







    Out[45]:





0



In [ ]: