Skip to content

OTBTF: Orfeo ToolBox meets TensorFlow


Documentation Status

This remote module of the Orfeo ToolBox provides a generic, multipurpose deep learning framework, targeting remote sensing images processing. It contains a set of new process objects for OTB that internally invoke Tensorflow, and new OTB applications to perform deep learning with real-world remote sensing images. Applications can be used to build OTB pipelines from Python or C++ APIs. OTBTF also includes a python API to build Keras compliant models, easy to train in distributed environments.

Features

OTB Applications

  • Sample patches in remote sensing images with PatchesExtraction,
  • Inference with support of OTB streaming mechanism with TensorflowModelServe: this means that inference is not limited by images number, size, of channels depths, and can be used as a "lego" in any pipeline composed of OTB applications and preserving streaming.
  • Model training, supporting save/restore/import operations (a model can be trained from scratch or fine-tuned) with TensorflowModelTrain. This application targets mostly newcomers and is nice for educational purpose, but deep learning practitioners will for sure prefer the Python API of OTBTF.

Python API

The otbtf module targets python developers that want to train their own model from python with TensorFlow or Keras. It provides various classes for datasets and iterators to handle the patches images generated from the PatchesExtraction OTB application. For instance, the otbtf.DatasetFromPatchesImages can be instantiated from a set of patches images and delivering samples as tf.dataset that can be used in your favorite TensorFlow pipelines, or convert your patches into TFRecords. The otbtf.TFRecords enables you train networks from TFRecords files, which is quite suited for distributed training. Read more in the tutorial for keras.

Examples

Below are some screen captures of deep learning applications performed at large scale with OTBTF.

  • Landcover mapping (Spot-7 images --> Building map using semantic segmentation)

Landcover mapping

  • Super resolution (Sentinel-2 images upsampled with the SR4RS software, which is based on OTBTF)

Super resolution

  • Sentinel-2 reconstruction with Sentinel-1 VV/VH with the Decloud software, which is based on OTBTF

Decloud

  • Image to image translation (Spot-7 image --> Wikimedia Map using CGAN. So unnecessary but fun!)

Pix2pix

Contribute

Every one can contribute to OTBTF. Just open a PR :)

Cite

@article{cresson2018framework,
  title={A framework for remote sensing images processing using deep learning techniques},
  author={Cresson, R{\'e}mi},
  journal={IEEE Geoscience and Remote Sensing Letters},
  volume={16},
  number={1},
  pages={25--29},
  year={2018},
  publisher={IEEE}
}

Additional resources

  • The otbtf/keras tutorial shows how to create a dataset, build, train, evaluate, and apply a model to perform semantic segmentation of a remote sensing image,
  • The test folder of this repository contains various use-cases with commands, python codes, and input/baseline data,
  • This book contains 130 pages to learn how to use OTBTF with OTB and QGIS to perform various kind of deep learning sorcery on remote sensing images (patch-based classification for landcover mapping, semantic segmentation of buildings, optical image restoration from joint SAR/Optical time series): Cresson, R. (2020). Deep Learning for Remote Sensing Images with Open Source Software. CRC Press.
  • A small tutorial on MDL4EO's blog
  • Check our repository containing stuff (data and models) to begin with!