OTBTF: Orfeo ToolBox meets TensorFlow¶
This remote module of the Orfeo ToolBox provides a generic, multipurpose deep learning framework, targeting remote sensing images processing. It contains a set of new process objects for OTB that internally invoke Tensorflow, and new OTB applications to perform deep learning with real-world remote sensing images. Applications can be used to build OTB pipelines from Python or C++ APIs. OTBTF also includes a python API to build Keras compliant models, easy to train in distributed environments.
Features¶
OTB Applications¶
- Sample patches in remote sensing images with
PatchesExtraction
, - Inference with support of OTB streaming mechanism with
TensorflowModelServe
: this means that inference is not limited by images number, size, of channels depths, and can be used as a "lego" in any pipeline composed of OTB applications and preserving streaming. - Model training, supporting save/restore/import operations (a model can be
trained from scratch or fine-tuned) with
TensorflowModelTrain
. This application targets mostly newcomers and is nice for educational purpose, but deep learning practitioners will for sure prefer the Python API of OTBTF.
Python API¶
The otbtf
module targets python developers that want to train their own
model from python with TensorFlow or Keras.
It provides various classes for datasets and iterators to handle the
patches images generated from the PatchesExtraction
OTB application.
For instance, the otbtf.DatasetFromPatchesImages
can be instantiated from a
set of patches images and delivering samples as tf.dataset
that can be
used in your favorite TensorFlow pipelines, or convert your patches into
TFRecords. The otbtf.TFRecords
enables you train networks from TFRecords
files, which is quite suited for distributed training. Read more in the
tutorial for keras.
Examples¶
Below are some screen captures of deep learning applications performed at large scale with OTBTF.
- Landcover mapping (Spot-7 images --> Building map using semantic segmentation)
- Super resolution (Sentinel-2 images upsampled with the SR4RS software, which is based on OTBTF)
- Sentinel-2 reconstruction with Sentinel-1 VV/VH with the Decloud software, which is based on OTBTF
- Image to image translation (Spot-7 image --> Wikimedia Map using CGAN. So unnecessary but fun!)
Contribute¶
Every one can contribute to OTBTF. Just open a PR :)
Cite¶
@article{cresson2018framework,
title={A framework for remote sensing images processing using deep learning techniques},
author={Cresson, R{\'e}mi},
journal={IEEE Geoscience and Remote Sensing Letters},
volume={16},
number={1},
pages={25--29},
year={2018},
publisher={IEEE}
}
Additional resources¶
- The otbtf/keras tutorial shows how to create a dataset, build, train, evaluate, and apply a model to perform semantic segmentation of a remote sensing image,
- The test folder of this repository contains various use-cases with commands, python codes, and input/baseline data,
- This book contains 130 pages to learn how to use OTBTF with OTB and QGIS to perform various kind of deep learning sorcery on remote sensing images (patch-based classification for landcover mapping, semantic segmentation of buildings, optical image restoration from joint SAR/Optical time series): Cresson, R. (2020). Deep Learning for Remote Sensing Images with Open Source Software. CRC Press.
- A small tutorial on MDL4EO's blog
- Check our repository containing stuff (data and models) to begin with!