Abstract: Deep learning has made significant contributions for agriculture and environmental monitoring. However, it is often viewed as “unexplainable black box”, which limits its adoption and trust.