PlantMDE

The repo for ``3D Plant Phenotyping from a Single Image: Generalizable Plant Monocular Depth Estimation via Dataset Mixing''.

🌱 PlantDepth Benchmark Dataset

The first benchmark dataset designed for plant depth estimation and 3D reconstruction tasks. It renders RGB-D plant images from the mainstream publicly available plant 3D datasets.

Download

• Google Drive: download the complete ZIP archive.
• Zenodo: download the dataset as 30 ZIP parts, with checksums and reconstruction instructions.

Dataset DOI: 10.5281/zenodo.23017847. The PlantDepth dataset is released under the MIT License.

The Zenodo copy contains 30 binary parts of the original 14.6 GB ZIP archive (up to 500 MB per part), plus SHA256SUMS and README_DOWNLOAD.txt. Individual parts cannot be extracted separately.

Download all parts (PlantDepth.zip.part-000 through PlantDepth.zip.part-029) and SHA256SUMS into the same directory. On Linux, reconstruct, verify, and extract the dataset:

cat PlantDepth.zip.part-[0-9][0-9][0-9] > PlantDepth.zip &&
sha256sum -c SHA256SUMS &&
unzip PlantDepth.zip

On macOS, replace sha256sum -c SHA256SUMS with shasum -a 256 -c SHA256SUMS. Allow approximately 45 GB of free disk space to keep the downloaded parts, reconstructed ZIP, and extracted dataset together.

📂 Dataset Structure

PlantDepth/
├── Crops3D/ # Sub-dataset
    ├── depth/                # Depth maps 
    ├── rgb/                  # RGB images 
    ├── segmentation/         # Segmentation labels 
    ├── test_file_list.txt    # List of test files used for evaluation
    ├── train_file_list.txt   # List of train files used for model training
├── ETH_BeetRoot/
    ├── ...
    ├── mask/ # Binary masks for the RGB-D datasets with backgrounds
├── GScatter/
├── PLANest/
├── Plant3D/
├── PlantStereo/
├── Soybeanmvs/
└── WUR_DwarfTomato/

🔗 Usage

We provide pytorch dataloader in dataset/plantdepth.py. The dataset can be loaded by:

from dataset.plantdepth import PLANTDEPTH_MIXED
mix_set = ['GScatter','Crops3D','PLANest','Soybeanmvs','Plant3D'] # The sub-datasets you want load
root_dir = '.../PlantDepth' # The path for PlantDepth
trainset = PLANTDEPTH_MIXED(root_dir, [f'/{d}/train_file_list.txt' for d in mix_set], 'train', size=(518,518))
trainloader = DataLoader(trainset)
for i, sample in enumerate(trainloader):
    # Load RGB, Depth in disparity space, mask for background, and organ segmentation
    img, disparity, valid_mask, segmentation = sample['image'], sample['disparity'], sample['mask'], sample['segmentation']
    ...
    ...

🤖PlantMDE model

Trained PlantMDE checkpoint can be downloaded in the link.

As a comparion, Depth Anything v2 checkpoint can be downloaded in the repo with link.

MDE_organ_wise.ipynb provides a tuition for depth estiamtion on spinach image. The Pearson correlation coefficient to the GT depth at each organ is computed. The notebook compares PlantMDE vs. Depth Anything.

📢 Citation

Please cite the paper when using PlantMDE or PlantDepth in your work. """