PlantGuide/Docs/plant-identification-model-plan.md

Plant Identification Core ML Model - Development Plan

Overview

Build a plant knowledge base from a curated plant list, then source/create an image dataset to train the Core ML model for visual plant identification.

---

Phase 1: Knowledge Base Creation from Plant List

Goal: Build structured plant knowledge from a curated plant list (CSV/JSON), enriching with taxonomy and characteristics.

Task	Description
1.1	Load and validate plant list file (CSV/JSON)
1.2	Normalize and standardize plant names
1.3	Create a master plant list with deduplicated entries
1.4	Enrich with physical characteristics (leaf shape, flower color, height, etc.)
1.5	Categorize plants by type (flower, tree, shrub, vegetable, herb, succulent)
1.6	Map common names to scientific names (binomial nomenclature)
1.7	Add regional/seasonal information from external sources

Deliverable: Structured plant knowledge base (JSON/SQLite) with ~500-2000 plant entries

---

Phase 2: Image Dataset Acquisition

Goal: Gather labeled plant images matching our knowledge base.

Task	Description
2.1	Research public plant image datasets (PlantCLEF, iNaturalist, PlantNet, Pl@ntNet)
2.2	Cross-reference available datasets with Phase 1 plant list
2.3	Download and organize images by species/category
2.4	Establish minimum image count per class (target: 100+ images per plant)
2.5	Identify gaps - plants in our knowledge base without sufficient images
2.6	Source supplementary images for gap plants (Flickr API, Wikimedia Commons)
2.7	Verify image quality and label accuracy (remove mislabeled/low-quality)
2.8	Split dataset: 70% training, 15% validation, 15% test

Deliverable: Labeled image dataset with 50,000-200,000 images across target plant classes

---

Phase 3: Dataset Preprocessing & Augmentation

Goal: Prepare images for training with consistent formatting and augmentation.

Task	Description
3.1	Standardize image dimensions (e.g., 224x224 or 299x299)
3.2	Normalize color channels and handle various image formats
3.3	Implement data augmentation pipeline (rotation, flip, brightness, crop)
3.4	Create augmented variants to balance underrepresented classes
3.5	Generate image manifest files mapping paths to labels
3.6	Validate dataset integrity (no corrupted files, correct labels)

Deliverable: Training-ready dataset with augmentation pipeline

---

Phase 4: Model Architecture Selection

Goal: Choose and configure the optimal model architecture for on-device inference.

Task	Description
4.1	Evaluate architectures: MobileNetV3, EfficientNet-Lite, ResNet50, Vision Transformer
4.2	Benchmark model size vs accuracy tradeoffs for mobile deployment
4.3	Select base architecture (recommend: MobileNetV3 or EfficientNet-Lite for iOS)
4.4	Configure transfer learning from ImageNet pretrained weights
4.5	Design classification head for our plant class count
4.6	Define target metrics: accuracy >85%, model size <50MB, inference <100ms

Deliverable: Model architecture specification document

---

Phase 5: Initial Training Run

Goal: Train baseline model and establish performance benchmarks.

Task	Description
5.1	Set up training environment (PyTorch/TensorFlow with GPU)
5.2	Implement training loop with learning rate scheduling
5.3	Train baseline model for 50 epochs
5.4	Log training/validation loss and accuracy curves
5.5	Evaluate on test set - document per-class accuracy
5.6	Identify problematic classes (low accuracy, high confusion)
5.7	Generate confusion matrix to find commonly confused plant pairs

Deliverable: Baseline model with documented accuracy metrics

---

Phase 6: Model Refinement & Iteration

Goal: Improve model through iterative refinement cycles.

Task	Description
6.1	Address class imbalance with weighted loss or oversampling
6.2	Fine-tune hyperparameters (learning rate, batch size, dropout)
6.3	Experiment with different augmentation strategies
6.4	Add more training data for underperforming classes
6.5	Consider hierarchical classification (family -> genus -> species)
6.6	Implement hard negative mining for confused pairs
6.7	Re-train and evaluate until target accuracy achieved
6.8	Perform k-fold cross-validation for robust metrics

Deliverable: Refined model meeting accuracy targets (>85% top-1, >95% top-5)

---

Phase 7: Core ML Conversion & Optimization

Goal: Convert trained model to Core ML format optimized for iOS.

Task	Description
7.1	Export trained model to ONNX or saved model format
7.2	Convert to Core ML using coremltools
7.3	Apply quantization (Float16 or Int8) to reduce model size
7.4	Configure model metadata (class labels, input/output specs)
7.5	Test converted model accuracy matches original
7.6	Optimize for Neural Engine execution
7.7	Benchmark inference speed on target devices (iPhone 12+)

Deliverable: Optimized .mlmodel or .mlpackage file

---

Phase 8: iOS Integration Testing

Goal: Validate model performance in real iOS environment.

Task	Description
8.1	Create test iOS app with camera capture
8.2	Integrate Core ML model with Vision framework
8.3	Test with real-world plant photos (not from training set)
8.4	Measure on-device inference latency
8.5	Test edge cases (partial plants, multiple plants, poor lighting)
8.6	Gather user feedback on identification accuracy
8.7	Document failure modes and edge cases

Deliverable: Validated model with real-world accuracy report

---

Phase 9: Knowledge Integration

Goal: Combine visual model with plant knowledge base for rich results.

Task	Description
9.1	Link model class predictions to Phase 1 knowledge base
9.2	Design result payload (name, description, care tips, characteristics)
9.3	Add confidence thresholds and "unknown plant" handling
9.4	Implement top-N predictions with confidence scores
9.5	Create fallback for low-confidence identifications

Deliverable: Complete plant identification system with rich metadata

---

Phase 10: Final Validation & Documentation

Goal: Comprehensive testing and production readiness.

Task	Description
10.1	Run full test suite across diverse plant images
10.2	Document supported plant list with accuracy per species
10.3	Create model card (training data, limitations, biases)
10.4	Write iOS integration guide
10.5	Package final .mlmodel with metadata and labels
10.6	Establish model versioning and update strategy

Deliverable: Production-ready Core ML model with documentation

---

Summary

Phase	Focus	Key Deliverable
1	Knowledge Base Creation	Plant knowledge base from plant list
2	Image Acquisition	Labeled dataset (50K-200K images)
3	Preprocessing	Training-ready augmented dataset
4	Architecture	Model design specification
5	Initial Training	Baseline model + benchmarks
6	Refinement	Optimized model (>85% accuracy)
7	Core ML Conversion	Quantized .mlmodel file
8	iOS Testing	Real-world validation report
9	Knowledge Integration	Rich identification results
10	Final Validation	Production-ready package

---

Key Insights

The plant list provides structured plant data (names, characteristics) but visual identification requires image training data. The plan combines the plant knowledge base with external image datasets to create a complete plant identification system.

Target Specifications

Metric	Target
Plant Classes	200-500 species
Top-1 Accuracy	>85%
Top-5 Accuracy	>95%
Model Size	<50MB
Inference Time	<100ms on iPhone 12+

Recommended Datasets

• PlantCLEF - Annual plant identification challenge dataset
• iNaturalist - Community-sourced plant observations
• PlantNet - Botanical research dataset
• Oxford Flowers - 102 flower categories
• Wikimedia Commons - Supplementary images

Recommended Architecture

MobileNetV3-Large or EfficientNet-Lite for optimal balance of:

• On-device performance
• Model size constraints
• Classification accuracy
• Neural Engine compatibility