AI & Full StackIn ProgressFeatured SystemAug 2026 – Present

AeroCrop.ai — Multi-Modal Deep Learning & Field Economics Platform

Bridging computer vision and field economics to deliver real-time foliar disease diagnostics, ICAR fertilizer schedules, and mandi revenue intelligence for Indian agriculture.

Source Code Live Demo Documentation

AeroCrop.ai — Multi-Modal Deep Learning & Field Economics Platform

Technologies & Architecture Stack

PyTorchtorchvisionResNet-18FastAPIPython 3.12MongoDBMotorReact 18TypeScriptViteTailwind CSSChart.jsLucide IconsDockerCoolifyNginxOpen-Meteo REST APIWeb Speech APIPDFKitJWTbcrypt

Project Overview

AeroCrop.ai is an end-to-end, multi-modal precision agriculture and agro-economic decision platform engineered specifically for smallholder and commercial farmers across Maharashtra's 36 districts. While conventional agronomic tools treat pathology detection and crop economics as disconnected concerns, AeroCrop.ai unifies computer vision, microclimatic telemetry, and stoichiometric field economics into a cohesive pipeline. From a single leaf photograph, the system executes real-time pathology classification across 50 canonical disease categories using an optimized ResNet-18 vision backbone, fused with tabular microclimate and soil NPK telemetry through a 3-layer Multi-Layer Perceptron (MLP). The platform simultaneously forecasts harvest yield (t/ha), formulates Indian Council of Agricultural Research (ICAR) compliant fertilizer split-schedules in physical 50kg commercial bags (Urea, DAP, MOP), monitors real-time foliar spraying hazard windows to eliminate pesticide runoff, and projects gross mandi cash revenues against official Government of India Minimum Support Price (MSP) benchmarks.

The Engineering Problem

Modern smallholder agriculture faces a severe breakdown across visual disease triage, input resource optimization, and economic planning: 1. Asymmetric Diagnostics & Yield Decay: Foliar plant infections cause 30–40% annual yield devastation across India because visual symptoms are either misidentified by local agronomists or diagnosed after irreversible pathogen incubation. 2. The Multi-Modal Disconnect in Academic AI: Existing academic models perform isolated image classification on laboratory datasets (like raw PlantVillage) without factoring in microclimatic variables (humidity, precipitation, ambient heat), causing severe domain shift failure under real field conditions. Furthermore, yield estimation models rely strictly on historical macroscopic climate tables, entirely unaware of localized plant pathology. 3. Agronomic Usability Failure: Scientific papers express fertilizer recommendations in abstract chemical metrics (e.g., kg/ha of pure N, P, and K). Rural farmers purchase straight fertilizers in standardized 50kg commercial bags (Urea, DAP, MOP) under Government of India subsidies. Abstract recommendations lead to arbitrary nitrogen over-fertilization, soil toxicity, and wasted capital. 4. Chemical Runoff & Financial Waste: Farmers frequently spray expensive foliar fungicides hours before heavy rainfall or during high wind velocities, causing 100% chemical wash-off or toxic drift onto non-target parcels. 5. Vernacular Accessibility Barriers: Most decision support systems are desktop-oriented, written exclusively in technical English, and require tedious manual entry of soil laboratory parameters, alienating the 80%+ of Indian farmers who lack Soil Health Cards.

The Solution

To solve these challenges, I designed and deployed AeroCrop.ai with a photo-first, multi-modal, and production-grade architecture: 1. Multi-Modal Joint Embedding Architecture (MultiModalAeroCropNet): Built a dual-encoder neural network combining a ResNet-18 visual feature extractor (512-dimensional output) with a 3-layer tabular MLP (64-dimensional output). Modalities are projected into a unified 128-dimensional shared latent manifold, simultaneously powering a 50-class disease classification head and a non-negative harvest yield regression head. 2. ICAR Commercial Stoichiometry Engine: Engineered a chemical deficit calculator that maps soil nutrient gaps into physical 50kg commercial bags of Urea, DAP, and MOP. The algorithm strictly accounts for DAP's dual composition (18% N, 46% P2O5) to prevent nitrogen over-application, schedules applications across 3 physiological growth stages (Basal, Vegetative, Flowering), and computes real-time purchase costs under GoI subsidized retail prices. 3. Real-Time Foliar Spray Hazard Decision Engine: Integrated the Open-Meteo REST API to monitor hourly rainfall, 10m wind speeds, humidity, and temperature. The engine issues instant safety badges (Safe, Caution, Hold Spray) preventing wash-off (>1.0 mm rain) and drift (>15 km/h wind). 4. APMC Mandi Revenue Intelligence: Aggregated live modal commodity prices across key Maharashtra APMC trading hubs (Lasalgaon, Jalgaon, Pune, Nagpur, Latur) to translate predicted yield (quintals/acre) into real-time gross revenue projections alongside official MSP safety floors. 5. Accessible Trilingual Experience: Delivered a high-performance React 18 + Vite client featuring Web Speech API text-to-speech audio narration in Marathi, Hindi, and English, coupled with automated 3-page trilingual PDF advisory generation for Pradhan Mantri Fasal Bima Yojana (PMFBY) insurance loss documentation.

System Architecture & Data Flow

AeroCrop.ai follows an asynchronous Service-Oriented Model-View-Controller (MVC) architecture optimized for cloud deployment: [Farmer Client / Browser] (React 18 + TypeScript + Vite + Web Speech API) │ ▼ (Multipart Form: RGB Image + District + Land Plot Size) [Reverse Proxy / Edge Gateway] (Nginx / Coolify SSL Termination) │ ▼ (Async ASGI HTTP Requests) [FastAPI Backend Core] (Python 3.12, Uvicorn Workers) │ ├──► [InferenceService (Singleton)] │ ├── Visual Stream: Leaf Image -> Resize (224x224) -> ImageNet Norm -> ResNet-18 -> 512-d │ ├── Tabular Stream: Soil NPK + Live Weather -> Z-score Norm -> 3-Layer MLP -> 64-d │ └── Fusion Layer: Concatenation (576-d) -> Linear + BatchNorm + Dropout(0.3) -> 128-d │ ├── Head A: Linear(128, 50) -> Softmax -> Disease Class & Confidence │ └── Head B: Linear(128, 32) -> ReLU -> Linear(32, 1) -> Yield (t/ha) │ ├──► [Open-Meteo Telemetry Client] (TTL In-Memory Cache: 1800s) -> Rainfall, Wind, Temp, Humidity ├──► [ICAR Fertilizer Service] -> NPK Deficit Matrix -> DAP/Urea/MOP 50kg Bags & Growth Stages ├──► [APMC Mandi Service] -> Regional Modal Prices & MSP -> Gross Revenue Calculation ├──► [MongoDB Database (Motor)] -> Users, Cadastral Farm Plots, Historical Telemetry Audits └──► [Microservice Dispatcher] -> Native Trilingual PDF Report Generation (Devanagari PDFKit)

Core Engineering Features

MultiModalAeroCropNet: PyTorch dual-backbone network fusing ResNet-18 visual embeddings (512-d) and a 3-layer tabular MLP (64-d) into a shared 128-d latent manifold.

Multi-Task Optimization: Simultaneous 50-class disease classification and non-negative crop yield regression (t/ha) using AdamW and CosineAnnealingLR.

Frictionless Zero-Login Diagnostics: Instant photo-first disease triage with automatic crop detection and district-level agronomic baseline auto-imputation.

ICAR Growth-Stage Fertilizer Engine: Converts soil nutrient deficits into integer 50kg bags of Urea, DAP, and MOP, factoring in DAP's 18% nitrogen credit.

Dynamic Multi-Unit Land Scaler: Real-time agricultural area conversions across Hectares, Standard Acres, and vernacular Gunthas.

Foliar Spraying Hazard Engine: Evaluates live Open-Meteo rainfall (>1.0 mm) and wind speed (>15 km/h) to prevent pesticide wash-off and chemical drift.

APMC Mandi Intelligence: Bridges harvest yield forecasts with live Maharashtra APMC market rates and GoI Minimum Support Price (MSP) benchmarks.

Vernacular Text-to-Speech Engine: Hands-free auditory narration of disease pathology and actionable remedies using the browser Web Speech API (mr-IN, hi-IN, en-IN).

PMFBY Insurance Loss Exporter: Generates formal trilingual 3-page PDF claim documentation containing leaf specimen imagery, geo-district timestamps, and surveyor signature blocks.

Cadastral Multi-Plot Registry: Full CRUD farmer land management tracking acreage, crop variety, sowing dates, and baseline soil health metrics.

Asynchronous FastAPI Architecture: High-throughput async API handling multipart image streams with Pydantic v2 validation and OpenAPI auto-documentation.

High-Density ARM64 Cloud Deployment: Containerized multi-stage Docker builds orchestrated via Coolify on Oracle Cloud Ampere A1 (ARM64) consuming <1.5 GB RAM.

Technical Challenges & Overcoming Them

Cross-Modal Dimensional Asymmetry: Concatenating high-dimensional image pixels (150,528 values) directly with 6 tabular scalars causes the network to ignore tabular features; solved by implementing intermediate feature fusion where ResNet-18 (512-d) and MLP (64-d) compress modalities into commensurate representations before joining at a 128-d manifold.

Overfitting to Laboratory Backgrounds: PlantVillage dataset images feature sterile studio backgrounds that degrade under variable farm lighting; mitigated by training with extensive data augmentations including aggressive ColorJitter (+-30% brightness, contrast, saturation), random rotations, flips, and tabular microclimate regularization.

Simultaneous Classification and Regression Gradient Drift: Multi-task loss imbalance can cause the regression gradient to overpower classification updates; solved by formulating a weighted joint objective (1.0 * CrossEntropy with 0.1 label smoothing + 0.5 * MSE) optimized via CosineAnnealingLR and gradient clipping (max norm 5.0).

Compound Fertilizer Nitrogen Toxicity: Calculating Urea first to satisfy Nitrogen deficits before adding DAP leads to toxic Nitrogen over-application due to DAP's 18% N content; solved by structuring the chemical stoichiometry to fulfill Phosphorus with DAP first, calculating the DAP nitrogen contribution, and applying Urea only for residual nitrogen requirements.

Extreme Rural Data Scarcity (Missing Soil Health Cards): Over 80% of Indian farmers lack numerical soil NPK data, leading to user drop-offs; solved by engineering an automatic proxy fallback that injects ICAR regional median soil baselines by district and crop, preserving neural network vector integrity without model retraining or performance degradation.

Cold Start Memory Spikes in Container Environments: Reloading PyTorch weights on every incoming HTTP inference call caused multi-second latencies and OOM crashes; solved by implementing a thread-safe Singleton InferenceService pattern that caches the 45.1 MB weights in RAM at startup for sub-150ms responses.

Trilingual Complex Font Rendering in Headless Containers: Standard headless PDF libraries failed to render Devanagari ligatures (Marathi/Hindi) correctly inside minimal Alpine/Debian Docker containers; solved by engineering a native PDFKit microservice embedding custom Google Noto Sans Devanagari font glyphs.

Results & Impact

- Disease Classification Accuracy: 90.82% Top-1 validation accuracy achieved across 50 canonical pathology classes. - Yield Forecasting Precision: 6.72 t/ha Root Mean Square Error (RMSE) on multi-modal test distribution. - Model Parameter Efficiency: 11.3 Million total parameters with a compact 45.1 MB weight footprint (aerocrop_weights.pth). - End-to-End Latency: <150ms inference time on standard CPU / ARM64 environments; sub-15ms on consumer GPU. - Verified Image Corpus: 62,836 clean, deduplicated agricultural pathology images across 11 primary commercial crops. - Geographic Coverage: 100% telemetry and APMC mandi support across all 36 administrative districts of Maharashtra. - Codebase Reliability: 155 automated unit, integration, and security test cases passing with 100% pass rate. - Cloud Resource Footprint: Complete 4-tier containerized stack (FastAPI, React, MongoDB, PDF Service) operates under 1.5 GB runtime RAM on Oracle Cloud A1 ARM64.

Key Takeaways

- Multi-Task Intermediate Fusion Dynamics: Gained deep practical mastery over multi-modal neural architectures, discovering that intermediate bottleneck fusion significantly outperforms early pixel concatenation or late decision voting when fusing disparate data structures (spatial vision vs. scalar telemetry). - The Art of Multi-Task Loss Balancing: Learned that multi-task learning requires meticulous loss weighting and regularization (label smoothing and decoupled weight decay) to prevent regression loss surfaces from dominating categorical classification heads. - Domain-First Engineering Over Pure Accuracy: Translating mathematical output into physical 50kg subsidized bags, growth-stage split calendars, and local currency (INR) is what transforms an AI model into a practical enterprise solution. - Resilient Systems Design & Graceful Degradation: Built robust production fallback patterns: if external weather APIs experience outages, regional historical meteorological defaults activate; if model weights fail integrity checks, deterministic agronomic heuristics ensure zero downtime. - Microservice Decoupling & Cloud Architecture: Mastered cross-platform Docker containerization, configuring multi-stage builds for ARM64 server architectures and deploying zero-downtime CI/CD workflows via Coolify and Nginx.

Future Roadmap

- Edge Quantization & Mobile Deployment: Quantize the PyTorch model to INT8 ONNX and TFLite runtimes for sub-30ms offline inference on low-cost Android mobile devices and Raspberry Pi / Jetson edge devices. - Drone (UAV) Canopy Lesion Segmentation: Integrate YOLOv9-seg and Mask R-CNN to process aerial multispectral orthomosaics, enabling commercial plantations to map hectare-scale infection hotspots from drone flights. - LoRaWAN IoT Soil Telemetry: Interface with optical ESP32-based NPK soil probes over LoRaWAN networks to feed live, real-time nutrient readings directly into the tabular inference pipeline. - Offline-First Progressive Web App (PWA): Implement Service Workers and IndexedDB local caching so field diagnosis works without uninterrupted 4G connectivity in remote agricultural valleys. - Conversational Vernacular Ag-Agent: Integrate lightweight regional LLMs fine-tuned on ICAR and MPKV Rahuri agronomic publications with Whisper ASR for fluid two-way voice consultations in rural Marathi dialects.

Back to Projects Discuss This Project