PolliSense: AI-powered Habitat Quality Assessment and Biodiversity Improvement
PI: Mehrdad Mahdavi ( Computer Science and Engineering)
Plan for funding tuition for graduate students, or the remainder of the researcher’s salary for postdoc and research faculty: We are seeking one year of funding to support a post-completion PhD student (stipend only) or an equivalent amount of funding to partially support a postdoctoral researcher, with the remaining funding to be covered by other grants held by the PIs.
In an era of rapidly changing environmental conditions, the health and quality of ecosystems are increasingly threatened by factors such as habitat destruction, climate change, and pollution. One of the most pressing challenges in maintaining biodiversity is the degradation of habitats that support key species, particularly pollinators like bees, butterflies, and birds. These species play a crucial role in maintaining ecological balance and ensuring food security by facilitating the pollination of many plants, including the crops that provide vital micronutrients for human health. In response to these challenges, there have been efforts to conserve and protect pollinators. Strategies include planting pollinator-friendly habitats, reducing pesticide use, and encouraging the adoption of sustainable agricultural practices. Additionally, scientific research is focused on understanding the factors driving pollinator declines and finding solutions to support their populations. However, these efforts have traditionally been a resource-intensive and time-consuming task, relying on field surveys, expert observations, and manual data collection.
The long-term vision for this project is to create an accessible, user-friendly tool that empowers land managers, farmers, and conservationists to quickly and easily assess their landscapes, make informed decisions, and collaborate on creating environments that support both human and ecological health. To this end, we first aim to develop an AI-powered tool that can analyze images of landscapes, such as gardens, farms, and urban green spaces, to extract key environmental features and provide an overall Habitat Quality Score (HQS). The tool will use advanced computer vision and machine learning techniques to identify individual features like plant species, water bodies, structures, and pollinator-friendly areas, offering both detailed information on these components and an actionable HQS. This HQS will allow land managers, farmers, and conservationists to assess the ecological health of landscapes and make informed decisions to improve biodiversity and promote sustainable land use.
In addition to developing the core AI system for assessing habitat quality, we aim to create an AI model to provide users with actionable insights on how to improve HQS. Unlike previous AI model that classify and analyze the features currently present in a landscape, this algorithm will focus on identifying what features are missing and offer guidance on how to enhance the habitat’s overall HQS. This approach addresses a crucial gap in landscape management: knowing not just what is present but also what is needed to optimize habitat quality for biodiversity, particularly pollinators. Identifying “what is missing” and understanding how to act upon this knowledge is a uniquely complex challenge in AI, given the interpretability issues and the dynamic nature of real-world environments.
We anticipate these AI-powered tools will have a broad and transformative impact on both ecological conservation and agricultural management. However, the development of them presents several challenges:
1. Diversity of landscape features : Landscapes can vary widely in terms of size, vegetation types, and ecological characteristics. Also, the data from user-supplied images will be heterogeneous (habitat, season, lighting condition, image resolution, image quality) and likely irregularly sampled, both spatially and temporally. Capturing this diversity in a dataset that allows for accurate feature detection is a significant challenge.
2. Image quality and variability : The quality of images can vary due to factors such as lighting, angle, and resolution. Developing a robust model that can function effectively across different environments and image qualities is crucial.
3. Complexity of habitat quality metrics : Habitat quality is a multifaceted concept that encompasses not only biodiversity but also factors such as connectivity, water availability, and vegetation health. Designing an AI system that can evaluate all of these elements and synthesize them into a single HQS is a complex task.
4. Data labeling and annotation : Annotating large datasets of landscape images for training purposes requires expert knowledge to identify various species, habitat types, and environmental features. Ensuring the accuracy and completeness of this labeling and automating it is key to training an effective model.
5. Scalability : The tool needs to be scalable to process large volumes of images and handle a wide variety of environments, from small urban gardens to expansive agricultural landscapes.
To address these issues, we will develop and leverage advanced machine learning tools such as convolutional neural networks and vision Transformers, contrastive learning, active learning, and counterfactual (CF) explanation techniques.
Expertise/skills of interest:
Experience in using and developing deep learning techniques, particularly Convolutional Neural Networks (CNNs) and Vision Transformers, for image classification and analysis
Experience with deep learning tools such as PyTorch, hyperparameter tuning, model optimization, and working with large (possibly noisy) image datasets
Familiarity with the following would be helpful, but is not required: contrastive learning, transfer learning, and counterfactual explanation
Expectations:
A graduate student or postdocs with at least some experience and/or training in machine learning and familiarity with deep learning tools, training, fine-tuning pretrained models, and deploying AI models
Willingness to learn new (possibly advanced) methods in machine learning and implement them
Weekly meeting and project updates with faculty advisor(s). Participate in group meetings (~1 hour roughly twice a month).
Engagement:
Christina Grozinger, Mehrdad Mahdavi, and Harland Patch will co-lead the project. Grozinger is the Director of Huck Institute and the Director of the Center for Pollinator Research (2000present) and Director of the Technology for Living Systems Center (2024-present). These Centers include >50 faculty from 9 Colleges. Mahdavi is the director of AI Hub at ICDS, and an Associate Director for the Center for Artificial Intelligence Foundations and Engineered Systems (CAFÉ). Path is the Director of Pollinator Programming, Arboretum at Penn State. Grozinger, Mahdavi, and Patch have been actively working together on projects focused on AI for Living Systems, from which this project has emerged. They have also made significant progress in developing tools for data gathering to support this research and lunch it.