LLM-Augmented Digital Twin Framework for Building Material Reuse and Recycling Assessment
PI: Yuqing Hu (Architectural Engineering)
PI will use start-up and other funded project to cover the rest of tuition.
Project Description
The construction sector is facing dual challenges: ongoing material shortages—especially in structural steel, engineered wood, and insulation—and the growing volume of construction and demolition (C&D) waste, which accounts for over 650 million tons annually in the U.S. alone. While a substantial portion of this material could be reused or recycled, most building renovation and demolition projects fail to capitalize on recovery opportunities. A central knowledge gap is the lack of scalable, automated tools to assess material recovery potential early in the planning process. Effective reuse decisions depend on understanding material type, condition, installation method, and disassembly feasibility, factors that are typically buried in inconsistent or unstructured project documentation. While Building Information Modeling (BIM) provides digital representations of building components and their properties, many older buildings lack BIM models altogether or have incomplete and outdated files. This results in reliance on time-consuming manual inspections or post-demolition audits, which often occur too late to inform design and procurement decisions. Without accessible, structured insights, potentially recoverable materials can be destroyed during demolition phase and routinely sent to landfills, missing opportunities to reduce waste and offset supply constraints. This project proposes to develop a digital twin framework powered by large language models (LLMs) and large vision models (LVMs) to support component-level material reuse and recycling assessment. The framework will include: 1) A semantic extraction module that leverages fine-tuned LLMs to interpret unstructured BIM/IFC metadata and free-text specifications, extracting material-relevant attributes such as component type, fastening method, installation context, and age; 2) A classification model that evaluates each component’s suitability for reuse, recycling, or disposal, and generates indicators including disassembly complexity, recovery value, and potential embodied carbon savings; 3) A reality capture and visualization layer powered by LVMs, which integrates spatial imagery (e.g., 360° scans or photos) and embeds recovery insights into an interactive digital twin, allowing users to explore material recovery potential systematically and spatially. The ICDS-supported phase of this research will focus on prototyping the LLM-LVM pipeline, applying it to sample IFC models and selected PSU buildings (e.g., Engineering Units). This phase will lay the technical foundation for scalable, AI-enabled recovery modeling and support future proposals in circular construction, smart infrastructure, and digital lifecycle management. An ICDS Junior Researcher will assist with core technical tasks during this prototyping phase, including:
• Preprocessing and structuring IFC/BIM data for AI model input.
• Assisting in fine-tuning or adapting LLMs/LVMs for material attribute extraction tasks (e.g., using OpenAI, Hugging Face models, or instruction-tuned variants);
• Supporting development of recovery classification logic using domain-informed rules or ML classifiers.
• Building proof-of-concept visualizations of material recovery insights using 3D digital twin.
Desired Skills and Expertise
• Experience with BIM and IFC schemas, including working with tools such as Revit, IfcOpenShell, or related BIM processing APIs
• Familiarity with fine-tuning or adapting LLMs and LVMs using frameworks such as Hugging Face Transformers or OpenAI APIs
Other Requirements
• Preferred: Post-comps PhD student, such as Architectural Engineering, CS, or IST.
Objectives of Work Supported by This Call
The research team plans to submit an NSF proposal to the cyber-physical program by Dec 2025
Medium- to Long-Term Goal
This project aims to develop a scalable, AI-powered digital twin framework for early-stage assessment of material reuse and recycling potential. By integrating LLMs for semantic extraction and LVMs for visual analysis, the system will automate recovery insights and reduce reliance on manual inspections. In the long term, this work will support broader applications in circular construction, adaptive reuse, and sustainable lifecycle planning.