The University of Illinois at Urbana-Champaign has developed an AI teaching assistant system that uses 11 models for retrieval and generation, achieving a median 2-second response time. This system, designed for ECE 120 (intro to Electrical Engineering), utilizes a dataset created by hiring students, but does not have rights to the textbook data it uses.
The University of Illinois at Urbana-Champaign has created an AI teaching assistant designed for its ECE 120 Electrical Engineering course. The system employs 11 distinct models that operate in parallel to facilitate text and image retrieval, generation, moderation, and ranking. This setup allows the assistant to respond to queries with a median delay of just 2 seconds, significantly enhancing the learning experience for students.
Key to the project's effectiveness is the data sourced from textbooks, lecture videos, and student QA forums; however, the team does not have public rights to this data. Instead, the group developed a novel dataset specifically for semantic search retrieval during reinforcement learning from human feedback (RLHF) utilizing Electrical Engineering students to curate the information. Unlike protected textbook materials, this dataset is accessible on Huggingface.
The system's models are continually evaluated to determine their effectiveness. The evaluation process involves using a dataset of QA pairs crafted by experts in the field. Each model's performance is then compared against baseline answers provided by human engineers using GPT-3 for assessment. While GPT-3's self-evaluation introduces potential bias, the iterative nature of the evaluation is vital for refining the assistant's features.
The project emphasizes open-source principles, providing various scripts and utilities for others to integrate their document databases into the system. Users can build their own retrieval-augmented generation framework with custom materials, although the proprietary nature of some textbook contents remains a barrier to complete openness.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The University of Illinois at Urbana-Champaign has developed an AI teaching assistant system that uses 11 models for retrieval and generation, achieving a median 2-second response time. This system, designed for ECE 120 (intro to Electrical Engineering), utilizes a dataset created by hiring students, but does not have rights to the textbook data it uses.