Developing a complex system is never easy, but it’s the job of systems engineers to understand the intricate interplay of its dynamic components and build solutions that do not yet exist. In GW Engineering’s new Model-Based Engineering (MBE) course, systems engineering undergraduate students gained hands-on experience in doing just this as they took a unique sensor system idea from initial concept to a fully functional prototype.
Taught by Associate Professor of Practice Eric Dano, this is the first undergraduate systems engineering class to focus on modeling and hands-on sensor system development while incorporating artificial intelligence/machine learning (AI/ML) algorithms.
Each student team chose a specific objective for their prototype, from passive SONAR detection of moving objects to a home security system that alerts homeowners if an intruder is detected. Using advanced modeling tools, including the open-source ARCADIA Capella, AI4SE Python Libraries, and the Siemens Teamcenter PLM environment, they first co-developed and critically assessed the operational architecture of their system.
Teams then moved from computer modeling to the real world by building their prototypes using a Raspberry Pi controller and associated sensors. The challenges they faced in this process quickly highlighted to students like junior Elena Ahmadi that while modeling is great for planning, it may provide a false sense of confidence.
“Building the prototype forced us to solve real problems instead of just assuming they wouldn’t exist,” said Ahmadi. “I think part of what made this experience so special was solving the gap between a good model and an actual working system.”
Alongside teammates junior Sasha Green, sophomore Valeria Grullon, and senior Arman Naseh, Ahmadi developed a system that converts American Sign Language (ASL) letters into on-screen text. Their system features three main layers, with the first being the hardware layer comprised of the Raspberry Pi, monitor, and keyboard.
The team’s primary obstacles stemmed from integrating the hardware and software. For instance, when they plugged in the hardware, they found that the Raspberry Pi couldn’t support some of the software they planned to use, like TensorFlow Lite, and the image libraries were so large they needed to be downloaded to a separate PC, pushing only the required run parameters to the Raspberry Pi–both things that didn’t show up in the computer model.
“Instead of trying to force it, we asked what that component actually needed to do and found a completely different approach that did the same job but fit our hardware better. We ended up with a more accurate system because of that pivot,” Ahmadi stated.
The second layer is the system’s software, where other disciplines like AI/ML and computer vision begin to come into play to make the system more complete. The MBE Lab’s high-performance compute node enables prototyping AI/ML algorithms for deployment on the Raspberry Pi sensors.
Once the team’s camera sensor captures the signer’s hand, AI/ML frameworks crop it from the image and identify the letter based on the landmarks shown in the ML Mediapipe library they trained their AI model on. In the final layer, the ML algorithm, a Random Forest Classifier, creates decision trees to determine the letter signed and displays it onscreen in real time.
The third layer is training their model. The team first trained it on Kaggle image datasets with perfect white backgrounds and studio lighting. Since this does not match the environment in which their system is used, they had to add images to the training data they captured themselves, demonstrating valuable lessons on the use of AI/ML for systems engineering.
“In terms of AI, you have to define your real operating environment early and make sure to train your ML to match it, otherwise your model and system won’t work together,” Ahmadi stated.
Using systems thinking, Ahmadi and team were able to problem-solve each time an obstacle arose because they understood that all components are interconnected and constantly asked, "If we change one thing, what else could break?"
Other students in the class, like senior Bogdan Bunea, noted that systems thinking also helped his team overcome the limitations of the physical sensors they used, forcing him to adapt to the systems at hand and teaching him to work on the riskiest aspects of the prototype first.
A systems mindset is not only useful for engineers, though. Ian Milko, a senior accounting major and systems engineering minor, shared that, while it is often seen as a niche, the concepts he’s learned in systems engineering courses like this one have value across many fields at GW.
“Departments and schools have to talk to each other because there’s a lot of value in these systems classes,” Milko stated.
This inaugural course marks a milestone for GW Engineering, providing students not only with invaluable experience working in the multi-domain model-based environment they’ll encounter in their careers but also with the opportunity to go beyond theory and integrate critical disciplines of complex system development.
“Capella taught me how to think through a system architecturally before touching any hardware, which is how large programs are run. Then building the Pi sensor taught me that models have to be grounded in reality to mean anything,” said Ahmadi. “These two things together, rigorous modeling and practical problem solving, feel like exactly what I’ll need in the field.”