Ph.D. Proposal: Michael Herman 

Wed May 25 2022 09:00 AM
Weber CoVE & BlueJeans
"Development of an Interpretable Digital Twin for the Predictive Calibration of Sun Sensors via Bayesian Convolutional Neural Network"

You're invited to attend


Ph.D. Proposal


Michael Herman 

(Advisor: Prof. Dimitri Mavris) 




"Development of an Interpretable Digital Twin for the Predictive Calibration of Sun Sensors via Bayesian Convolutional Neural Network"


Wednesday, May 25 at 9:00 a.m. 
Collaborative Visualization Environment (CoVE) 
Weber Space Science and Technology Building 

The Digital Twin, a virtual representation of a connected physical asset, accelerates the sensor modeling and calibration process through the combination of real-time data, physics-informed models and intelligence from various sources. Recent developments of AI techniques for space applications mirror the success achieved in terrestrial applications. Machine learning, which scales in data rich environments, is particularly well suited to space-based computer vision applications such as space optical attitude sensing. Attitude sensors determine the spacecraft attitude through the sensing of an astronomical object. The sun and fixed stars are the two primary astronomical sensing objects. Attitude sensors are critical components for the survival and knowledge improvement of spacecraft. Of these, sun sensors are the most common and important sensor for spacecraft attitude determination. The sun sensor measures the Sun vector in spacecraft coordinates. The two main categories of sun sensors are analog and digital. 

The development of small satellites requires highly accurate and reliable sensing techniques of satellite attitude. Most often the attitude determination needs for these performance critical missions are supported by star trackers, however star trackers are not always practical for small satellite operations. This motivates the improvement of sun sensor calibration and modeling algorithms to enable attitude determination for high-performance small satellite platforms. Deep learning for attitude determination applications has received limited attention thus far, however some work has been demonstrated for Earth sensors. One such example is an improved three-axis Earth sensor using deep learning image recognition. The use of deep learning to Earth sensing increases the detected feature space beyond the Earth-limb, thereby greatly increasing the accuracy possible of the sensor. This motivates the improvement of sun sensor calibration and modeling algorithms by increasing the richness of the detected feature space beyond traditional centroiding and modeling techniques. 

The sun sensor calibration process is especially difficult due to the complex nature of the uncertainties involved. The uncertainties are small, difficult to observe, and vary spatiotemporally over the lifecycle of the sensor. During ground testing and in-orbit operation, the sensors are affected by numerous sources of uncertainties, including manufacturing, electrical, environmental and interference sources. This motivates the development of improved calibration techniques to minimize uncertainty over the sensor lifecycle. However, the traditional sun sensor calibration process is costly, slow, labor intensive and inefficient. A Digital Twin can improve the modeling and calibration process through predictive calibration and an evolving flight-tested model.  

The harshness of the space environment leads to inherently more risk averse operational conditions. These conditions motivate the development of a comprehensive model credibility framework to support mission critical operations under uncertainty. Unfortunately, the current verification and validation (V&V) standards for space-based systems are insufficient for embedded deep learning space applications. The qualification of space sensors involves analysis, testing, inspection, and demonstration. However, the process of qualification for deep learning systems differs greatly compared to traditional modeling techniques. This motivates the development of a framework to combine the formal verification process with modeling and interpretability. The inspection of deep learning attitude sensors with interpretable models enables human-in-the-loop understanding for V&V ground testing and mission control in-orbit. 

The objective of this dissertation is to develop an interpretable Digital Twin predictive calibration methodology for digital sun sensors to solve 2-axis state estimates under uncertainty utilizing deep regression. A methodology is proposed to address the gaps in traditional sun sensor calibration through: (i) generation and augmentation of datasets, (ii) modeling and training, and (iii) assessment of model credibility. In the first step, synthetic data is generated with a physics-informed model and then augmented for initial model validation. Next, physical asset data is efficiently collected and augmented with the use of an experimental testbed. The second step involves the development and training of a novel Digital Twin predictive calibration model via convolutional neural network (CNN) to address model uncertainty, depth, and feature limitations. The model to be trained is a modified multi-input/multi-output VGG-16 CNN architecture. Finally, the third step assesses the virtual model credibility using verification and validation, Bayesian uncertainty quantification with Monte Carlo Dropout (MCDO) method, and saliency techniques to address model uncertainty and interpretability. The development of this methodology will contribute to the improvement of digital fine sun sensor modeling and calibration accuracy and efficiency, especially for encoded mask configurations. Furthermore, implementation of this methodology will enable on-board adaptive calibration to emergent uncertainties and human-in-the-loop understanding of the uncertainties during testing and operation. 


  • Prof. Dimitri Mavris – School of Aerospace Engineering (advisor) 
  • Prof. Glenn Lightsey – School of Aerospace Engineering 
  • Prof. John Christian – School of Aerospace Engineering 
  • Dr. Olivia Pinon Fischer – School of Aerospace Engineering 


Weber CoVE & BlueJeans