By Oloruntobi Oluwaseun Ojumu
As software engineering teams strive to inject artificial intelligence into more aspects of modern software systems, there is a significant trade-off of velocity/scalability for a sense of responsibility. Many engineering teams focus a great deal on model velocity/scalability; they often prioritize this value at the expense of not being responsible, interpretable, and fair enough before being safely deployed in production. As a software engineer with deep insight into interactions of machine learning systems with production-grade deployments, I have firsthand seen how a lack of focus on these imperatives leads to brittle, biased systems that erode user confidence, as well as fail to comply with regulations accurately.
Responsible engineering of artificial intelligence is not some theoretical exercise but a concrete, engineering, as well as ethical, imperative that starts at design time and persists well after a product release. It calls for deep interplay between software designers, data scientists, subject matter specialists, as well as product stakeholders, to ensure that artificial intelligence systems not only function at an optimum level but also behave responsibly in line with societal mores as well as business ethics.
On one of the high-level business applications for which I helped with the architecture, our goal was to roll out a machine learning model created with the goal of automating applicant screening on an online recruitment site that was worldwide. Trained on past applicant datasets, the model showed stunning statistics of precision. Yet, a more in-depth analysis revealed an alarming discrimination: it consistently underrated candidates with non-traditional education levels, especially from developing nations. Even though no explicit programming for discrimination existed for the model, it absorbed existing biases in the set used for training. Without an explicit design of engineering for fairness, this problem would have gone undetected during production, thus discriminating against a significant population group and opening the site to possible reputational harm as well as litigation.
To deal with this problem, we created a pipeline that included bias detection checkpoints during the model retraining loops. Through using a continuous integration and continuous deployment (CI/CD) pipeline with open-source fairness libraries, we tracked the behavior of the model with regard to protected attributes like sex, ethnicity, and education level. This effort was not a point fix; instead, it was an integral part of the engineering workflow. Just as a test catches a performance regression, we treated fairness as a software quality metric in need of systematic enforcement.
But fairness is only one aspect of responsibility. For production settings where end-user consequences rely on models, like those for loan applications, hiring processes, medical diagnoses, or content moderation, there is a need for explainability. We must not only understand what decision was made, but also why a given decision was made. Such gives rise to a unique engineering problem, especially for black-box models like deep neural networks.
As a second example of a separate deployment, a customer-facing application used a sentiment classifier for support ticket routing. Business managers were interested in having support tickets generated from this model audited, especially for those that were subject to escalation. We incorporated a locally run explainability module using LIME (Local Interpretable Model-agnostic Explanations), which provided interpretable explanations for each prediction from the model. We displayed these explanations via the user interface and were consistent with internal logging. Integration was successful because our model-serving infrastructure was in very tight sync with our observability tooling. We did not treat explainability as an afterthought, but rather explicitly included it as a core design consideration.
The third pillar, accountability, is often the least discussed because it overlaps with both technology and organizational realms. Who takes ownership of a model after it is deployed? Who is responsible for keeping an eye on its behavior, redeploying it when concept drift is observed, or checking its predictions on demand? We define software systems with explicit operational ownership in mature engineering organizations. Similar principles should be used for artificial intelligence. In a specific multi-tenant SaaS application for which I was responsible, we used versioned model governance processes. Every deployed model was accompanied by a model card, which included its training data lineage, its measurements of performance, noted constraints, as well as an assigned owner. Our versioned model cards were automatically generated as a natural part of our machine learning pipeline and were saved alongside API documentation, thus making them easily accessible to engineers, quality assurance analysts, as well as compliance officials.
The deployment of responsible AI isn’t a problem to be solved in silos but a change required in engineering culture. It involves disabusing ourselves of thinking models reside only with data scientists and recognizing production AI systems as software systems, subject to the usual quality requirements of reliability, auditability, and maintainability. And it involves building internal tools that flag ethically problematic behavior as quickly as they flag performance issues. Most importantly, it involves building in feedback loops whereby models can be changed responsibly alongside systems they intend to support, as well as society at large.
As governments move towards regulating artificial intelligence and companies confront more demands from their clients and citizens, the concept of responsibility will ever more be a necessity, not an option. For engineers, this is not a limitation but an opportunity. It presents a distinct chance to define the yardsticks by which intelligent systems are developed, examined, as well as trusted.
My experiences with production systems of AI have shed some light on understanding that sustainable innovations are not innovations with speed alone or of extreme complexity; they are innovations designed to be fair, explainable, as well as accountable from the design stage onwards. We embark on a new century in which engineering excellence must be accompanied by ethical thinking about the future. For practitioners hoping to design future infrastructure, this alignment is not only desirable but necessary.