Agencies are increasingly bringing AI into their analytical workflows, drawn by its promise to handle larger datasets, streamline processes, and speed up research. But while the role of analysts is changing, their contribution to meaningful insights is as important as ever.

The specific demands of qualitative analysis, especially within the government's need to handle sensitive information and adhere to strict regulations, necessitate a careful evaluation of whether open-source tools designed for broader use can adequately meet key requirements.

Systematic analysis of internal qualitative data offers a powerful, yet often underutilized, approach for federal agency leadership to gain deeper insights into the human factors driving organizational performance, thereby enabling the identification and mitigation of waste, inefficiencies, and redundancies.

It is commonly proposed that a human (or a group of humans) has the final say on any AI-based decision. While the methods and merits of this mitigation are debatable (a subject for another time) what is not debatable is the importance of appropriate trust: In order for human oversight to function, the individuals providing oversight must be given the appropriate tools and motivations to correct the model when it is wrong and trust it when it is correct.

This combination of capabilities and dramatic failures is surprising if you expect the (perfect but limited) behavior of a computer or the (imperfect, but self-aware) behavior of a human, but they are a predictable consequence of how these models are trained.

Public discussions around broken AI or fixing AI are in line with how computers have performed in recent history: either you get what you need, or a result is returned that obviously indicates a failure.