Deep Insight Think Deeper. See Clearer
On 30th November 2022, with the launch of ChatGPT to the general public1, generative AI left the laboratory and entered meeting rooms, financial services, hospitals, schools, and more. The main advantage of this technology is well known – with just a few clicks, it can transform a mass of data into fluid, intelligible text. Today, with this tool, a financial director can obtain an automatic commentary on his margins in a matter of seconds, a doctor can obtain a report based on examinations, and a student can generate an essay from a simple statement.
This convenience and ease of use are a game-changer. Where business intelligence mainly produced figures and graphs, generative models add a layer of interpretation. They prioritise signals, offer explanations and sometimes suggest forecasts. However, a clear narrative gives the impression of obviousness : the conclusion seems robust because it is well formulated, even though it is based on just one model among many2.
The risk lies not in the use of AI, but in the excessive credibility given to texts for which we often do not know the conditions of production. In other words, can we decide on an investment of several million pounds or make a medical diagnosis based on the recommendations and interpretations of generative AI ?
Re-examining the trust given
The trust given to a numerical response is usually based on two conditions : the quality of the source data and the transparency of the calculation method. However, in the case of a literal response such as that produced with generative AI, a third layer is added : the interpretation of the model3.
Indeed, the model decides what to highlight, discards certain elements and implicitly combines variables. The final product is an automated narrative that bears the mark of invisible statistical and linguistic choices. These choices may be related to the frequency of the data used to build the model, problem-solving methods or any other cause. To ensure confidence in the answer given, these steps should be auditable, i.e. indicated by the user, who can then verify them.
It is now important to imagine a form of algorithmic audit, no longer just verifying data but controlling the entire chain
This solution, which allows for verification, already exists in similar situations. First of all, showing the thought process is a common approach in teaching mathematics, as it allows the teacher to ensure that the student has understood the steps involved in the reasoning. Similarly, in financial analysis, audits are used to verify compliance with accounting rules. Financial audits guarantee that the published figures correspond to a measurable reality.
Thus, it is now necessary to imagine a form of “algorithmic audit”: no longer just verifying data but controlling the entire chain that leads from the raw flow to the final narrative. Take the example of a hospital where generative AI summarises patient records. If it systematically omits certain clinical parameters deemed rare, it produces attractive but incomplete reports. The audit must therefore test the robustness of the model, assess its ability to reproduce atypical cases and verify the traceability of sources. Similarly, an automatic energy report that ignores abnormal consumption peaks can give a false impression of stability. Here again, the audit must ensure that anomalies are taken into account.
Technical protocols to be optimised and deployed more widely
Trust-based engineering cannot rely solely on declarations of principle. It must be translated into specific protocols. A number of approaches are already emerging :
- Traceability of algorithmic choices : each indicator must be linked to the source data and the processing applied. This involves documenting transformations, as we currently document a supply chain. Circuit tracing methods can provide traceability that is understandable to humans4. Traceability then becomes an educational tool as well as a control mechanism.
- Model stress tests : exposing AI to unusual scenarios to measure its ability to reflect uncertainty rather than smooth it out. For example, it is very useful to use a sample that does not follow the classic distribution to check whether the deep AI model has been integrated, independently of the test set provided5. This could involve providing a set of lung X‑rays from smokers only. This makes it possible to verify that the AI does not generate an excess of ‘false negatives’ to return to a statistical average.
- Minimum explainability guaranteed : Without revealing the algorithmic secrets of companies providing AI solutions, it is envisaged that they will be asked to provide at least a summary of the main variables used in their models to reach a conclusion. This explainability could be subject to either ISO-type certification for AI quality or validation by a regulatory body (preferably an existing one so as not to multiply the number of authorities).
These methods will not remove the confidentiality associated with the specific and differentiating settings of the manufacturers who develop large language models, but they will reduce the risk of blindness and unjustified confidence. The issue is not to make AI completely transparent, but to create sufficient safeguards to maintain trust.
An organisational culture in need of transformation
Beyond the technical dimension, it is necessary to promote a major cultural shift. For decades, organisations have been accustomed to viewing figures as certainties. Dashboards are often perceived as indisputable truths. With generative AI and its extension to literal and subjective productions, this stance is becoming untenable.
Decision-makers, as well as all digital stakeholders, must learn to read an automatic report as a statistical response based on known or unknown assumptions, and above all not as a definitive conclusion. This means training users of AI solutions to formulate demanding requests (asking for the AI’s ‘reasoning’ process) and to read responses critically : identifying margins of error, questioning omissions, and asking for alternative scenarios. In other words, reintroducing uncertainty into the very heart of the decision-making process.
The European Union has begun to lay the groundwork with the AI Act, which classifies the use of AI in finance and public governance as ‘high risk’. This regulation imposes an obligation of transparency and auditability. But the law will not be enough if organisations do not cultivate active vigilance. Generative AI must be controlled not only by standards, but also by a daily practice of critical reading.
Moving towards a measure of vigilance
Generative AI is neither a mirage nor a panacea. It speeds up access to information and provides clarity on volumes of data that are unmanageable for humans, but it also transforms our relationship with decision-making. Where we used to see numbers, we now read stories.
The challenge is therefore not to turn back the clock, but to invent a new engineering of trust. Traceability of calculations, stress tests, minimal explainability : these are all technical building blocks that need to be put in place, bearing in mind that an AI model is likely to be the target of multiple cyberattacks 6,7.
But the key lies in adopting a new organisational culture : accepting that uncertainty is a given and not a failure of the system. Only then can generative AI become a reliable tool to support human decision-making, rather than a producer of illusory certainties.
Bandi, A., Adapa, P. V. S. R., & Kuchi, Y. E. V. P. K. (2023). The Power of Generative AI : A Review of Requirements, Models, Input–Output Formats, Evaluation Metrics, and Challenges. Future Internet, 15(8), 260.
https://doi.org/10.3390/fi15080260↑
Eche, T., Schwartz, L. H., Mokrane, F., & Dercle, L. (2021). Toward generalizability in the deployment of artificial intelligence in radiology : Role of computation stress Testing to overcome underspecification. Radiology Artificial Intelligence, 3(6). https://doi.org/10.1148/ryai.2021210097↑
Esmradi, A., Yip, D.W., Chan, C.F. (2024). A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models. In : Wang, G., Wang, H., Min, G., Georgalas, N., Meng, W. (eds) Ubiquitous Security. UbiSec 2023. Communications in Computer and Information Science, vol 2034. Springer, Singapore.https://doi.org/10.1007/978–981-97–1274-8_6↑
