The presentation was convincing. The licenses were ordered quickly. And after the first few weeks, questions suddenly arose that had apparently been left out of the product demonstration: Why are some of the answers incorrect? Why does the verification process take so long? And who actually decided what data the system is allowed to use?
So if you encounter initial setbacks when implementing AI, that doesn't mean you have to throw in the towel. But simply carrying on as usual isn't a strategy either.
AI isn't a sprint; it's a boxing match that goes several rounds. It's not just the first performance that matters. What matters is what a company learns between rounds.
Implementing AI involves integrating technology with business processes, data, people, and legal requirements. These elements do not automatically fit together simply because an application is available.
The goal, therefore, is not endless experimentation, but a controlled learning process: defining a specific benefit, testing it under appropriate conditions, evaluating the results, and making targeted improvements to its implementation. Legal requirements and protective measures are an integral part of this process from the very beginning.
However, the "box" metaphor has its limits: Neither the employees nor the technology are the enemy. The key is to deal with uncertainty without losing your bearings every time you face a challenge.
“The AI isn’t working” is an understandable reaction. As a troubleshooting statement, it’s about as helpful as “Something’s wrong with the engine.”
Was the task unclear? Was there a lack of up-to-date information? Were results accepted without being verified? Or was the system expected to do something for which it simply isn't designed?
A qualitative RAND study from 2024 involving 65 experienced professionals in AI development and research identified, among other things, unclear problem statements, insufficient data, and an excessive focus on technology as causes of failed AI projects. The study provides insights into typical mistakes, but does not offer a universally applicable failure rate.
For companies, this means: First, collect specific examples of errors and investigate their causes. Only then should you decide whether a different tool, better data, a more focused task, or a fundamentally different approach is needed.
After a disappointing pilot project, the next major attempt is sometimes surprisingly close at hand: more features, more data, an even more powerful model. It may be more helpful to limit the scope of the project at first.
A hypothetical example: A medium-sized retailer is testing an AI assistant to handle customer inquiries. It is designed to answer product questions, address delivery issues, and handle complaints. During testing, it confuses product variants and makes promises that the company did not intend to make.
For the second round, the task will be modified. The assistant will now only prepare internal draft responses for a selected product group. He will receive reviewed, up-to-date documentation. Complaints and binding commitments remain outside his scope of responsibility. Employees will review the drafts before they are sent.
That doesn't prove success just yet. But the next test may show whether this limited use actually reduces the load.
A narrower scope of application is not a step backward if it enables a more sound decision.
In the workplace, protection doesn't come from pulling up your gloves. It comes from appropriate safety measures and clear decisions.
What information may be processed? Who has access? For what tasks is the application authorized? Which results must be reviewed by subject matter experts? And when is its use suspended?
The legal assessment depends on the specific use case. The EU AI Act distinguishes, in particular, based on risks and the company’s role. When it comes to personal data, data protection requirements must also be taken into account. An internal approval does not replace this assessment.
This should result in clear guidelines. “Please use responsibly” sounds friendly, but leaves the crucial questions unanswered.
For example, a useful set of guidelines describes acceptable data sources, excluded tasks, necessary checks, and a contact person who can be reached. This is especially helpful when there is little time for policy discussions in day-to-day operations.
Just because someone knows how to use a tool doesn't mean they can automatically evaluate the results. For use in the workplace, employees must also be able to recognize when an answer sounds plausible but isn't technically sound.
Application-oriented AI training should therefore focus on typical tasks and scenarios where errors might occur. The challenges in customer service differ from those in human resources or software development. Effective exercises address not only successful outcomes but also missing sources, contradictory information, and situations in which AI should not be used.
Equally important is the organization behind the tool. Who collects feedback? Who initiates new tests? Who coordinates changes with IT, data protection, and the business unit?
An AI Officer can coordinate these tasks and bring together the expertise of various departments. The role promotes clear lines of responsibility; it does not take decision-making away from management or the relevant departments.
That position needs to be filled. A new title on a business card alone isn't enough.
It's easy to tell whether an AI application is impressive. Determining whether it's useful in a business setting requires a comparison.
To that end, clear evaluation criteria should be established before the next test:
The evaluation should also include difficult and unusual cases. It is understandable to showcase only the most successful examples in a product presentation, but that is not enough to support an investment decision.
After that, there are three reasonable options: expand the scope of the project, make targeted improvements, or discontinue the project. The fact that part of the budget has already been spent does not mean that further spending is justified.
Perseverance means staying committed to your business goal—but not necessarily to the first tool you choose.
Even a well-functioning AI implementation is not set in stone. Providers update their models, internal knowledge bases grow, and specialized departments discover additional applications.
For this reason, it is advisable to establish a set schedule for reviews. Significant changes—such as new data sources or a different intended use—should also serve as grounds for retesting and an updated assessment.
The voluntary NIST AI Risk Management Framework explicitly describes AI risk management as an ongoing task throughout a system’s lifecycle. This is a technical guideline, not a European legal requirement.
ISO/IEC 42001 also emphasizes continuous improvement in an AI management system. The concept behind it aligns with best practices: plan, implement, review, and adapt. However, the standard does not replace the legal review of a specific project.
Anyone who wants to use AI effectively needs not only practical experience but also an understanding of its technical, legal, and organizational foundations.
This is where the book *AI Regulation—Made Easy* by Alexander Deicke, Francois Heynike, and Hannes Deuerling comes in. Among other topics, it covers the fundamentals of artificial intelligence, the EU AI Act, data handling, and relevant standards. The conversational style makes it easier to get started with a topic whose terminology sometimes seems more complicated than the questions behind it.
A book is no substitute for hands-on practice or assessing current requirements. However, it can help you ask the right questions before a small-scale trial turns into a larger-scale operational deployment.
K11 Consulting helps companies integrate this knowledge into their day-to-day operations through training and appropriate governance structures.
A disappointing first experience with AI is neither proof against the technology nor a call to give it unlimited further chances.
A successful AI implementation requires a willingness to learn and clear boundaries: analyzing mistakes, tailoring tasks appropriately, empowering people, and honestly measuring benefits.
If you took a “hit” in round one, that doesn’t mean you have to give up. For round two, however, you’ll need more than just the same plan with a bigger budget.