Managing AI after deployment: five companies and the engineering questions to ask

An AI feature can return valid JSON and still produce the wrong result. Production checks therefore need to test behavior as well as availability.

GeekyAnts’ published examples describe version-controlled prompts, evaluation baselines, and request-level records. These are project-specific practices, rather than proof of a universal implementation.

Five companies to evaluate, without implying a performance ranking:

• GeekyAnts: Application-level AI integration and documented operational controls.
• IBM: Model choice through watsonx.ai.
• Accenture: Foundation-model customization and management.
• Deloitte: Enterprise generative AI implementation.
• TCS: Model comparison and integration through WisdomNext.

For an extraction service, a useful acceptance test is whether a model update still handles incomplete documents correctly. A successful HTTP response says little about that.

Teams comparing providers should request sample evaluation results and a demonstration of how a failed update is handled.

The source article on managing AI after launch provides implementation examples.

Which has caught more production issues in practice: output validation or evaluation datasets?