Agentic AI: lessons learnt from a high-street implementation

Having access to clear and available nutritional information on something as basic as a sandwich is critical for the over 2 million people in the UK who have a visual impairment.  

The recent European Accessibility Act (EAA) set out to ensure digital accessibility. Food and drink companies must ensure that product descriptions, nutritional information, and allergen warnings provided on websites and mobile apps are accessible via screen readers and other assisted technologies. 

We recently worked with a leading high-street food retailer who needed to meet these standards; otherwise, they would risk significant fines and penalties. 

Data complexity creates challenges beyond traditional automation

Manually checking this information would take time and effort. That’s where automation came in, but we soon discovered that traditional automation couldn’t handle the sheer visual and structural diversity of the food packaging. 

Every product’s back of pack was different in layout, design and content structure. Take a Christmas food selection pack, offering treats such as goose-fat roasting potatoes, red cabbage and cauliflower cheese. 

Each item had its own set of information, but allergen information was presented differently. Some listings had storage information, others didn’t. Even headings differed across the same packaging.

Automation rules break down in the face of such variances.

So we had to take a different approach. And that approach was Agentic AI.

Using multiple AI agents, we assigned each one a distinct role: extracting, validating and reconciling. This created a collaborative workflow that mirrors how skilled humans check one another’s work. It was a way to scale judgment without sacrificing accuracy. 

Agentic AI presented its own set of challenges

This was a new approach for the retailer and us. So what challenges did we face along the way?

The main one was maintaining absolute trust in the data. Every image was unique; fonts, lighting, print quality and layout conventions. Even the presence of smudges or reflections could distort meanings. 

We were dealing with English text, but no two labels looked alike. This variability meant AI could never operate unsupervised. Human verification was needed for every output to ensure allergens and other regulatory information were represented correctly. 

We solved it in a way that we often don’t hear about: AI and humans working together. AI handled the heavy lifting - OCR (optical character recognition), structuring, and cross-checking.

Humans validated the final result. In the words of Meza, one of our Engineering Managers: “The blend of AI scale with human oversight achieved both speed and certainty”. 

Lessons learnt - good AI design is knowing when not to use AI

As with any Crosstide engagement, there were some lessons learnt along the way that we're already applying:

• Good AI design is about knowing when not to use AI - for example, things like image fetching are best kept deterministic. AI shines where interpretation is required, not where rules are clear.
• The ‘power of structure’ - by this, we mean giving each agent a clear mission, input, and expected output. This led to reliable behaviour. Building explainability into every step made it possible to trace and improve the process instead of treating it as a black box.

Results - speed, scalability and accuracy

So, what have been the results for our retail client?

They’ve got a faster, more reliable data flow that transforms raw packaging images into structured, auditable information. They can now capture ingredients, allergens, and nutritional data at scale while maintaining human-grade accuracy. 

Processes that once took hours of manual transcription are now completed in minutes, with a transparent review trail that satisfies both internal governance and external regulations.

The ingredients for successful Agentic AI rollout

For retailers, this is just the start in terms of using Agentic AI. But what’s key to scaling these types of solutions is simplicity and focus. Single-purpose agents that perform one job well are easier to reuse, test, and adapt than broad, generalist models. 

Once an agent is proven - say, one that validates ingredient lists - it can be redeployed across new product lines or even new retailers with minimal retraining. This modularity turns a complex AI pipeline into a library of reliable components that can grow organically within the business. 

Some workflows naturally lend themselves to Agentic AI. Any that rely on humans to interpret unstructured data are ideal candidates. 

Supplier documentation, sustainability claims, recipe digitisation, and even product photography metadata all have the same pain point: humans bridging the gap between messy reality and clean systems. Agentic AI offers a way to automate that bridge while keeping accountability intact.

Food for thought - recommendations for other retailers

We know Agentic AI offers opportunities, but having seen the challenges first-hand on this project, Meza shares his invaluable advice for other retailers (and non-retailers):

• Before going down an Agentic AI route, map your judgment work. Identify where staff are making repeat interpretive decisions rather than mechanical ones. It's those repeat decisions that will benefit most from an Agentic AI approach.
• Prototype with accountability in mind - each agent must have a defined purpose, a way to explain its reasoning, and a clear path for human review. This clarity builds trust and makes scaling safe.
• Design the thinking before you design the doing. Each agent’s system message should describe how it should reason, not just what to do.
• Lay out each agent's decision protocols so it can explain itself. If you find you are writing long English instructions that resemble an algorithm, that is usually a sign that traditional code would be more reliable. AI should interpret, not imitate code.
  
  

Finally, as we prepare for this year’s festive food shop - one of the busiest times for food retailers - humans and technology will be working behind the scenes to ensure that everyone gets the essentials (maybe with a few treats) that they need. 

Follow me on LinkedIn

An audio version of this blog is also available here.