AI in Conversion Rate Optimization: Tasks vs. Strategy (Part-One)
Intro
With the rise of AI, many people and fledgling startups are looking to AI to simply look at a website, tell you what to test and run the test automatically. In fact, just this morning on the day of writing this, I saw a company use AI to reference thousands of tests, look at your website and based on its test database, recommend changes without even running an experiment.
Conversion Rate Optimization (CRO) programs are complex. They require leadership, coordination, deep understanding of customers and the correct mindset. CRO programs, when done correctly are not tools, they are systems.
As of this writing, AI is very powerful. But still it’s most powerful as a partner, guided by humans. That may change some day, but right now, AI is a power that needs to be managed by people.
The Nature of CRO: More than just tests
Strong CRO programs are systems, not just tools or tactics. There are some key ingredients that go into best-in-class experimentation programs. These include but are not limited to; The correct mindset – how you think and approach experimentation. Executive Involvement – are they involved, as champions, leaders, giving direction and coordinating teams? Governance – the right documentation, the right iteration, the right processes and approaches. Customers Centricity – deep understanding of customers pains, needs and wants and what they are trying to accomplish. And many other aspects like ideation, strategy, communication, team alignment, technology, skill set, reach, personalization, etc.
Without these aspects of the program aligned around a proper strategy, the CRO program can become misaligned, or worse a dysfunctional compass, pointing people down incorrect paths.
Can AI solve this? Well, yes and no. There are levels of AI, and it depends on the people who use them or make them. AI tools like ChatGPT are one thing, agentic AI tools are another altogether.
Prompt based AI tools are like a smart intern. Agentic AI is like a project manager. But they both still require strategic guidance.
What AI Can and Cannot Do
AI can excel at:
Among other things.
Where AI struggles right now
A mature experimentation program is there for discovery, exploration and challenging the status quo. A good strategist will ask “what if” and then try different experiments, learn from those experiments, and document that approach into compounded learning that moves the business forward. It’s a series of exploring, learning and applying what you’ve learned repeatedly, based on data and results.
Can AI do that? I’m going to say, not yet.
Can AI help strategist do this job? Absolutely!
Prompt-based vs Agentic AI
Prompt-based AI is what most people think of when they look to use an AI tool. Usually going to a site like ChatGPT, Claude, Perplexity or others. These are great for one-off tasks (e.g., “Write a test hypothesis,” “Summarize a report”). Usually, these tools and their one-off tasks are only as good as the details of the prompt itself. So, what you get out of them depends a lot on you, what you know, how you phrase it and how you iterate using it.
Agentic AI is more of an autonomous system that sets goals, plans and takes action to achieve them with minimal human intervention. But it is also only as good as the people who design it in the beginning. In terms of CRO, think of tasks like pulling analytics, preparing weekly CRO documentation, generating first-pass reports, etc.
Agentic AI will help operationalize tasks but still requires strategy at the helm.
Essentially, Prompt-based AI is like your intern, Agentic AI is like your project manager, and both are as good as the direction you give them.
Why Humans and Strategy are still essential
AI at this point is the worst it’s ever going to be, which means it’s only going to get better. But right now, leaving AI to decide what matters to test, is like letting a GPS choose your destination. It can get you there efficiently, but only if you’ve told it where you want to go.
The Human + AI partnership
A human sets the vision; the AI executes tasks faster. For example, a strategist looks at the roadmap and decides what to prioritize, agentic AI can maintain it, run updates and communicate to stakeholders.
A human analyst interprets data and AI compiles, cleans and formats it for delivery. However, the analysts still has to tell the story to executives, which is an art form that entails more than just charts and raw data.
Or maybe a strategist runs an ideation workshop, but prompt-based AI helps to draft the hypotheses.
I know what you are going to say, AI can do these parts of the strategist and analysts. Maybe, but at this juncture, a human can still direct the big picture. I believe that at this point in time, it’s strongest when led by a human strategist and supported by AI.
Looking Ahead: The Future of AI and CRO
As mentioned above, right now is the worst AI is ever going to be. It’s only going to get better. Agentic AI is moving rapidly, expanding into tasks that are ripe for automation. But strategy, alignment and culture are still human driven. Especially since AI, when it doesn’t know what to do, hallucinates to fill its knowledge gap.
If you want to run a bigger program and you lack the human resources to do so, right now with AI, one or two people can act like more. AI is a force-multiplier, so is a good experimentation program. Get a good strategist and use AI as a workflow engine.
Conclusion
CRO is not a set it and forget it function. There is no autopilot. But AI can be a co-pilot.
Prompt-based AI can help you today, Agentic AI is going to unlock even more workflow efficiencies. But neither can replace strategy, customer empathy, mindset and the ever-changing landscape of competition and market forces (at least not now).
I still ascertain that a good program is built on the correct mindset, the correct culture and leadership involvement. Embrace AI, but do so thoughtfully, while still maintaining the aspects that really help us adapt and overcome.
Thanks for reading part 1, please continue the journey to Part 2: AI & Personalization, where conversion rate optimization programs meet tailored, individual experiences.
