Most of what we talk about in experimentation circles starts with your own data. Heatmaps, session recordings, post-purchase surveys, customer interviews. The idea being: if you want to test something that moves the needle for your brand, start with your customers. @Andy Costes pushed back on that last week on Live with Intelligems. His take: your competitors are running their own experiments too. They have different teams, different budgets, different areas of focus. And when you watch what they ship and keep for two to three months, across multiple brands, you start to see patterns worth questioning. Not copying. Forming hypotheses. Curious where you land on this: - Do you actively do competitor research as part of your testing process? How? - How do you decide if something a competitor is doing is worth testing for your own brand? - Is there a risk of optimizing toward the wrong benchmark?

Hi guys, I'm Rajesh. I'm a paid media first growth marketer for the last 5 years I have worked with every possible niche out there and loved ecom and went all in. CRO was my 5/10 skill because I ideate the overview part of the strategy and leave it to my team. But I wanted to take a voluntary jump into CRO to become a 9/10. Currently working as CRO manager for a supplement brand. New here and want to learn from everyone & Nate wasn't surprised to see you here lol..

We had @Shane Roach on Live with Intelligems 💎 last week and I keep thinking about one thing he said: "60-80% of your homepage traffic is new visitors. But most brands have one homepage built for everyone, which means it's really built for no one." Shane's take: you need two experiences. One for new visitors. One for returning ones. New visitors need to be sold. Hero product up front. Brand story. Reasons to buy. Multiple add-to-cart entry points. Returning visitors already know who you are. They don't need the brand story again. They need a fast path to a PDP or collection page. Shane spent years in physical retail before ecommerce. Nordstrom's suiting department, specifically. While the tailor was measuring a customer in a $1,000 suit, he'd go grab shirts and ties, show them how six combinations work, stack the add-ons before the credit card came out. That's merchandising. That's also exactly what a well-designed homepage should do. My pushbask was: "it's hard enough to maintain one homepage." Shane's response: the new-visitor version is set-it-and-forget-it. Revisit seasonally when imagery or hero products change. It's not a live maintenance burden. The other insight worth flagging: you can't control all the traffic entry points to your homepage. Paid? Sure, send them to a landing page. But organic traffic, social shares, a friend dropping your URL via text all of them land on the homepage. That's why the homepage itself has to do this job. The execution: - Build the new-visitor experience - Run it as an A/B test targeted to new visitors only - If it lifts, turn it into a personalization - Revisit seasonally Full stream linked below. Shane covers this plus a speed run on what Intelligems is shipping right now. What's your homepage strategy for new vs returning? Curious if anyone here has tested this split and what you found.

It blows my mind how many companies run experimentation without a naming convention. Not an imperfect one. None. Tests called "homepage revamp" or "Q2 pricing," somewhere in a spreadsheet or just inside Intelligems, with no structure, no ID, no way to find them six months later without scrolling through 40 rows trying to remember what was actually being tested. And I think it's a hard way to scale an experimentation program. The fix is simple: give every test a unique ID and a structured name before you do anything else. The ID part is what I'm most emphatic about. Something as simple as #1, #2, #3. Or more structured: EXP-23. Looks bureaucratic. Feels unnecessary on your first few tests. But here's what I noticed when I was on the agency side: stakeholders started referencing the IDs in their own conversations. Instead of "that price test from March," they'd say "test 47." That moment, when a client uses a test ID unprompted in a meeting, that's when you know the program has actually landed. They're thinking in experiments now, not in one-off projects. It just makes everyone's lives easier. Stakeholder updates, Slack threads, QA handoffs, everything gets cleaner when there's a shared shorthand. And when you need to iterate on a test, same hypothesis, new variant, you just add a suffix: Ex47a, Ex47b. No confusion, no "wait, which version of that test are we talking about?" Beyond the ID, a naming structure worth trying: 'Ex[#] | Surface | Short Name | Date' Example: 'Ex47 | PDP | Anchor Price Removal | May 2026' The best part: this is trivial to auto-generate with a formula in whatever tool you're using, Notion, Airtable, Google Sheets. One formula column that builds the name from your existing fields. You fill in the surface and a short description, the ID increments automatically, and the full experiment name writes itself. One line. You know what number it is, where it ran, and what changed. A year from now you can look back across your test library and actually see patterns, which surfaces you've tested most, which hypotheses keep winning, where you've put a lot of effort with not much to show for it.

Attached is what a rigorous CRO workflow looked like at my old agency a few years back. Every step had a dedicated human. Research, ideation, technical feasibility, documentation, design, coding, QA. If anyone was slow or busy, the whole thing stalled. And this is assuming the team actually had time to go through all of it — sometimes you had to cut corners just to keep the program moving. Last week @Victor Paytuvi and I went live to talk about how much this has changed. Full replay here if you missed it: How AI is Reshaping Ecomm Optimization Workflows — Live with Intelligems A few things that came up: - Automated test reporting was both of our first real AI use case (~2023). The mechanical part of writing up results doesn't need a human. That time goes back to strategy. - Prototyping with AI closed the communication gap between CRO strategists and designers. Days of back-and-forth became hours. - AI-coded variants are no longer science fiction. One practitioner at Jones Road is running his entire program solo today. We also talked about what won't change: the contextual judgment that comes from knowing a brand deeply, its customers, its history, what's keeping the founder up at night. That's still on us. Curious where this community is at. What's the first step in that process you stopped doing manually?