yeah i've been doing something similar. the biggest unlock for me was having the agent compare against a specific reference image rather than just evaluating "does this look good" — without a concrete target it tends to loop forever making tiny tweaks that don't converge. also found that keeping the feedback to 2-3 concrete items per iteration (instead of a full eval) speeds things up a lot. the thin frontend approach you mentioned is key though, i tried this on a project with server-rendered components and it was a nightmare.