, and it’s transforming the way we live and work. For companies, this revolution presents a dual opportunity. On one hand, the chance to solve previously really complex problems and to build incredible new products and features. On the other hand, the promise to boost efficiency across a wide range of tasks.
Many resources cover one opportunity or the other, but as a ML/AI Product Manager, I am actually interested in considering the two opportunities at the same time. What this actually means is: how to successfully develop AI-powered solutions, accelerated by AI itself. Based on my experience and learnings, I’ve split the process into five phases: ideation, design & plan, development, deployment, and impact and monitoring. In each phase, we’ll cover “what” needs to happen, but also “how” to use AI to boost efficiency and quality.
Phase 1: Ideation
The goal of this phase is to move from a high-level objective to a specific, prioritized solution.
The first and most critical step is to remember that AI is a tool, not a solution. Always start with the problem you need to solve. It should be directly aligned with your company’s high-level OKRs and validated with evidence from user research and data.
Once the problem is clearly defined, brainstorm a variety of solutions. This should include both traditional non-AI approaches and potential AI-powered features. Prioritize these solutions using a structured method. A framework like RICE (Reach, Impact, Confidence, Effort) allows you to make a data-informed decision by weighing the potential value of each solution against its cost. For AI solutions, remember that “Effort” includes dealing with AI’s complexity such as data acquisition, system evaluation, or determining required guardrails.
To make sure this blog post is not too abstract, I’ll use my favorite marketplace use case as an example. A common user pain in marketplaces is the time and effort it takes to list new items (e.g. determining the right price, category, writing the description…). Data allows to quantify this problem: a high percentage of users who start creating a listing but never finish.
To address this, you could consider traditional non-AI solutions like offering templates, providing tips for each field, or creating a better onboarding process. Or, you could explore AI-powered solutions, such as using a large language model (LLM) to generate a product description or suggest a category. AI is a really cool tool though, as it can be applied to multiple use cases and applications, problems that used to be hard are now feasible, and it lowers the entry barriers to predictive models.
Building AI products always seems easier than it really is. The main hidden challenges are the integration of non-deterministic, probabilistic models (ensuring their outputs are relevant, consistent, safe…) while aiming for competitive advantage (your core enabler is an external provider any competitor can user).
⚡️ Accelerating the Ideation Phase with AI
- Tools for Brainstorming: AI chatbots like OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, or Perplexity can act as an extra brainstorming team member. You can prompt them with your user’s pain points and ask for a wide range of potential solutions, both traditional and AI-based. Consider and test different AI chatbot “flavors” available: getting answers from simple LLMs, getting answers with LLM leveraging “reasoning” capabilities (Chain of Thought prompting or “reasoning” model versions, e.g. Gemini 2.5 Pro, OpenAI o3,…), getting answers with LLMs using web search results, and Deep Research functionalities.
- Tools for Knowledge Management: Platforms like Notion AI, Mem, Tettra, or Glean can help you organize your research and ideas, using AI to connect to relevant internal knowledge and information.
Phase 2: Design & Plan solution
The goal of this phase is to take a prioritized solution, assess and mitigate potential risks, define its MVP and create a concrete project plan and initial designs.
With a prioritized GenAI solution in hand, implementation should be designed holistically across four dimensions:
- Capturing user input and relevant context (moving from prompt to context engineering)
- Selecting and configuring the right model (balancing cost, latency, and performance)
- Generating and evaluating outputs for quality and safety
- Delivering results through effective UX/UI that supports user trust and feedback.
Throughout, teams must embed monitoring, evaluation, and risk management practices (addressing bias, compliance, and observability) to ensure reliability, scalability, and trustworthiness. If your are building an AI Product, another crucial part of this phase is assessing the four big risks of product management: value, usability, feasibility and viability.
With all this in mind, we start further designing and planning for the project. In this step it is key to understand how to start small (Minimum Valuable Product) to expand and iterate once value has been proven.
For our marketplace example, let’s consider a feature that suggests a description and category based on the product title. The flow could look like this: a user inputs their item’s title, which is then used to construct a prompt for an LLM. The model returns a suggested description and category, which are pre-filled for the user to edit.
The feature’s risks can be broken down by category. For usability and value, the feature is familiar and editable, which is good for user experience. The main risk is AI hallucinating or producing non-relevant suggestions, which must be measured with evaluations during the development phase. For feasibility, generating a description and category with current LLM capabilities should be feasible. And for viability, this includes considering ethical risks, for instance, generating biased or discriminatory suggestions (e.g. a cleaning product -> “perfect for women”), which also needs to be specifically evaluated during development.
All the AI system’s parts need to be connected to ensure trust & value. For every potential risk you identify now, ensure you include it into your future evaluation plan.
⚡️ Accelerating the Phase with AI
- Tools for writing: Accelerate the creation of your Product Requirements Document (PRD) with tools like ChatPRD. You can also improve the clarity and quality of your writing with assistants like Grammarly or Quillbot, and even get specific feedback on your writing with Quarkle.
- Tools for preparing presentations: Different tools like Gamma, Pitch, or beautiful.ai, are offering generation of slides from simple text and other documents.
- Tools for prototyping: AI can help you create everything from simple front-end mockups to complex, fully functional prototypes. Tools like Figma Make and Uizard are great for design-focused prototypes, and Claude artifacts is also great to prototype UIs really fast. Platforms like Replit, Lovable, V0, Bolt can generate prototypes closer to fully functional MVPs, by generating the code full stack.
Phase 3: Development
In this phase, you’ll move from experimenting with prompts to getting the system ready to be integrated into your platform.
This is where prompt engineering and trying out different models and approaches takes place, with the goal to get the best possible outputs your use case needs. The key is to prepare a request to an LLM with specific instructions, which will return generated text in the requested format (e.g. JSON object containing the suggested description and category).
Avoid adding unnecessary complexity. Although AI Agents are a hot topic, they add costs, complexity and non-determinism. In many cases it is best to build a predefined workflow that concatenates LLM calls in a happy path.
A critical and often overlooked part of development is evaluation. You need to be sure that the model’s predictions and generations meet a certain quality bar before they go live, and that the risks identified in phase two are mitigated. This involves defining use-case-specific evals to measure things like hallucination, correctness, bias, and task-specific performance. For a deep dive into this topic you can check my previous post:
For our marketplace example, we might want to track the percentage of times the output is in the correct format, the accuracy of the category predictions, the relevancy of the generated descriptions, the percentage of times our outputs where biased or discriminatory…
Evals are the hardest but most necessary step in the development phase. Try to obtain inputs as close to future reality as possible, start with manual evaluations to then escale to more automated ones.
⚡️ Accelerating the Phase with AI
- Tools for Coding: Many software development tools now include Generative AI features that help lower the entry barrier to coding. Assistants like Github Copilot, Cursor, Windsurf, or Claude Code are widely used to suggest code, complete functions, and solve coding problems. The use of AI Chatbots is also widely extended among programmers to accelerate code implementations.
- Tools for Evaluation: LLMs are both used to generate input datasets when real data is not available, and to design metrics that scale through the technique LLM as a judge.
Phase 4: Deployment
Once your solution is developed and evaluated, you’re ready to deploy it to production. This is where you integrate the solution into your company’s processes and platform so it can be used automatically.
In our example, deploying the solution would allow that, when a user in the platform publishes a product, this triggers the call to an LLM to obtain the description and category from the title, and those are displayed in their corresponding touch-points.
Cloud providers like AWS, Azure, and Google have dedicated tools to accelerate the process of integrating LLMs into your platform in a scalable way. On top of using these tools, you will need to care about service metrics like latency to ensure a good user experience.
A major challenge with Generative AI is the “free input/free output” nature of the technology, which can introduce new risks. For example, users might accidentally enter personal information or even try to attack your system through “prompt injection”. This is where guardrails come in. Guardrails are checks you put in place to ensure the robustness of your solution. They can be used to detect and block unwanted input, and to ensure outputs follow certain predefined rules like avoiding profanity or mentioning competitors.
Don’t just deploy the AI feature: your go-live plan isn’t complete without being ready for what can go wrong in production and therefore ensuring observability (service performance, security, quality…).
⚡️ Accelerating the Phase with AI
- Tools for Guardrails: You can implement safety checks using specific open-source libraries like Guardrails AI and LangChain, or use managed services from cloud providers like Microsoft Azure AI Content Safety. These tools, similarly to evals, many times include LLM calls to automate the guardrail check.
Phase 5: Impact and Monitoring
Your solution is live, but the work isn’t over. Now you need to make sure it continues to perform as expected while having a positive impact to users and to your product and business.
This involves a combination of:
- Service monitoring, where you use tools like Datadog or specialized platforms like WhyLabs and Arize to track the operational health and quality of your AI system in production.
- Quantitative data with analytics dashboards to measure the feature’s impact on key product metrics like user retention and engagement. For our marketplace example, you’d want to see if the new feature leads to a decrease in the number of users who abandon the listing process.
- Qualitative feedback from users to understand further their experience and identify areas for improvement.
⚡️ Accelerating the Phase with AI
- Tools for Qualitative Analysis: Many vendors that help collect user feedback, such as Typeform and Canny, are now incorporating AI features to automatically analyze and categorize responses. You can also leverage LLMs directly to analyze large volumes of qualitative feedback. Instead of manually reading thousands of comments, you can use an LLM to summarize themes, classify feedback by sentiment or topic (e.g. “inaccurate suggestion,” “UI feedback,” “latency issue”) , and identify emerging issues.
Not all GenAI features are shiny✨, but even the less shiny ones can be very impactful. AI solutions integrated in the backend and not user-facing, might have a lot of potential too.
Wrapping it up
Developing an AI-powered solution is a journey from a user problem to a measurable impact. By moving through these five phases, you can navigate complexity and risks, while significantly improving the odds of building something of value.
In a meta twist, AI itself has become your creative partner on this journey, ready to help you and your team brainstorm, code, and analyze feedback faster than ever before.
AI is making this a very exciting time to be building. The only question left is: What will you build next?