The Framework for AI PM lens. Product of choice: ChatGPT Use Case: summarizing long reports and documents, e.g. NPS report, with ChatGPT Use these six questions to guide your thinking: 1. What's the real job this solves? Not what the AI does technically. What does the user actually need? Ans: The user needs to save time in extracting the key information out of documents so that they can share with stakeholders or look good in front of their management team. 2. How does the system stay grounded? What keeps it from making stuff up? (Data? Limited scope? Sources? Guardrails?) Ans: The system uses the data that the user provides. In this case it would be documents or reports to summarize. It can also show reasoning if asked by the user. 3. What's the context the model receives? What instructions or information did the team give the model about how to behave? Ans: The instructions were clear in the user prompt, “summarize this NPS report and provide the top five most critical users concerns and recommended follow-up action items.” If the output from the system did not provide clear info, then the user would follow-up with a refined prompt. 4. What are the failure modes? What could go wrong? And when it does, how does the product handle it? Ans: The user could provide a vague prompt, like summarize this report. In this case, the system might not give the user the specific summary that they are looking for. The other thing that could go wrong is the user ask for specific information from the documents, but the AI system could not find that information. In this case the AI system will either hallucinate or say “sorry I don’t have that information.” 5. What trade-offs did the team make? Did they choose speed or accuracy? Consistency or flexibility? Safety or usefulness? How can you tell? Ans: They made the following trade-offs: -- They picked speed over accuracy because the system answers instantly -- They picked flexibility over consistency because the system sometimes give different answers

The Three Dimensions Of AI Fit Score on the AI job finder for PMs who are in transition idea. Data Readiness: Score = 4: Good data, but some gaps. Lot of Job listings, and resumes, but you have to find a good way to aggregate them because you can’t use the job boards APIs. Output type: Score = 5: Subjective/judgment-based (AI is built for this.). The output is a conversation on what’s right for an individual based on their experience, skill-sets, goals and a set of job listings. Error tolerance: Score = 3: Errors acceptable with human review or correction (Human-in-the-loop works.). If the LLM gets the recommendation wrong, the user can correct refine it and ask a better prompt or provide more context. The user will be tolerant of errors because they view this as exploratory. Overall Score = 12: This is a strong AI candidate based on the overall fit score falling between 11 and 15. AI is a great solution for this type of user exploratory problem where users want to talk with the AI to get better ideas on how to position themselves for certain job roles.

@Peculiar Ediomo-Abasi I see data readiness and error tolerance as your two biggest challenges. I'm guessing error tolerance would be the biggest blocker because errors could be expensive for both hospitals and patients. This is going to be an interesting area to explore with hypothesis testing, e.g., will stakeholders handle the potential errors for upside benefits or do they fall back to old solutions?

@Peculiar Ediomo-Abasi RAG can enhance this solution. I image hospitals keep a lot of information about patients, medications, subscriptions, etc. I can see a scenario where you might look at this information to help you improve your forecasts and predictions on medicine supply.

@Akshun Gulati This is a great problem to solve. I see many places where a solution like this would be impactful. Bad invoices and purchase orders information can lead to expensive problems for companies. One of the big blockers you would need to overcome is the level of accuracy of the results, (e.g., many companies have penalties associated with missing due dates so you won't want to mis-read a due date).

@Masahiro Teramoto, this is an interesting problem to solve. I often ask myself whether my learnings are translating into meaningful growth. It's hard to see sometimes especially if the improvement is gradual over time. A system that provides a set of reference criterias and feedback over time would help me to have someway of judging my progress. One of the challenges is the system would need to find creative ways to help me stay encouraged, e.g., think Duolingo versus traditional language learning programs.

@Peculiar Ediomo-Abasi I see a similar problem in a number of countries. AI has made good progress in predictive analytics within big industries and companies. Largely due to those places having lots of good data. The challenge I see we would need to overcome in health care is getting the data and ensuring continuous discipline in feedback loops(more good data) into the system. It would also take time to get this system to be good enough. People would need to have the right expectations about what the system can do in the short versus mid term.

@Peculiar Ediomo-Abasi, Netflix recommendation is a great example. They have one of the best recommendation systems that I have seen. It's also amazing to see how they do this when they have a family with different users. For example, they don't confuse my recommendations with those for my son. Other systems miss this idea of recommendations based on the specific user from a given family. Netflix has mastered this idea of separating users and providing specific recommendations based on early signals.

@Manasa Shetty, Grammarly is an great example. It's amazing that they can handle use cases for students, professionals, and writers. These groups have a wide variety in context, tone, and writing style. Normally traditional PM would say let's serve these groups through different products, especially the students versus professionals. But Grammarly has found an effective way through AI to meet this set of diversify needs.