The release of Perplexity, OpenAI and Google's Deep Research capability (discussed last week in data pro.news), is proving to be significant productivity enhancement for Data Professionals. To give you a sense of its power I set it to work to troll reddit to see: What are the top 10 most discussed Data Engineering Topics in Online Communities? And the winner is... 1. ETL/ELT Processes and Modern Tooling The shift from traditional ETL to ELT frameworks, enabled by tools like dbt and Apache Airflow, remains a cornerstone of data engineering discussions. Communities emphasize the importance of modular pipeline design, idempotency, and partitioning strategies to handle large-scale data transformations. The rise of dbt as a transformation layer for SQL-centric workflows has sparked debates about its limitations in complex orchestration scenarios. Meanwhile, Apache Spark continues to dominate batch and stream processing use cases, with PySpark adoption outpacing Scala. The full list of the top 10 is published here https://www.perplexity.ai/page/top-10-most-frequently-discuss-vwndmAbdQVC_ANozGkGP9A

Retrieval-Augmented Generation (RAG): The Definitive Guide [2025] https://www.chitika.com/retrieval-augmented-generation-rag-the-definitive-guide-2025/ Utilising RAG in Enterprise AI deployments is moving at a rapid clip. In this week's edition of www.datapro.news we take a look at the latest developments in the field.

https://blog.google/technology/google-labs/whisk/ (image generated with Whisk) FYI, to generate the image I uploaded my profile pic, the skool logo and a "high tech lab" image i found on google along with a super simple prompt "subject working on an AI project for Skool". Most likely could produce better results with a more descriptive prompt.

Check out DSharp Studio @Konsta Weber our latest sponsor making a splash with their no-code/low-code approach to developing data warehouses in this week's DataPro.news Stay tuned to The Data Radio Shows for an interview with @Kim Johnsson on the approach. https://www.datapro.news/p/improving-your-data-models-with-ai

Top amongst the challenges for Data Engineers in 2025 will be the improving Data Quality. In this weeks newsletter we explore how the rise of near-infinite context windows for LLM's will be a major driver to improve Data Quality in the Enterprise https://www.datapro.news/p/why-data-quality-will-matter-even-more-in-2025