Hello! Welcome to The Modern Data Community. The goal of this community is to help Data Engineers on small (or solo) teams confidently build modern architectures by simplifying key concepts, clarifying common strategies & learning from others. Pumped to have you here! ==================== HOW IT WORKS ==================== By joining, you get instant access to tons of free content (see Classroom). Dive right in. But even more can be unlocked by contributing to the Community, which I encourage you to do. It works like this: Contribute (post/comment) >> Get points (likes) >> Reach new levels >> Unlock content ==================== 6 SIMPLE GUIDELINES ==================== ❌ Do not post error messages looking for others to debug your code. That's why Stack Overflow and other tool-specific Slack channels exist. ❌ Do not use this community for self-promotion (unless Admin approved). We all know it when we see it. ❌ Do not create low-quality posts with poor grammar/spelling or that provide little (or no) value. These will be deleted. You can do better! ✅ Ask questions, share your experiences & overall be a good person. This not only helps everyone get better, but can help you unlock bonus content faster. Win-Win. ✅ Speaking of wins, share yours! Whether it's finally solving a complex problem, hitting a team milestone or starting a new gig - post about it. You'll get the props you deserve and it just might inspire somebody else. ✅ Take the time to craft thoughtful posts & always proof-read before hitting submit. We're all about quality here. High quality posts --> more engagement (aka you'll climb the leaderboard & unlock content) --> ensures the community stays enjoyable for everyone. ==================== QUICK LINKS ==================== Here are a few links to help you get going: - Classroom - What's Your Data Stack? - Leaderboard - Work with me (Kahan Data Solutions)

All of us are at different parts of the journey. What part of Data Engineering are you most focused on this year & why?

I tend to waste a lot of time over analyzing things - especially when it comes to new content. So rather than continue to do that, this time I figured I'd share a raw behind-the-scenes look at something new I'm working on directly with you. Your feedback can make sure it's on the right track and will help you overcome real challenges you're facing. (or let me know if it's off the mark) Rather than bore you with text, I've recorded a Loom video breaking it down. Heads up - this was a completely unplanned video and I ramble a bit. But hopefully the main points still come across. Any feedback at all on this would be incredibly valuable. Feel free to leave a comment here or DM me directly if you'd prefer to remain private. Thanks as always!

Over the past few years I've been fortunate to meet a lot of great engineers & work with a variety of companies. As a whole, most devs/teams aspire to create architectures that are easy to understand, fun/engaging to work on & dependable for analytics. Nothing surprising there. But actually getting to that point can be challenging. And the challenges you face can vary drastically depending on where on the journey you're at. Why am I bringing this up? As mentioned recently, I'm working on putting together something to better address these common challenges and help teams more effectively build what I call a "Simple Modern Stack". The goal is to show you how to stand up a foundational data stack (faster) while avoiding the traps of over-complicating by guiding you through 5 steps: 1. Design 2. Setup 3. Build  4. Automate 5. Maintain To help prepare, this morning I wrote out a bunch of common questions I've heard over the years and often address during consulting engagements. The goal is to then answer each of them over time in the form of more free videos, community posts and/or more hands-on paid offerings like mentioned above. But rather than debate with myself on what's important, below I've decided to share my list of common questions with you. Perhaps seeing it will stir up new ideas for you/your team, or at the very least make you feel better knowing others are in the same boat. After reading this, my questions to you are: - Are these in line with what you've experienced? - Is anything else missing? Would love to her your thoughts. With that said, here's my current list: ======= Common Questions By Step ======= Design - What high-level approach should I follow - What tools should I use - How much will it cost - Which cloud provider is best - What MDS components do I need - What about [this thing] I see on social media/articles - How does it all work together end-to-end - What data modeling approach is best - What if I need or already have a custom solution - How do I get buy-in from decision makers 

If you're like me, most of your day-to-day life in the data world is heads-down working. Maybe it's learning a new tool or skill. Maybe it's building your company's data architecture. Maybe it's working on stakeholder requests or fixing bugs. In the moment, we're so focused that it's hard to lift our heads up and see a wider view. To analyze what we've done or where we want to go next. If there's one good thing about the end of the year, is it gives a clear point in time to do just that. With that said, I'd like to open up this thread to give you an opportunity to share things you've done this year and what you hope to do next year. Maybe you'll find that somebody else has already walked the path that you're about to start. Or at the very least write something to look back at 12 months from now. Happy New Years!