Activity
Mon
Wed
Fri
Sun
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
What is this?
Less
More
Memberships
Learn Microsoft Fabric
Public • 1.5k • Free
70 contributions to Learn Microsoft Fabric
1d ago
in
becoming 'that new guy' who uses extensive chatgpt to learn
Not sure if anyone can relate but here goes: I've started using chatgpt extensively to get up the learning curve on spark & python specifically (coming from a reasonable sql base). What started off has mere curiosity morphed into full blown multi-hour gpt sessions querying and re-querying code with heavy in-line formatting to help me understand what does what within a library or specific transformation. It's been great, but I've started to pick up a bit of scorn from the more 'traditional' data engineers in our business. I don't actually mind too much, but this category of assisted learning is quite incredible, and strange to think it didn't exist even 6-months ago. It makes me wonder about the future of this space given how fast things are moving 😅
4
5
New comment 8h ago
1 like • 8h
@Santosh Pothnak In my experience, I found GPT being inaccurate a lot of times specifically on PySpark and it also hallucinates a lot. Well, for understanding or writing small snippets of code it works well, can't do much of heavy lifting accurately.
1 like • 8h
Hey All, One more suggestion if moving from SQL to PySpark (I used it personally when I was moving from PySpark to SQL) is, you can do this exercise of translating a code from SQL to PySpark or vice versa, as almost every PySpark code can be written in SQL (SparkSQL). These links can help you with some examples: https://towardsdatascience.com/sql-to-pyspark-d7966e3c15b3 https://www.linkedin.com/posts/mrabhijitsahoo_sql-pyspark-equivalent-activity-7031479783263715328-ijVm?utm_source=share&utm_medium=member_desktop
17h ago
in
Fabric-to-be
Hello from another Fabric-to-be professional 🤩 Just got my DP-600 and after the summer holidays, we are starting a big project at work of migrating from on-prem to Fabric. Gonna be some fun months/years 😄 Thank you Will for all the great content!!! Keep it up👏
6
2
New comment 5h ago
0 likes • 8h
Congratulations @Petko Ts 🎉
19h ago
in
Data Workflows (Apache Airflow integration into Fabric) 👀👀👀
This could be quite a fun week of announcements - including this one - a new Data Factory item for creating Airflow DAGs. What's Airflow? It's a python-based orchestration framework (similar-ish to data pipelines, but python-based and open-source). Any Airflow wizards out there??
11
9
New comment 58m ago
0 likes • 9h
Amazing!! They named it "Data workflow", seems like this would be MS version of Airflow and not the regular one. Would be interesting to see as it comes up
4d ago
in
Help with notebook reference.
Hello Skool, I need a little help. I'm not sure if it's possible to do what I'm thinking. Is it possible to run a cell from another notebook? I know that I can run a whole notebook from another notebook, using the command %run 'Name of the notebook' But is it possible to run only a cell? and how should I insert the reference in the "source" cell? I've been looking for this on these links: Introduction to Microsoft Spark utilities - Azure Synapse Analytics | Microsoft Learn How to use notebooks - Microsoft Fabric | Microsoft Learn Thanks.
0
5
New comment 9h ago
2 likes • 3d
Hi @Luiz Fernando , I think its not possible. Well, i could think of these workarounds. Let say you want to run code of cell1 from notebook1 in notebook2. 1. You can create notebook3 with code of cell1. Call notebook3 in notebook1 and notebook2. 2. You can create function1 with code of cell1 in notebook1. In notebook2, %run notebook1, and call for the function1 in notebook2.
0 likes • 9h
@Luiz Fernando no, we %run the notebook first, so the function gets defined in the notebook2. Then just call the function()
1d ago
in
Real-time Hub
Hey Guys! Just found this new tab in Fabric. It's called "Real-time hub". It looks like a really interesting place to kind of manage all the streaming data sources for Fabric [Image attached]. It has segregated all sources into 3 further sections, you can see what each of them is for in the image. Also, one more thing I noticed is Fabric events has got integration with Data activator, and we can set alerts on Fabric events, and it provides us 3 options for action. [Image attached]
8
11
New comment 22h ago
0 likes • 1d
@Will Needham Yeah, that really opens up lot of new possibilities!!
0 likes • 22h
@Will Needham 😂😂
1-10 of 70
@vinayak-k-8946
Inquisitive Data Engineer | Aspiring Solution Architect & MS Fabric Expert
Active 10m ago
Joined Mar 15, 2024
powered by