NOV 2022 - Modernize your Data wrangling approach for Datalakes by Dan King (@SQLSWAGG)

Abstract
The evolution of technology and the sheer amount of data increases the use of modern-day architectures. Carefully using the term modern-day architecture because the data lake architecture isn’t just for data warehousing and prescriptive analytics. It spans multiple modern-day use cases and architectural designs. Data wrangling and choosing your tools effectively continue to change as well. Think about Azure data factory v1 and how it evolved into ADF V2. Look at some of the additional data movement components and options. Databricks and notebooks modernized data movement and ACID transactions with delta lake. We have witnessed the next shift with Power Query and data wrangling within your ADF pipelines. In this session, we will look at some of these services and open mic and discuss the pros and cons of each. I will demo different data movement techniques and provide a forum to discuss experiences.

Speaker Bio
Dan King is currently a Solutions Architect Competency lead at 3Cloud Solutions. His daily activities are not only working with customers but with a primary focus on repeatable processes and standardizing go-to-market analytics architectural design to fit the needs of different industries. From a Microsoft mindset, you can relate it to common data models with common ETL/ELT processes.
His experience spans over a decade that started from networking and telecommunication to being a SQL database administrator with a final landing zone in data analytics. Today is his core focus and goal. With expertise in analyzing data, data movement, ETL design, data modeling, and visualization, in his own words, he has been honored to see how organizations take data from various sources through ingestion as it flows to storage and eventually lands in the presentation layer.
Dan holds multiple Microsoft certifications and continues to progress with the changes in technology. He loves to work out, and do outdoor sports, and finds tremendous value in training his peers in areas in which he holds knowledge in. Family is a core value, and he loves family. He views the SQL community as his extended family.

Slide deck and code
TBD