Archive - Junaid Effendi | Sharing knowledge for Engineers

Transition: Data Analyst to Data Engineer

Interested in working in the data engineering landscape, check out this natural transition for Data Analyst.

Nov 9 •

October 2024

Terraform vs Asset Bundles for Databricks Workflows

Sharing the experience of moving from Terraform to Asset Bundle for Databricks workflow deployment, covering challenges and benefits.

Oct 28 •

LinkedIn Data Tech Stack

Learn how LinkedIn handle trillions of events per day from billion customers using Apache Beam and more.

Oct 16 •

Transition: Data Scientist to Data Engineer

Interested in working in the data engineering landscape, check out this natural transition for Data Scientist.

Oct 5 •

September 2024

Challenges: From Databricks to Open Source Spark & Delta

Sharing the challenges to save hours when doing migration from Databricks to Open Source.

Sep 25 •

How to avoid pipeline backfill nightmares

Tired of the backfill process? Checkout these best practices to help you ease the pain of backfilling the data.

Published on DataEngineer.io Newsletter • Sep 17

August 2024

July 2024

Types of Data Engineers

Learn different types of focus areas for a Data Engineer in the modern industry with future articles covering the specific roadmaps.

Jul 27 •

Practical Data Engineering using AWS Cloud Technologies

Learn how to build end-to-end data engineering projects using pure AWS cloud technologies like S3, SNS, Lambda, Step Function, and more.

Published on VuTrinh. • Jul 24

Data Modelling Using Complex Data Types

Complex data types like struct, array, map in modern warehouses are game changer, learn the useful aspects from a Data Engineer.

Jul 6 •

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts