Inside Data Engineering with Vu Trinh
Join Vu Trinh as he navigates the world of data engineering, sharing insights, challenges, and emerging industry trends.
Continuing the series ‘Inside Data Engineering’ with the second article with
, who is a Data Engineer working in mobile gaming industry. He shares his knowledge at VuTrinh. If you missed the first article from the series then checkout here.To recap: the series follows a Q&A format, featuring professionals who share their journeys, insights, and challenges.
What to Expect:
Real-world insights – Learn what data engineers actually do on a daily basis.
Industry trends – Stay updated on evolving technologies and best practices.
Challenges – Discover what real-world challenges engineers face.
Common misconceptions – Debunk myths about data engineering and clarify its role.
⭐ If you're curious about data engineering or considering it as a career, this series is for you!
Let’s dive into Inside Data Engineering:
How would you describe Data Engineering?
To me, data engineering is all about designing, building, and maintaining the foundation that enables efficient data storage, retrieval, and processing, ensuring that data is organized, and ready for analysis.
How did you end up being a Data Engineer?
Honestly, it’s a bit of an unexpected journey.
Back in 2019, I graduated in Electrical and Telecommunications but quickly realized I had no interest in working in that field. So, like many fresh grads, I went online searching for high-paying jobs, and “Data X” (whether scientist, engineer, or analyst) sounded cool.
Then, a company offered me a “data” job, even though they weren’t sure what the exact role was. (To this day, I’m not sure how I got that offer!)
But that job introduced me to Docker, Spark, HDFS, and Airflow, though mostly for building POCs. Eventually, the company shut down, forcing me to look for a new job.
That’s when I really took the time to research the differences between the “Data X” roles. And it finally clicked: I wanted to be a data engineer.
What's your day to day look like?
I start my day by checking Slack—just in case someone is complaining about a data bug. (Kidding… mostly!)
A typical day involves:
Coding data pipelines
Fixing bugs
Helping data analysts optimize queries
Lately, I’ve also been working on an internal data app focused on the semantic layer, which has been an interesting challenge.
What are some stakeholders that you work with?
Primary owners that I closely work with are Data Analysts and Project Owners.
What kind of data do you work with?
I work for a music game company with mobile games played globally.
Most of the data comes from:
In-app event tracking
Third-party services that track user acquisition
Revenue collection processes
This means handling large volumes of player interactions, engagement metrics, and monetization data to drive insights and optimize the game's performance.
What data size do you work with?
My day to day work involves a couple of TBs.
What tech stack do you use?
We primarily use BigQuery, DBT and Airflow. (I’m a fan of BigQuery)
What programming languages do you use?
We are a Python and SQL shop.
What tools do you leverage for GenAI?
I primarily use ChatGPT, but I also leverage Perplexity and NotebookLM when I need deeper research.
What is your favorite area of Data Engineering?
My favorite thing about Data Engineering is that you never run out of things to learn. The field is constantly evolving, with new tools, architectures, and best practices emerging all the time.
What advice would you give your past self as a beginner Data Engineer?
Spend time understanding the real value a data engineer provides. This awareness will give you clarity on your contributions and help you avoid roles that expect you to train ML models instead. It will also guide you in prioritizing which skills to learn and which ones to skip.
Next, I’d tell myself to build strong data engineering fundamentals at all costs, especially data modeling, even if it feels tedious at first. It pays off in the long run.
What are some challenging aspects of Data Engineering?
There are two challenges that I can talk about:
First, in Data Engineering there is the sheer amount of things to learn. That’s why understanding the true value a data engineer brings is crucial, it helps you focus on what’s worth mastering instead of getting lost in an endless tech stack.
Second, I observe is that many companies know they need a data team but have no clear direction on how to build or utilize it effectively.
What is the next big thing in Data Engineering?
I believe Data Lake Houses will continue to grow, with table formats and query engines becoming even more efficient. The adoption of object storage as the primary storage layer will keep expanding, driven by its scalability and cost-effectiveness.
What are some common misconceptions about data engineering?
A common misconception is that data engineering is always about big data. In reality, the core goal is to build a solid data foundation, ensuring data is stored, retrieved, and delivered efficiently for insights.
Sometimes, this means working with massive datasets, but often, you must work with "not-so-big" data.
Reach out if you like:
To be the guest and share your experiences & journey.
To provide feedback and suggestions on how we can improve the quality of questions.
To suggest guests for the future articles.
 | A guest post by
|
I’m a big fan of VuTrinh. I recently transitioned to a Data Engineer role, and your articles are making this transition easier!
Shoutout to you ! This is a very interesting discussion! I sincerely hope that one day I will be able to join in one of those!