Everyone's Data Infrastructure Is A Mess - The Truth About Working As A Data Engineer

Опубликовано: 04 Октябрь 2024
на канале: Seattle Data Guy

7,728

219

Is everyone’s data a mess?

Recently, I came across a post in the data engineering subreddit that asked the question.

The answer is yes, but no.

As someone who has seen data infrastructure at FAANGs, Enterprises, start-ups, and every other company in between, all companies need to make some concessions that can build up and become messy over a long period of time.

So let’s discuss some of the causes of data infrastructure becoming messy and how some companies are trying to deal with it.

Also, I forgot to cover a very important topic!

That is all of the mess often starts at the data source.

You can read the fuller version of this topic here

https://seattledataguy.substack.com/p...

If you need consulting help, set up some time with me here -
https://calendly.com/seattledataguy/3...

If you enjoyed this video, check out some of my other top videos.

Top Courses To Become A Data Engineer In 2022
   • Top Courses To Become A Data Engineer...

What Is The Modern Data Stack - Intro To Data Infrastructure Part 1
   • What Is The Modern Data Stack - Intro...

If you would like to learn more about data engineering, then check out Googles GCP certificate
https://bit.ly/3NQVn7V

If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.

https://seattledataguy.substack.com/

Or check out my blog
https://www.theseattledataguy.com/

And if you want to support the channel, then you can become a paid member of my newsletter
https://seattledataguy.substack.com/s...

Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio

_____________________________________________________________
Subscribe:    / @seattledataguy
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.

*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.