Let's build a Service Oriented Data Pipeline
Author: Yasha Podeswa
Abstract:
Like many software projects, data pipelines built by Business Intelligence teams often start out as quickly built monoliths, but over time they can be made simpler and more maintainable by splitting them into a series of tightly defined services. In this talk we'll look at the challenges we've faced scaling Data Analytics at Hootsuite, then move into a live coding session, where we'll stitch together a data pipeline as a series of Scala apps, deployed to AWS Lambda, connected using Airbnb's open source Airflow tool.
ACM DL: http://dl.acm.org/citation.cfm?id=295...
DOI: http://dx.doi.org/10.1145/2959689.295...
