AEG Presents is a seasoned veteran when it comes to staging high-profile concerts with first-class artists and venues like Coachella and the New Orleans Jazz Festival. But the organization recently realized it had a problem getting the most out of its sales and marketing efforts, and that problem was centered around its data.
Even though AEG’s systems generated several different types of data including geographic, demographic, historical ticket sales, clickstream, and social media, the organization’s legacy data warehouse (an on-prem MS SQL Server and related integration services, along with Teradata) wasn’t capable of efficiently harnessing it all to scale analysis and improve ticket sales. AEG’s legacy on-prem warehouse presented several problems for analysts:
- The data model was hard to navigate: To glean basic information about certain demographics or shows, they often had to join ten or more data tables together – a time-consuming and frustrating exercise.
- Making changes was difficult: New tables or data sources required wholesale changes to the entire upstream process.
- It couldn’t handle semi-structured (but vital) data: Real-time streaming data was difficult to integrate without time-consuming processes.
- It was slow and expensive: The system’s rapidly declining performance couldn’t justify the ongoing hardware and support costs it was incurring.
Because AEG Presents inevitably collects a notable amount of sensitive personal data, data privacy was also a major concern. All PII (personally identifiable information) needed encryption, masking, or anonymization.
So the organization turned to Pythian to help migrate its legacy data warehouse to the cloud.
In our latest 2-part on-demand webinar, Danil Zburivsky, Pythian’s Director of Engineering, explains the steps AEG took to achieve better insights and customer engagement. Here’s a brief summary of how they did it.
Step 1: Pick your platform
AEG’s first order of business was choosing a cloud platform between Amazon Web Services, Microsoft Azure and Google Cloud platform. All three offer compelling services and have invested significantly in data management, processing, and housing.
But when choosing a cloud platform, it often also comes down to strategic considerations such as vendor relationship (and if you can get a deal), along with which platform is the most natural fit based on the technologies you currently use. Since AEG Presents already had experience with MS products, it made the most sense to move to Azure.
Step 2: Data lake or data warehouse… or both?
AEG’s next big decision was whether to use a data lake or data warehouse. Both have significant advantages and disadvantages: While data warehouses are typically well organized, relational, and support SQL, data lakes easily support new (streaming) data types and allow administrators to make changes to the system quickly. Unlike warehouses, data lakes also allow data collection from multiple, differently formatted sources to a single repository.
Pythian’s advice was to use both because bolting a data lake onto a data warehouse allows organizations to take advantage of the combined benefits. By combining both, AEG can use its data lake’s flexibility to bring in any data type, move complex and computationally expensive data transformations outside the data warehouse, and keep multiple representations of data in the lake for different needs. It can also take advantage of fast and slow storage for both its real-time and permanent data.
Step 3: Implementation
Pythian worked with AEG to implement a cloud-native data platform with an Azure backbone. It allows inputs from third-party SaaS sources like Salesforce, Google Analytics, social media, and Microsoft Dynamics along with customer on-prem relational databases, JSON-based API systems, and file-based sources.
AEG’s new data platform provides the flexibility to do more with its data, including:
- Collection, integration, storage and analysis of Clickstream and other semi-structured sources
- Faster query response time, allowing for real-time analysis and decisions
- A data model that makes sense to analysts, not just IT administrators
- Ad-hoc data exploration for data scientists without worrying about crashing the data warehouse
- The ability to export large datasets for marketing activation use cases
This allows AEG to tailor certain shows and artists to specific venues, geographies and demographics for even better ticket sales and improved customer experience.
Pythian helps organizations upgrade their legacy, on-prem data warehouses to cloud-based data platforms every day. Thinking about taking the leap? Give us a shout to schedule a quick discovery call and put Pythian expertise to work for you.
Or watch Danil Zburivsky’s webinar that explains the AEG case study in more detail.