Your data warehouse modernization strategy can make or break your ability to derive value from traditional data sources (for example, your operations and financial data) as well as emerging data sources such as IoT data from devices and sensors, or social media data. So, you need to plan carefully, consider your needs, and align them with the right technologies.
What is a modern data warehouse?
At Pythian we define a modern data warehouse as one that extends the functionality of a traditional data warehouse to a system that includes a data lake, built-in ETL (extract, transform, load), and support for advanced analytics and machine learning. It’s not one thing. It’s composed of several carefully selected technologies that work together to ingest, store, process and present data.
The result is a platform that is scalable, has the flexibility to support different data types, and enables analytics in addition to reporting.
Options for modernizing
There are different strategies that you can apply in a data warehouse modernization project, from updating the current systems, completely replacing it, or expanding the existing data warehouse capabilities with complementary technologies.
While you can modernize on-premises using technologies like Oracle Exadata, many organizations are looking to highly scalable open-source options like the Hadoop framework, or to flexible and cost-effective cloud platforms, to extend or replace their data warehouses.
The ability to access structured, semi-structured and unstructured data on-demand and to integrate a range of both internal and external data are easily accommodated by both Hadoop and cloud platforms. But the key issue with Hadoop is that it’s open source and requires you to build your data warehouse from the ground up. So while you save on hardware, engineering your solution can be costly. So it’s best to use Hadoop at a scale that justifies the engineering effort and expense.
Benefits of Choosing the Cloud
If your organization plans to keep all its critical data on-premise, it may be in the minority. Constellation Research estimates that by 2020, 60 percent of mission-critical data will reside outside a business’ walls.
Moving to the cloud also provides a level of elasticity not available with on-premises solutions.
Worried about security? Don’t be.
Many companies, particularly those in traditional industries such as finance and insurance, have concerns about the security of data on the cloud, particularly PII (personally identifiable information). But gone are the days when a company’s data was all contained within its firewall. And if you think that statement isn’t true of your organization, think again. If you have company social media accounts or use SaaS for things such as CRM or marketing automation, you already have external data in the cloud.
The dollars and sense of moving to the cloud
Many organizations are abandoning the approach of purchasing large, expensive, proprietary appliances that are capitalized over several years. Cloud-based technologies that separate storage and compute are cheaper to operate, and they scale faster and more efficiently than the alternatives. In addition, cloud-based technologies promote innovation at a speed we’ve never seen before.
With a cloud-based data warehouse, IT personnel can better support business objectives by enabling:
- bigger and better functionality
- wider scope
- improved user service
- better business value
Modernizing on the cloud reduces the time-to-answer and the cost-per-answer, supports an increasing number of data types, integrates enterprise data with external digital data, and supports self-service analytics.
In addition, cloud platforms provide automation features to speed the integration of large data sets. And by enabling more users to access more data through self-service, cloud can help businesses provide in-depth insights, enabling unique observations for business users with varying technical abilities across the organization. Empowering users with data can have a positive impact on your ability to compete, on the customer experience, and on employee performance and satisfaction.
Ways to move to the cloud
A data warehouse move to the cloud can take one of two directions: hosting on an IaaS platform or using a more fully featured DBaaS solution. The major providers—Amazon Web Services (AWS), Google Cloud Platform (GCP) and Microsoft Azure—offer solutions to manage everything from small data warehousing needs to very large workloads. These data warehouses limit the burden of administration to the organization while making capacity scaling easy.
If you’re considering modernizing your data warehouse environment for analytics, download this eBook to determine if your existing warehouse is up to the challenge:
No comments