What makes a good data warehouse strategy?

Cole Lehman -

Your data warehouse strategy sets the foundation for your modern data infrastructure. If you get your strategy right, your whole company has access to better data and can make important decisions faster. If you get it wrong, your business might suffer long-term from poor data quality and muddled business intelligence.

When defining or revamping your organization’s data warehouse strategy, it can be tempting to pick a popular cloud data warehouse solution and run with it. But it’s worth taking the time to build your own data warehouse strategy before making a choice that impacts every part of your business for years to come.

A well-built modern data warehouse strategy will help your organization thrive in today’s increasingly tech-driven world.

In this guide, we’ll explore some of the most important questions you should ask yourself as you build your data warehouse strategy:

  • What is a data warehouse?
  • Why implement a data warehouse?
  • What are the key areas of a data warehouse?
  • What’s better—a cloud data warehouse or an on-site data warehouse?
  • What data tools do you need in order to unify data sources in your warehouse?

Let’s start with a short definition of a data warehouse.

What is a data warehouse?

A data warehouse is the central storage location for all of your data. It provides a single version of the truth and aggregates data from disparate sources, making it easier to analyze and access. You can choose between a cloud data warehouse solution or an on-site data warehouse—the right choice depends on your needs and budget.

Each data ingestion pipeline leads to your data warehouse. They unify all of your raw data sources through a central location for storage, transformation, and analysis. Your data warehouse is where data scientists, machine learning specialists, and business analysts perform their data modeling—thus creating advanced metrics like lifetime customer value from many different data sources.

Your data analytics team relies on a data warehouse for their day-to-day responsibilities. Your whole organization relies on it for business intelligence, product development capabilities (e.g. ecommerce search features), and customer experience quality.

Because your data warehouse impacts so many parts of your business, it’s important to come up with a clear strategy.

Why implement a data warehousing strategy?

Choosing the right data warehouse is often the first step in a larger effort to modernize data platforms and data infrastructure. Data warehouse implementations help businesses stay competitive by enabling them to extract value from their data. They can provide strategic insights that inform business decisions and make more accurate projections about future possibilities.

When you implement a data warehouse strategy, you gain all the benefits of a modern data warehouse for your business.

Here are some of the main benefits:

  • Consolidate datasets into a central location
  • Improve data quality
  • Faster data transformation and data integration
  • Scalability and flexibility
  • Better decision-making capabilities
  • Increased data analytics accuracy

Your data warehouse strategy expands and enables your existing data management efforts. Data management involves everything from modernizing your data stack to putting together the right data analytics team. It’s a combination of all the tools, processes, and people who make data useful to your business. Solid data warehouse implementation ensures optimal performance, reduced costs, and increased ROI in your data management initiatives.

Let’s take a look at the key areas needed in every data warehouse strategy to realize these benefits.

What are the key areas of a data warehouse strategy?

A good data warehouse strategy provides an organization with more than just the insights they need to make better decisions. It also creates space for future growth, next-generation data capabilities, and support for emergent technology.

Here are the main areas you need to cover in your data warehouse strategy:

  • Map your data sources and destinations
  • Outline your data stack: including ETL process, data pipeline tools, and data science pipeline
  • Define data lifecycle requirements
  • Determine actions you expect to take on the data
  • Sketch out dashboards you expect to build with the data
  • Build your data analytics team
  • Explore data warehouse solutions

Once you have these mapped out, you’ll know much more about how your data warehouse fits as the central hub of your data sources, data destinations, data tools, data pipelines, and data team. From here, define your current and future states. Then ask more detailed questions, like “How long do I need to keep my customer data from Salesforce?” and “What are the security requirements for my industry?”

Main questions to answer in your data warehouse strategy

The trick at this point is to ask enough detailed questions to ensure your data warehouse strategy functions without getting stuck in the weeds. For example, you need to know how long you’re going to retain your financial data but you don’t need to know each custom field associated with your financial data (yet).

Here’s a list of questions a modern data warehouse strategy should be able to answer:

  • What is the goal of your data warehouse? Do you want it for operational or analytical purposes? Are you looking for real-time insights or historical analytics? Both?
  • How much storage space will your warehouse need?
  • Do you have enough storage space available internally to keep all of the data you’ll collect over time, or should you consider storing some data off-site instead? If so, how much and which types of data should go where?
  • What data needs to be in hot storage vs cold storage?
  • What kind of data do you need for your business?
  • How much can you spend on hardware and software for your data warehouse?
  • How many human resources can you devote to support your data warehouse strategy?
  • Who will be accessing the data? How often will they access it?
  • How long should your data retention period be?
  • Will this be a short-term or long-term endeavor for your organization?

Talk with your stakeholders about their business requirements, different data sources, and existing datasets. They’ll answer questions to fill in your strategy and likely give you a few new questions you haven’t thought of.

And the biggest question of all: Do you need a cloud data warehouse or an on-site data warehouse?

The advantages and disadvantages vary depending on your business use cases. Sometimes, a hybrid cloud data warehouse is the answer. Here’s how you can choose between a cloud data warehouse and an on-site data warehouse.

Should you choose a cloud data warehouse or an on-site data warehouse?

Cloud data warehouses offer many advantages in terms of time, cost, and scalability. They don’t require nearly as many infrastructure resources to manage them. However, they don’t always meet the requirements for data governance.

An on-site data warehouse will be more expensive to set up and maintain, but it can be faster for data retrieval. On the other hand, a cloud warehouse is cheaper and requires less maintenance, but there might be a slight delay when retrieving data.

Ultimately, the choice is based on your specific business requirements. Here are some of the basic benefits of each solution:

Benefits of on-site data warehouse

  • Complete control over the tech stack and security
  • Local speed and performance
  • Governance and regulatory compliance

Benefits of cloud data warehouse

  • On-demand scalability
  • Cost efficiency
  • Bundled capabilities such as IAM and analytics
  • Security
  • System uptime and availability

It’s likely you’ll find all the performance, security, and capability you need with a cloud data warehouse. According to Yellowbrick’s Data Warehousing Survey in 2021, overall data warehouse performance (31%) and improved agility for faster innovation (29%) are the most common reasons people move to cloud-based systems.

Even if you think your company might benefit from an on-premises data warehouse, it’s worth exploring cloud solutions before you make a decision. In general, we've found that modern businesses greatly benefit from starting with cloud warehouses.

More and more companies are leaning toward cloud data warehouses to form the center of their strategy or supplement their on-premise data warehouses.

Here are some of our favorite solutions on the market:

Snowflake Data Cloud

Snowflake is a fully managed cloud data service that’s simple to use but powers a near-unlimited number of concurrent workloads. It’s your solution for data warehouses, data lakes, data engineering, data science, data application development, and securely sharing and consuming shared data.

Top features:

  • Governed data for all users
  • Bring development to data
  • Work global & cross-cloud
  • Create new revenue streams

Amazon Redshift

Amazon Redshift uses SQL to analyze structured and semi-structured data across data warehouses, operational databases, and data lakes. It uses AWS-designed hardware and machine learning to deliver the best price-performance at any scale.

Top features:

  • Easy analytics for everyone
  • Analyze all of your data
  • Performance at any scale
  • Secure and compliant

Yellowbrick Data Warehouse

Yellowbrick’s Data Warehouse is unlike others. It’s an elastic, cloud-native data warehouse that runs in your own cloud account and lets you pay with predictable dollars instead of credits. Migrate incrementally while paying for your own cloud or on-premises infrastructure. Get the same experience across data center, cloud, and edge deployments—and radically simplify migrations.

Top features:

  • Leverage a distributed data cloud
  • Faster analytics
  • No sticker shock/clear pricing

Google BigQuery

BigQuery is a serverless, cost-effective, and multi-cloud data warehouse designed to help you turn big data into valuable business insights. Power business decisions from data across clouds with a flexible, multi-cloud analytics solution. Democratize insights with built-in business intelligence and machine learning. BigQuery is at the core of Google's unified data cloud to help you drive data transformation.

Top features:

  • Gain insights with real-time and predictive analytics
  • Protect your data and operate with trust
  • Break data silos and prevent lock-in
  • Analyze and share data across clouds

Microsoft Azure Cloud Databases

Use Azure to simplify application development with fully managed databases. Choose among fully managed relational, NoSQL, and in-memory databases that span proprietary and open-source engines. Fully managed databases automate tasks like configuring and managing high availability, disaster recovery, backups, and data replication across regions, saving you time and money.

Top features:

  • Fully managed
  • Limitless scale
  • Open and flexible
  • Intelligent security

What’s next?

Ready for your business to receive useful insights, new revenue opportunities, and fewer data silos? Now you’ve got the info to start working on your own data warehouse strategy. Start with the key strategy areas above and then track down answers to your main questions. With that accomplished, you’ll have the framework you need to get your data warehouse strategy right.

Once you get a data warehouse in place, you’ll need data pipeline tools to bring data together from all of your sources. We built Shipyard’s data automation tools and integrations to work with your existing data warehouse strategy or modernize your legacy systems.

If you’re ready to ingest data into your data warehouse or action on it right from the source, sign up to demo the Shipyard app with our free Developer plan—no credit card required. Start to build data workflows in 10 minutes or less, automate them, and see if Shipyard fits your business needs.