Author: Scott McCormick

Near Real-Time Data Processing for BigQuery: Part Two

This post is part two of describing (near) real-time data processing for BigQuery. In this post, I will use Dataform to implement transforms as well as ASSERTS on the data and unit testing of BigQuery code and SQL statements. Part…

Read More >

Near Real-Time Data Processing for BigQuery: Part One

This post describes (near) real-time data processing for BigQuery with unique and other check constraints, and unit testing. This is part one of two, and describes the real-time ingestion of the data. Part two will describe how to implement ASSERTS…

Read More >

Google Cloud Composer Costs and Performance

Controlling Cloud Composer Costs and Performance Managing, optimizing and balancing cloud cost vs. performance is an ongoing challenge for all cloud architects and administrators. The variety and complexity of tools available can sometimes be daunting, so much so that many…

Read More >

Google Cloud (GC) Cloud SQL Disaster Recovery

This blog post describes how to enable roll-your-own Disaster Recovery in GCP Cloud SQL. This process is automated and will save money. However, recovery is manual. Introduction One of the benefits of using the cloud is the ability to track…

Read More >

Top talks to watch for at Google Next London

On October 10 & 11, 2018, Google Next London will be running in full force. Pythian will be attending with a booth and many of our top executives and technical leaders ready to discuss our cloud strategies and offerings. This…

Read More >

A look at GCP Helsinki Data Center’s opening event

Pythian was recently invited to attend the very successful opening of the new GCP Helsinki Data Center. Based on the number of attendees and their excitement level, Finland is very ready to jump on a local cloud provider with over…

Read More >

SQL Server and Azure soft delete

Microsoft just announced the public preview for Azure Soft Delete of Storage Blobs. From the documentation, “When turned on, soft delete enables you to save and recover your data when blobs or blob snapshots are deleted. This protection extends to…

Read More >

Azure total cost of ownership calculator

Microsoft has released a new Azure Total Cost Ownership (TCO) Calculator. This calculator allows you to quickly enter in your current on-premises workload and review the expected savings or costs of moving to Azure. Workloads You simply enter in your…

Read More >

Building data tests in PowerShell

While working with a client recently, we came across a problem while testing data for completeness or errors after running an ETL process to import & manipulate the data. The main issue we ran across was that the overall client…

Read More >

Sharding a SQL Server database

This blog post covers sharding a SQL Server database using Azure tools and PowerShell script snippets. Sharding, at its core, is breaking up a single, large database into multiple smaller, self-contained ones. This is usually done by companies that need…

Read More >
Page 1 of 3123