Techniques for tumbles, fills, and linear interpolation — (Update 12/01/2020: The user defined functions in this article have been added to the BigQuery community UDFs and slightly reworked. As an example, the tumble function can be invoked with bqutil.fn.ts_tumble()) As BigQuery is increasingly used as a store for real time analytic data such as web events and IoT…

The BigQuery data warehouse has extensive flexibility in how an administrator sets up access control and column level security. …

Reliably adding and removing short term slot capacity on a schedule — Following up on an earlier post about creating and assigning Flex Slots commitments through the client libraries. Users already taking advantage of flat rate pricing often ask how they can automate Flex Slots purchases for short-term capacity increases. For organizations making use of flat rate pricing, slot scheduling is a…

Wrapping faker.js with a Javascript UDF — (Update 11/20/2020: Examples now use a generated a sequence table instead of using the wise_all_sky public dataset as a base table) When creating demos and proofs of concept, I frequently run into the problem of finding suitable sample data to get started. …

Using integer range partitions with BigQuery scripting — I recently worked on a BigQuery use case, which required a very large, skewed, many-to-many join resulting in a fan-out of over 180 Billion records. On-Demand projects, limited to 2000 slots, would frequently time-out before the query could be completed. …

(Update 5/13/2020: Code now uses the Reservations client libraries) Google Cloud recently added Flex Slots as a new pricing option for BigQuery. Users on Flat Rate commitments no longer pay for queries by bytes scanned and instead pay for reserved compute resources; and using Flex Slots commitments, users can now…