Data and cloud engineer by trade. She mainly writes about Tech. Interested in Cloud technology and personal finance. Consultant @ Servian.

My journey in evaluating Great Expectations, a data quality framework – Part 1

Photo by Mika Baumeister on Unsplash

I’m a data engineer currently working in the banking industry. I play a big part in delivering an essential project that focuses on risk assessments to protect the institution and its customers from financial crimes. It is particularly crucial to produce periodical risk reports that are accurate, complete, consistent, time-relevant…

Provisioning Snowpipe using Terraform

Photo by Tom Wilson on Unsplash

This is the last article in the 3-part blog series where I demonstrated how to build a serverless ingestion pipeline from S3 into Snowflake using Snowpipe and deployed the infrastructure using Terraform. The build is divided into 3 phases:

Configuring storage integration in Snowflake to access Amazon S3 using Terraform

Photo by Zoey Tian on Unsplash

In Part 1, we have produced the Terraform code to provision the S3 bucket (Snowflake external stage) and the event-driven pipeline.

  1. An S3 bucket, a role for Snowflake, and the policies required to give Snowflake access to the bucket.
  2. An event notification that triggers when new files arrive in the…

Karen Bajador Valencia

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store