Photo by Braňo on Unsplash

Over the past month, I have started investing in crypto. This decision lead me into a whole new world of technological innovation that has evolved too quickly that I could barely fathom the entirety of it. I initially just intended to purchase a couple of coins that have potential, HODL

Data engineering Data Quality

Photo by Mika Baumeister on Unsplash

I’m a data engineer currently working in the banking industry. I play a big part in delivering an essential project that focuses on risk assessments to protect the institution and its customers from financial crimes. It is particularly crucial to produce periodical risk reports that are accurate, complete, consistent, time-relevant…

Event-driven pipeline Infrastructure-as-Code Series

Photo by Tom Wilson on Unsplash

This is the last article in the 3-part blog series where I demonstrated how to build a serverless ingestion pipeline from S3 into Snowflake using Snowpipe and deployed the infrastructure using Terraform. The build is divided into 3 phases:

Event-driven pipeline Infrastructure-as-Code Series

Photo by Zoey Tian on Unsplash

In Part 1, we have produced the Terraform code to provision the S3 bucket (Snowflake external stage) and the event-driven pipeline.

  1. An S3 bucket, a role for Snowflake, and the policies required to give Snowflake access to the bucket.
  2. An event notification that triggers when new files arrive in the…

Karen Bajador Valencia

Data and cloud engineer. Also interested in blockchain and personal finance. Senior Consultant @ Servian.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store