ETL – EXTRACT, TRANSFORM, LOAD

Here we have a perfect example of an ETL in action:

One of the robot fetches daily data from Yahoo Finance about stock prices via Python (hosted at wayscript.com)

Store it into SQL Server (hosted at AWS)

Presenting it via Power BI (hosted at Office 365)

The result is here: Stock Prices

FETCHING DATA FROM YAHOO FINANCE

VERSION 1

FETCHING DATA WITH BEAUTIFULSOUP AND SAVING INTO SQL SERVER

VERSION 2

Less code

VERSION 3 MUCH SIMPLER

Fetching the last stored date from SQL Server and comparing with last available date from Yahoo

VERSION 4 FETCHING COUNTRY

With a new column “Country” we (me) now have to fetch for the existing country, if not for now it exits

Current Version