Using environment variables saved in a .env file

dotenv is a package that finds and reads a .env file and then creates environment variables from the contents.


In [1]:
import os
from dotenv import load_dotenv, find_dotenv

# find .env automagically by walking up directories until it's found
dotenv_path = find_dotenv()

# load up the entries as environment variables
load_dotenv(dotenv_path)


Out[1]:
True

Once the .env file has been loaded you can use the environment variables and the dotenv_path


In [2]:
# Get the project folders that we are interested in
PROJECT_DIR = os.path.dirname(dotenv_path)
EXTERNAL_DATA_DIR = PROJECT_DIR + os.environ.get("EXTERNAL_DATA_DIR")

# Get the base URL of the data
BASE_URL = os.environ.get("BASE_URL")

# Get the list of filenames
files=os.environ.get("FILES").split()

print("Project directory is  : {0}".format(PROJECT_DIR))
print("External directory is : {0}".format(EXTERNAL_DATA_DIR))
print("Source data URL is    : {0}".format(BASE_URL))
print("Base names of files   : {0}".format(" ".join(files)))


Project directory is  : /home/gsentveld/lunch_and_learn
External directory is : /home/gsentveld/lunch_and_learn/data/external
Source data URL is    : https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/
Base names of files   : fmlydisb funcdisb familyxx househld injpoiep personsx samadult samchild paradata cancerxx

Getting the files is done with urllib.request


In [4]:
import urllib.request

for file in files:
    
    # transform the base filename into a URL and a target filename
    url=BASE_URL + file + '.zip'
    targetfile=EXTERNAL_DATA_DIR + '/' + file + '.zip'

    # By specifying the end=" ... " parameter to the print function, you surpress the newline.
    # You can also specify end="" to have subsequent print statements connect
    print('Get {0}'.format(url))
    
    urllib.request.urlretrieve(url, targetfile)

print('Done.')


Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/fmlydisb.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/funcdisb.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/familyxx.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/househld.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/injpoiep.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/personsx.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/samadult.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/samchild.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/paradata.zip
Get https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHIS/2015/cancerxx.zip
Done.