Omar Aref

Introduction: In this article I will be using Selenium to extract data from http://www.oecdbetterlifeindex.org I will be getting data for each country regarding: Population Visitors per year Renewable Energy Indices for things like: Housing Income Jobs Community  Education Environment Civic Engagement Health  Life Satisfaction Safety Work-life Balance    Importing Libraries: from selenium import webdriver from selenium.webdriver.common.by import By import time Loading chrome driver: We need to use the chrome driver exe since I will be using google chrome to crawl our webpage (drivers for each web browser are available online and can be downloaded) browser = webdriver . Chrome(executable_path = '/chromedriver' ) Define our 1st function to: 1.         Access our link and load it according to the country we pass into the function 2.         Use xpath to look for 'td' tags which have our desired values (

Introduction: In this tutorial I will show you the easiest way to fetch tweets from twitter using twitter's API. We will fetch some tweets, store them into a DataFrame and do some interactive visualization to get some insights. In order to access twitter's API you need to have the below: Consumer API key Consumer API secret key Access token Access token secret It is a very simple 5 minutes process getting those. You just need to log into your twitter account and go to this link click create an app and just follow the steps. After you create an app and fill the app details, you can just go into your app details and go to the keys & tokens tab and click generate and you'll have your keys & tokens. Getting the tweets: We start by storing our keys & tokens into variables access_token = 'paste your token here' access_token_secret = 'paste your token secret here' consumer_key = 'paste your consumer key here' consume