Automating the IMPORTANT stuff with Python: Creating a #FreeBritney Vaccination Twitter Bot
Trawling through job descriptions for several hours a day, you soon get a feel for the current trends in programming. The popularity of Ruby has seemingly reached a plateau, with the demand for Python programmers as much as five times greater than for Rubyists. With all the time in the world, I had no excuse but to try my hand at learning it!
I was very surprised to discover that I had already “purchased” arguably the best Python beginners course out there: Al Sweigart’s “Automate the Boring Stuff with Python”. Who would have thought that stockpiling flash-sale Udemy courses would actually be fruitful? Not only was this course a great refresher for programming fundamentals, but it had me writing Python in no time. I was very happy to learn that Python is syntactically very similar to Ruby, but as with all online tutorials, I didn’t want to fall into the trap of entering “tutorial hell”. As I am very much a person who learns by doing, I decided to jump straight into a personal project.
I thought it might be fun to create a Twitter bot to tweet the UK vaccination rates once a day. This way I could utilise my newfound web scraping skills to obtain live information on the number of people who have received their first COVID vaccinations from the government website, and write a short Python script to tweet this every 24 hours. Simple enough.
But wait! This original concept didn’t sit well with me. Having recently watched the Framing Britney documentary, I knew that it was my moral duty to improve awareness of the #freebritney campaign wherever possible. Combining both ideas into one project, I set out to create a Britney Spears-themed vaccination bot — naturally!
Applying for a Twitter developer account allows access to their API, permitting users to programmatically engage with Twitter’s features. This seemingly easy task actually took a lot longer than I anticipated. I had a back and forth dialogue with Twitter that lasted about a week before I was approved. Having to describe in detail the intention of my project multiple times felt flat-out insulting! I don’t think Twitter fully appreciates the importance of spreading awareness about Britney Spears’ conservatorship! But fortunately, I was eventually granted my API keys and I could begin constructing my bot. I set up a new python project and stored the API keys as environment variables in a file called credentials.py. I also imported the Tweepy library, and made a test tweet to make sure that the API keys were working as intended:
The UK Government Coronavirus webpage contains live information regarding vaccination rates. Initially, I tried to scrape the information using the lxml library technique outlined in this digital ocean tutorial Yet since the information on this website is dynamically updated with JavaScript, I ended up having to use selenium to scrape the data. Selenium is a Python library that can be used to automate tasks in web browsers. I downloaded the chromium webdriver to facilitate the automation since other browsers are not supported by Heroku, which is where I intended to deploy my bot. “Inspecting” the UK gov website using google chrome’s dev tools allowed me to locate the specific XPath of the vaccination rates. I was then able to use the find_elements_by_xpath method to glean the relevant information. Why didn’t I use the government’s API, you might ask?! Good question! It definitely wasn’t because I hadn’t thought about it. However, it was definitely because I wanted this to be an exercise in web scraping.
I created a sample method to randomly select a youtube video of one of Britney’s greatest hits. The intention being that every time the bot tweeted, it would share a different video to keep the tweets varied. To achieve this I created a list of youtube URL strings and used the random.choice method to select one at random.
I created a separate method to construct the tweet, containing freshly scraped information and the random video.
To ensure that the bot tweets every 24 hours, I created a simple while loop, and set a sleep interval of 86400 seconds.
And in practice…
In terms of hosting my bot on Heroku, this video was an absolute life-saver. Essentially it involved adding the google chrome and chromedriver buildpacks to my project, and ensuring that the webdriver was set up as headless. All I needed to do was enable my bot.py file in the Heroku Dynos, and the app was up and running! Posting once a day… forever and ever
Please do check out the Britney Bot for yourselves, or take a look at the source code on my GitHub! 💞
