Mary Dickson Diaz

Code, Life, Learning

build a twitter bot with python

This year I set out to learn how to make twitter bots, accounts that automatically interact with twitter in some way, whether through tweeting set content at regular intervals, watching for and responding to code words, or responding in a certain way when tweeted at. I learned the basics of Twitter API interaction at UW’s Community Data Science Workshop last November, and have since been inspired by other people doing cool stuff with bots.

Witness:

For more great examples, check out 52bot project.

A few nights ago, as I was reading about the reunion of the women of Bletchley Park, I clicked through to the Honour Roll and found a database that looked like it might translate well into a twitterbot. Using a subset of data and some internet tutorials, a test bot took maybe an hour or two tops to set up (and I was so thrilled when it worked!). The time-consuming part has been 1) finding a way to reliably keep it tweeting; and 2) pulling and cleaning up the full dataset to fit twitter’s 140 character limit (in my test run, I just told twitter “print the first 140 characters and ignore the rest” but for the full deployment, I wanted to edit the longer bios to make them fit).

Ok, so without further ado, here’s the bot:

And here’s how I made it!:

STEP 1: Get you some data The first thing I did was to sort by gender = female (I am interested in the oft-neglected lady coders of Bletchley). From there, I just cut and pasted the results (about 7,000 records in all) into an Excel spreadsheet. I then saved it as a comma separated values (.csv) file so that Python can read it.

*(Optional) Advanced data clean-up! In my second round, I used the excel formula “concatenate” to combine first name, last name, and service summary for each record and then “length” to show me which ones exceeded the 140 character limit. About 1/6th of my records were too long. Shortening “Bletchley Park” to “BP” (for those records only) took care of another 30%. This left me about 600 records to go through and try to get rid of characters, from abbreviating months and years to making tough choices about which text to cut out. Argh. Painful. Especially since these are real women and I feel a responsibility to accurately represent their service, or else I might have just said “first 140 characters and we’re done here.”

*(Optional) Advanced data collection: I know there is probably a better way to access that database than copy, paste (which took forever), but I don’t know how to do it yet. Soon!  

STEP 2 (python optional): Use Python or some other means to clean up your .csv data into a text file (.txt). Each tweet should be on a new line, no longer than 140 characters. Spoiler: I ultimately wound up just copying my 140 character sentences from the .csv to a text file. For this project, I could do that. BUT the better thing to do is to write or find a Python program that will collect dictionary information from a .csv and sort/export it into the format you want and the kind of file you want. So now I have a text file, that looks something like this:

Audrey M Abbot (Weston) Bletchley Park, Wavendon House and Stanmore mid 1942 - 1945. Bombe operator.
Hilda Abbott Bletchley Park. Block E, Communications Centre, probably Morse Slip Reader.
Barbara Abernethy (Eachus) Bletchley Park 1939 - 1945. Mansion, Naval Section. PA to Head of GCCS, Cdr Denniston from c 1941. 
...

It’s ok if your lines wrap over, but each tweet has to start on a new line.

Step 3: Make a twitter account, get developer keys

I followed the directions in this great tutorial to get everything working. Where they talk about the file “helloworld.txt”? That’s where I used my Bletchley bios instead. Scooch on over there and set yours up.

Step 4: You should now have a dedicated twitter account, text file with tweets, and a python file with your twitter credentials filled in.

Again I used this tutorial and the template for “helloworld.py” which worked like a charm. You can adjust how frequently you’d like your bot to tweet. I set mine for every 30 minutes, or a wait time of 1800 (15 minutes is 900). I saved it as “bletchley.py” for easy reference and made sure it was saved in the same folder as my text file, on my desktop.

Step 5: Run it!

From the command window (the c:\ black box, or type “cmd” into windows search), navigate to the folder where your files live, and type

python yourfilename.py yourtextfile.txt

This tells the computer: use this python file (tweet every 30 minutes) and apply it to this text file.

Step 6: Don’t touch anything and never turn your computer off

As long as the console window is on and open, everything appears to be working. But I want to tweet 7,000 records+ and at a rate of every 30 minutes, that will take my bot about half a year. I might want to restart my laptop at some point before then.

I am also running into the problem where if I DO need to restart the program, it won’t run because Twitter is like “dude, you already tweeted that.” And I’m like, yeah, I know, skip over those and just go to the next. And it’s like: does not compute.

Step 6: Cloud?? Cloud!

So surely there must be a way to host my little bot somewhere where it will happily tweet away while I do other things, and where I can monitor and make adjustments via an error log. Early investigation shows that heroku is the way to go. I’ve reached out for some help on this after getting 3/4ths of the way through the heroku python tutorial and thinking “this is way too complicated. I just want to upload my two files and then a command window to say ‘run that.'” So stay tuned for that.

UPDATED 1/26/15: Bot successfully deployed via Heroku. Click here to learn how.

In the meantime, cheers to the original women in technology who served at Bletchley Park, and cheers to your newfound power to make a twitter bot! Please use it for good and not evil (or I may be forced to make a bot to destroy your bot.)

UPDATE 6/25/15

This is part 1 of the tutorial.

Part 2: Finishing the Bot

Part 3: Top Bot

1 Comment

  1. This is brilliant. I love that you thought of using a bot to share the details of Bletchley Park’s inspirational woman. I especially like it that you have also shared about the idea and process, which I find as fascinating as the outcome.

Leave a Reply

© 2017 Mary Dickson Diaz

Theme by Anders NorenUp ↑