Creating Cactool – Reflections on developing a research tool

This is a supplementary blog for the launch of Cactool, which you can find here at https://cactool.github.io . It goes over the process of how Cactool came to be, and some of the issues faced along the way.

I can’t pinpoint when I originally came up with the idea of Cactool (Content Analysis Coding Tool), but I can certainly think of two points in time where I grew extremely frustrated for the ‘well established’ workflows for coding social media posts. The first was while annotating abusive messages to MPs, which went on to be published as Turds, Traitors and Tossers. The second was while researching political memes.

While both papers required a lot of manual data labour, they also highlight that there is clearly space, and need, for manually coded social media data. While automated approaches hold their weight, and are certainly valuable, there is still data and questions in which context, nuance, and ultimately, the human touch is needed (for example, in trying to understand sarcasm).

The problem with manual approaches to content analysisis the two most common workflows pose challenges to social media researchers. The first typically seen approach involves downloading social media content into a spreadsheet format, with researchers going down row-by-row, adding in their codes to the correct column. This approach gives you speed, but often strips important data. The second is through Nvivo, which while very powerful, isn’t quite suited for flicking quickly between downloaded data in these projects.

From my experiences with these approaches, I scribbled down some notes of what exactly I’d want a platform for helping with social media content analysis. I wanted it to retain the look and feel of the post, I wanted the speed of going through posts in a spreadsheet, while maintaining the visual elements often found in social media posts.

“Skill Issue”

My first approach *kinda* worked, but quickly highlighted that my code is terrible when it comes to developing something like this. Or as you say in the online video gaming community when moaning about a loss: “Skill Issue”.

 Luckily, I came across an extremely talented software developer by the name of Sam Ezeh. Sam initially worked for way below market rates for the proof of concept. But luckily I able to give a much better market rate with the help of the RDIF grant from the University of Liverpool to create a much more finished platform. Their effort was *amazing* and quite simply wouldn’t have been possible without them. 50% of the credit very much goes their way.

The changing nature of social media

Issues we encountered throughout the way, other than, the typical bugs you find when creating something like this was the rapidly changing world of social media. When I first started, Reddit and Twitter had openly accessible APIs for academic research, both of these are now seemingly locked behind overly prohibitive API fees.

Likewise, how much of these platforms operated have changed. When we first launched Cactool in 2022, the platforms suddenly changed how some of their embedding features worked. Cactool still worked, but changed were needed to bring it back to being 100%.

Scope creep

Scope creep refers to circumstances where a project or product keeps having new features added until it becomes too complicated or difficult to complete (or use). When we first launched Cactool, we had some amazing, well considered, feedback. Much of these I really saw value in. Many of these made it into this release (such features to allow inter-coder reliability, and visual analysis, and able to skip rows). Many others, which were great ideas, I really wanted to add. However, the realisation shook me that a) I only had enough resources for a few additions, and b) each addition took more time to add in. There was also a third issue c) new features means more bugs. There was a point where you have to say “no more” and get the damn thing out the door.

So the three lessons I learned from Cactool:

1. Quickly assess what skills you do, and don’t have. Then become very lucky by finding yourself a Samwise Gamgee Ezeh.
2. You can’t predict where social media is going next, even if it has been relatively stable for multiple years.
3. Have a feature plan and stick to it. Add in additional features down the line if necessary.

Overall, I *really* enjoyed working on this project. And I really hope this is a tool which the community finds valuable and gets to use. I know I’ve already used Cactool a few times, and it’s been a lifesaver!