Proposal for a half-day Pre-Conference workshop for up to 30 participants
Workshop Title: "A primer in automating visual social media analysis with deep learning techniques"
Organisers: Fabrizio Poltronieri & Max Hänska
Workshop Description:
With contemporary online communications increasingly generating large volumes of visual data, scholars face the challenge of retrieving, wrangling, and analysing visual social media content at scale. Indeed, visual social media is attracting increasing attention in the field (Highfield & Leaver, 2016; Svensson & Russmann, 2017). While 'big data' techniques for analysing textual content (e.g. topic modelling) are already wide-spread, and quite advanced, similar techniques for analysing visual social media content (e.g. memes, images, videos) are nascent, and have not yet been widely deployed. How can we analyse visual social media content, given that the sheer volume of visual material requiring our consideration vastly exceed the strictures manual analysis impose on the amount of material that can realistically be analysed? This workshop offers an introduction and overview to available deep learning techniques for automating visual analysis, an introduction to tools for retrieving social media images, and the opportunity to plan an image retrieval, processing, and analysis pipeline. The workshop will focus on the following:
- Tools and techniques for image retrieval, ingestion, and processing:
The workshop will introduce participants to the parameters of the Twitter, Instagram, and Flickr APIs for image collection and retrieval. The workshop will also introduce online tools and software packages for retrieving images from these various APIs, including Echosec (https://www.echosec.net), Netlytic (https://netlytic.org), Nvivo, and Boston University BU-TCAT.
- Deep Learning techniques for image classification:
The workshop will Introduce the basic principles of using machine learning algorithms to train Convolutional Neural Networks (CNN) for image classification, the requirement of training data sets (including recommended parameters for the training data), and machine learning libraries (e.g. TensorFlow and Nvidia Digits). These techniques are particularly useful for researchers who want to train their own neural network to perform basic image classifications at scale, who have specific classification needs, and who have access to appropriate training data.
- Cloud-based visual analysis tools:
Researchers who want to avoid training their own Neural Network, and require standard descriptions of visual material (including images and videos) can also rely on a set of cloud-based computer-vision toolkits for image analysis. These typically perform, inter alia, object recognition (does visual material depict people or animals, men or women, does it contain obscenities, etc.). The advantage of these services is that they generally offer much greater level of granularity than can be achieved through a self-trained CNN, though they are mostly not designed for highly specific classification needs. We will focus in particular on the operation, and output of following APIs: Amazon Rekognition, Google Vision, IBM Vision Recognition, and Microsoft Azure.
Organiser Bios:
Fabrizio Poltronieri is an award-winning computer artist (some of his works are included in the collection of London's Victoria & Albert Museum). His artistic practice applies machine and deep learning techniques to the production and design of narratives, moving images and objects. He is has extensive programming experience, and has worked with computer vision techniques for over a decade. Currently Fabrizio is a Lecturer in the Institute of Creative Technologies at De Montfort University, he holds a PhD in Semiotics and a BSc in Mathematics.
Max Hänska is a Senior Lecturer in Digital Journalism at De Montfort University, and has extensive experience working with Twitter data on European politics (including the use of geocoding toolkits), the Brexit referendum, and other political events. He also has a longstanding interest in the way visual social media content is used in newsrooms to report breaking stories. He holds a PhD in Media and Communications from the LSE. Max and Fabrizio are collaborating on a project that automates the analysis of visual social media content.
