Example demonstration showcasing the significance of television character importance levels
In the ever-evolving world of broadcast media, a new tool is making waves: computer vision. This technology is set to widen the evidence base of on-screen representation across various types of programs, ultimately improving diversity in the industry.
The power of computer vision lies in its ability to automatically analyze, categorize, and quantify visual content at scale. One key aspect is automated detection and classification. Using deep learning and vision-language models (VLMs), computer vision can recognize faces, genders, ethnicities, and other attributes in video content. This systematic measurement of on-screen presence generates large datasets, providing valuable insights into representation patterns across multiple shows.
Advanced models that combine visual and textual understanding (e.g., VLMs with OCR) allow not only the detection of characters but also the understanding of the context and roles they play on-screen. This moves beyond mere counting appearances to assessing the quality and nuance of representation.
Tools like FastVLM optimize accuracy and efficiency in processing high-resolution video frames, enabling near real-time or large-scale retrospective analysis of broadcasts. This means that computer vision can be applied to a wide range of programs—from news and dramas to reality TV—providing a comprehensive view of on-screen diversity.
The generated detailed quantitative data can help broadcasters and regulators identify biases, gaps, or over/under-represented groups. With this information, they can take informed actions to promote diversity. Moreover, techniques like Grad-CAM enhance trust in AI-driven analyses by highlighting which visual cues contributed to classification decisions, facilitating industry acceptance of automated monitoring tools.
The potential of computer vision extends beyond the broadcast industry. It can be a valuable tool for diversity leads and monitors, content producers, editors and commissioners, and researchers, providing an objective, large-scale, and nuanced measurement of on-screen diversity patterns.
Two examples of measuring character prominence in broadcast TV using computer vision have been demonstrated in a recent blog series. The first example used an episode of the TV show Mock the Week, while the second example took an episode from the American sitcom Black-ish to test the feasibility of generating character prominence metrics in the face of greater variety of camera angles and face sizes.
The future of computer vision in the broadcast industry looks promising. The next issue of ViewFinder, Learning on Screen's specialist online magazine dedicated to the moving image and education, will delve deeper into AI and its relationship with audiovisual media.
As the UK's departure from the EU changes the way British firms trade and work with European counterparts, the focus on diversity data and computer vision remains crucial. The economic consequences and potential market failures of overseas mergers and acquisitions in the UK video games industry are being explored in a new scoping study conducted by the BFI.
Moreover, the importance of representation is not limited to on-screen diversity. The focus on diversity data and computer vision serves as a reminder that representation is just one part of inclusion. In 2020, a new BAFTA diversity steering group was established, and several broadcasters renewed their inclusion and diversity commitments, referencing the global anti-racism movement.
A report on the migrant and skills needs of creative businesses in the UK details the results of a survey of employers commissioned by the Creative Industries Council. Meanwhile, a report on the worldwide exports of creative goods exceeded 500 billion USD in 2015, with a 150% increase since 2000, according to a report on 12 facts about the UK's international trade in creative goods and services.
Raphael Leung, a Data Science Fellow at Nesta, and Bartolomeo Meletti, the Creative Director for CREATe at the University of Glasgow, are at the forefront of this revolution, using computer vision to drive change in the broadcast industry.
- Using computer vision, data from various shows can offer evidence of representation patterns across diverse broadcast programs, promoting education about the importance of on-screen diversity.
- Computer vision technology, equipped with deep learning and vision-language models, can analyze data and cloud computing resources to research and classify visual content in the context of education and lifestyle, including home-and-garden programs and sustainable-living documentaries.
- The ability to automatically analyze and categorize visual content makes computer vision essential for industries that prioritize diversity and representation, like education, arts, and technology.
- Governments, researchers, and broadcasters alike can utilize data generated by computer vision to enhance their understanding of representation patterns in the creative industries, fueling more informed analysis and action towards a more diverse, inclusive world.
- Advanced computer vision models like FastVLM and those incorporating textual understanding help understand character roles and contexts, ensuring robust, accurate, and nuanced data to inform skill development, talent acquisitions, and strategic planning.
- In the near future, artificial intelligence and technological advances in data-and-cloud computing will drive the widespread adoption of computer vision across creative industries, facilitating more creative and sustainable living solutions.
- The benefits of computer vision extend beyond data collection and analysis, as it serves as a creative platform for leaders in the technology and arts industries to develop solutions that drive change, promote diversity, and improve accessibility in education and entertainment.
