Enhancing visual on-screen presentation via computer-based vision technologies
In the realm of the creative industries, the importance of representation and diversity has never been more significant. A new research series, led by Nesta in partnership with Learning on Screen, is leveraging computer vision technology to address this issue head-on.
The project, which builds upon previous research by Nesta, aims to measure on-screen representation and diversity in character portrayal. By applying automated visual analysis techniques to audiovisual media, the team hopes to provide a new method for quantifying representation and identifying patterns or disparities.
This approach overcomes the manual limitations of reviewing large volumes of media. Computer vision algorithms can detect and classify faces, estimate demographic attributes, and track screen time and interaction, providing objective data on how diverse groups are portrayed in television and film content.
The system can also integrate with subtitle and script analysis to relate visual appearances to dialogue and roles, deepening insights into character portrayal. The research is inspired by Nesta's previous work on generating more subtle, nuanced measures of creative diversity using machine learning, such as "She said more," which analyzed gendered pronouns in news articles about the creative industries.
The project team includes Raphael Leung, Bartolomeo Meletti, Dr Cath Sleeman, Gabriel A. Hernández, and Gil Toffell. The survey was commissioned by the Creative Industries Council and the report, though not mentioned in the provided paragraph, details the results of the survey of employers in the UK's creative industries regarding their migrant and skills needs.
An illustrative example of measuring character prominence using a short video clip is provided. Common computer vision techniques include face detection and recognition to identify characters, attribute classification models trained on annotated data sets to predict demographic features (gender, ethnicity, approximate age), screen time and interaction metrics through temporal video analysis, and scene segmentation for context-aware insights into how and when characters appear.
This methodology could prove invaluable for policy makers, content creators, and advocacy groups alike. It provides quantitative evidence to inform decisions about on-screen diversity and helps track progress over time. By making media portrayal more inclusive and equitable, this research series supports efforts to foster a more diverse and representative creative industries landscape.
The blog series is an output of a pilot project conducted by Nesta in partnership with Learning on Screen. This type of application fits the known research frontier Nesta explores in tech-enabled research for social impact. Such projects typically combine computer vision with social science methods to study inclusivity and representation in digital media.
- The new research series, led by Nesta in collaboration with Learning on Screen, uses computer vision technology to assess and promote diversity in the creative industries.
- The project investigates character portrayal in audiovisual media by incorporating automated visual analysis techniques, examining patterns of representation and identifying disparities.
- The team's objective is to provide data on how diverse groups are represented on screen, as well as insights into character portrayal by integrating subtitle and script analysis with visual appearances.
- This approach overcomes the limitations of manually reviewing large volumes of media by utilizing computer vision algorithms for face detection, demographic attribute prediction, tracking screen time and interaction, and scene segmentation.
- By supplying quantitative evidence, this research can aid policy makers, content creators, and advocacy groups in making informed decisions about on-screen diversity and monitoring progress over time.
- The research series aligns with Nesta's exploration of tech-enabled research for social impact, typically combining computer vision with social science methods to study inclusivity and representation in digital media.
