Subscribe now

New Scientist Default Image

By the people, for the people

(Image: Jim Dyson/Redferns/Getty Images)

ONE limitation of video footage is that it tends to reflect the singular interests of the person who shot it. What if the viewer wants to see a public event – a children’s play, say, or a political protest – from a different perspective?

Pablo Cesar at the National Research Institute for Mathematics and Computer Science in the Netherlands has a solution. He and his colleagues have developed a system for creating mash-ups of crowd-sourced video clips that can be personalised for you on the fly.

Cesar starts with raw video footage recorded at a specific event by different people, at different moments and from different perspectives. Next, software called a “narrative engine” uses what it knows of the relationships between people to create dynamic stories. These are tailored to an individual’s preferences, interests and social connections by automatically stitching together parts of clips into a seamless video stream. “The stories are highly personal depending on the recipient of the story,” says Cesar.

“Dynamic stories are tailored to an individual’s preferences, interests and social connections”

The system works by synchronising all the video clips with a master audio track that is recorded at the event. The audio of each clip gives it a digital fingerprint allowing similar footage in different clips to be matched up. The software analyses the video content – applying facial recognition techniques, for example – and contextual information added by the film-maker. It then puts together clips or partial clips, producing a bespoke video edit for every user.

The system was tested at a school concert that was filmed with 12 cameras – some fixed, some belonging to parents in the audience – generating more than 300 raw video clips. These clips were pooled and annotated with personal details, including who was in the clip, or which musical instrument was shown. Most parents agreed that the tailored films made the viewing experience more personal. The team presented the work at the 2012 Symposium on Document Engineering in Paris, earlier this month.

“We’re living in a world of abundant content,” says Mor Naaman of Mahaya, a start-up that is developing software to find and organise social media shared from real-world events. “The real technical challenge is to do this at scale.”

Sign up to our weekly newsletter

Receive a weekly dose of discovery in your inbox! We'll also keep you up to date with New Scientist events and special offers.

Sign up