This post was originally published on this site
Children love to draw and express their thoughts and ideas with the help of doodles and pictures. This is the way people, from a very young age, portray their emotions along with creativity. Kids put their abstract ideas into a drawing, which helps their cognitive development. The field of Artificial Intelligence has been making noteworthy strides after the increasing popularity of Large Language Models like ChatGPT and DALL-E. With a new research paper and model getting released almost every day, now comes a new AI model that can convert a doodle into an animation.
Traditional AI models, trained on images of real-life objects, usually find it difficult to detect and recognize an abstract or a non-realistic drawing. To overcome the limitations of these AI models, a team of researchers from Meta developed an AI system research demo that can bring artwork to life through animation.
The doodles get converted into animation in four main steps. In the first step, the system detects the human figure in the photograph of the drawing. In the second step, the system uses a segmentation mask to separate the figure from the background. The third step consists of the system estimating the pose and rigging of the figure, making it possible to animate it. Lastly, in the fourth and final step, the system animates the figure using motion capture data, which is retargeted onto the character in a unique and appealing way.
The team has developed a dataset of 178,166 annotated amateur drawings. This dataset of abstract images can help other AI researchers and creators innovate further. For creating this dataset, the researchers released an Animated Drawings Demo in the year 2021 and invited people to contribute their drawings to the dataset. People could upload images, verify or fix annotation predictions, and receive a short animation of their humanlike character within their drawing with the help of the browser-based demo. More than 3.2 million people worldwide visited the site, and 6.7 million images were uploaded. The pictures chosen by people to share with the team were filtered by human reviewers.
The researchers also implemented privacy safeguards to ensure participants’ anonymity and the dataset’s quality. The team shared the animation code for the model and the fine-tuned model weights for drawn human figure detection and pose estimation, which can be accessed here.
Anyone can use the open-source code and the dataset to expand upon their methods of analyzing and augmenting amateur drawings. This can unlock new forms of storytelling and greater accessibility in art. The system could have applications in animation and game development, as well as in educational settings, where it could be used to engage children in creative activities. The system is fast, intuitive, and robust and is definitely a great development in AI to cater to human creativity.
Check out the Paper, Github, Project, and Blog. Don’t forget to join our 18k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more. If you have any questions regarding the above article or if we missed anything, feel free to email us at [email protected]
🚀 Check Out 100’s AI Tools in AI Tools Club
Tanya Malhotra is a final year undergrad from the University of Petroleum & Energy Studies, Dehradun, pursuing BTech in Computer Science Engineering with a specialization in Artificial Intelligence and Machine Learning.
She is a Data Science enthusiast with good analytical and critical thinking, along with an ardent interest in acquiring new skills, leading groups, and managing work in an organized manner.