Mexico

SOVIA, an interactive system that endows Claude Monet’s art with responsive auditory experiences. SOVIA uses computer vision trained on Monet’s artwork to take the user ``into the painting.&#39;&#39; When the user interacts with a digital version of Monet&#39;s landscapes, their mouse positions are mapped to sounds that artistically represent the objects that the user is currently exploring in the art. These interactive musical journeys have the potential to make classical art more captivating for modern audiences.

ICCC&#39;21

SOVIA: Sonification of Visual Interactive Art

### **Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!**

<iframe width=1280 height=720 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/player/crQbitV66dw?width=1280&height=720&ff=1&title=0" allowfullscreen="true"></iframe>

Computational creativity is an interdisciplinary endeavor that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, social sciences and the arts. The goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends:

* to construct a program or computer capable of showing some level of creativity
* to better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans
* to design programs that can enhance human creativity without necessarily being creative themselves

<br>The ICCC conference series organized by the [**Association for Computational Creativity**](https://computationalcreativity.net/) since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.



To access the Twelfth International Conference on Computational Creativity, ICCC’21, you need to register for the conference.  Instructions on how to access the event will be provided directly to attendees and included on the Twelfth International Conference on Computational Creativity, ICCC’21 conference website. 

[Register now](https://computationalcreativity.net/iccc21/registration/)

Register now

ICCC'21

Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!

SOVIA, an interactive system that endows Claude Monet’s art with responsive auditory experiences. SOVIA uses computer vision trained on Monet’s artwork to take the user ``into the painting.'' When the user interacts with a digital version of Monet's landscapes, their mouse positions are mapped to sounds that artistically represent the objects that the user is currently exploring in the art. These interactive musical journeys have the potential to make classical art more captivating for modern audiences.

poster

What if creative machines could show the story told by a live orchestra? In this paper, we propose a vision for a co-creative orchestra performance partner that enlivens this classical art form and has the potential to increase audience engagement and reach broader audiences. We present MIST, an early version of our vision, which creates real-time visualizations as live music is performed, and showcase it in a violin performance.

MIST: ``You Play, I'll Draw'

The ability to produce surprising outputs is a cornerstone of creative behavior. In this paper we propose deep learning architectures that are trained in an adversarial setting to learn patterns of expectations and surprise from data. We introduce benchmark datasets of geometrical shapes that represent well defined patterns of violations of expectations, and use them to verify empirically that the separation between learning of expectations and learning of surprise is essential for achieving good generalization performance.

Adversarial Learning of Expectation and Surprise: Experiments with Geometric Shapes

The study of Ukiyo-e, an important genre of pre-modern Japanese art, focuses on the object and style like other artwork researches. Such study has benefited from the renewed interest by the machine learning community in culturally important topics, leading to interdisciplinary works including collections of images, quantitative approaches, and machine learning-based creativities. They, however, have several drawbacks, and it remains challenging to integrate these works into a comprehensive view. To bridge this gap, we propose a holistic approach We first present a large-scale Ukiyo-e dataset with coherent semantic labels and geometric annotations, then show its value in a quantitative study of Ukiyo-e paintings' object using these labels and annotations. We further demonstrate the machine learning methods could help style study through soft color decomposition of Ukiyo-e, and finally provides joint insights into object and style by composing sketches and colors using colorization

Ukiyo-e Analysis and Creativity with Attribute and Geometry Annotation

Metaphor is a powerful tool in the performer’s tool box, not least because it can operate at several levels at once. As our linguistic metaphors deliver rhetorical flourishes, conceptual metaphors change the way we see the world, while our metaphorical body language allows us to take postures and stances that are both figurative and literal. Each kind of metaphor is another expressive choice that creative performers can make to convey their meanings. This choice sits at the heart of what it means to be a creative agent, for agents who lack choice and do exactly as they are told cannot exercise genuine intent for their actions. We shall explore how interpretation is wrapped up with choice to appreciate and to achieve emotional creativity in a system for generating and enacting automated tales. We explain here how three levels of description – linguistic, conceptual and physical – are integrated in a framework that motivates the use of metaphor or irony as a creative choice, by robot performers aiming to go beyond a predetermined script.

Metaphor, Blending and Irony in Action: Creative Performance as Interpretation and Emotionally-Grounded Choice

Architecture search optimizes the structure of a neural network for some task instead of relying on manual authoring. However, it is slow, as each potential architecture is typically trained from scratch. In this paper we present an approach called Conceptual Expansion Neural Architecture Search (CENAS) that combines a sample-efficient, computational creativityinspired transfer learning approach with neural architecture search. This approach finds models faster than naive architecture search via transferring existing weights to approximate the parameters of the new model. It outperforms standard transfer learning by allowing for the addition of features instead of only modifying existing features. We demonstrate that our approach outperforms standard neural architecture search and transfer learning methods in terms of efficiency, performance, and parameter counts on a variety of transfer learning tasks.

Conceptual Expansion Neural Architecture Search (CENAS)

Punch Card Knitting Pattern Design in Collaboration with GAN

Designers increasingly rely on procedural generation for automatic generation of content in various industries. These techniques require extensive knowledge of the desired content, and about how to actually implement such procedural methods. Algorithms for learning interpretable generative models from example content could alleviate both difficulties. We propose SIGI, a novel method for inferring shapes and inducing a shape grammar from grid-based 3D building examples. This interpretable grammar is well-suited for co-creative design. Applied to Minecraft buildings, we show how the shape grammar can be used to automatically generate new buildings in a similar style.

Shape Inference and Grammar Induction for Example-based Procedural Generation

We present a framework for automating generative deep learning with a specific focus on artistic applications. The framework provides opportunities to hand over creative responsibilities to a generative system as *targets* for automation. For the definition of targets, we adopt core concepts from automated machine learning and an analysis of generative deep learning pipelines, both in standard and artistic settings. To motivate the framework, we argue that automation aligns well with the goal of increasing the *creative responsibility* of a generative system, a central theme in computational creativity research. We understand automation as the challenge of granting a generative system more *creative autonomy*, by framing the interaction between the user and the system as a co-creative process. The development of the framework is informed by our analysis of the relationship between automation and creative autonomy. An illustrative example shows how the framework can give inspiration and guidance in the process of handing over creative responsibility.

Automating Generative Deep Learning for Artistic Purposes: Challenges and Opportunities

We posit entrepreneurship as a new frontier in creativity: the problem is ill-defined and the criteria for evaluating the product are initially unavailable; the product is novel, useful, and often non-obvious; success requires producers as well as receivers; the upfront investment for the producer is significant, but the results of the evaluation are delayed; the cost of being incorrect can be considerable but the gains for being correct can be significant as well. How, then, may entrepreneurship be fostered, and how might computing be used to help teach entrepreneurship? In this paper, we describe a preliminary experiment in designing AI technology for teaching entrepreneurship. When a novice entrepreneur creates a business model for a startup, our AI agent named Errol uses semantic and lexical analysis of the entries in the model to ask questions. By attempting to categorize and correct the errors that novices typically make, Errol seeks to accelerate the process by which a novice can start creating more expert-like business models.

Entrepreneurship: A New Frontier in Creativity: Towards A Virtual Coach For Startup Engineering

Portmanteaus are a type of neologism combining two source words, for example brunch, and are popular for naming all kinds of phenomena. While coming up with suitable portmanteaus is a difficult creative endeavor, several portmanteau generators already exist for assistance in this process. When using these systems, it is often hard to find out which of the generated portmanteau is likely to be the best, and consequently also hard to automatically compare the quality of different portmanteau generators. In this paper, we create a model that can rank portmanteaus for two given source words, which thus aims to help find the best portmanteau to help further improve portmanteau generators. Our model first uses XGBoost trained on unlabeled generated outputs and existing portmanteaus to learn to rank portmanteaus and shows that this already greatly improves the performance of the initial generator. By ranking outputs of a state-of-the-art generator and a new simple portmanteau generator, we show by validating its quality in a human evaluation that the ranker can help visually identify the better generator, thus providing an alternative to only calculating real portmanteau generation frequency. Additionally, we find that this first model performs almost as well as a model trained on more fine-grained human-labeled portmanteaus. This indicates that just using generated and real portmanteaus is enough to create a ranker that can in turn improve the quality of the initial generator, and could additionally be of use in comparing different portmanteau generators.

Downloads

Next from ICCC'21

MIST: ``You Play, I'll Draw'

Similar lecture

SOVIA: Sonification of Visual Interactive Art

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES