Portugal

A growing number of visual artists use neural networks in their practice. While these networks show promise as an art form, the lack of interpretability limits control to high level decisions based on observations. As an alternative, this research investigates the hand-crafting of network weights coupled with explanatory visualizations as a form of creative control over the internal and lower level processes. Two experimental tools were developed: one for parametrically generating first layer kernels and the second for editing multiple layers. These tools attempt to transform the hand-crafting of features into ``crafting&#39;&#39; in a richer sense by bringing network weights and visual materials into a tight feedback loop. The first author extensively engaged with these tools and these case studies serve to examine the affordances of internal interaction for art-making. The findings suggest that direct manipulation can be used intentionally and can yield insights into network representations, but that hand-crafting networks of greater sophistication would likely require a hybrid approach integrating data-driven methods.

ICCC&#39;20

Hand-Crafting Neural Networks for Art-Making

interactive visualizations

hand-crafting

explainable ai

neural networks

Computational creativity is the art, science, philosophy and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative. As a field of research, this area is thriving, with progress in formalising what it means for software to be creative, along with many exciting and valuable applications of creative software in the sciences, the arts, literature, gaming and elsewhere.

The ICCC conference series organized by the Association for Computational Creativity since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.


ICCC'20

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

A growing number of visual artists use neural networks in their practice. While these networks show promise as an art form, the lack of interpretability limits control to high level decisions based on observations. As an alternative, this research investigates the hand-crafting of network weights coupled with explanatory visualizations as a form of creative control over the internal and lower level processes. Two experimental tools were developed: one for parametrically generating first layer kernels and the second for editing multiple layers. These tools attempt to transform the hand-crafting of features into ``crafting'' in a richer sense by bringing network weights and visual materials into a tight feedback loop. The first author extensively engaged with these tools and these case studies serve to examine the affordances of internal interaction for art-making. The findings suggest that direct manipulation can be used intentionally and can yield insights into network representations, but that hand-crafting networks of greater sophistication would likely require a hybrid approach integrating data-driven methods.

technical paper

In human-human collaboration, gesture, verbal communications, and emotional responses are among the communication strategies that shape the interactions between collaborators to negotiate the contributions. Emotional feedback allows human collaborators to passively communicate their stance about the experience and convey their perception of the process without distracting the flow of the task. In human-human co-creative collaboration, participants inter-act and contribute to the task based on their perception of the collaboration over time.  In human-AI co-creativity, perceiving the cognitive and emotional state of the user in order to determine the dynamics of collaboration and decide what the agent should contribute to the artifact are two primary challenges of building effective co-creative Artificial Intelligence systems. This paper addresses these challenges with the design of a co-creative agent that can sense the user’s emotional state through facial expression.

ARNY: A Co-Creative System Design Based On Emotional Feedback

We present a general computational approach that enables a machine to generate a dance for any input music. We encode intuitive, flexible heuristics for what a 'good' dance is: the structure of the dance should align with the structure of the music. This flexibility allows the agent to discover creative dances. Human studies show that participants find our dances to be more creative and inspiring compared to meaningful baselines. We also evaluate how perception of creativity changes based on different presentations of the dance. Our code is available at github.com/purvaten/feel-the-music.

Feel The Music: Automatically Generating A Dance For An Input Song

Cross-Domain   creative   systems   can   learn   how   to creatively  map  and/or  transform  semantic  concepts from remote domains to their own. This project sets out to explore how a computational system relies on what it has learned from analyzing images to produce creative work.  Specifically, the system presented here extracts semantic content and themes from an input image and uses this knowledge to creatively generate emotionally-appropriate musical artifacts to accompany the image. An independent experiment was conducted to provide a performance assessment and direct future work.

Emotive Music Composition from Visual Sources of Inspiration

"Flow 1 is the first part of two pieces of music, Flow 1 and Flow 2 
While Flow 1 is more classically inspired, Flow 2 is more experimental.

This piece is influenced by the way in which humans, technology and intelligent systems work together to bring music into being. It works both as personal experiment in compositional style and as a way to bring together tradition and technology to explore the interplay between the composer, the tools they use and they ways in which we can explore this. In all works of this kind there are challenges that the composer has to deal with, using new technologies to invoke feelings is a difficult and thoughtful process. It is important to both be in the ‘flow’ of the music creation process and to understand the ways in which working technology can bend your practices to the ‘workflow’ of the system. 

"

Flow 1

Flow 2 is the second piece in the series of works and is a more experimental piece which is the counterpoint to Flow 1. These pieces are influenced by the way in which humans, technology and intelligent systems work together to bring music into being. It works both as personal experiment in compositional style and as a way to bring together tradition and technology to explore the interplay between the composer, the tools they use and they ways in which we can explore this. In all works of this kind there are challenges that the composer has to deal with, using new technologies to invoke feelings is a difficult and thoughtful process. It is important to both be in the ‘flow’ of the music creation process and to understand the ways in which working technology can bend your practices to the ‘workflow’ of the system.

Flow 2

Generative models for singing voice have been mostly concerned with the task of “singing voice synthesis,” i.e., to produce singing voice waveforms given musical scores and text lyrics. In this work, we explore a novel yet challenging alternative: singing voice generation without pre-assigned scores and lyrics, in both training and inference time. In particular, we outline three such generation schemes, and propose a pipeline to tackle these new tasks. Moreover, we implement such models using generative adversarial networks and evaluate them both objectively and subjectively.

Score and Lyrics-Free Singing Voice Generation

We consider the challenge of designing an artificial agent capable of interacting with humans in collaborative dialogue to produce creative, engaging narratives. Collaborative dialogue is distinct from chit-chat in that it is knowledge building, each utterance provides just enough information to add specificity and reduce ambiguity without limiting the conversation. We use concepts from information theory to define a narrative arc function which models dialogue progression. We demonstrate that this function can be used to modulate a generative conversation model and make it produce more interesting dialogues, compared to baseline outputs. We focus on two modes of modulation: reveal and conceal. Empirically, we show how the narrative arc function can model existing dialogues and shape conversation models towards either mode. We conclude with quantitative evidence suggesting that these modulated models provide interesting and engaging dialogue partners for improvisational theatre performers.

Shaping the Narrative Arc: Information-Theoretic Collaborative Dialogue

The Creative Systems Framework (CSF) formalises creativity as search through a space of concepts.  As a formal account of Margaret Boden’s descriptive hierarchy of creativity, it is at the basis of multiple studies dealing with diverse aspects ofComputational Creativity (CC) systems. However, the CSF at present neither formalises action nor action selection during search, limiting its use in analysing creative processes.  We extend the CSF by explicitly modelling these missing components in the search space traversal function.  We furthermore integrate the distinction between a concept and an artefact, and provide stopping criteria for creative search.  Our extension,  the  Creative  Action  Selection  Framework  (CASF),  is informed by previous studies in CC and draws on concepts from Markov Decision Processes (MDPs). It allows us to de-scribe a creative system as an agent selecting actions based on the value, validity and novelty of concepts and artefacts. TheCASF brings more analytical depth for creative systems that can be modelled as utilising an action selection procedure.

Action Selection in the Creative Systems Framework

"This paper presents a domain independent framework for discussing human–computer co-creativity. It expands on Rhodes’ (1961) four perspectives on creativity
and their later adaptations to socio-cultural views of creativity and computational creativity. The new framework allows the attribution of creativity not only to individual creators but to a collective of creators, recognising the importance of meta-level communication to the creative collaboration, and the variety of creative
contributions that emerge during a co-creative process. It also elaborates on the different communities and contexts surrounding co-creative collaboration and thus facilitates the analysis, evaluation and study of human–computer co-creativity by allowing researchers to describe and situate their work in the field."

Five C’s for Human–Computer Co-Creativity — An Update on Classical Creativity Perspectives

Rosetta Code provides improv theatre performers with artificial intelligence (AI)-based technology to perform shows understandable across many different languages. We combine speech recognition, improv chatbots and language translation tools to enable improvisers to communicate with each other while being understood—or comically misunderstood—by multilingual audiences. We describe the technology underlying Rosetta Code, detailing the speech recognition, machine translation, text generation and text-to-speech subsystems. We then describe scene structures that feature the system in performances in multilingual shows (9 languages). We provide evaluative feedback from performers, audiences, and critics. From this feedback, we draw analogies between surrealism, absurdism, and multilingual AI improv. Rosetta Code creates a new form of language-based absurdist improv. The performance re- mains ephemeral and performers of different languages can express themselves and their culture while accommodating the linguistic diversity of audiences.

Hand-Crafting Neural Networks for Art-Making

Downloads

Next from ICCC'20

ARNY: A Co-Creative System Design Based On Emotional Feedback

Similar lecture

Neural Networks approaches focused on French Spoken Language Understanding: application to MEDIA Evaluation Task

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Hand-Crafting Neural Networks for Art-Making

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ICCC'20

ARNY: A Co-Creative System Design Based On Emotional Feedback

Similar lecture

Neural Networks approaches focused on French Spoken Language Understanding: application to MEDIA Evaluation Task

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads