Text-guided 3D object generation aims to generate 3D objects described by user-defined captions, which paves a flexible way to visualize what we imagined. Although some works have been devoted to solving this challenging task, these works either utilize some explicit 3D representations (e.g., mesh), which lack texture and require post-processing for rendering photo-realistic views; or require individual time-consuming optimization for every single case. Here, we make the first attempt to achieve generic text-guided cross-category 3D object generation via a new 3D-TOGO model, which integrates a text-to-views generation module and a views-to-3D generation module. The text-to-views generation module is designed to generate different views of the target 3D object given an input caption. prior-guidance, caption-guidance and view contrastive learning are proposed for achieving better view-consistency and caption similarity. Meanwhile, a pixelNeRF model is adopted for the views-to-3D generation module to obtain the implicit 3D neural representation from the previously-generated views. Our 3D-TOGO model generates 3D objects in the form of the neural radiance field with good texture and requires no time-cost optimization for every single caption. Besides, 3D-TOGO can control the category, color and shape of generated 3D objects with the input caption. Extensive experiments on the largest 3D object dataset (i.e., ABO) are conducted to verify that 3D-TOGO can better generate high-quality 3D objects according to the input captions across 98 different categories, in terms of PSNR, SSIM, LPIPS and CLIP-score, compared with text-NeRF and Dreamfields.

3D-TOGO: Towards Text-Guided Cross-Category 3D Object Generation

We describe a real-time system that receives a live audio stream from a jam session and generates lyric lines that are congruent with the live music being played. Two novel approaches are proposed to align the learned latent spaces of audio and text representations that allow the system to generate novel lyric lines matching live instrumental music. One approach is based on adversarial alignment of latent representations of audio and lyrics, while the other approach learns to transfer the topology from the music latent space to the lyric latent space. A user study with music artists using the system showed that the system was useful not only in lyric composition, but also encouraged the artists to improvise and find new musical expressions. Another user study demonstrated that users preferred the lines generated using the proposed methods to the lines generated by a baseline model.

LyricJam: A system for generating lyrics for live instrumental music

In conceptual blending, the transfer of properties from the input spaces relies on a shared semantic base. At the same time, interesting blends are supposed to resolve deep semantic clashes where many concept combinations correspond to impossible blends, i.e. blends whose input spaces lack any obvious similarities. Instead of a shared structure, the blends are based on bi- directional affordance structures. While humans can easily map this information, computational systems for creative constructions require an understanding of how these features relate to one another. In this paper, we discuss this problem from the perspective of linguistics and computational blending and propose a method combining theory weakening and semantic prioritisation. To demonstrate the problem space, we look at the Sesame Street character ‘The Cookie Monster’ and formalise the blending process using description logic.

Deciphering The Cookie Monster: A case study in impossible combinations

This article presents preliminary work on a creative chess engine that can be used to produce creative chess games or sequences. The contribution in this article is the creation of a creative chess engine that is then pitted against itself to form a creative system that outputs chess games. The chess engine is an extension to an existing chess engine that consists of forcing the existing engine to play more creative moves. It is in no way an improvement when compared to existing chess engines, even though it is based on the world's best: Stockfish. Letting the supposedly creative chess engine play against itself forms a creative system that outputs chess games. Through analyzing these games it might be possible to discover new chess openings or principles.

Producing creative chess through chess engine selfplay

Within the field of Computational Creativity, evaluation is one of the most important and more difficult tasks. Sometimes evaluation is part of the creative systems themselves, becoming an internal evaluation. Being a module of a creative system, it is useful to evaluate how effective this internal evaluation is.
In this paper, we propose a procedure for the (meta-)evaluation of internal evaluation modules, that allows for incremental development of both the evaluation module and the creative system, which are considered fully independent of each other. The procedure works by statistically comparing the evaluation of the average output of the generation system with the best results from the same, to see if the evaluation procedure can statistically distinguish the two. We then show how to apply the procedure giving one example evaluating a module we designed to assess structural coherence in generated folk music.

Meta-Evaluating Quantitative Internal Evaluation: a Practical Approach for Developers

Software design patterns can be helpful in describing the architecture of a system. Our objective is to obtain a broad overview of the current state-of-the-art of software design patterns used in Computational Creative (CC) systems. We conducted a systematic mapping study using manual and snowballing search techniques. Only 7 primary studies are identified in the CC community that explicitly mention the use of design patterns. Within these primary studies, 14 design patterns are mentioned, 12 of which are user-interaction design patterns rather than software design patterns describing the architecture of the system. The small number of primary studies indicates a gap in CC literature regarding the use of software design patterns in CC systems and motivates the need for research to identify software design patterns specific to CC systems.

Software Design Patterns of Computational Creativity: A Systematic Mapping Study

This paper describes a system for the design or redesign of movie posters. The main reasoning strategy used by the system is case-based reasoning (Leake 1996), which requires two important sub-tasks to be performed: case retrieval and case adaptation. We have used the random forest algorithm to implement the case retrieval subtask, as it allows the system to formulate a generalization of the features of all the top matching cases that are deter-mined to be most relevant to the new (re)design desired. We have used heuristic rules to implement the case ad-aptation task which results in generating the suggestion for poster composition. These heuristic rules are based on the Gestalt theory of composition (Arnheim 1974) and the requirements specified by the user.

A Case-Based Approach to Creating Movie Poster Compositions

In this paper we argue that the computational creativity community would benefit by working together to create a methodology for a systematic assessment and comparison of our creative agents. To achieve this goal, we suggest collaborating with experts in real-world environments to develop what we refer to as Rich Creative Environment Experience (RICEE). We describe a work in progress, where two artists and one CC scientist collaborate, to illustrate a possible way to build a RICEE. Based on this experience, we suggest some initial steps to start the development of this methodology. Thus, the purpose of this paper is to motivate the CC community to collaborate in the development of a methodology for the design and implementation of RICEEs.

The stone of madness meets AI

We conjecture that creativity and the perception of creativity are, at least to some extent, shaped by embodiment. This makes embodiment highly relevant for computational creativity (CC) research, but existing research is scarce and the use of the concept highly ambiguous. We overcome this situation by means of a systematic review and a prescriptive analysis of publications at the International Conference on Computational Creativity. We adopt and extend an established typology of embodiment to resolve ambiguity through identifying and comparing different usages of the concept. We collect, contextualise and highlight opportunities and challenges in embracing embodiment in CC as a reference for research, and put forward important directions to further the embodied CC research programme.

Embodiment and Computational Creativity (Study Paper)

In this presentation we focus on two of the major concepts we present in the main paper, namely: how we can understand the Typicality and Novelty of an object with Models created by an observer, and sophistication which lets us analyse the creator of a creative object. We also talk about highly sophisticated objects which can be regarded as masterpieces of the creator. Finally, we review some related works in computational aesthetics and how they can be improved. 



Incorporating Algorithmic Information Theory into Fundamental Concepts of Computational Creativity

Getting machines to generate text perceived as creative is a long-pursued goal. A growing body of research directs this goal towards augmenting the creative writing abilities of human authors. In this paper, we pursue this objective by analyzing how observing examples of automatically generated text influences writing. In particular, we examine a task referred to as sentence infilling, which involves transforming a list of words into a complete sentence. We emphasize “storiability” as a desirable feature of the resulting sentences, where “storiable” sentences are those that suggest a story a reader would be curious to hear about. Both humans and an automated system (based on a neural language model) performed this sentence infilling task. In one setting, people wrote sentences on their own; in a different setting, people observed the sentences produced by the model while writing their own sentences. Readers then assigned storiability preferences to the resulting sentences in a subsequent evaluation. We find that human-authored sentences were judged as more storiable when authors observed the generated examples, and that storiability increased as authors derived more semantic content from the examples. This result gives evidence of an “inspiration through observation” paradigm for human-computer collaborative writing, through which human writing can be enhanced by text generation models without directly copying their output.

Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writing

Recent interest in story generators capable of combining more than on plot line into an elaborate story have been handicapped by the lack of either theoretical material or quantitative metrics to ascertain the quality of outputs of such attempts. The present short paper postulates a set of metrics designed to capture some of the insights elaborated during a formative evaluation of an existing attempt at plot weaving.

Underline digital video library

3D-TOGO: Towards Text-Guided Cross-Category 3D Object Generation

LyricJam: A system for generating lyrics for live instrumental music

Deciphering The Cookie Monster: A case study in impossible combinations

Producing creative chess through chess engine selfplay

Meta-Evaluating Quantitative Internal Evaluation: a Practical Approach for Developers

Software Design Patterns of Computational Creativity: A Systematic Mapping Study

A Case-Based Approach to Creating Movie Poster Compositions

The stone of madness meets AI

Embodiment and Computational Creativity (Study Paper)

Incorporating Algorithmic Information Theory into Fundamental Concepts of Computational Creativity

Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writing

Assessing MultiPlot Stories: from Formative Analysis to Computational Metrics

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES