-Jayson Gylen & Parham Ghalamdar
The full initial questions of our proposal are available here:
[Drafted between Feb and April 2024]
In the realm of artistic inquiry, our project embarked on exploring the recursive challenges posed by artificial intelligence (AI) and its intersection with traditional art forms, focusing on oil painting. The primary goal was to scrutinize and challenge the evolving dynamics between human creativity and AI's generative capabilities. This endeavour was anchored in the metaphor of the Ouroboros, symbolizing the self-referential loop, recursive learning, reusing the output as input, and the model collapse of AI. This raises significant questions about autonomy, adaptability, and the potential for both symbiotic and parasitic relationships between AI and painting. This is especially important to us since the term "painting" is being used to advertise AI image generators (Fig. 1).
Fig. 2: Screenshots of our Instagram ads, the two images on the left, and the loading page on DALL-E 2, showcasing what people prompted before using the service.
We recognize that artificial intelligence (AI) is not some extraterrestrial technology that appeared out of the blue. Rather, the inception of AI can be traced back to the very moment humans first made a mark on a surface, signaling the beginning of our journey to externalize computational and memory tasks. We view the acts of mark-making and oil painting as early forms of technology aimed at outsourcing computational and memory work. Furthermore, these acts serve as means to transfer our capacity for pattern recognition (akin to imagination) into a static format, much like a "PDF," where the entirety of time, movement, and brushstrokes are compressed into a single plane, imbued with a profound density of existential qualities. What are pen and paper, if not primitive forms of technology that externalize memory, minimize computational demands, and convert the most abstract notions from the mind into tangible, material evidence? AI represents merely the latest milestone in this grand human endeavor to extend our cognitive reach. Painting, with its ability to flatten complex concepts onto a single surface, is a prime example of this longstanding tradition.
Thus, our academic backgrounds in painting do not deter us from embracing AI. Our perspectives are significantly shaped by Howard Halle's 1999 essay, "Photo Unrealism." Halle's proposition that "Painting is a philosophical enterprise that doesn’t necessarily involve paint" opens up vast avenues for creative exploration. This viewpoint provides us with ample flexibility to navigate and integrate AI into our artistic practice, underscoring the adaptability and broad conceptual foundation of painting beyond its traditional mediums.
In our exploration, we initiated our project with a unique dataset created from fifty oil paintings. These paintings, identical in landscape, were crafted in China and then transported to the UK for commercial distribution: A foreign object (Fig. 2). We used digital photographs of these paintings to construct the dataset for training our AI model. The training process unfolded as a captivating duel between an imposter (the AI) and a gatekeeper, engaging in a rigorous exchange of attempts to produce images that mirrored the original input. It took around 50 iterations for the imposter to generate an image that closely resembled the original painting. The outcome of this intricate process yielded a collection of dynamic visuals.
Fig.2 mass-produced oil painting created in China for export.
1. Some of these visuals animate snapshots that highlight a specific phase in the generative process across all iterations, offering a glimpse into the evolutionary journey of the images. (Fig. 3)
Fig.3 Compilation of phase 50, the last phase, of the training of all iterations.
2. Alternatively, the visuals could document the entire developmental sequence of a single image, charting its progression from the imposter's initial attempt to its final, successful emulation. (Fig. 4)
Fig. 4 A heavily compressed and slowed compilation of all 50 phases of one iteration from the training to mimic the process of painting and adding brush strokes down on the canvas. This piece of work began as a fast-paced moving image, comprising 50 frames exported as a GIF. However, we decided to slow it down by generating many more transitional frames between each original frame, creating a pace where the slowness and gradual addition of colors feel more painterly. This way the image is produced slowly by being slowly painted in and painted out of the frame, which is more loyal to the truth of painting in the studio. The original file of this moving image is 400 megabytes; what you see here is a heavily compressed version, occupying only 30 megabytes. We believe we have entered a twilight zone where the animated GIF image is evolving into a video art form due to computational costs. Processing this 400 megabyte GIF using software was slow, while exporting it as a video and dealing with a video of the work was immediate and easy. "Computational cost" defines the medium; "computational cost" is the message.
Building upon our initial experiment with mass-produced oil paintings, we decided to take a divergence to refine our approach. Although we appreciated the concept of mass-produced oil paintings exported from China as a foreign object, or a Deep Object, whose cultural, economic, and political impacts would be revealed over time, we ultimately decided to transition to a fully digital approach. This decision was made in favor of cost efficiency, opting to use only digital images gathered from online archives for training our models:
1. First we began by fine-tuning established AI models like Stable Diffusion by retraining three models. Each model was retrained with hundreds of images only one of the categories of Persian Miniatures, Chinese drawings, and British landscape oil paintings. We then used the models to generate new images to be used as input data again rather than acquiring and photographing mass-produced oil paintings from China. This generated iterations of images then served as the foundation for creating animated representations of those landscapes (Fig. 5).
Fig. 5 The image on the left compiles all the phases of one iteration. On the right, the image gathers all the iterations of one image produced by prompting a tuned AI model for British Landscape Oil Painting.
We found it fascinating that the concept of British Landscape oil painting included a frame- a cultural signifier of its origins and existence as a commodity - a movable and exchangeable art object. The inclusion of the frame in this case also breaks the illusory nature of the image, as its objectness is asserted, its presence as a thing in the world affirmed.
The painting, as it shifts and pulsates before us, starts to give the appearance of something self-aware, of possessing an interiority or inner world. It's as if it’s figuring itself out, pushing out against its boundaries. We are reminded of the artifice of landscape painting as an act of fragmentation of the whole. An unnatural severing of a part from the larger body.
The fluctuating image is restless, reaching beyond the frame, it turns its ‘intelligent’ gaze back on us. This is the gaze of the MACHINE looking through the painting, through the screen, its sophisticated set of processes hidden just beneath the surface. Of course, any suggestion that AI has agency is a misnomer. We ascribe agency to Artificial intelligence by anthropomorphising its functions which themselves, mimic certain functions and characteristics of our brains. The data set, built in our own image, is thus reflected back to us and we mistake this reflection for something other than what it is; a technological extension of our mind’s capacities and agency. We fall into our image.
‘But computers produce so enchanting a simulacrum of mental agency that sometimes we fall under their spell, and begin to think there must really be someone there’
(David Bentley Hart)
The presence of the frame reminds us of the selective choices executed by every painter when 'framing' a landscape. The prioritizing of certain viewpoints over others implies a value judgment on the part of the artist but also embodies a host of other motivations and considerations beyond the agency of the artist. Historically, a myriad of socio-political factors have shaped the representation of landscape in Western oil paintings:
1.1 Ownership:
Historically, landscape paintings were often commissioned by wealthy patrons or aristocrats who had the means to acquire and display such artworks. Depictions were shaped to reflect the preferences of the elite, often portraying landscapes as picturesque scenes, serving as status symbols showcasing the owner's wealth and social standing.
Landscape painters were influenced by socio-political forces like patronage systems and artistic movements. Patrons often dictated subject matter and style, shaping landscapes to reflect their desires and agendas, thus reflecting the power dynamics of the time.
1.3 Accessibility:
Much like access to the countryside itself, landscape paintings were primarily accessible to the affluent, reinforcing social hierarchies. A closing circle, where the representations of the landscape reinforce the ownership of that landscape. This is compounded by the particular ability of oil paint to render land in all its substantiality, further strengthening the sense of possession and entitlement.
Questions of ownership, agency, and accessibility are relevant to the landscape of AI and can be considered in relation to the specifics of landscape images or images of landscape paintings:
1.4 Representation Bias:
Despite the wealth of easily accessible digital images on the internet, there are still problems with representation in Ai generated images. AI relies heavily on the data it is trained on. If the dataset predominantly features landscapes from specific geographic regions, climates, or cultural contexts, it may struggle to accurately generate landscapes outside of those parameters. This can result in biased representations that do not fully reflect the diversity of landscapes worldwide.
Landscape images generated using Ai are scenes “imagined” by a piece of code. (Fig. 7)Their unfixed/untethered viewpoint is unlike any rendering of the landscape that’s come before. They have an affinity with the idealised landscape painting of the 17th century, which seeked to present a view of nature more beautiful and harmonious than nature itself (Fig. 6). Ai Landscapes are idealised amalgamations, on the one hand appearing convincingly real and on the other too perfect and polished to be natural. At best, they are convincing impersonations of the real.
Fig. 6 Claude Lorrain: Pastoral Landscape with a Mill. Pastoral Landscape with a Mill, oil on canvas by Claude Lorrain, 1634; An example of 17th century idealised landscapes which set scenes in the mythic and idyllic Arcadia of ancient Greece.
Fig. 7 Ai generated image made using www.getimg.ai . Stable diffusion 1.5. Prompt: ‘British Landscape’
1.5 Cultural Distinctions in Landscape painting:
Persian Miniatures employ an isometric perspective, allowing the viewer to be omnipresent with no fixed standing point, everything all at once - god mode. Chinese drawings, on the other hand, exude a more wondrous quality, akin to observing assets within a video game rather than the entirety of a map. This effect is achieved through the significant blank areas of the paper surrounding the marks, evoking a sense of unexplored territories awaiting discovery, akin to unlocked areas in a game that the player must navigate.
AGENCY of the painter, AGENCY of the painting. AGENCY of the viewer.
Writing this caption, we realized the extent to which these images, both still and moving, turn their intelligent gaze inward; they are responsive. We believe this effect is borrowed from painting. It begins with the question, "How do we know when a painting is finished?" When one paints, information is added to the surface, and stepping back to observe allows for the reception of information from the painting's surface: it becomes a dialogue with the material. The conversation concludes when no further valuable information can be added to or derived from the surface. Of course, one can always continue to paint, potentially exhausting the painting, much like how one can exhaust a conversation with another human by being overly chatty. Thus, a painting always converses and reflects back, akin to a mirror perhaps, as we direct our intelligence towards it. This notion aligns well with "Why We Need the Demonic: A Phenomenological Analysis of Negative Religious Experience" by James M. Nelson* and Jonah Koetke, which explores the phenomenology of the demonic as perceived by early Christians, particularly those in Egyptian desert communities. Specific principles associated with what is deemed demonic include intentionality, where consciousness is directed towards an object, even in its absence. Could we argue that AI-generated content falls within the realm of the demonic?
2. Further experimentation involved the integration of multiple datasets from the distinct categories. This approach allowed us to observe the interactions and potential conflicts between different landscape perspectives when merged. When presented with two disparate datasets, the AI model displayed confusion over perspectives, angles, and perception, rendering the viewer "ungrounded" in a manner reminiscent of Cubist artworks. Interestingly, upon reflection, we realized that while many Cubist works focus on figures or objects, landscapes are seldom depicted within this style (Fig. 8).
Fig. 8 The dataset included a hundred images of British landscape oil paintings gathered from ArtUK.org and another hundred images of Chinese drawings held at the Harvard University Art Museum.
Another layer involved compiling early-stage snapshots from the image generation process, specifically when the AI struggled to accurately replicate the input. We deliberately chose to juxtapose images from different cultural contexts—Chinese drawings against British landscape oil paintings—to highlight the contrast in their inherent compositional lines; the former's vertical or curvy lines against the latter's straight horizontals (Fig. 9).
Fig. 9 presents a compilation of snapshots from the early stages of the model training, featuring images generated by Stable Diffusion. These images were prompted to amalgamate Chinese drawings and British oil paintings with a focus on landscapes. The horizontal lines captured our attention.
In a subsequent test (Fig. 10), despite the ambitious use of a vast and distinguished dataset that included Chinese landscape paintings sourced from prestigious collections like the Princeton University Art Museum (362 images), Harvard University Art Museum (101 images), the Metropolitan Museum of Art (428 images), and the Smithsonian's Freer Gallery of Ar (1301 images)t, the results were inconclusive. The generated images were often distorted and glitched, posing a challenge to clear interpretation. The images looked more like grainy aerial shots of deserts. We realized that this reflected the "isometric perspective," used in Eastern landscape painting, which essentially serves as a "god mode." This perspective embodies how a deity might view creation, with the observer having an equal presence everywhere. Such a god mode perspective in Western art could be likened to aerial shots, couldn’t it? If the stereotypical old man with a white beard were to sit atop his cloud and look down, he would see something akin to drone shots. These images strongly resembled the maps from the Dune 2000 video game (Fig. 11). We were tempted to create a small video game using these images as maps. In the game, you would wander around the map to collect whale fossils from the depths of the desert, only to bring them to a character resembling "Reza Negarestani" to receive philosophical quotes: "The desert reveals depth, akin to Noah's ark settling once more after the storm. [Reza lights his cigarette with the punchline]."
Fig. 10 Compilation of all the iterations at the last phase.
Fig. 11 Screenshot of Dune 2000 video game.
2. 1 The surface of the pool
At this juncture, our reflections took us to the myth of Narcissus, considered by Renaissance theorist Leon Battista Alberti to be the progenitor of painting: (Fig. 12.)
Alberti's musings, "the inventor of painting ... was Narcissus ... What is painting but the act of embracing by means of art the surface of the pool?" sparked a series of contemplations. This perspective sheds light on contemporary practices, such as Gerhard Richter's installation of mirrors in a gallery space, or Roy Fox Lichtenstein's adaptation of comic book aesthetics to paint mirrors. (It prompted us to question whether Lichtenstein's graphically rendered mirrors might be seen as precursors to today's digital "Filters" or "Avatars" used on platforms like Zoom, Google Meet, and TikTok.)
Fig. 12 “Narcissus,” Painting by Caravaggio.
In this digital era, the interface of AI acts as a polished mirror, reflecting our collective human output—our desires, fears, achievements, failures, creativity, intelligence, culture, ideas, and attitudes. This reflection forms a composite image of humanity, showcasing our progress and intellectual evolution. AI's development is intricately linked to our cultural and intellectual heritage, built upon the vast datasets derived from human achievements. Conversely, the rapid advancement of AI algorithms are influencing and reshaping the way we live, work, and interact, transforming our cultural landscape and highlighting the cyclical relationship between AI and human culture.
Fig. 13. “Narcissus,” Painting by Caravaggio with mirrored snapshots from the early stages of model training using datasets of Western and Eastern landscape paintings. The mirrored snapshots resemble the inkblots used in Rorschach tests which are used to analyse the personality characteristics and emotional functioning of patients. In the context of Ai generation, they can be read as a visualisation of the ‘thinking process’ and functioning of the neural networks that are used in GAN (Generative Adversarial Networks).
This cyclical nature is metaphorically similar to the Ouroboros, depicting a serpent eating its own tail. AI has entered a recursive loop, consuming and regenerating its content in a continuous cycle. The moving images, or GIFs, created from our project are compilations of snapshots from AI models trained on datasets of Western and Eastern landscape paintings. These images, selected from the earliest and most abstract stages of the training process, represent the field of pure potentiality before they converge into more coherent forms. They are the ‘chewing’ over of images, a visual interpretation of the process of digital digestion and assimilation. Generative Ai is a vast Metabolic machine. (Fig. 13.)
AI's capability to produce an endless array of outcomes instantaneously, is akin to ripples spreading across a pond. These reverberations, originating from the deep well of collective human knowledge and creativity (deep objects) , interact and coalesce, forming new artifacts. AI accelerates this dynamic process, increasing the rate of these reverberations, the rate of historical reactions acting as an extension of human cognitive functions like intelligence, imagination, and creativity. (Fig. 14).
Fig. 14. Narcissus reverberations in a deep well. Visual idea illustrating the rippling effect of cultural artifacts across time. These artifacts are deep objects which thrust towards us from the depth of history whilst we are simultaneously pulled back into its body much like a gravitational tidal force.
‘..when we realize that we have always been artifacts of our decisions, thoughts and actions’ (Reza Negarestani)
2.2 Landscape of perception
These cognitive ‘artefacts’ are constantly being formed and as they accumulate leave their trace on the collective landscape of the real/artificial.
The digital proliferation of images complicates our perception, Imagining the 'physicality' and spatial dimensions of images, despite their dematerialised nature as digital images speaks to the fusion or bleeding between the realms of the 'real'/ physical' world and the artificial/ 'virtual' one. Employing spatial or architectural analogies helps us grasp the complexity and perceptual distortions arising from the accumulation of digital images, highlighting the tangible implications of this digital amalgamation.
With AI's accelerated generation of digital imagery, the landscape of digital art and representation is rapidly expanding. The staggering volume of images created by text-to-image algorithms in a single year dwarfs the cumulative efforts of human creativity over centuries, illustrating the profound impact of AI on the digital and cultural landscape. This expansion not only showcases the immense potential of AI in art but also challenges us to reconsider our understanding of creativity, authorship, and the essence of human expression in the age of digital replication and innovation.
Fig. 15. fig.14. glitching
In conclusion...
Since the invention of Stone tools; humanity's earliest technology, invented more than 2 million years ago, and the first man made drawings of the same ‘‘Stone age’ period, humanity has been on a journey of technological evolution. We’ve learnt to harness and control our physical surroundings through a process of externalisation and development of our interior functions. Each stage of technological invention has left its traces on the world and its indelible marks on human history, ushering in new eras of progress, innovation, and connectivity.
Today, we stand on the precipice of another such transformative moment, with Artificial Intelligence (AI) poised to redefine the boundaries of possibility.
Landscape paintings themselves embody a sense of these possibilities and freedoms. The Horizon is the line at which the earth's surface and the sky appear to meet. It is also the limit of a person's knowledge, experience, or interest. To ‘broaden one’s horizons’ is to venture forth into the unknown, unlocking areas of the map previously unknown or off limits. This is much like how the framing of a landscape painting delimits space, artificially separating it and restricting our focus and attention, whilst simultaneously offering the possibility for invention, imagination and creativity within these boundaries.
Developments in Ai have the potential to extend the frontiers of knowledge, reshaping the contours of our world as it goes. Welcome to the brave new world.
Comments