The training

As said, our model generates new images that imitate the da Vinci illustrations based on data from different topics in his notebooks. One way of viewing these images is through a latent space walk, a video that combines similarly generated images into a frame-by-frame animation, creating a morphing effect:

When inspiration struck

While looking at this output from our model, one frame in particular caught my eye because I saw a profile of a person hidden in the seemingly abstract sketch. Was this a lucky catch where the model was succesfully trained on the handful of portraits in the dataset, or perhaps a figure of my imagination after years of anatomy practice?

I decided to use this image as the start of a collaborative artwork, but before painting I set myself some constraints:

1. Any colours I want to use need to be colour picked from the original image.
2. The existing shapes should guide me in the final design. No excessive erasing or creating new shapes where there were none.
3. Try to stay true to the da Vinci sketches, while allowing for both the ML model and my own style to co-exist.

The painting process

I started with cleaning up the space around the face, highlighting the profile that excited me to begin with, after which it quite naturally continued in defining the spine and the more ‘android’ like features on the head.

During the process, I tried to be led by the shapes that were already there in the image. This was a fundamentally different approach than doing a normal painting, where I’d have a specific goal in mind. At times it felt nice to go with the flow and forget about the final result, but at others it felt uneasy and I’d frustratedly tell myself “I have no clue what I’m doing or where this is going”.

One of the moments I deliberately chose a certain direction for the artwork, was when I saw the typical ‘android’ features like knots, bolts and tubes appear in the head shape. Working in the field of AI I have negative connotations with representing Machine Learning in such a way. Instead, I wanted to focus on the organic, imperfect shapes that were produced by the model almost ‘naturally’. Inspired by a leaf-like shape underneath the face of the subject, I turned the lower half of the image in branches and leaves, highlighting the wild ‘growth’ of the model. It was this lower part that really made me surrender to the shapes that already existed, and gave me a feeling of flow and collaboration.

The results

Because of my style and way of working, a lot of the noise, originally produced by the model, was reduced significantly. It made me question how far I wanted to take over the outcome of the image. Was it fair to the model to be hiding the original image for the benefit of my own process? And where did Leonardo da Vinci come in during all of this? As a hint to him, I added his characteristic left-handed hatching (luckily I share this rare trait with him). I would also like to believe that Leonardo would have enjoyed the design in a way, as he worked on similar automatons and theatre contraptions in his lifetime.

Coming out of the flow of painting, and thinking about my contribution compared to that of the ML model and da Vinci, I decided to make an experiment to show my painting in contrast with the original. Instead of keeping to the existing colours of the image, I changed the hue and saturation of the painted elements. From there on I did another iteration of painting and erasing parts of my illustration. In a second iteration of giving my contribution more I created a line art of the painting, a technique that I usually use to construct my artworks.

So, I would like to come back to my goal, to combine technology and art through collaboration with a ML model. The process allows for new possibilities, especially forcing you to rely on your instincts and to trust the process. It can be beneficial as an inspiration or even a whole new style. Balancing this is the control that you have as an artist by training the model on a certain style, and the amount of painting you do after the image generation. A limitation in this, however, is the mining of enough data to create an interesting model.

Personally, I’m mostly enjoying the search to a good balance. How much do I want to show my own style, or do I want the generated image or the original artist to take centre stage? As artists I believe we get to choose our answers to these questions. If you are an artist working with these techniques or interested in exploring it with me, let’s get in touch! I would love to hear your thoughts.

‍