Machine Learning Abstracts

Creating abstract painting composition using TensorFlow Classification

Classification

Classification is my first use of Artificial Intelligence in my artistic process. I have found it to be a powerful tool for art. As an artist, I am constantly experimenting, trying new things and iterating. I think the most accomplished artists keep painting very similar things repeatedly that they become known for. I have been creating large datasets of similar images of my art that are then processed and sorted by algorithms. I found using an image processor to sort through digital concept ideas allowed me to create more ideas faster, better understand my artistic preferences and create better art. Below is a description of how I use TensorFlow Classification to create color-field abstract paintings.  

Machine Learning Abstract Painting Series by Tech Artist Joanne Hastie
Figure 1: Hand painted abstract paintings created using compositions created with Python code and ranked with Google TensorFlow classification trained with my preferences

 

Starting Point

My starting point for this project was an abstract painting I create one evening that had 12 different smaller compositions of varied shapes. I love minimalism painting that focuses on color and value, so I was excited and hopeful to create new abstract artwork in this vain.

I photographed the painting and used python code to extract shapes from my existing abstract paintings. The extracting shapes was done with a K-Means clustering algorithm that examined every pixel in the image and graphed it on 3 axes (R, G, B), it then identified the most common clusters and those were the colors in the image.  It would re-process the image only using the pixels from that color to create a new PNG file that only included that shape, this process was

Original painting that was used to extract all of the shapes for the compositions.
Figure 2: The original abstract painting I created with 12 abstract sketches

 

Composition Generation

The simplest code, but most exciting part was creating random abstract compositions with the shapes. I wrote code layered the shapes by randomly moving, rotating, resizing and recoloring the shapes on top of each other. Each iteration was saved as a separate file. I would let the software run overnight creating thousands of potential abstract compositions. At one point I had over 40,000 composition files that I was sorting through.

Classification

Data Classification

Early in my research into classification I was classifying the data used to create the composition. For instance the R,G,B values, the size of the shapes, the position of the shapes. I had just taken a Udemy course on Python & Machine Learning so I was keen to sort through eveyr parameter. My first set of 12 paintings were created by doing this and I loved the results (perhaps due to the new project)… but as my dataset grew I quickly realized that the raw numbers used to create a painting composition had very little correlation to whether the actual painting composition was interesting. My early work even had me using Linear Regression, Random Forests and Decision Trees with all the data and then averaging the results but the results were grim.

TensorFlow

I then stumbled on Google’s open-source TensorFlow. The basic examples of TensorFlow when you start learning to program is using MNIST. MNIST is a dataset of hand written numbers that can then be sorted or re-generated. It is like the “Hello World” of Machine Learning. Hello World refers to the quick code used in programming to check everything is working as expected. With all the different modules to sort through to start programming in Machine Learning it is a time saver to make sure MNIST works before you go deep into hacking code together.
For my painting classification I downloaded TensorFlow Inception v3. Inception v3 is a pre-trained database of images that can then be quickly retrained with your specific task. I retrained the Classification with Yes (image compositions I liked) and No (image compositions I did not like). Each time it ranked an image high that I did not like (or ranked an image low that I did like) I would retrain the classifier. Retraining could take upwards of 3 hours on my laptop, then after training, one “tests” the algorithm which give a rating for each image (% Yes and % no). I used the % as a score for ranking the compositions.
One thing that I quickly learned by then painting some of the compositions by hand, is that a composition that stands-out in a sea of digital images doesn’t often make a good painting. Likewise a good painting might not be interesting as a digital image. So I found my preferences are continually changing as this project progresses.

Titles of the paintings

The paintings are titled based on Google’s untrained TensorFlow Inception Classification model. Untrained means it is downloaded from Google and that I have not trained the model for my specific classification task. Their model classifies images to daily objects, so the software suggest what the abstract painting could be. It is interesting to take a second look at abstract art and consider how it’s been seen by the code that has no bias. For example, hockey puck, bird house, sweet potato, bulletproof vest are only some of the objects it has found in the paintings.

Christmas Stocking Abstract Painting by Tech Artist Joanne Hastie

Shipping Container, Abstract Painting created by tech artist Joanne Hastie

Figure 3: Original paintings with their detailed TensorFlow Inception results

Resources for this project

 

If you want to dig into some of the resources I used in this project here are the links: