Interactive Art Installation:
notMONDRIAN is an interactive installation. The inspiration comes from the famous red, yellow-blue blocks and grid abstract paintings by the Dutch artist Mondrian during the 1930's. In notMONDRIAN, the artist first breaks the static grid into non-static, followed by the introduction of movement detecting technology, therefore the audience can move different parts of the body to control the positions of the red, yellow or blue blocks on the screen. At the same time, the magnitude of the action will also affect the rhythm and frequency of the sound produced by these patches during the collision. Traditional paintings mainly rely on the coordination of the artists' hands and their tools, as well as professional painting techniques and experiences. What a traditional painting could present to the audience is the last frozen second of the artist's creative process. notMONDRIAN extends painting into the process of "painting" as improvised dance-style performances. Meanwhile, in notMONDRIAN, the painting is no longer presented in a single still way in front of the audience, the audience is being involved. Since 2013, the artist began to use interactive installations as a means to explore the forms of stitching, penetration, and even conflict between art and technology. In addition to thinking about the relationship between art and technology in a broad sense, in this work, she hopes to explore how interactive technology can bring new forms of presentation and experience to traditional paintings, and how to make the audience from "the bystander" of a painting to become "the participant" and "the co-creator" of an experience.
notMONDRIAN was invited and exhibited in BEIJING DESIGN WEEK (Beijing, China) during 23 September - 7 October in 2017.
In the early versionof 2019, the installation notMONDRIAN uses a Kinect 2 device as input, which is a special type of camera that can not only film a video but also detect things in the video. It can detect persons, faces and the depth of the space in front of the camera. The installation uses the person detection and gets data about up to 3 persons in the camera view. This data is a set of 3D coordinates per person that give the accurate position of head, shoulders, arms, hands, hips, legs, feet and all joints that connect them.
In 2019, the newest version of notMONDRIAN uses this data to calculate four vectors (think of pointed arrows) from the center of the body of a person to the hands and feet. These "arrows" have a length and direction. To better process the information we normalise the vectors, so their length is always same and therefore can be easier compared. Then the machine learning part start:
We control two different objects in the screen with the input of the four "arrows": Let's talk about the bigger object first. It is positioned on the screen by a two-dimensional projection of the 8-dimensional input data. Why 8 dimensions? Because each of the four "arrows" has two components: the x and y coordinates, and we are ignoring the z coordinate here. The two-dimensional projection is a mathematical operation that reduces the input of 8 dimensions into two dimensions, so we can visualise it on the screen easily. On top of this dimension reduction, which uses a Random Projection algorithm, we smoothen the movement of the object according to the projected coordinates. That's it.
The second smaller object is locked to the movement of the first object and we are simply changing its size according to the novelty of how the person in the video moves. What do we do to make this work? We train a neural network with the 8-dimensional input data and it will give us a rating how "new" or "surprising" the current input is. This rating is then used to calculate the size of the second object. The precise algorithm to do this is Neural Gas, a "simple algorithm for finding optimal data representations based on feature vectors" (Wikipedia). Think of a map that is constructed from the input data and that can be used to determine the distance of any new input to the existing points on the map. If this distance is larger then the map is more surprised about the new data, if it is smaller, then less.
This explanation is about a single player, there are different rules for two or three players. notMONDRIAN will adapt fluently between different rules depending on how many people are using the installation.
The new AI version of notMONDRIAN is invited by Kulturhack and exhibited at Kulturhuset i Ås, Olso, Norway, from 30 of March to 2 of April in 2019. Related link: https://www.kulturhack.no/l/kinesisk-kunstner-med-kunst-skapt-av-ai/?fbclid=IwAR30OVyGzuPNCHUIlqxfPO3GipWfhcgkF6DemzH64qEY0s-jVTHDTgixSz0
notMONDRIAN was set up and experienced in LG -1.17a LaPlace in Eindhoven University of Technology (the Netherlands). The entire installation space was 7m*8m*6m.
notMONDRIAN exhibition decumetation in Beijing Design Week 2017.
notMONDRIAN Beijing trip documentation.
notMONDRIAN was exhibited at BEIJING DESIGN WEEK 2017 and opened to the public in #49 Moshikou Street, in Beijing, China. The entire installation space was 2.5m*6m*3m.
notMONDRIAN is invited by Kulturhack and exhibited at Kulturhuset i Ås, Olso, Norway, from 30 of March to 2 of April in 2019.