Hacking Heuristics

SENSING TOGETHER

Today was the first test to get the sensor's data over to a python server which was receiving this wirelessly, and visualising this on the graphs seen in the image where each person has their own colour.

ALL HOOKED UP!

Here I am, all hooked up to the device with one of the prototype headsets - this will measure my heart rate, sweat response, eye movement, and brain activity and provide me with 'translated' information in the form of electrical stimulation, sound, and hot and cold on my forehead and the back of my neck.

ATTEMPTS AT CABLE MANAGEMENT AND LONGER WIRES

TWO BABIES!

Right now there has been a lot of developing and testing with hardware and prototypes but I now need to pull the focus back to the initial intentions of the project - the interaction. Only I couldn't do that very well with only parts of hardware. Now finally I have one pair, meaning at least an experience with two people.

MANY WIRES

I think I need some serious cable management - but the issue is also that the sensors are sensitive to interferences - including movement or tangling/crossing each other

HELLOO

Adding a screen to display information and data - a simple o-led 0.91 screen.

NEW OUTPUT ADDITION!

After some discussion with Marcel, I have decided to add another output to the experience: temperature. The little white square at in the top corner of the image is something called a Peltier device - they often use it in mini fridges for example. When you run electricity through it, one side of the peltier gets extremely cold, and the other extremely hot. Using an H-bridge (a device which is able to swap the polarity (+) (-) of the electricity) I am able to create both hot and cold sensation on...

FIRST COMPLETE(ISH) DEVICE

This includes 2 bioamp sensors, one for EEG(brainwaves), and one for EOG(eye movement), two objects to hold which act as GSR (sweat response) sensors, and the stimulator and power source.

NEW RUBBER STIMULATION ELECTRODES

Although these were actually designed for breast stimulation, when cut in half they make perfect ear cuffs for the stimulation I am looking for.

PULSE SENSOR ADDITION

Since it is such an easy thing to measure, and it was missing from the set up, I decided to add the simple Pulse Sensor, which uses light to measure heart rate.

EEG FILTERING !

Today we had a breakthrough! I have been working with a creative coder - Arran Lyon - to help with filtering the data coming in from the sensors, and to set up the framework for the machine learning. This filtering is used to extract the information which represents specific aspects of the internal responses occuring in the body. What you see in the image is a segment of recorded data with sensors connected to my prefrontal cortex. For this I used 1024Hz sampling rate and Arran applied a band pa...

EOG WITH CUSTOM HEADSET

BUMPER CAR WIRING

Imagining how this 'partly wireless' experience could be done in such a way that there is no wires to trip over or get tangled, or that the users have to wear some sort of backpack which would contain all the devices. One such option would be to have a system almost similar to how bumper cars work in the sense that only the wires go down to the headset, and the rest is held safely on rails above the users.

NEW RUBBER SUCTION ELECTRODES

Unfortunately they are a little too big for what I am looking for, but the experience is great!

NEW REINTERPRETATION

So after making some big steps with the hardware side of the project, it becomes clear that there is the option of the experience becoming partly wireless - this will not only allow for movement of the body caused by the electrical stimulation, but also will enable a new level of 'freedom' of the users whilst they are being 'controlled'. It will offer the possibility of exploring more than just the senses of a static body, but rather of a moving body situated in space, with other...

GSR-SENSORS & SPEAKERS

Using copper sheets as sensors that the user will hold in their hands, they will also contain speakers which are connected to the stimulator, so that each user will have not only the physical feedback, but also the auditory feedback. My hope is that although it is very soft, perhaps when there are multiple of these sounds playing at the same time, it creates a collective symphony that represents the physical stimulation each person is feeling, but is not able to share with the others were it not...

FIRST UNIT PROTOTYPE

In the image you can see the NeurostimDuino, two 18650 Li-on batteries that power the stimulator, an ESP32 used for controlling the stimulator and reading the sensor data, and one bioamp sensor which I have attached to a prototype headset via button snaps.

NEUROSTIMDUINO

Introducing the NeuroStimDuino! Developed by Neuralaxy - https://www.neuralgalaxy.com/ - this device offers a super affordable (if you compare it to the devices used in labs) way to control the frequency, strength, pulse width, and duration of electrical stimulations with a microcontroller! Their device and software is all open source too! https://github.com/neuralaxy/NeuroStimDuino The device also came from the same site that the little sensors came from - https://www.crowdsupply.com/ ...

BRUISES FROM SUCTION ELECTRODES

I'll have to reconsider this one..

SUCTION EFFECT WITH CHECK VALVES

PROTOTYPE HEADGEAR #7

PROTOTYPE HEADGEAR #6

PROTOTYPE HEADGEAR #5

PROTOTYPE HEADGEAR #4

ALTERNATIVE HEADGEAR PROTOTYPE #3

HEADGEAR PROTOTYPE #2

PROTOTYPE HEADSET #1

COPPER OXIDATION SENSORS

UPDATED SENSORS

After much experimentation and prototyping and testing, I have decided to narrow down on the types of sensors and the location of the electrical stimulation. For the sensors I have decided: 1. Pre-frontal cortex activity 2. Eye movement 3. Heart Rate and Sweat Response And for the stimulation location - because of the experience of having my cerebellum (the back of my brain responsible for balance) and because of the difficulty to have contact with the skin on most parts of the body, I...

PROTOTYPE HEADSET

Using these suction electrodes, can all the desired locations on the head be contacted. Here is a first prototype!

TAKING FORMS

Using polymorph, a reusable moudable plastic, the placement of these electrodes are beginning to take form in a structure.

SUCTION MARKS

Looking back at the initial illustrations where the work took the form of an octopus, seeing the marks these suction electrodes leave act as a sort of indicator of a new experience, a marking of a new communication system.

ELECTRODE PROTOTYPES

SUCTION ELECTRODES

In order to have sensors and electrodes work properly, without using single use sticky electrodes, looking into suction electrodes seems like a very viable option, allowing visitors to move freely and still have effective placement.

TRANSLATING BIOSIGNALS

Today I met with Arran Lyon, a creative coder with whom I have also worked with in the past. Together we discussed how to develop the 'translator' algorithm which will transform biosignals into electrical stimulations. Because I do not want the neural network to work with pre-defined data sets, the algorithm must learn by 'doing', and then determine if it's translation was effective by monitoring the biosignals. However, such an algorithm needs to still have a goal, in o...

STIMULATING THE BRAIN WITH 'NOISE'

On the screen you can see a visualisation of the impulses that were sent to my head. What was so interesting was that usually you receive a pulse, at a certain frequency, so its very linear. This, however, was 'noise', meaning that it was many frequencies at the same time, in a random order. The experience of this was also very bizzare. Unlike the other impulses I had felt that made me sway from side to side, this one was much less of a bodily reaction and more of a visual and physical r...

VIDEO -> BRAIN STIMULATION

Link to video of stimulation taking place, while Patrick explains what is happening. If you look closely you will notice my body swaying from side to side - responding to the stimulation delivered to the left and right side of my head. Stimulating the Brain from Marlot Meyer on Vimeo.

BACK AT THE LAB FOR SOME STIMULATION

Today I met with Patrick (the same person who created the balancing robot -- see earlier post), who also works at the department of neuroscience at the Erasmus medical centre in Rotterdam. His area of research mainly focuses on our experience and perception of balance. By placing two electrodes on either side of my head, behind my ears, we stimulated a part of my brain responsible for these 'balance resposponses', to simluate the sensation of rotating my head.

PRE FRONTAL CORTEX BREAST ELECTRODES

It might be a strange sensation to rest your head on a breast, just like almost all of us once rested our head on our mothers breast...

SOFT ELECTRODES

Another option for keeping the skin in contact with the electrodes would be to create soft, comfortable sufaces which take in the pressure of the body, allowing the person to be comforatble, and to make efficient contract with the electrodes. Here is a silicone breast implant, with two metal electrodes attached to it. The feeling in and of itself is silightly uncomfotable but very pleasent.