How to make interactive audiovisual effect in 5 minutes using Blender and MaxMSP

Blender is now a new force in 3D art. Although new but not young, about twenties.

blender2.8

In [Experimental Programming], I generally use Blender as a Python runtime environment out of the box, like:
Using Blender to run Python and visualizing the Fourier Series.

And this time, it is a simple and crude VJ / music visualization / audio-visual interaction software:

blender-osc-finger-s

Fingers hurt a little, change a prop:

blender-osc-mb1-s

One more:

blender-osc-mb2-s

Controlled by mobile of course:

blender-osc-bugosc-s

The OSC Controller above is BugOSC, which is based on “Wechat Mini Program” and developed by myself.

The full video only takes a few minutes:

(Chinese B站:https://www.bilibili.com/video/av92795224/


The following is a detailed explanation of the text.

Several core modules:

Blender Eevee Engine: responsible for real-time rendering
NodeOSC Blender addon: responsible for OSC data sending and receiving
MaxMSP: responsible for sound processing
BugOSC: OSC controller

Chapter One

Step 1, download the Blender Eevee demo “Wasp Bot”

https://www.blender.org/download/demo-files/

blender-osc-step1

Step 2, download and install NodeOSC addon

https://github.com/maybites/blender.NodeOSC

blender-osc-step2

Install the addon in Preferences:

blender-osc-addon1

After installing and modifying the options, remember to save the settings:

blender-osc-addon2

Step 3, config NodeOSC

NodeOSC opens from here:

blender-osc-pic-config1

Firstly make a simple Cube to practice.

Set the IP and port of the PC host in NodeOSC, and bind the OSC address to the Cube’s scale attribute:

blender-osc-pic-config2

Note the two fields datapath and property in Message handlers.

For the default Cube, NodeOSC has already filled it for you:

datapath:`bpy.data.objects['Cube']`
property: `location`

Just change the location in the property to scale, because here I show the scaling effect.

Step 4, make a simple OSC sender in MaxMSP

blender-osc-pic-max-sender

The /cube/scale corresponds to address on NodeOSC.

Step 5, got it!

blender-osc-max-scale

The real-time interaction between MaxMSP and Blender is completed, only two steps more than putting the elephant in the refrigerator.


However, this article is not over, and the excitement has just begun.

Chapter Two

Now let’s try Eevee demo:Wasp Bot

1. Open the Wasp Bot file downloaded before, and config NodeOSC

blender-osc-pic-eevee-demo.png

OSC is used to control the eyes, so first find the node of the eye parameter:

blender-osc-pic-eevee-demo-eye.png

Try adjusting the Color parameter in the node. Well, that’s the taste:

blender-osc-eye

How to fill in the parameters of Eye Object in NodeOSC?
Don’t panic, here I have a family heirloom recipe.

Click on the parameters you want to set in Color Node, and then go to the script information window to find the corresponding command line parameters!

blender-osc-pic-eevee-demo-eye-param.png

Then copy and paste into datapath and property of NodeOSC!

datapath:`bpy.data.materials["Head"].node_tree.nodes["Emission"].inputs[1]`
property: `default_value`

If the script information window is not open, you can refer to the related operations in the video.

2. config MaxMSP

Next, play a piece of rhythmic music in MaxMSP, or collect the sound of the microphone:

blender-osc-pic-max-sender-2.png

3. got it!

blender-osc-mb1-s

Chapter Three

Now try to control the model in Blender with mobile phone.

First, in NodeOSC, change the IP from 127.0.0.1 to the IP of the PC in the LAN, such as 10.0.0.11, or 192.168.1.123.
Make sure your phone and PC are in the same LAN.

blender-osc-pic-lan-ip

The OSC message address may also need to be changed to keep consistent with the OSC controller App in the phone.

For OSC App, you can use TouchOSC.
Of course, I use my BugOSC.

blender-osc-bugosc-record

I use the first Toggle button in BugOSC, and its address is /bug_slider1.

BugOSC will send two numbers 1 and 0 to NodeOSC according to the on / off state of the Toggle button.

The datapath of NodeOSC has been bound to the parameters of the model eyes, so as to achieve the blinking effect controlled by the mobile phone.

blender-osc-pic-max-toggle

blender-osc-bugosc-s

References


Talk is cheap. Show me the code!

The entire process is included in the video.

If you still need the source files (MaxMSP and Blender), you can get them from my Patreon.
There are many articles, patches, source code and some advanced Patron-only content there.

Source code: https://www.patreon.com/posts/34449785

中文源代码请访问微信公众号/知识星球:实验编程

You can find more resources in my site floatbug.com.

Your encouragement is my driving energy!

Cheers~

Contra

Website: floatbug.com
Github: github.com/avantcontra
Facebook: facebook.com/avantcontra
Twitter: twitter.com/avantcontra
Instagram: instagram.com/avantcontra
Patreon: patreon.com/avantcontra
微信公众号/知识星球:实验编程