Punctual & Pictures: Pixels and Variables

There are two more topics to cover before we move onto more active practice. The first is that we haven’t explained what the [fx,fy] we’ve been using to render pictures means. So fx and fy have special meaning in Punctual: they represent the coordinates of the current pixel being rendered, as scaled to be between -1 and 1. On the other hand, the coordinates we’re giving the tex operation are telling it what part of the picture to pick out. For example, if we did something like

tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [-0.5,0] >> video;

then we’re saying to create a texture that, for every pixel on the screen, draws from the color of the pixel halfway to the left of the photo and in the center. That’s a light green, if you look at the original photo I’m using, and sure enough if you run this code the whole screen becomes a light green.

So we’ve been doing something pretty sophisticated with tex! Whenever we say

tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [fx,fy] >> video;

we’re actually writing a program that says “for every pixel on the screen, look for the corresponding pixel in the original image and put that color here”. This also means we can start doing weird things. Like if you want to skew the image you can do something like

tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [fx+fy,fy] >> video;

And if we want to zoom in we just multiply both fx and fy by a number smaller than 1

tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [0.8*fx,0.8*fy] >> video;

What do you think happens if you multiply fx and fy by different numbers? It stretches or squishes the image!

You can also combine the tricks we’ve seen so far:

tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [fx + lo * fy, fy] >> video;

The last trick is one that will make doing more complicated things easier and that’s variables. In programming, a variable is a name you give something to reference it later. It lets you write more complicated code easily. For example, we can do something like

pic << tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [fx,fy];

rect [sin 0.1,0] [0.3,0.3] * pic >> video;
circle [sin 0.2, sin 0.3] 0.3*lo * pic >> video;

And if you run this code and play a beat with Tidal, you’ll see that the rectangle acts like a sliding window into the picture and that with every low pitched beat you can see the circle travelling around also revealing the picture.

So now we’re ready to try some practice exercises!