{"id":18308,"date":"2024-12-05T14:47:16","date_gmt":"2024-12-05T14:47:16","guid":{"rendered":"https:\/\/astoryisnotatree.net\/?p=18308"},"modified":"2024-12-05T14:47:16","modified_gmt":"2024-12-05T14:47:16","slug":"art-from-code-audio-reactive-images","status":"publish","type":"post","link":"https:\/\/astoryisnotatree.net\/?p=18308","title":{"rendered":"ART FROM CODE: Audio Reactive Images"},"content":{"rendered":"\n

Since the beginning of the semester, I knew I wanted to work with audio-reactivity. Initially using the Processing Library called Minim, I found resources to be difficult to understand and implement considerin the time I was able to allocate to working on this. As such I turned to the processing website and decided to switch to using Processing Sound, which is Processing’s native sound library. This came with the added bonus of reference and example code directly in Processing itself. I began by trying to understand the code’s logic by making small changes and observing what responds. <\/p>\n\n\n\n

The project itself contains an image rasterizer that changes the background image’s pixel count as you move your mouse around the screeen. In the meantime, music can be heard playing and several layers of audio reactive “streams” and shapes sit on top of the images actively borrowing colour from the image as they are moved around. The bezier, is mapped to the mouse position and will move as the mouse is moved. Using FFT(Fast Fourier Transform) data, the amplitutde and shape of the bezier pulses to the music. On top of the bezier is two waveform visualizations that show the soundwave at 256 samples. <\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

These visualizations are not tied into any particular audio. In fact, the audio itself can manipulated using mouse and keyboard. When the mouse is left clicked, the sample speeds up, and when right clicked it slows down. The visuals will reflect these audio changes. Additionally, there is a bandpass and highpass filter mapped to the screen, so upon pressing the space key, moving the mouse around the screen also changes the frequencies that are playing. <\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

<\/p>\n\n\n\n

import processing.sound.*;\n\nPImage img;\nSoundFile sample;\nBandPass bandPass;\nLowPass lowPass;\nFFT fft;\nWaveform waveform;\n\n\/\/ Number of waveform samples\nint samples = 256;\n\nint bands = 512; \/\/ FFT bands\nint doubleCup = bands \/ 4;\nfloat smoothingFactor = 0.4; \/\/ FFT smoothing factor (1: highest, 0.1: lowest)\nfloat playbackRate = 1.0;\n\nvoid setup() {\n  size(1000, 800);\n  img = loadImage(\"SLO.jpg\");\n  img.resize(1000, 800);\n\n  sample = new SoundFile(this, \"please remix.mp3\");\n  sample.loop();\n\n  fft = new FFT(this, bands);\n  fft.input(sample);\n\n  waveform = new Waveform(this, samples);\n  waveform.input(sample);\n\n  bandPass = new BandPass(this);\n  bandPass.process(sample);\n\n  lowPass = new LowPass(this);\n  lowPass.process(sample);\n}\n\nvoid draw() {\n  background(11, 88);\n  noFill();\n  filter();\n  render();\n  blendMode(DIFFERENCE);\n  bezel();\n  waveTube();\n}\n\nvoid filter() {\n  if (keyPressed) {\n    float ratio = float (height)\/float (width);\n    float freq = map(mouseX, 0, width, 250, 10000); \/\/ Frequency range: 250 Hz to 10 kHz\n    float bw = map(mouseY, 0, height, 10, 5000);   \/\/ Bandwidth range: 10 Hz to 5 kHz\n    float res = map(ratio, 0, ratio, 0.1, 0.4);\n\n    bandPass.freq(freq);\n    bandPass.bw(bw);\n    bandPass.res(res);\n\n    float lowpassFreq = map(mouseX, 0, height \/ 2, 250, 10000);\n    lowPass.freq(lowpassFreq);\n    lowPass.res(res);\n  }\n}\n\nvoid render() {\n  float tilesX = map(mouseX, 0, width, 10, 100); \/\/ Number of horizontal tiles\n  float tileSize = height \/ tilesX;\n\n  for (int y = 0; y < img.height; y += int(tileSize)) {\n    for (int x = 0; x < img.width; x += int(tileSize)) {\n      color c = img.get(x, y);\n      float b = map(brightness(c), 50, 200, 1, 0.5);\n\n      pushMatrix();\n      translate(x, y);\n      fill(c);\n      rect(0, 0, b * tileSize, b * tileSize);\n      popMatrix();\n    }\n  }\n}\n\nvoid bezel() {\n  fft.analyze();\n\n  for (int i = 0; i < doubleCup; i++) {\n    float bandValue = fft.spectrum[i] * (height * 290);\n    stroke(255, 255, 255, i % 2 == 0 ? 68 : 55);\n    bezier(\n      mouseX - i * 10, height \/ 2,\n      width \/ 2 - 100, height \/ 2 - bandValue,\n      width \/ 2 + 100 + 55, mouseY + bandValue + 55,\n      width \/ 4 + i * 10, height \/ 1\n    );\n  }\n}\n\nvoid waveTube() {\n  waveform.analyze();\n\n   \/\/ Perform waveform analysis and FFT analysis\n  waveform.analyze();\n  \n  for (int j = 0; j < samples -1; j++) { \/\/offset waveform\n    float x3 = map(j, 0, samples, 0, width);\n    float y3 = map(waveform.data[j], -1, 1, height \/ 2 + 450 , height \/ 2 - 400);\n    float x4 = map(j , 0, samples, 0, width);\n    float y4 = map(waveform.data[j], -1, 1, height \/ 2 + 450 , height \/ 2 - 400);\n\n    \/\/ Color based on amplitude of waveform\n    float amplitude = abs(waveform.data[j]);\n    int alphaValue = (int) map(amplitude, 0, 0.5, 85, 200); \/\/ Adjust transparency\n    strokeWeight(4);\n    stroke(lerpColor(color(255, 235, 250), color(255, 235, 250), amplitude), alphaValue); \/\/ Ghost trail effect\n    line(x3, y3, x4, y4);\n }\n for (int i = 0; i < samples - 1; i++) {\n    float x1 = map(i, 0, samples, 0, width);\n    float y1 = map(waveform.data[i], -1, 1, random(height \/ 2 - 400, height \/ 4 -400), height \/ 2 + 400);\n    float x2 = map(i * 0.2, 0, samples, 0, width);\n    float y2 = map(waveform.data[i], -1, 1, random(height \/ 2 - 400, height \/ 4 -400), height \/ 2 + 400);\n\n    \/\/ Color based on amplitude of waveform\n    float amplitude = abs(waveform.data[i]);\n    int alphaValue = (int) map(amplitude, 0, 0.5, 75, 220); \/\/ Adjust transparency\n    strokeWeight(5);\n    stroke(lerpColor(color(120, 45, 170), color(130, 15, 185), amplitude), alphaValue); \/\/ Ghost trail effect\n    line(x1, y1, x2, y2);\n  }\n}\nvoid mousePressed() {\n  if (mouseButton == LEFT) {\n    playbackRate += 0.1;\n  } else if (mouseButton == RIGHT) {\n    playbackRate = max(playbackRate - 0.1, 0.1);\n  }\n  sample.rate(playbackRate);\n}<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
Since the beginning of the semester, I knew I wanted to work with audio-reactivity. Initially using the Processing Library called Minim, I found resources to be difficult to understand and implement considerin the time I was able to allocate to…<\/p>\n","protected":false},"author":410,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[365],"tags":[],"class_list":["post-18308","post","type-post","status-publish","format-standard","hentry","category-new-genres-2","wpcat-365-id"],"_links":{"self":[{"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=\/wp\/v2\/posts\/18308","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=\/wp\/v2\/users\/410"}],"replies":[{"embeddable":true,"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=18308"}],"version-history":[{"count":2,"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=\/wp\/v2\/posts\/18308\/revisions"}],"predecessor-version":[{"id":18547,"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=\/wp\/v2\/posts\/18308\/revisions\/18547"}],"wp:attachment":[{"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=18308"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=18308"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/astoryisnotatree.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=18308"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}