Instructions for audioToolkit

(produced at 15:29 UTC on 2021-10-19)

This task is part of ps05 which is due at 23:59 EDT on 2021-10-19.

You have the option to work with a partner on this task if you wish. Working with a partner requires more work to coordinate schedules, but if you work together and make sure that you are both understanding the code you write, you will make progress faster and learn more.

You can submit this task using this link.

Put all of your work for this task into the file audioToolkit.py
(which is provided among the starter files)

Note: Except to comment or uncomment tests, do not modify any of the starter code or the python files audio_config.py or waveTools.py.

General Overview

This task involves writing several functions that will process audio files with digital effects. Our program will work only on .wav audio files. Most of the information in a .wav audio file is simply a list of numbers that represent the audio signals of songs, speech, or noise. Yes, the audio you listen to on your computer is stored as a list of numbers!

Most audio files, including .wav files, have two lists of numbers, one for the left speaker and one for the right. These two lists are called channels and the numbers in each list are called samples. We will design most of our audio effects to work on a single channel and the starter code will handle processing both channels.

The effects you will write will just manipulate list of numbers, but the coolest part is that you can (optionally) apply the effects to some real audio files by using the supplied applyEffects function near the end of audioToolkt.py. You will need comment out or uncomment lines in applyEffects to select which effect(s) to apply to which file(s).

(Optional but very cool) Install the simpleaudio package to hear the sound files on your computer

In order for your computer to be able to play the sound files used in the applyEffects function, you will need to install an extra Python library called simpleaudio. Oherwise no sound will be played.

In Thonny, you can install simpleaudio using the following steps:

  1. In the "Tools" menu, select the "Manage packages" option.
  2. In the window that pops up, type simpleaudio into the search field and press ENTER.
  3. Information about the simpleaudio package should be displayed; click on the Install button to install it, which should be a fairly quick process.

Audio Effects to Write

As usual, you must document each function you write.

As a general rule, all effect functions must return a new list, and you must not modify the original channel or samples that is provided as an argument.

Also note that the rubric requires that your code does not waste fruit or waste boxes: you shouldn't ignore the results of a fruitful function, nor should you create any variables that you don't use. This applies to loop variables defined by a for loop as well as regular variables and parameters... if you need to create a for loop but you don't need to use the loop variable, use _ (a single underscore) as the name of the variable to indicate that you are intentionally not using it.

Due to the representation of floating point numbers in computers, certain mathematical calculations will lead to very tiny errors. These tiny errors are okay. DO NOT ROUND YOUR ANSWERS. If your answers differ only ever so slightly from our output below, then your answer is almost certainly correct. The version of optimism provided for this week should account for this.

Audio Effect 1: makeSofter

The function makeSofter is designed to make your audio file softer. makeSofter takes a channel (i.e., a list of numbers) and must return a new list where each value has been multiplied by a factor of 0.1. In general, the numbers representing audio (i.e., samples) should lie in the range of -1 to 1. Audio files are perceived as louder the closer those samples get to -1 and 1. Therefore, multiplying all values by 0.1 will narrow the range of the samples, making the audio softer. This example of makeSofter shows how it works.

Notice from the example how some of the answers have very small errors when attempting to multiply by 0.1. Again that is okay; we will be testing the accuracy of your results out to only 3 decimal places.

Audio Effect 2: chipmunk

The function chipmunk is designed to make everything sound like it was made by Alvin and the Chipmunks! Technically, all the pitches and frequency of the sound become higher. When applied to the human voice, it gives a chipmunk-like effect. chipmunk takes a channel and returns a new channel of only those samples at even indices. The other samples are discarded. This example shows how chipmunk works.

Audio Effect 3: removeVocals

The function removeVocals is designed to remove the vocal track from a song. The approach is crude and simple: it only works for certain songs when the vocal track is evenly split between the left channel and the right channel. Therefore, we can subtract one channel from the other and the remaining result should be the rest of the song. Accordingly, removeVocals requires two channels as arguments: the left and the right channel. removeVocals will return a single channel.

To illustrate the approach, let's take two channels as an example: a left channel of [0.1, 0.5, -0.2, 0.3] and a right channel of [-0.1, -0.6, 0.3, 0.1]. To remove the vocals subtract all the values from the right channel from the corresponding values of the left channel. That should yield a result of [0.2, 1.1, -0.5, 0.2]. You'll notice that with the right combination of numbers it is possible for the subtraction to produce a result outside of our preferred range of -1 to 1. The final step to the approach is to divide all the results by two. Therefore, the final result should be [0.1, 0.55, -0.25, 0.1].

You may assume that the left channel and right channels are the same length.

This example demonstrates how removeVocals works.

As a final note about this effect, it works decently well on sound file "distance.wav" for "The Distance" by Cake, but poorly on the others. Even so with the Cake example, you can hear that some of the other instrumentation is degraded, meaning that those instruments were relatively evenly split between the left and right channels. It is incredibly difficult to devise a strategy for this that works universally.

Audio Effect 4: reverse

The function reverse takes a channel and reverses the order of the samples. Like all effect functions, reverse must return a new channel rather than modifying the channel it is given. This is an old audio technique that can be used to create swells. It has also been used in some contexts to create hidden messages that can be revealed only when the listener plays the audio backwards. Check out this Wikipedia article on Backmasking.

For this function, you may not use the reversed function or the method .reverse(). In any other context, you should use either of these tools for a task like this, since it is usually inadvisable to try and rewrite something that has already been written for you. However, for pedagogical purposes, implementing reverse is great practice for working with loops and indices.

These examples show how reverse should work.

Audio Effect 5: twoSampleDelay

The function twoSampleDelay creates an underwater effect by removing the higher pitches in the audio file. In technical terms, this is called a Lowpass Filter. A filter is a processing technique that removes parts of an audio signal. Filtering is a common technique in lots of data processing especially for images and video.

twoSampleDelay works by adding together the original channel and the original channel shifted by two indices to the right. For example, suppose we had a channel of [0.1, 0.4, -0.1, 0.3]. A shifted version of that channel would be [?, ?, 0.1, 0.4]. We can see that the original 0.1 has now moved two indices to the right as well as the sample 0.4. Notice how -0.1 and 0.3 "fall off" and are no longer retained when we shift. We want to ensure that the shifted version is the same length as the original.

There is also a question about how to fill the two spots on the left of the shifted version, represented above with question marks. Those will be filled by arguments passed to twoSampleDelay. The parameter twoSampleBack will be used for index 0 and oneSampleBack will be used for index 1. Suppose then that twoSampleBack had a value of -0.9 and oneSampleBack had a value of -0.7. Then if channel [0.1, 0.4, -0.1, 0.3] is shifted, we would get [-0.9, -0.7, 0.1, 0.4].

The final step to the strategy is to add the original and shifted version together and divide by two. When we add, we add at each index as shown below to produce a new list.

   |  0.1 |  0.4 | -0.1 | 0.3 |
 + | -0.9 | -0.7 |  0.1 | 0.4 |
-----------------------------
   | -0.8 | -0.3 |  0.0 | 0.7 |

Then we divide each sample by two. In this example, that would give us a final result of [-0.4, -0.15, 0, 0.35].

twoSampleDelay should work with lists of size zero to two as well!

Some hints:

  • The strategy for twoSampleBack can be a little complicated. Make sure you understand how the shifted version is constructed before you start writing your code.
  • The above strategy can be implemented exactly as described using several loops. As an extra goal, use exactly one loop. If you attempt this, think about updating the parameters oneSampleBack and twoSampleBack as you progress through the loop to accomplish the same task. Reducing the number of loops in your program can pay big dividends in terms of efficiency, especially when working with large files like audio files.

These examples demonstrate how twoSampleDelay should work.

Ungraded Challenges

If you'd like to challenge yourself, here are a few more functions you could write. These are not part of the rubric and will not be graded; they are purely for your own benefit if you have extra time.

Audio Effect 6: ohYeah (ungraded)

Note: ohYeah is not part of the rubric and does not count towards your grade. It is an optional challenge problem.

The function ohYeah will be used to drop the pitch, similar to how chipmunk raised the pitch. There is no particular named associated with this effect but a famous example comes from the song "Oh Yeah" by Yello popularized in Ferris Bueller's Day Off. You can listen here.

ohYeah works by retaining all the samples of the original channel and adds new samples in between. The values of the new samples are always halfway between the original samples. ohYeah accepts two parameters: a channel and a sample called prevSample.

Suppose we call ohYeah with the channel [0.1, -0.3, 0.3] and a previous sample of 0.6 as in ohYeah([0.1, -0.3, 0.3], 0.6). We'll keep all the samples from the original channel and we will add values that are halfway between. Halfway between 0.1 and -0.3 is -0.1 and halfway between -0.3 and 0.3 is 0. Therefore the final output will contain [0.1, -0.1, -0.3, 0.0, 0.3].

The final step is to prepend a sample that is halfway between the previous sample and the 0th sample of the original list. In this example, 0.35 is halfway between the previous sample of 0.6 and the 0th index from the original channel of 0.1. Notice that ohYeah will always retain the original samples but double the length by adding samples in between the originals and attaching a new one to the front.

The image below gives a visual depiction of how each sample in the output is generated by the inputs. The blue dotted lines show how the output sample is the halfway point between two of the original samples. The red arrow shows where the sample from the original channel is preserved. A diagram showing how the previous sample and the samples from the
original channel are used to create the final output

This process of generating new sample points between others is called interpolation and is used often for many audio effects.

If you are curious why we need this previous sample, the starter code for audioToolkit.py processes an audio file in chunks. In order to connect each chunk smoothly and without any glitches, some of the effects like ohYeah need information from the previous chunk. Here we need the last sample of the previous chunk in order to properly calculate the halfway point to the start of the next chunk.

Audio Effect 7: crescendo (ungraded)

Note: crescendo is not part of the rubric and does not count towards your grade. It is an optional challenge problem.

The function crescendo creates a fade-in on the audio track for the duration of the song length. The simplest way to produce a fade-in is to generate a list of samples that incrementally grow from some start point up to some end point, very similar to what the range function does, except that they're not integers.

For example, suppose we want to generate 4 samples that grow from 0.2 up to 0.5. Such a list of samples would be [0.2, 0.3, 0.4, 0.5]. Note how we include both 0.2 and 0.4 at the ends, and that with 4 numbers in the list, there are 3 spaces between them, so the intermediate numbers are 1/3 and 2/3 of the way in between the endpoints. Also notice that the rate of increase is always the same (i.e., 0.1) from one value to the next.

To create a fade-in, then, we multiply our increasing samples by the original samples at each index and return a new list. For example, suppose our original samples are [0.2, -0.4, 0.6, 0.1] and we want fade-in from 0.2 to 0.5, then we would multiply each index as follows:

   | 0.2  | -0.4  | 0.6  | 0.1  |
 * | 0.2  |  0.3  | 0.4  | 0.5  |
---------------------------------
   | 0.04 | -0.12 | 0.24 | 0.05 |

The final result is [0.04, -0.12, 0.24, 0.05]. The length of the increasing samples should always match the length of the original samples.

The function crescendo takes three arguments: channel, startVolume, and endVolume. It should return a channel of the same length. startVolume represents the starting point of the ramping samples and endVolume represents the ending point of the ramping samples. startVolume and endVolume are values between 0 and 1 that represent what fraction of the song's overall volume should be achieved with 1 being full volume.

For Fun: Custom Audio Effect (ungraded)

Note: custom is not part of the rubric and does not count towards your grade. It is an optional challenge problem.

The function custom is a function to allow you to explore your own audio effect. But it can be interesting to see how manipulation of samples produces interesting audio effects. Remember though to keep your range of numbers between -1 and 1. Otherwise you could damage your speakers or your ears!

Do not add any parameters to custom otherwise it will not work with the starter code (you could also modify the starter code to compensate for this, but doing so is complex).

To process an audio file using your custom effect, make sure to call applyEffect with the second argument as "custom". For example,

applyEffect("poker.wav", "custom")

Show us all the fun ways you can experiment with music!

Using applyEffect and applyEffects

The starter code contains a folder called sounds which contains four audio excerpts from the following songs:

  1. "poker.wav": "Poker Face" by Lady Gaga
  2. "hello.wav": "Hello" by Adele
  3. "prayer.wav": "I Say A Little Prayer" by Aretha Franklin
  4. "distance.wav": "The Distance" by Cake

The function applyEffect is called to process one of the four audio files from above with one of the audio effects you will write. You do not need to write applyEffect. It has already been written for you. The function applyEffect has two parameters:

  1. The name of the audio file to process (e.g., "poker.wav")
  2. The name of the function that will process the audio as a string (e.g., "makeSofter")

applyEffect will process the sound file using the given effect, and store the processed sound file in the sounds folder with _proc after the base file name. For example,

applyEffect("poker.wav", "makeSofter")

will make the volume of the song "Poker Face" by Lady Gaga softer and place the new file in the sounds folder under the name "poker\_proc.wav". You could rename this to "poker\_softer.wav" if you want to save this file and prevent it from being overwritten by another call to applyEffect.

The starter code under applyEffects provides function calls to applyEffect to apply effects to the above songs. Uncomment any call to try your effect! You could also copy-paste them and modify the arguments to apply different effects to different audio excerpts.

In the code for applyEffect, you can see how it uses an if/elif structure to look at the string provided and decide which function to call.

Examples

makeSofter Examples

These examples show how makeSofter works. Notice how the number of samples in the output channel is the same as the number of samples in the input channel, but each number is scaled. Also note that we will ignore the small rounding errors that Python sometimes produces.

In []:
makeSofter([0, 0.1, 3, -1.3])
Out[]:
[0.0, 0.010000000000000002, 0.30000000000000004, -0.13]
In []:
makeSofter([-0.2, 0.4, 0.1, 0.2])
Out[]:
[-0.020000000000000004, 0.04000000000000001, 0.010000000000000002, 0.020000000000000004]
In []:
makeSofter([-0.2, 0.1])
Out[]:
[-0.020000000000000004, 0.010000000000000002]

chipmunk Examples

These examples show how chipmunk works. Notice that the result has exactly half as many samples (rounded up) as in the input channel.

In []:
chipmunk([0, -0.7, 0.3, 0.2])
Out[]:
[0, 0.3]
In []:
chipmunk([-0.2])
Out[]:
[-0.2]
In []:
chipmunk([-0.3, 0.4, -0.2])
Out[]:
[-0.3, -0.2]

removeVocals Examples

These examples show how removeVocals works. Notice that there are two input channels but only one output channel. Also note that when combining the channels, the results are divided by 2 after subtraction to ensure that no value will be above 1 or below -1.

In []:
removeVocals([0, 0.1, -0.1, 0.4], [-0.2, 0.2, 0.1, 0.3])
Out[]:
[0.1, -0.05, -0.1, 0.05000000000000002]
In []:
removeVocals([0.9], [0.2])
Out[]:
[0.35]
In []:
removeVocals([-0.2, 0.1], [-0.2, 0.1])
Out[]:
[0.0, 0.0]

reverse Examples

These examples show how reverse works.

In []:
reverse([0.1, 0.4, 0.2, -0.5])
Out[]:
[-0.5, 0.2, 0.4, 0.1]
In []:
reverse([-0.9, 0.4, 0.2])
Out[]:
[0.2, 0.4, -0.9]
In []:
reverse([-0.3])
Out[]:
[-0.3]

twoSampleDelay Examples

These examples show how twoSampleDelay works. Notice that the two extra arguments get combined with the first and second numbers in the channel, respectively. They are necessary because those first two samples don't have a sample in the channel that's two places to the left of them to combine with. Remember that the results come from adding two samples together and then dividing by 2, so for example, the first sample in the first result (0.25) is the result of the first input sample (0.1) plus the twoSampleBack argument (0.4) divided by 2. The next sample is (-0.2 + -0.3)/2, using oneSampleBack, and then samples from the channel itself can be used, so we have (0.9 + 0.1)/2 followed by (-0.1 + -0.2)/2 to complete the result.

In []:
twoSampleDelay([0.1, -0.2, 0.9, -0.1], 0.4, -0.3)
Out[]:
[0.25, -0.25, 0.5, -0.15000000000000002]
In []:
twoSampleDelay([0.9, 0.3, -0.2, 0.4, 0.5], 0.2, -0.1)
Out[]:
[0.55, 0.09999999999999999, 0.35, 0.35, 0.15]

Rubric

 
unknown Style Requirements
How your code is written.
 
unknown Core goals
Complete all core goals for core credit. Get partial credit for completing at least half, and more partial credit for completing at least 90%.
 
unknown All functions are documented
Each function you define must include a non-empty documentation string as the very first thing in the function.
 
unknown Procedure Requirements
What code you use to solve the problem.
 
unknown Core goals
Complete all core goals for core credit. Get partial credit for completing at least half, and more partial credit for completing at least 90%.
 
unknown Define makeSofter
Use def to define makeSofter
 
unknown Use a loop
Within the definition of makeSofter, use any kind of loop in at least once place.
 
unknown Define chipmunk
Use def to define chipmunk
 
unknown Use a loop
Within the definition of chipmunk, use any kind of loop in at least once place.
 
unknown Define removeVocals
Use def to define removeVocals
 
unknown Use a loop
Within the definition of removeVocals, use any kind of loop in at least once place.
 
unknown Define reverse
Use def to define reverse
 
unknown Use a loop
Within the definition of reverse, use any kind of loop in at least once place.
 
unknown Do not use the built-in reversed function
Within the definition of reverse, you must not use the built-in reversed function; instead, write your own code to reverse the samples using a loop.
 
unknown Do not use the reverse method of a list
Within the definition of reverse, you must not use the built-in reverse method of lists; instead, write your own code to reverse the samples using a loop.
 
unknown Define twoSampleDelay
Use def to define twoSampleDelay
 
unknown Use a loop
Within the definition of twoSampleDelay, use any kind of loop in at least once place.
 
unknown Extra goals
Complete all extra goals in addition to the core goals for a perfect score.
 
unknown Do not ignore the results of any fruitful function calls
According to the "Don't waste fruit" principle, every place you call a fruitful function (built-in or custom) you must store the result in a variable, or that function call must be part of a larger expression that uses its return value.
 
unknown Do not create any variables that you never make use of
According to the "Don't waste boxes" principle, every time you create a variable (using = or by defining a parameter for a function) you must also later use that variable as part of another expression. If you need to create a variable that you won't use, it must have the name _, but you should only do this if absolutely necessary.
 
unknown Define makeSofter
Use def to define makeSofter
 
unknown Use a loop
Within the definition of makeSofter, use any kind of loop in exactly one place.
 
unknown Define chipmunk
Use def to define chipmunk
 
unknown Use a loop
Within the definition of chipmunk, use any kind of loop in exactly one place.
 
unknown Define removeVocals
Use def to define removeVocals
 
unknown Use a loop
Within the definition of removeVocals, use any kind of loop in exactly one place.
 
unknown Define reverse
Use def to define reverse
 
unknown Use a loop
Within the definition of reverse, use any kind of loop in exactly one place.
 
unknown Define twoSampleDelay
Use def to define twoSampleDelay
 
unknown Use a loop
Within the definition of twoSampleDelay, use any kind of loop in exactly one place.
 
unknown Product Requirements
Your code's result values.
 
unknown Core goals
Complete all core goals for core credit. Get partial credit for completing at least half, and more partial credit for completing at least 90%.
 
unknown makeSofter returns the correct result
The result returned when your makeSofter function is run must match the solution result.
 
unknown chipmunk returns the correct result
The result returned when your chipmunk function is run must match the solution result.
 
unknown removeVocals returns the correct result
The result returned when your removeVocals function is run must match the solution result.
 
unknown reverse returns the correct result
The result returned when your reverse function is run must match the solution result.
 
unknown twoSampleDelay returns the correct result
The result returned when your twoSampleDelay function is run must match the solution result.
 
unknown Extra goals
Complete all extra goals in addition to the core goals for a perfect score.
 
unknown makeSofter returns the correct result
The result returned when your makeSofter function is run must match the solution result.
 
unknown makeSofter must not modify provided channel data.
We will call makeSofter to make sure it does not alter values in the channel(s) it is given.
 
unknown makeSofter must return a fresh list.
We will call makeSofter to make sure it returns a fresh list instead of one of the channel(s) it was given.
 
unknown chipmunk returns the correct result
The result returned when your chipmunk function is run must match the solution result.
 
unknown chipmunk must not modify provided channel data.
We will call chipmunk to make sure it does not alter values in the channel(s) it is given.
 
unknown chipmunk must return a fresh list.
We will call chipmunk to make sure it returns a fresh list instead of one of the channel(s) it was given.
 
unknown removeVocals returns the correct result
The result returned when your removeVocals function is run must match the solution result.
 
unknown removeVocals must not modify provided channel data.
We will call removeVocals to make sure it does not alter values in the channel(s) it is given.
 
unknown removeVocals must return a fresh list.
We will call removeVocals to make sure it returns a fresh list instead of one of the channel(s) it was given.
 
unknown reverse returns the correct result
The result returned when your reverse function is run must match the solution result.
 
unknown reverse must not modify provided channel data.
We will call reverse to make sure it does not alter values in the channel(s) it is given.
 
unknown reverse must return a fresh list.
We will call reverse to make sure it returns a fresh list instead of one of the channel(s) it was given.
 
unknown twoSampleDelay returns the correct result
The result returned when your twoSampleDelay function is run must match the solution result.
 
unknown twoSampleDelay must not modify provided channel data.
We will call twoSampleDelay to make sure it does not alter values in the channel(s) it is given.
 
unknown twoSampleDelay must return a fresh list.
We will call twoSampleDelay to make sure it returns a fresh list instead of one of the channel(s) it was given.