irrKlang object (v1.0.2)

an advanced sound extension for Clickteam Fusion based on the irrKlang audio library (© Nikolaus Gebhardt)

Introduction

The irrKlang object is an advanced sound extension for Clickteam Fusion. From it's features is allowing Fusion developers to feasibly implement real-time 3D and 2D sound with realistic effects, recording the microphone, playing it back and saving it locally to a WAV file.

Please note that this extension must not be used in commercial products (see the irrKlang license). If you wish to do so, you must purchase the appropriate license from Ambiera and recompile the extension from source with your irrKlang Pro files (see the extension source code here).

Note: This extension currently doesn't include native support for Fusion's embedded binary files feature. To use these, you must extract/release your binary files yourself and pass the temporary filenames for playback. Refer to The objects -> Inventory -> Special objects -> The Special Object -> Actions section in the Clickteam Fusion help file to learn more about this.

How to use : Actions

- Creating the irrKlang device (optional)

Before doing anything with the object, you must create the irrKlang device first. By default, it is created automatically (see object properties), but you can create it manually in events if you want to do specific things like play sounds using a specific device driver and on a specific sound device. Assuming you're already familar with placing events in Fusion:

Add the Create irrKlang device action:

This action contains 2 parameters:

Enter the device driver used for sound output - this parameter requires you to pass in the device driver used to output sounds. Options are "Automatic", "DirectSound", "DirectSound8", and "WinMM". If you pass in other than these, the object will just use the device driver selected in the object properties.

The recommended device driver to use is DirectSound8, as it is the most feature-rich out of all the other drivers.
Enter the device ID of the sound device to use - this parameter requires you to pass in the ID of the computer's sound device you want to output sounds to (or just set it to "0" to use the default). To get the ID of a sound device, you use the Sound devices > Get sound device ID expression. Then you pass in the index of the sound device (1 would be the first device, 2 would be the second and so on).

- Playing a sound

Playing a sound can be done as follows:

Add the Play a 2D sound (or Play a looping 2D sound if you would like it looped) action in the 2D sounds submenu:

Playing sounds contains these 3 parameters:

Enter the path to the audio file to play - this is a filename parameter which requests the filename of the sound to play. Supported formats are .WAV, .OGG, .MP3, .FLAC, .MOD, .IT, .S3D and .XM files.
Enter an ID used to identify the sound - this is used if you want to manipulate the sound using a numeric ID instead of the sound's path name. This is optional, you can keep it at 0 if you won't use IDs.
Enable sound effect support for this sound? - this is used if you want to enable real-time sound effect support for the sound you're playing. Set this to 1 if you want it enabled, otherwise pass in 0.

- Playing a 3D sound

You can play a 3D sound to simulate sound in a three-dimensional space. Playing a 3D sound can be done as follows:

Add the Play a 3D sound (or Play a looping 3D sound if you would like it looped) action in the 3D sounds submenu:

Unlike 2D sounds, playing 3D sounds contains these unique parameters:

Enter the X position of the sound - the virtual X position of the sound.
Enter the Y position of the sound - the virtual Y position of the sound.
Enter the Z position of the sound - the virtual Z position of the sound.

3D sounds can be manipulated using the 7 actions listed below:

Set listener position/rotation/velocity - this action allows you to set the position, rotation, and velocity (used for doppler effects) of the listener. If you're using this extension in a game, think of the listener as the player.
Set 3D sound position - this action sets the position of a 3D sound at runtime.
Set sound minimum distance - this action sets the minimum distance of a 3D sound at runtime. It is the distance of which a sound stops getting louder.
Set sound maximum distance - this action sets the maximum distance of a 3D sound at runtime. This value usually does not need to be set, this will stop the sound from attenuating after reaching the maximum distance you set.
Set rolloff factor - this action sets the rolloff factor which influences the amount of attenuation applied to 3D sounds. It can range from 0.0 to 10.0, where 0 is no rolloff, and 1.0 is the default. 2.0 would mean two times the real-world rolloff.
Set doppler parameters - this action sets parameters related to doppler effects applied to 3D sounds. The doppler factor is a value you should set between 0 and 10 which multiplies the doppler effect. Default value is 1.0, and 10.0 would mean ten times the real-world doppler effect. The distance factor is the number of meters in a vector unit. The default value is 1.0. Doppler effects are calculated in meters per second, changing the distance factor manipulates this, and all velocities and positions of 3D sounds will be influenced by this. If you want to measure doppler effects in foot instead of meters, you should set this parameter to 0.3048.
Set sound velocity - this action sets the velocity (in other words, speed) of a 3D sound. This is used only if you're using doppler effects, the pitch of the sound changes based on the velocity and doppler parameters you set.

- Manipulating sounds

3D and 2D sounds share actions for general sound manipulation. They are listed below:

Set master volume - this action sets the volume influenced by all sounds.
Set sound volume - this action sets the volume of a sound by it's pathname. Values range from 0 to 1, meaning low to high volume. This action also accepts floats.
Stop a sound - this action stops a sound by it's pathname.
Pause a sound - this action pauses a sound by it's pathname.
Unpause a sound - this action unpauses a sound by it's pathname.
Pause all sounds - this action pauses all sounds being played.
Unpause all sounds - this action unpauses all sounds being played.
Set sound playback position - this action sets the playback position (not to be confused with 3D sound position) of a sound in milliseconds by it's pathname.
Set sound playback speed - this action sets the speed of a sound by it's pathname. Default value is 1.0. Note that changing the speed also changes the pitch of the sound.
Set sound pan - this action sets the pan of a sound by it's pathname. Values range from -1 being the far right, 1 being the far left, and 0 being center. This action also accepts floats.

NOTE: You can also manipulate sounds by referring to them using a numeric ID specified when initially playing the sound rather than their pathname. To do this, you use the same actions but in the Modify sounds by ID submenu instead. This applies to Conditions/Expressions as well:

- Realtime sound effects

You can apply realtime sound effects such as chorus and reverb to playing sounds and manipulate their sound effect parameters at runtime.

The full list of available sound effects are:

Chorus
Compressor
Echo
Distortion
Flanger
Gargle
Interactive 3D Audio Level 2 reverbation
Parametic equalizer
Waves reverb

IMPORTANT NOTE: Sound effects are only available when using the DirectSound8 device driver.

You can apply these effects to sounds by using the Enable XXX sound effect for a sound action (where XXX is the name of the sound effect you want to use).

If a sound has a sound effect applied, you can modify it's parameters by using the Set XXX parameters action.

The full list of parameters for each sound is displayed below, and an explanation for them via the irrKlang library documentation:

Chorus

Wet dry mix (default value 50.0) - ratio of wet (processed) signal to dry (unprocessed) signal. Minimum value is 0, maximum is 100.0.
Depth (default value 10.0) - percentage by which the delay time is modulated by the low-frequency oscillator, in hundredths of a percentage point. Minimum value is 0, maximum is 100.0.
Feedback (default value 25.0) - percentage of output signal to feed back into the effect's input. Minimum value is -99, maximum is 99.0.
Frequency (default value 1.1) - frequency of the LFO. Minimum value is 0, maximum is 10.0.
Sinus wave form (default value 1) - pass 1 for sinus wave form, or 0 for triangle.
Delay (default value 16.0) - number of milliseconds the input is delayed before it is played back. Minimum value is 0, maximum is 20.0.
Phase (default value 90) - phase differential between left and right LFOs. Possible values are -180, -90, 0, 90, and 180.

Compressor

Gain (default value 0) - output gain of signal after the compressor. Minimum value is -60, maximum is 0.0.
Attack (default value 10.0) - time before the compressor reaches its full value. Minimum value is 0.01, maximum is 500.0.
Release (default value 200.0) - speed at which the compressor is stopped after input drops below the threshold parameter. Minimum value is 50, maximum is 3000.0.
Threshold (default value -20.0) - point at which the compressor begins in decibels. Minimum value is -60, maximum is 0.0.
Ratio (default value 3.0) - the compressor ratio. Minimum value is 1, maximum is 100.0.
Predelay (default value 4.0) - time after the threshold parameter is reached before the attack phase is started in milliseconds. Minimum value is 0, maximum is 4.0.

Echo

Wet dry mix (default value 50.0) - ratio of wet (processed) signal to dry (unprocessed) signal. Minimum value is 0, maximum is 100.0.
Feedback (default value 50.0) - percentage of output fed back into input. Minimum value is 0, maximum is 100.0.
Left delay (default value 500.0) - delay for left channel in milliseconds. Minimum value is 1, maximum is 2000.0.
Right delay (default value 500.0) - delay for right channel in milliseconds. Minimum value is 1, maximum is 2000.0.
Pan delay (default value 0) - value that specifies whether to swap left and right delays with each successive echo. Minimum value is 0, maximum is 1.

Distortion

Gain (default value -18.0) - amount of signal change after distortion. Minimum value is -60, maximum is 0.
Edge (default value 15.0) - percentage of distortion intensity. Minimum value is 0, maximum is 100.
Post EQ center frequency (default value 2400.0) - center frequency of content addition. Minimum value is 100, maximum is 8000.
Post EQ bandwidth (default value 2400.0) - width of frequency band that determines range of content addition. Minimum value is 100, maximum is 8000.
Pre lowpass cutoff (default value 8000.0) - filter cutoff for high-frequency attenuation. Minimum value is 100, maximum is 8000.

Flanger

Wet dry mix (default value 50.0) - ratio of wet (processed) signal to dry (unprocessed) signal. Minimum value is 0, maximum is 100.0.
Depth (default value 100.0) - percentage by which the delay time is modulated by the low-frequency oscillator, in hundredths of a percentage point. Minimum value is 0, maximum is 100.0.
Feedback (default value 50.0) - percentage of output signal to feed back into the effect's input. Minimum value is -99, maximum is 99.0.
Frequency (default value 0.25) - frequency of the LFO. Minimum value is 0, maximum is 10.0.
Triangle wave form (default value 1) - pass 1 for triangle wave form, or 0 for triangle.
Delay (default value 2.0) - number of milliseconds the input is delayed before it is played back. Minimum value is 0, maximum is 20.0.
Phase (default value 0) - phase differential between left and right LFOs. Possible values are -180, -90, 0, 90, and 180.

Gargle

Rate (default value 20) - rate of modulation in hertz. Minimum value is 1, maximum is 1000.
Sinus wave form (default value 1) - pass 1 for sinus wave form, or 0 for triangle.

Interactive 3D Audio Level 2 reverbation

Room (default value -1000) - attenuation of the room effect, in millibels. Interval is (-10000, 0).
Room high frequency (default value -100) - attenuation of the room high-frequency effect. Interval is (-10000, 0).
Room rolloff factor (default value 0.0) - rolloff factor for the reflected signals. Interval is (0.0, 1.0).
Decay time (default value 1.49) - decay time in seconds. Interval is (0.1, 20.0).
Decay time high frequency ratio (default value 0.83) - ratio of the decay time at high frequencies to the decay time at low frequencies. Interval is (0.1, 2.0).
Reflections (default value -2602) - attenuation of early reflections relative to the Room parameter. Interval is (-10000, 1000).
Reflections delay (default value 0.007) - delay time of the first reflection relative to the direct path in seconds. Interval is (0.0, 0.3).
Reverb (default value 200) - attenuation of late reverberation relative to the Room parameter. Interval is (-10000, 2000).
Reverb delay (default value 0.011) - time limit between the early reflections and the late reverberation relative to the time of the first reflection. Interval is (0.0, 0.1).
Diffusion (default value 100.0) - echo density in the late reverberation decay in percent. Interval is (0.0, 100.0).
Density (default value 100.0) - modal density in the late reverberation decay in percent. Interval is (0.0, 100.0).
High frequency reference (default value 5000.0) - reference high frequency in hertz. Interval is (20.0, 20000.0).

Parametric Equalizer

Center frequency (default value 8000.0) - center frequency in hertz. Minimum value is 80, maximum is 16000.0.
Bandwidth (default value 12.0) - bandwidth, in semitones. Minimum value is 1.0, maximum is 36.0.
Gain (default value 0.0) - amount of gain. Minimum value is -15.0, maximum is 15.0.

Waves reverb

Input gain (default value 0.0) - input gain of signal in decibels. Minimum value is -96.0, maximum is 0.0.
Reverb mix (default value 0.0) - reverb mix in decibels. Minimum value is -96.0, maximum is 0.0.
Reverb time (default value 1000.0) - reverb time in milliseconds. Minimum value is 0.001, maximum is 3000.0.
High frequency reverb time ratio (default value 0.001) - the high frequency reverb time ratio. Minimum value is 0.001, maximum is 0.999.

- Recording audio using the microphone

You can record and play back audio using one of the microphones present on the computer. This can be done by first creating a microphone recorder using the Microphone > Create microphone recorder action (or by enabling the Automatically create microphone recorder checkbox in the object properties), and then using the Start recording microphone action to begin recording the microphone. You must then call the Stop recording microphone action when you are finished recording.

IMPORTANT NOTE: The microphone recorder is only compatible with the DirectSound8 device driver.

All actions are listed below:

The Create microphone recorder action has 1 parameter:

Enter the device ID of the microphone to use - this parameter requires you to pass in the ID of the computer's microphone device you want to record audio from (or just set it to "0" to use the default). To get the ID of a microphone device, you use the Microphones > Get microphone ID expression. Then you pass in the index of the microphone (1 would be the first microphone connected to the computer, 2 would be the second and so on).

The Start recording microphone action has 1 parameter:

Enter the sample rate of the audio to record - this parameter requires you to pass in the sample rate of the audio to be recorded. The recommended sample rate is 48000hz, but this depends if your microphone is capable of recording at the sample rate you provide.

The Stop recording microphone action has 1 parameter:

Enter a name for the recorded audio - once the microphone is finished being recorded, you must enter a name for the recorded audio so you may use it to play it back (e.g, passing in the name "MyRecordedAudio" and then using the Play a 2D/3D sound action using that name rather than a sound file pathname).

The Save last microphone recording to WAV file action has 1 parameter:

Enter the file path to save the recording to - the file pathname to save the last microphone recording to. You can use the Use expression button at the bottom right of Fusion's file selector to enter an evaluated path.

- Loading plugins from a folder

irrKlang supports plugins, which allows to extend the possible file formats you can play (see the irrKlang library documentation for creating your own plugins, which requires general C++ and sound knowledge). To utilize plugins, you use the Load plugins from a folder action which will load all plugin DLLs in that folder. It contains 1 parameter:

Enter the folder path of the plugins - the path to the folder containing the plugins. Recommended to use Apppath$ (which is the path of your EXE/MFA).

How to use : Conditions

Using Conditions in the Event Editor, you can check if the sound/microphone device is active and check sound states like if a sound is paused or if it is currently playing. Each condition is documented below:

Check if the irrKlang device is active

You can use the Is irrKlang device created? condition to check if irrKlang is active.

Check if the microphone recorder is active

You can use the Is microphone recorder created? condition to check if the microphone recorder is active.

Check if a sound is playing

You can use the Is a specific sound playing? (or Check sounds by ID -> Is a specific sound playing by ID? to check by ID rather than filename) condition to check if a sound is playing. This condition takes a filename parameter which indicates what sound to be checked.

Check if a sound is paused

You can use the Is a specific sound paused? (or Check sounds by ID -> Is a specific sound paused by ID? to check by ID rather than filename) condition to check if a sound is paused. This condition takes a filename parameter which indicates what sound to be checked.

How to use : Expressions

You can use Expressions to get values of things like 3D sound/listener position, sound volume, etc. Each expression is documented below:

Retrieving listener states

The Listener submenu (or By ID -> Listener submenu for refrencing sounds by ID) contains a list of expressions related to the 3D sound listener (remember, it's like the position of the player around the sounds), such as Get the listener's X/Y/Z position to get the listener position, Get the listener's X/Y/Z rotation to get the listener rotation, and Get the listener's X/Y/Z velocity to get the listener velocity (only used for doppler effects).

Retrieving sound states

The Sound submenu contains a list of expressions related to 3D sounds, such as Get a sound's X/Y/Z position to get the position of a 3D sound, Get a sounds's minimum/maximum distance to get the minimum/maximum distance of a 3D sound, and Get a sounds's X/Y/Z velocity to get the velocity (the speed of a sound, only used for doppler effects) of a 3D sound.

Generic expressions

The rest of the general expressions are documented below:

Get master volume - this expression retrieves the master volume influenced by all sounds.
Get a sound's current volume (or Get a sound's current volume by ID) - this expression retrieves the volume of a sound by it's pathname or ID.
Get a sound's current playback position (or Get a sound's current playback position by ID) - this expression retrieves the playback position of a sound in milliseconds by it's pathname or ID..
Get a sound's length (or Get a sound's length by ID) - this expression retrieves the total length of a sound in milliseconds by it's pathname or ID.
Get a sound's current playback speed (or Get a sound's current playback speed by ID) - this expression retrieves the playback speed of a sound by it's pathname or ID.
Get a sound's current pan (or Get a sound's current pan by ID) - this expression retrieves the pan of a sound by it's pathname or ID.

Microphone expressions

The Microphones submenu contains a list of expressions related to the microphone devices currently connected to the computer. They are documented below:

Get number of microphones - this expression returns the number of microphones currently connected to the computer. Note that microphone number 1 is always the default microphone, so the number returned may be 1 more than the number of microphones actually connected.
Get a microphone's description - this expression returns the name of a microphone. The Index of microphone parameter is which microphone number to retrieve the description from (0 being the first, 1 being the second, etc). It ranges from 0 to GetMicCount() - 1.
Get a microphone's device ID - this expression returns the device ID of a microphone. The Index of microphone parameter is which microphone to retrieve the ID from (1 being the first, 2 being the second, etc). If there are microphones connected, it ranges from 1 to GetMicCount() - 1.

Sound device expressions

The Sound devices submenu contains a list of expressions related to the sound devices currently connected to the computer. They are documented below:

Get number of sound devices - this expression returns the number of sound devices currently connected to the computer. Note that sound device number 1 is always the default device, so the number returned may be 1 more than the number of devices actually connected.
Get a sound device's description - this expression returns the name of a sound device. The Index of sound device parameter is which device number to retrieve the description from (0 being the first, 1 being the second, etc). It ranges from 0 to GetSoundDevCount() - 1.
Get a sound device's device ID - this expression returns the device ID of a sound device. The Index of sound device parameter is which device to retrieve the ID from (1 being the first, 2 being the second, etc). If there are sound devices connected, it ranges from 1 to GetSoundDevCount() - 1.

Retrieve active device driver name

The Get active device driver name expression simply returns the name of the device driver actually being used by the irrklang device. Possible returned strings are "DirectSound", "DirectSound8", "winMM" or "" if the irrKlang device isn't created.