AudioMixer Class Reference

Class for mixing together 0-n audio streams. More...

#include <audiomixer.h>

Inheritance diagram for AudioMixer:

Collaboration diagram for AudioMixer:

Classes
struct	AudioFrameStruct
struct	AudioStreamMetadata

Public Slots
void	insertCapturedAudioFrame (const QByteArray &aFrame, quint32 aSeqNo, float aVolumeLevel)
void	insertReceivedAudioFrame (quint32 aCallId, quint32 aSeqNo, const QByteArray &aFrame, Hash aOriginatingNode)

Signals
void	frameReadyForLocalSpeaker (const QByteArray &aFrame)
void	frameReadyForRemoteSend (quint32 aCallId, quint32 aSeqNo, const QByteArray &aFrame, Hash aForNode)

Public Member Functions
	AudioMixer (Model &aModel)
	~AudioMixer ()
void	insertStream (quint32 aCallId, quint32 aStartingSeq, bool aIsLocallyCaptured, const Hash &aOriginatingNode)
void	removeStream (quint32 aCallId)

Private Types
typedef struct AudioMixer::AudioFrameStruct	AudioFrame
typedef struct AudioMixer::AudioStreamMetadata	AudioFrameMetadata

Private Member Functions
void	tryMixFrames (quint32 aCallId, quint32 aSeqNo)
void	configureMixedStreams ()
void	doInsertAudioFrame (const QByteArray &aFrame, bool aIsLocallyCaptured, quint32 aSeqNo, quint32 aCallId, float aVolumeLevel, Hash &aOriginatingNode)

Private Attributes
QList< QQueue< AudioFrame > >	iAudioDataBuffer
QList< AudioFrameMetadata >	iStreams
QList< int >	iIndexesToMixForLocalSpeaker
QList< int >	iIndexesToMixForRemoteOutput
Model &	iModel

Detailed Description

Class for mixing together 0-n audio streams.

This class has 2 types of input and 2 types of output. Input type 1: locally captured microphone data Input type 2: 0-n streams sent from remote nodes Output type 1: Audio intended for playback from local loudspeaker Output type 2: Audio intended to be sent to remote nodes Depending on situation this class will mix a bit differently. If there is only type 1 input and 1 type 2 stream then no mixing will occur: input type 2 becomes output type 1 and that's it. If there is more than 1 streams to mix together then mixer will actually mix.

Member Typedef Documentation

AudioFrame

typedef struct AudioMixer::AudioFrameStruct AudioMixer::AudioFrame

private

Carrier-structure for audio frame. These sit in a queue waiting to be mixed and sent to remote destinations or local playback

AudioFrameMetadata

typedef struct AudioMixer::AudioStreamMetadata AudioMixer::AudioFrameMetadata

private

Constructor & Destructor Documentation

AudioMixer()

AudioMixer::AudioMixer

(

Model &

aModel

)

Constructor.

~AudioMixer()

AudioMixer::~AudioMixer

(

)

Member Function Documentation

configureMixedStreams()

void AudioMixer::configureMixedStreams ( )

private

method for figuring out which stream to mix into which direction

doInsertAudioFrame()

void AudioMixer::doInsertAudioFrame ( const QByteArray &  aFrame, bool  aIsLocallyCaptured, quint32  aSeqNo, quint32  aCallId, float  aVolumeLevel, Hash &  aOriginatingNode  )

private

workhorse of insertCaptured/ReceivedAudioFrame slots

Parameters

aIsLocallyCaptured

is true if audio is from local microphone and false if received from remote node

frameReadyForLocalSpeaker

void AudioMixer::frameReadyForLocalSpeaker ( const QByteArray &  aFrame )

signal

emitted when mixing done for local playback

frameReadyForRemoteSend

void AudioMixer::frameReadyForRemoteSend ( quint32  aCallId, quint32  aSeqNo, const QByteArray &  aFrame, Hash  aForNode  )

signal

emitted when mixing done for remote nodes

insertCapturedAudioFrame

void AudioMixer::insertCapturedAudioFrame ( const QByteArray &  aFrame, quint32  aSeqNo, float  aVolumeLevel  )

slot

Method for inputting an audio frame from local microphone. Frame is 60 milliseconds raw audio data in floating point format with 8kHz sample rate. That means there are 480 samples, totalling 960*2 bytes. See constants in top of audiomixer.cpp file. Each floating point sample in array needs to be in range [-1,1]

Parameters

aFrame	is the audio data. floating point, native-encoding audio/pcm data.
aSeqNo	Sequence number of frame. This is a incrementing number that begins from 0 at stream start. Note that it is entirely possible to start mixing a stream from middle so when processing starts, this value does not necessarily start from 0
aVolumeLevel	calculated volume level of this 60 ms snippet

insertReceivedAudioFrame

void AudioMixer::insertReceivedAudioFrame ( quint32  aCallId, quint32  aSeqNo, const QByteArray &  aFrame, Hash  aOriginatingNode  )

slot

Method for inputting an audio frame that is received from network. See method insertCapturedAudioFrame for more information, same frame data format must be used.

Parameters

aFrame	is the audio data. float, native-encoding audio/pcm data.
aSeqNo	Sequence number of frame. This is a incrementing number that begins from 0 at stream start. Note that it is entirely possible to start mixing a stream from middle so when processing starts, this value does not necessarily start from 0
aCallId	Call id for keeping track which frame belongs to which stream. Typically inside mixer there are both locally captured frames and frames from remote that both have same call id ; if there are multiple calls in progress then one set of frames of local input and several from remote sources. If aIsLocallyCaptured is true, then value of this parameter has no effect
aOriginatingNode	node where frame is from.

insertStream()

void AudioMixer::insertStream	(	quint32	aCallId,
		quint32	aStartingSeq,
		bool	aIsLocallyCaptured,
		const Hash &	aOriginatingNode
	)

method for entering stream metadata so that stream with given callid gets included into mixer output

Parameters

aCallId	call identifier
aStartingSeq	first sequence number to expect
aIsLocallyCaptured	if true, stream is coming from local audio hardware, not from remote node
aOriginatingNode	Fingerprint of node sending this stream

removeStream()

void AudioMixer::removeStream

(

quint32

aCallId

)

method for removing stream metadata so that stream with given callid no longer gets included into mixer output

tryMixFrames()

void AudioMixer::tryMixFrames ( quint32  aCallId, quint32  aSeqNo  )

private

workhorse method. will check for buffers and see if we have frames for all streams to mix and if conditions evaluate true, will output the mixed frame, or several if possible.

Member Data Documentation

iAudioDataBuffer

QList<QQueue<AudioFrame> > AudioMixer::iAudioDataBuffer

private

Voice data buffer. Must share same indexing with iStreams e.g. if stream metadata for call x is in iStreams[y] then call data for that stream needs to go to queue strictly at iAudioDataBuffer[y]

iIndexesToMixForLocalSpeaker

QList<int> AudioMixer::iIndexesToMixForLocalSpeaker

private

List of indexes to iStreams that are mixed into speaker output

iIndexesToMixForRemoteOutput

QList<int> AudioMixer::iIndexesToMixForRemoteOutput

private

List of indexes to iStreams that are mixed into remote nodes

iModel

Model& AudioMixer::iModel

private

datamodel

iStreams

QList<AudioFrameMetadata> AudioMixer::iStreams

private

Metadata of streams being served

---

The documentation for this class was generated from the following file:

• audiomixer.h