Home > software > Realtime Signal Analysis in Perl

Realtime Signal Analysis in Perl

September 24th, 2011 Leave a comment Go to comments

About a month ago, we were working on the Fluffy Ball project – a computer input device that can react to fondling and punching. Thanks to a nice idea on the brmlab mailing list, we use a microphone and process the noise coming from the ball’s scratchy stuffing and an embedded jingle. The sounds from the outside are almost entirely dampened by the stuffing and for a human, the noise of fondling and punching is easily distinguishable.

Frequency spectrum, for our purposes, is just an array indexed by frequency, storing the amplitude of each frequency (in some range). A common variation is the power spectrum that describes the power of each frequency, i.e. the amplitude squared. Frequency spectrum is obtained by splitting the input signal to fixed-size samples and performing Discrete-Time Fourier Transform.

It turns out that trivial spectrum-based rules can be used to achieve reasonably high detection accuracy for a computer too (especially when the user is allowed to “train” her input based on feedback); I had big plans to use ANN and all the nifty things I have learned in our AI classes, but it turned out to be simply an overkill. The input signal is transformed to a frequency spectrum (see box) using real discrete FFT.

So, we have the audio signal coming in from a regular mic device and need to process it further. I chose Perl for quick prototyping and I have assumed that I would find some pre-made scaffolding for this ready. But it turns out that noone really published a simple example of even just showing a real-time frequency spectrum. So, here you go! :-)

First, we need some reasonable way to continuously display the spectrum. Most GUI paradigms are event-driven, but events are usually user interaction pieces and while it would be possible to incorporate continuous data-based updates in this model, it feels quite backwards. So we use a trick:

use warnings;
use strict;
 
init_dsp(); init_fft();
 
use Tk;
our $mw = MainWindow->new;
$mw->after(1, \&ticks); # after 1ms, give control back
MainLoop;
 
sub ticks {
	while (1) {
		render_signal(process_signal(read_dsp()));
		$mw->idletasks();
	}
}

This circumvents the event-driven architecture of Tk and instead puts our main loop in control, processing any GUI events when it’s good time. For more complex programs, this is a bad idea and it will lead to poorly maintaineable code, but when writing simple tools, you should not succumb to grand frameworks and let your code overgrow you.

Okay, how to grab audio input signal in Perl? Unfortunately, there are not really any handy modules you could use thoughtlessly. Audio::DSP is a possibility, but using it is clumsy, especially in the current world of ALSA as you have to rely on the imperfect aoss wrapper. A simple alternative is to get the raw byte data through a pipeline from the ALSA arecord tool:

our ($devname, $fmt, $bitrate, $wps, $bps, $bufsize, $dsp);
BEGIN {
	$devname = "default"; # or e.g. hw:1,0 for an additional USB soundcard input
	$fmt = 16;            # sample format (bits per sample)
	$bitrate = 16384;     # sample rate (number of samples per second)
	$wps = 8;             # FFT windows per second (rate of FFT updates)
	$bps = ($fmt * $bitrate) / 8; # bytes per second
	$bufsize = $bps / $wps;   # window buffer size in bytes
}
 
sub init_dsp {
	open ($dsp, '-|', 'arecord', '-D', $devname, '-t', 'raw',
		'-r', $bitrate, '-f', 'S'.$fmt) or die "arecord: $!";
	use IO::Handle;
	$dsp->autoflush(1);
}
 
sub read_dsp {
	my $w;
	read $dsp, $w, $bufsize or die "read: $!";
	return $w;
}

read_dsp will return one signal window per call, the window being a binary blob consisting of one two-byte word per sample. We want to magically convert this to a spectrogram.

Audio::Analyze is again the simple way to get a signal spectrum. If you are after analyzing a pure audio signal, you probably want to use it since it can easily filter the signal based on relative human perception of frequencies etc. But for us, it is inconvenient to feed it data through a pipe and we will directly use Math::FFT. It will still handle all the gory math for our case (and we care about the actual noise, not the way people would hear it).

use Math::FFT;
use List::Util qw(sum);
 
our @freqs;
sub init_fft {
	my $dft_size = $bitrate / $wps;
	for (my $i = 0; $i < $dft_size / 2; $i++) {
		$freqs[$i] = $i / $dft_size * $bitrate;
	}
}
 
sub process_signal {
	my ($bytes) = @_;
 
	# Convert raw bytes to a list of numerical values.
	$fmt == 16 or die "unsupported $fmt bits per sample\n";
	my @samples;
	while (length($bytes) > 0) {
		my $sample = unpack('s<', substr($bytes, 0, 2, ''));
		push(@samples, $sample);
	}
 
	# Perform RDFT
	my $fft = Math::FFT->new(\@samples);
	my $coeff = $fft->rdft;
 
	# The output are complex numbers describing the exactly phased
	# sin/cos waves. By taking an abs value of the complex numbers,
	# we just measure the amplitude of a wave for each frequency.
	my @mag;
	$mag[0] = sqrt($coeff->[0]**2);
	for (my $k = 1; $k < @$coeff / 2; $k++) {
		$mag[$k] = sqrt(($coeff->[$k * 2] ** 2)
		                + ($coeff->[$k * 2 + 1] ** 2));
	}
 
	# Rescale to 0..1. Many fancy strategies are possible, this is
	# extremely silly.
	my $avgmag = sum (@mag) / @mag;
	@mag = map { $_ / $avgmag * 0.3 } @mag;
	return @mag;
}

Not much to add besides the inline comments. The input of the process_signal function is a raw byte stream, the output is a list of amplitudes; @freqs maps the list indices to the actual Hz frequencies. The normalization to [0,1] interval shown here (pitching the mean at 0.3) is extremely naive, again there are many possible strategies. Also, you certainly want to use a window function etc. in more serious applications.

Now, for the visualization. We have chosen Tk for our GUI (it looks ugly, but it is reasonably easy to use despite its Tcl antics). We will use its Canvas object where we can draw freely, and just plot a line for each frequency:

our $canvas;
sub render_signal {
	# Display parameters, tweak to taste:
	my $rows = 2;
	my $hspace = 20;
	my $height = 150;
	my $vspace = 20;
 
	my @spectrum = @_;
	my $row_freqn = @spectrum / $rows;
 
	$canvas;
	unless ($canvas) {
		$canvas = $mw->Canvas(
			-width => $row_freqn + $hspace * 2,
			-height => $height * $rows + $vspace * ($rows + 1));
		$canvas->pack;
	}
	$canvas->delete('all');
 
	for my $y (0..($rows-1)) {
		for my $x (0..($row_freqn-1)) {
			my $hb = ($height + $vspace) * ($y + 1);
			my $i = $row_freqn * $y + $x;
 
			# Draw line:
			my $ampl = $spectrum[$i];
			$ampl <= 1.0 or $ampl = 1.0;
			my $bar = $height * $ampl;
			$canvas->createLine($x + $hspace, $hb,
			                    $x + $hspace, $hb - $bar);
 
			# Draw label:
			if (!($x % ($row_freqn/4))) {
				$canvas->createLine($x + $hspace, $hb + 0,
				                    $x + $hspace, $hb + 5,
				                    -fill => 'blue');
				$canvas->createText($x + $hspace, $hb + 15,
				                    -fill => 'blue',
				                    -font => 'small',
				                    -text => $freqs[$i]);
			}
		}
	}
 
	$mw->update();
}

This suffices for a naive visualization, you can easily tweak it to do thresholding and whatever else you desire. I have found that on some of my computers, the X protocol is pushed to its limits by repeatedly drawing a large amount of lines, and sometimes the spectrum will start to lag behind the signal; either show wider bars averaging together multiple frequencies, or use something other than a Canvas object – raw pixmap transfer would likely be better than such a large amount of line drawing operations.

For a serious signal analysis work, you will also want a spectrogram – a time-based plot of amplitude of various frequencies.

To get a working script skeleton, simply piece the code snippets together (fb-simple.pl). See fb.pl for the real fluffy ball script. It is much uglier, but it maintains sample averages over longer time windows (essential for more complex signal analysis), it has simple sample recording capabilities, and an example of naive threshold-based classifier.

Categories: software Tags: , , , ,
  1. July 16th, 2014 at 09:07 | #1

    I am truly thankful to the owner of this website who has shared this fantastic piece of writing at at this
    place.

  2. August 2nd, 2014 at 08:24 | #2

    as an alternative to getting all these cons, ipad tablet markets just like scorching birthday cake. . -= Sourish | Wp Set up Guide’s continue blog… The reason Pick out Our Hubpages Install Assistance =-.
    chaussures louboutin outlet http://www.sacha.org/main_sp.htm

  3. August 5th, 2014 at 05:50 | #4

    it does not work we gain 6 insurance 64 touch. and still have mistake You ought to have referred to as this Titanic, doing this it can claim syncing Titanic.
    Mizuno Shoes Outlet http://www.bonapartesretreat.org/BonapartesRetreatAboutUs.html

  4. August 9th, 2014 at 11:30 | #5

    iPhone 4 launched simply by Maxis, DiGi; although which includes better options for customers? By LEONG HUNG YEE

  5. August 13th, 2014 at 05:59 | #6

    Thanks a great deal, these records has been just right personally. I will save this great site for upcoming upadtes. Cheers once more.

  6. August 13th, 2014 at 09:05 | #7

    At this time ATM skimming is generally a serious problem, Appreciate your discussing this posting. This can be a very useful in addition to useful for all.

  7. August 16th, 2014 at 07:36 | #8

    Hi there We are producing which you extremely gang of outstanding performers as well as hear the fantastic experience a person stunning noises combination along and as soon as I had an opportunity to ever previously head over to The united states and speak to you’d be better with our a single major fantasy is to satisfy a person once again if no I could be around a person even participate in therefore i could load my wish. I truly do not necessarily recognize when to return to The european union although feel that many but is not practically and so probably perhaps even much more yet me personally and if an individual actually seem to pay a visit to North america, as well as definitely I had always be pleased when we include your own routines for the sedentary ever before have experienced. If only a person all the best for the whole class every single achievement along with provides excellent shows mainly great in the operations. Once again I wish good luck to get a wonderful moment.

  8. August 16th, 2014 at 14:52 | #9

    David On the web working together with small businesses proprietors here in Upper ANAVA and wish to use your artwork with my conversations together. This is a more efficient 1 web site above look at from the sophisticated Web community this anyone has to be aware about and take part in. Can i attain your own personal agreement intended for is actually use? Thank anyone Doug

  9. August 17th, 2014 at 07:52 | #10

    Fantastic blog you may have below however I became questioning if you recognized involving any discussion boards that cover the identical matters spoken of right here? We would indulge to become a portion of set just where I will receive opinions from the other skilled individuals that share precisely the same desire. When you have almost any tips, please let me recognize. Thanks a lot!
    Canada Goose Parka http://www.koncreteboxing.com/canada-goose.html

  10. August 18th, 2014 at 07:07 | #11

    There are usually disadvantages when comparing a subject do some simple different object., From the above details I actually figure you will have in comparison some sort of apple ipad tablet with a Atom driven Notebook computer Nicely Some recognize issues which might be not really throughout Apple’s iPad. Nevertheless the ipad device has its restriction, it has a excellent characteristics. You can carry there are excellent effectiveness. We can say that ipad device will a good hit similar to i phone.
    Cheap Jordan Shoes http://www.laidbackrebels.com

  11. August 25th, 2014 at 13:43 | #12

    Theres beneficial on my phone called slacker radio stations… I use a google, and so idk when there is schooling internet site Yes deadbeat totally carries a computer web site. It is great. Like deadbeat. They get considerably more audio in comparison with pandora and also you dont find all the continuing goods.

  12. September 2nd, 2014 at 03:29 | #13

    With havin so much content do you ever run into any problems of plagorism or copyright violation? My blog has a lot of completely unique content I’ve either written myself or outsourced but it looks like a lot of it is popping it up all over the internet without my authorization. Do you know any techniques to help protect against content from being ripped off? I’d really appreciate it.

  13. September 4th, 2014 at 10:24 | #14

    男性のコスチュームは別売りです。あなたは、彼らが女性と男性のために個々に提供するすべてのコスチュームを通して見ることができるし楽しいし、そしてあなた自身のカップルの観察をつくってください。あなたは、おそらくあなたが経験豊かなプラスサイズを着るならば一式の狭い選択の失望をすでに持っています。幸い、コスチュームデザイナーのショッピングセンターでは、プラスサイズのコスチュームのための大きな人のために多くのオプションを作成しました。贅沢なマスクを必要として、独特の帽子、またはクレイジーカラーのウィッグですか?コスチュームのショッピングセンターとしてよくこれらのアクセサリーを提供しています。党のスターと驚異的な、愚かな、エレガントな、法外であるとあなたの友人の羨望の的である、またはセクシーなカップルハロウィンコスチュームは、コスチュームのショッピングセンターから。
    コスプレ通販 http://www.tigerbeerus.com/costumes.php

  14. September 6th, 2014 at 03:32 | #15

    I usually turn out playing Grooveshark though. <3 me a number of grooveshark!!
    jackets moncler http://www.zetroz.com/about/index.php

  15. September 10th, 2014 at 10:51 | #16

    藏密双宝http://www.baike.com/wiki/%E8%97%8F%E7%A7%98%E5%8F%8C%E5%AE%9D

  16. September 10th, 2014 at 14:30 | #17

    iphone4 unveiled by Maxis, DiGi; however containing a great deal better ideas regarding subscribers? By LEONG STRUNG YEE
    Mizuno Running Shoes http://www.neemjeevan.com/ureacoat.php

  17. September 12th, 2014 at 03:12 | #18

    Usually Therefore i’m just going to put it to use with regard to publishing blog post in any case, you will discover certainly trade offs yet No later than this still be investing in this. Price is probably the most challenging to agree to in my opinion. . -= Tini | Inzone Internet’s previous blog… Why I actually KeywordLuv =-.
    moncler coat http://www.keizervet.com/staff.html

  18. September 13th, 2014 at 03:14 | #19

    I tried looking at your web site together with my ipod itouch and the layout will not are most often proper. May wish to look it over with WAP in addition to this indicates the majority of cell phone styles aren’t going to be really working together with your internet site.

  19. September 15th, 2014 at 04:02 | #20

    At present TELLER MACHINES skimming it’s enormous problem, Many thanks sharing this write-up. This can be a worthwhile and also informative for everybody.

  20. September 15th, 2014 at 06:44 | #21

    Nice story, very good to see individuals taking some societal obligation.

  1. No trackbacks yet.