I’m doing some experiments with advertising my live looper app Loopy lately, and want a way to track the success of various approaches. It occurred to me that the totally freeform nature of the LinkShare “signature” field (which you can use to track traffic sources) lends itself to an even more flexible approach than that presented in the Mobile Orchard article.

Here’s a way to use that signature field to report the domain name of any referrer who links either to the app page, or to a download link (like, say, http://loopyapp.com/download).

This way, if, say, TUAW link to your app site, if someone clicks through then clicks the download link on your app site and buys, the resulting order will be reported as coming from TUAW. If someone clicks through from your Facebook page, it’ll come up as coming from Facebook. You can even modify the script further to report more precise details (like the path), if you like.

It assumes you’re using PHP, but the principle’s the same for any other language (BYO code, though ;-)).

Step 1: Sign up to LinkShare

First, if you haven’t already, Sign up to the LinkShare program — Once you’ve created a LinkShare account, join the Apple affiliate program via the “Programs” tab. After 3 days, you’ll get an email welcoming you to the program, and you’ll be good to go.

Step 2: Create a product link

Once you’re admitted to the program, open up the “My Advertisers” sub-tab from the LinkShare Programs tab, and open the “Link Maker Tool”. This lets you search for products, and create a link that will open up your app’s App Store page, and will be associated with your LinkShare account.

Step 3: Create a download redirection script

Now we’re going to set up a script on your app site which will redirect the visitor to the URL you just created (which in turn, redirects straight to the App Store page). It’ll add a “signature” parameter to the URL, which corresponds to the original referrer, so you can track where orders came from.

Create a file called ‘download.php’ in the root of your app site, with the following content, with your LinkShare URL inserted where indicated:

<?php
// Replace the following URL with the LinkShare URL you created
$linkshare_url = "http://click.linksynergy.com/fs-bin/stat?id=/yGrgMJzFG0&offerid=146261&type=3&subid=0&tmpid=1826&RD_PARM1=http%253A%252F%252Fitunes.apple.com%252Fapp%252Floopy%252Fid300257824%253Fmt%253D8%2526uo%253D4%2526partnerId%253D30";
 
session_start();
$referer = $_SESSION['original_referer'];
if ( !$referer ) $referer = $_SERVER["HTTP_REFERER"];
 
if ( $referer ) {
    $signature = preg_replace("@https?://(?:www\.)?([^/]+?)(?:\.com)?/.*@", "$1", $referer);
} else {
    $signature = preg_replace("@^(?:www\.)?(.+?)(?:\.com)?$@", "$1", $_SERVER["HTTP_HOST"]);
}
 
$signature = preg_replace("@[^a-zA-Z0-9]@", "", $signature);
 
header("Location: ".$linkshare_url."&u1=".$signature);
?>

This script looks for the original referrer in a session variable (which we’ll set up in the next step), so that the domain of the site that links to your app site is used, not just your app site’s domain. Then it creates a properly-formatted signature parameter (just alphanumeric), appends it to your LinkShare URL, and sends the viewer onwards.

Bonus points: I prefer to get rid of the ‘php’ extension to make the URL a bit cleaner. Pop the following into your .htaccess file to access ‘download.php’ as just ‘download’:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule . %{REQUEST_FILENAME}.php [L]
</IfModule>

Step 4: Remember the referrer

Now, on the landing page script for your app site (or the site header), pop this in at the very start:

<?php
session_start();
if ( !$_SESSION["original_referer"] ) $_SESSION["original_referer"] = $_SERVER["HTTP_REFERER"];
?>

This stores the original referrer URL in a session variable, to use when we actually link the viewer through to the App Store.

Step 5: Test it

To make sure everything’s working properly, open download.php again, and replace “header” at the bottom with “echo”, so that instead of redirecting the browser, we just print out the URL where we would be redirecting to.

Open your appsite/download URL, and make sure the URL ends with “&u1=appsite“. That’s for direct visitors. Now, click through to your app site from another site, then click your “download” link. You should now see the name of the original site you linked from as the “u1″ parameter at the end of the URL.

Once you’re satisfied that you’re good to go, make sure you replace “echo” with “header” again.

Step 6: Track

Now that you’re ready to track referrals, you can give out your http://yourappsite/download URL as your app’s direct iTunes download link (to reviewers, in your press releases, etc).

You can view a report showing clicks and orders associated with each referrer on the LinkShare page — create an advanced report by clicking the “Advanced Reports” sub-tab, then select your desired date range (I use “Since Last Payment”, and under “Report Type”, select “Signature Activity”. Hit “View Report”, and you’ll see your clicks and sales versus each referrer (“Member ID”, on the report).

Voila! Omnipotence achieved.

Addendum: This technique works for tracking referrers, but if you’re wanting to track the performance of ads (say, with AdMob), you’ll want to use your original LinkShare URL, with a custom “&u1″ signature parameter. As ad platforms like AdMob link directly (and don’t, as far as I know, send a referer parameter), this script won’t pick up that it’s from your ad platform.

Addendum 2: LinkShare’s reports don’t distinguish between products, so if you’ve multiple apps, you might want to add a prefix to your signature parameter to tell ‘em apart. You could, say, replace that header("Location: ".$linkshare_url."&u1=".$signature); line with something like header("Location: ".$linkshare_url."&u1=myapp".$signature);.

Any app that makes use of the Remote IO audio unit framework (which is generally necessary for the kind of responsiveness required in a realtime musical app) provides audio to the hardware via a callback, which is periodically called when the hardware is ready for more.

The trick here is to provide the right chunk of samples in this callback for the current time position.

Loopy achieves this by:

1. Keeping track of where in the timeline we are at the time the callback is called

This is easily accomplished by keeping a record of the time the clock was started, subtracting this from the current time, and possibly performing a modulus with the tempo. For example:

• (now - startTime) % timePerBar gives the number of time units into the current bar (lets call it timeIntoBar).
• timeIntoBar / (timePerBar/beatsPerBar) gives the number of beats into the current bar, and
• timeIntoBar % (timePerBar/beatsPerBar) gives us the time into the current beat.

2. Determining first if we should be playing audio at this time, and if so, which samples should be playing

This involves first converting our time units from step 1 into samples. For instance, you can convert microseconds to samples by dividing your time by 1000000/yourSampleRate. Aside: Of course, you can convert back from samples to time by multiplying instead of dividing.

Next, in the case of Loopy’s metronome, for example, we test for whether samplesIntoBeat < sound.lengthInSamples. If so, that means we should be playing audio. If the sound was a loop, of course, we could be always playing.

The offset into the sound, in samples, is just samplesIntoBeat, in the case of the simple metronome. In the case of a loop, you probably will be more interested in the number of samples into your loop — so instead of determining (now - startTime) % timePerBar, you may be interested in (now - startTime) % timePerLoop.

So, we want to return the requested number of samples starting from this offset into the sample array representing our audio.

3. Returning smooth audio in time

Note that if you just go returning any old set of samples, willy-nilly, you’re going to get nasty clicks and pops from discontinuities you get by not matching the start of your next buffer to the last one.

To ensure smoothness, Loopy keeps track of the offset of the last samples we returned, and just return the immediately following bunch of samples — unless we’re more than some threshold number of samples out of time, in which case we’ll suffer the pop in order to stay synchronised. Actually, you can even generally avoid the pop if you smoothly blend buffers over a short time, removing any discontinuity.

Final words

The example above was a relatively trivial one, for a metronome sound. For longer audio that may span multiple bars, you’ll probably want to perform a modulus by the length of your audio clip, possibly quantised to your time signature, and possibly using a per-loop time base, so you can start the loop at any point in your timeline and have it begin from the start. This is something Loopy doesn’t currently do — Loopy will keep your loops synchronised so when you start a loop playing, it’ll play whatever part corresponds to the current timeline, not from the start of the loop. Maybe it’ll be an option in the future?

I wrote a little about the timing of loops in my second article on Loopy’s implementation.

Talkie is my newest product, a Walkie Talkie for iPhone and Mac.

In Part 1 of this series, I wrote about basic broadcasting. This works fine with one network device, but it’s worth discussing how to send through all devices, so you can communicate with others connected via, say, Ethernet and WiFi simultaneously.

So, in Part 2 I’ll write about the approach I took in Talkie for broadcasting from all network devices (a.k.a. network interfaces), so that one can communicate with others connected via WiFi, Ethernet (on a Mac), and any other network devices simultaneously.

Bind them

From Part 1, we have a working broadcast mechanism, but it will only send through the default interface — whatever you’re connected to the network via. This is often sufficient, but if you have more than one device that you communicate through, like Ethernet and WiFi, then you will find that it only works with one.

In order to send through all your connected network interfaces, we need to create one socket for each interface, and bind the socket to its corresponding interface.

Here’s how:

First, we need to obtain a list of all network interfaces with getifaddrs.

#include <ifaddrs.h>
...
struct ifaddrs *addrs;
int result = getifaddrs(&addrs);
if ( result < 0 ) {
  // Error occurred
  return 0;
}

Now, addrs is a list of interfaces that we can iterate over. We now do so, picking out those devices that support broadcasting, and that aren’t loopback or point-to-point devices — loopback is an internal interface that is provided for your computer’s inner dialogue, and point-to-point (ppp) devices include dialup interfaces, 3G modems and the like. We can exclude those guys.

const struct ifaddrs *cursor = addrs;
while ( cursor != NULL ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_BROADCAST) ) {
 
    // We will do some stuff in here
 
  }
  cursor = cursor->ifa_next;
}

Now, for each interface that meets our criteria, we create a socket (which we covered in Part 1), then bind the socket to the network interface, to force transmission from that particular device. Finally, as we did in Part 1, we enable broadcasting using setsockopt with SO_BROADCAST.

We want to store the sockets we create in an array, so we can access them later. If we assume a maximum number of interfaces we will support (lets call it kMaxSockets), we can just use an array of that length. So, putting it together:

#define kMaxSockets 16
...
int sock_fds[kMaxSockets];
int number_sockets = 0;
 
while ( cursor != NULL && number_sockets < kMaxSockets ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_BROADCAST) ) {
 
    // Create socket
    sock_fds[number_sockets] = socket(AF_INET, SOCK_DGRAM, 0);
    if ( sock_fds[number_sockets] == -1 ) {
      // Error occurred
      return 0;
    }
 
    // Create address from which we want to send, and bind it
    struct sockaddr_in addr;
    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_addr = ((struct sockaddr_in *)cursor->ifa_addr)->sin_addr;
    addr.sin_port = htons(0);
 
    int result = bind(sock_fds[number_sockets], (struct sockaddr*)&addr, sizeof(addr));
    if ( result < 0 ) {
      // Error occurred
      return 0;
    }
 
    // Enable broadcast
    int flag = 1;
    result = setsockopt(sock_fds[number_sockets], SOL_SOCKET, SO_BROADCAST, &flag, sizeof(flag));
    if ( result != 0 ) {
      // Error occurred
      return 0;
    }
 
    number_sockets++;
  }
  cursor = cursor->ifa_next;
}

Finally, as before, we can setup a broadcast address to send to, and use sendto to broadcast, this time for each socket we created:

// Initialise broadcast address
struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_BROADCAST;
addr.sin_port = htons(kPortNumber);
 
// Send through each interface
int i;
for ( i=0; i<number_sockets; i++ ) {
  int result = sendto(sock_fds[i], data, length, 0, (struct sockaddr*)&addr, sizeof(addr));
  if ( result < 0 ) {
    // Error occurred
    return 0;
  }
}

Note that the receive routine only needs a single socket, as we can receive on any interface when we use INADDR_ANY. So, the receive routine needs no changes from the single-interface version we saw in Part 1.

Here’s the test app with the above modification: broadcast_sample_all_interfaces.c

Again, compile by opening Terminal, and typing make broadcast_sample_all_interfaces or cc -o broadcast_sample_all_interfaces broadcast_sample_all_interfaces.c, then run it with ./broadcast_sample_all_interfaces "Message to send" to send, or just ./broadcast_sample_all_interfaces with no arguments to listen.

You may notice that multiple messages may be received: These have probably arrived via multiple network interfaces, virtual or otherwise. It’s usually a good idea to check for duplicate messages, if this is an issue for program operation, by including a sequence number into the message — this will be discussed in Part 3.

It also may be a good idea to ignore your own messages, which may find their way back to you. One way to accomplish this is to examine the source address (addr in the example above) and compare it with your local interface addresses (stored in addrs, above). If you get a match, the message came from you, and you can drop it.

Multicast

Broadcast is fine when everyone on the local network is interested in what you have to say. If this isn’t the case, though (lets face it, those Chuck Norris jokes aren’t for everyone), effort is wasted delivering to those who aren’t particularly interested.

Multicast works by using a specific address that one ‘subscribes’ to in order to receive messages sent to that address. So, it’s opt-in, allowing for better efficiency and one day, Internet-wide support for ‘to-many’ communications.

Well, in theory. Actually, multicast is still quite new, and for the most part — from what I understand — it behaves pretty much like broadcast on a local area network. However, support can only increase, and given that many services already use it — Multicast DNS (mDNS), also known as Bonjour, being one of the most well-known examples — it seems a good idea to follow their lead. Note also that IPv6, the successor to IP as we know it, and our saviour-to-be from our little Internet overpopulation problem (among other things), doesn’t even have broadcast provisions — the future is all multicast.

So, for these reasons, Talkie speaks multicast, instead of plain ol’ broadcast.

Making use of multicast is relatively straightforward: To receive, you join a multicast group using setsockopt with IP_ADD_MEMBERSHIP, and the address of the multicast group, which is in the range 224.0.0.0-239.255.255.255 (for IPv4, of course). To send, you just use sendto to transmit data to a multicast group address.

Using multicast on all network interfaces is just a little more complicated. Here’s how it’s done with Talkie:

Sending

The send routine is very similar to the one above, using broadcast. However, instead of using bind to specify the outgoing network interface and enabling broadcast, we assign a multicast interface using setsockopt with IP_MULTICAST_IF. And, instead of transmitting to the broadcast address, we transmit to the multicast group address.

Again, we loop through all network interfaces. This time, we pick out those that support multicast (ifa_flags & IFF_MULTICAST):

const struct ifaddrs *cursor = addrs;
while ( cursor != NULL ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_MULTICAST) ) {
 
    // We will do some stuff in here
 
  }
  cursor = cursor->ifa_next;
}

And, after creating the socket, we assign the interface:

if ( setsockopt(sock_fds[number_sockets], IPPROTO_IP, IP_MULTICAST_IF, &((struct sockaddr_in *)cursor->ifa_addr)->sin_addr, sizeof(struct in_addr)) != 0  ) {
  // Error occurred
  return 0;
}

Finally, as a nicety, we can disable loopback so that we don’t receive our own messages. This isn’t 100% reliable, as certain network conditions can result in the local machine still receiving its outgoing messages, but it can improve efficiency:

u_char loop = 0;
if ( setsockopt(sock_fds[number_sockets], IPPROTO_IP, IP_MULTICAST_LOOP, &loop, sizeof(loop)) != 0 ) {
  // Error occurred
  return 0;
}

Now that our sockets are set up, we can prepare to send to the multicast address:

#define kMulticastAddress "224.0.0.123"
...
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = inet_addr(kMulticastAddress);
addr.sin_port = htons(kPortNumber);

And, as before, we send through each of the sockets we created:

int i;
for ( i=0; i<number_sockets; i++ ) {
  if ( sendto(sock_fds[i], data, length, 0, (struct sockaddr*)&addr, sizeof(addr)) < 0 ) {
    // Error occurred
    return 0;
  }
}

Receiving

Receiving messages from a multicast group on several network interfaces is a little more involved than doing so with broadcast: We need to subscribe to the multicast group from each network interface, explicitly. If we were to just specify no device in particular, the system would choose a single interface for us, neglecting the others.

Joining the multicast group for each interface takes place in a now-familiar loop:

const struct ifaddrs *cursor = addrs;
while ( cursor != NULL ) {
  if ( cursor->ifa_addr->sa_family == AF_INET 
          && !(cursor->ifa_flags & IFF_LOOPBACK) 
          && !(cursor->ifa_flags & IFF_POINTOPOINT) 
          &&  (cursor->ifa_flags & IFF_MULTICAST) ) {
 
    // We will do some stuff in here
 
  }
  cursor = cursor->ifa_next;
}

For each network device, we use the IP_ADD_MEMBERSHIP property with setsockopt to join — thereby subscribing to any messages sent to the multicast group address that reach that network interface.

First, we prepare the join request structure. This provides the multicast group address, and the network interface:

struct ip_mreq multicast_req;
memset(&multicast_req, 0, sizeof(multicast_req));
multicast_req.imr_multiaddr.s_addr = inet_addr(kMulticastAddress);
multicast_req.imr_interface = ((struct sockaddr_in *)cursor->ifa_addr)->sin_addr;

Now we use this structure to join the group:

if ( setsockopt(sock_fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &multicast_req, sizeof(multicast_req)) < 0 ) {
  // Error occurred
  return 0;
}

Now, a caveat: While it’s perfectly legal to join the same multicast group on more than one network interface, and up to 20 memberships may be added to the same socket (see ip(4)), for some reason, OS X spews ‘Address already in use’ errors when we actually attempt it.

As a workaround, we can ‘drop’ the membership first, which would normally have no effect, as we have not yet joined on this interface. However, it enables us to perform the subsequent join, without dropping prior memberships.

So, before we perform the above IP_ADD_MEMBERSHIP, we do a IP_DROP_MEMBERSHIP first:

setsockopt(sock_fd, IPPROTO_IP, IP_DROP_MEMBERSHIP, &multicast_req, sizeof(multicast_req));

This sets up our socket to receive messages sent to the multicast group that are received via any interface.

Here it is all put together: multicast_sample.c

Compile by opening Terminal, and typing make multicast_sample or cc -o multicast_sample multicast_sample .c, then run it with ./multicast_sample "Message to send" to send, or just ./multicast_sample with no arguments to listen.

Still to come

So, now we mostly have networking covered. There’s one obvious omission, though, for an iPhone app: Bluetooth. In Part 3, I’ll discuss how to perform communications over Bluetooth on the iPhone, in a way that’s compatible with the above generic network communications. I’ll also talk about how to connect to other devices automatically, without user intervention — This is one particularly popular feature of Talkie that allows it to behave more like a real walkie-talkie.

I promised in Part 1 that I’d talk about packet formats. We’ve covered a lot of ground in Part 2, however, so it shall be postponed to Part 3 — I’ll discuss how to ensure you get messages in the correct order by using sequence numbers, as well as providing for versioning and a few other bits and pieces.

Talkie is my newest product, the result of a collaboration with a good designer friend, Tim Churchward, who did the user interface.

Talkie is a little different from many of the other walkie talkie applications on the App Store (aside from the fact that much of it was written by me from our motorhome in Tunisia!), and I thought I’d write a little about some of the tech underpinning the app, and some of the choices we made. Along the way it may get a little tutorial-esque.

• This first part will introduce our initial motivations, and will talk about basic broadcast communications — the broadcast communications part may be very familiar to some, in which case it may be worth skipping to the next instalment.
• In the second part, I’ll continue the theme of networking, and will talk about what I ended up with for Talkie’s network code after addressing a couple of things, including switching to multicast.
• Finally, I’ll talk audio, dual platform development, and anything else I think of along the way (Actually, I’m aching to talk about one particular upcoming feature that had me jumping up and down when I first thought of it, but for now, mum’s the word on that one.)## Inspiration

Right from the start, we wanted a product that brought back the fun walkie talkie experience we remember from our childhoods. I’m talking colourful plastic, whip antennas and hiding in tree-houses. This was mostly Tim’s domain, so I shall leave it to him to discuss how he found that in the user interface.

It also meant stepping back from traditional voice chat, with manual call initialisation and termination and simple one-to-one calls. We wanted to mimic a radio, so that meant broadcasting as soon as you hit “Talk” — whereupon anyone in the neighbourhood would hear you.

Basically, we wanted to get as close to the real thing as was practical. This included the addition of a prominent ‘morse code’ button, of course, as well as a ‘squelch’ control for ‘fine tuning’, which simulated the static of bad reception.

Going dual-platform

Soon after I started developing Talkie, I realised I wanted a version for the Mac too.

Having Talkie both on the iPhone and on the Mac made sense, as we envisioned that a fairly common usage pattern would involve communication between a desktop and a handheld — say, someone wandering a campus with the iPhone in their pocket, staying in touch with friends at their desks.

We wanted to offer good value with Talkie, which is why we made Talkie for Mac available for free, when it’s used with Talkie for iPhone.

One of the very convenient things about developing for both iPhone and Mac is that the platforms are so similar, porting code is usually effortless. So, going dual-platform was an easy decision.

Finding common ground

The result of all of this was the need to develop or find a communication protocol and codec that would work across the iPhone and the Mac, over both Bluetooth for iPhone-iPhone communication, and Wifi, for communication between iPhones and iPhones, and iPhones and Macs.

Version 3.0 of the iPhone SDK introduced the GameKit framework and as part of it, GKVoiceChatService, which provides two person voice chat straight out of the box. Immediately it was clear that it wouldn’t serve our purposes — two person is not broadcast. I was also keen to provide a Mac version of Talkie, and given that GameKit is iPhone-only, it was time to go off the beaten track.

There are a variety of voice communication protocols out there, with varying licences and varying complexity and feature sets. I could’ve spent a week or two researching the options and evaluating them against our need for broadcast functionality, figuring out compilation and linking on the iPhone, resolving any compatibility or performance issues that arose, etc.

Or, I could spend a day putting together a few big building blocks provided by Apple (and the underlying Unix system) and have an easily tweakable working solution that precisely meets our needs and provides for future features.

It may not have been the most scientific approach, but the core of Talkie’s functionality was up and running on our home wi-fi network within about four hours, and it was working well.

Getting from a to b (and c, d, e, and f)

For those unfamiliar with the ins and outs of TCP/IP (the most common communication protocol — or rather, set of protocols — between computers, and the fundamental building block of the Internet), communication between computers can be connection-oriented, or connectionless.

Connection-oriented (a.k.a. ‘reliable’) communications are most common: They’re used when you open up a website, connect to iChat, or check your email. This is the TCP part of TCP/IP, and includes niceties like built-in guaranteed delivery via retransmission (providing your cat hasn’t eaten your network cable, of course) and packet ordering so you receive the messages in the right order. This only works between two computers, though – you and the server.

Connectionless (or ‘unreliable’) communications are much more open — it’s basically just a spray of messages: The data isn’t carefully ushered along, it’s just spat out and left to fend for itself. This is UDP, and is commonly used for time-sensitive applications like audio communication, network gaming, etc., where once a packet arrives late, it’s useless: No point mucking about re-sending lost data that’s going to be past its use-by date by the time it arrives. The other thing about UDP is that, because there’s no co-operation required from the destination (to acknowledge receipt, etc.), it lends itself to one-to-many communications — broadcast.

Just what we’re after.

So, we use connectionless communications (UDP) to send messages containing the audio, and anyone who receives them unpackages and plays the audio within.

The basic mechanics of this are fairly simple, once you get past the arguably cryptic C syntax.

Here’s how it works.

Sending

On the transmission end, on startup, you create a socket using the socket call:

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
...
int sock_fd = socket(AF_INET, SOCK_DGRAM, 0);
if ( sock_fd < 0 ) {
   // Error occurred
}

The socket, identified by sock_fd, will be created using the IPv4 domain (AF_INET), a ‘datagram’ type (SOCK_DFRAM: that’s connectionless communication — if we wanted connection-oriented, we could put in SOCK_STREAM here instead), using protocol 0, which is IP.

Then, we enable broadcasting:

int flag = 1;
int result = setsockopt(sock_fd, SOL_SOCKET, SO_BROADCAST, &flag, sizeof(flag));
if ( result < 0 ) {
   // Error occurred
}

That is, we set the SO_BROADCAST option at the SOL_SOCKET level to 1 (via the flag variable, which we pass in as a pointer), thereby requesting permission from the operating system kernel to broadcast.

Now we have the carrier pigeon coop at our disposal, we can start dispatching pigeons. First, we fill out the address:

#define kPortNumber 1234
...
struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_BROADCAST;
addr.sin_port = htons(kPortNumber);

(Note, htons converts kPortNumber into a format that can be understood by any computer, regardless of the way it represents numbers internally.)

Now, send:

NSData *dataToSend;
result = sendto(sock_fd, [data bytes], [data length], 0, (struct sockaddr*)&addr, sizeof(addr));
if ( result < 0 ) {
   // Error occurred
}

Receiving

That takes care of outgoing messages. To receive them, we want another socket:

int sock_fd = socket(AF_INET, SOCK_DGRAM, 0);
if ( sock_fd < 0 ) {
   // Error occurred
}

Now we specify where we want to receive messages from, by ‘binding’ the socket:

struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_ANY;
addr.sin_port = htons(kPortNumber);
result = bind(sock_fd, (struct sockaddr*)&addr, sizeof(addr));
if ( result < 0 ) {
  // Error occurred
}

And we pass a buffer to recvfrom to fill up with tasty morsels of data:

#define kBufferSize 1024
...
char buffer[kBufferSize];
int addr_len = sizeof(addr);
int bytes_received = recvfrom(sock_fd, buffer, kBufferSize, 0, (struck sockaddr*)&addr, &addr_len);
if ( bytes_received < 0 ) {
   // Error occurred
}

Voila, we have bytes_received bytes sitting in buffer for us to do something with. Note also that addr now also contains the address of the sender. Now we can loop and continue to receive data as it comes in, passing the data off to some other part of the application to deal with.

We can put it all together into a simple test app: broadcast_sample.c

Compile it by opening up Terminal, and typing make broadcast_sample (or cc -o broadcast_sample broadcast_sample.c, if you like), then run it with ./broadcast_sample "Message to send", or just ./broadcast_sample to listen.

$ ./broadcast_sample 
Listening...
Hello world!

$ ./broadcast_sample 'Hello world!'
"Hello world!" transmitted.

Coming next

This will work happily between computers with just one network interface. But, if you have more than one (say, wireless, and an Ethernet connection too), you’ll notice that it will only communicate through one of the interfaces. That’s because just the default interface is used. You have to explicitly attend to each interface, to broadcast out of each one, and listen on each one.

In the next part of this series, I’ll write about how I addressed that, and about multicast, which is used in things like Bonjour (MDNS). I’ll also write about designing packet formats.

Thanks for reading!

Drop me a line if you find this helpful.

TPAudioController iOS engine now available

If you’d rather not worry about this stuff, I’ve released TPAudioController, the iOS audio engine library that takes all the work out of creating an audio app.

TPAudioController is a flexible, easy-to-use and complete audio engine for iOS, built upon Core Audio. It handles all setup and management of the low-level Remote IO audio unit system, with support for capturing input via the microphone, automatic mixing of multiple audio signals with per-channel volume and pan controls, and capturing audio system output for session recording.

Read more, and download the sample app or evaluation framework here.

Update: Thanks to Joel Reymont, we now have an explanation for the “CrashIfClientProvidedBogusAudioBufferList” iPhone simulator bug: The simulator doesn’t like mono audio. Thanks, Joel!

Update: Happily, Apple have now created some excellent documentation on Remote IO, with some good sample projects. I recommend using that as a resource, now that it’s there, as that will continue to be updated.

Update: Tom Zicarelli has created a very extensive sample app that demonstrates the use of AUGraph, with all sorts of goodies.

So, we need to obtain an instance of the RemoteIO audio unit, configure it, and hook it up to a recording callback, which is used to notify you that there is data ready to be grabbed, and where you pull the data from the audio unit.

Overview

1. Identify the audio component (kAudioUnitType_Output/ kAudioUnitSubType_RemoteIO/ kAudioUnitManufacturerApple)
2. Use AudioComponentFindNext(NULL, &descriptionOfAudioComponent) to obtain the AudioComponent, which is like the factory with which you obtain the audio unit
   
3. Use AudioComponentInstanceNew(ourComponent, &audioUnit) to make an instance of the audio unit
   
4. Enable IO for recording and possibly playback with AudioUnitSetProperty
   
5. Describe the audio format in an AudioStreamBasicDescription structure, and apply the format using AudioUnitSetProperty
   
6. Provide a callback for recording, and possibly playback, again using AudioUnitSetProperty
   
7. Allocate some buffers
   
8. Initialise the audio unit
   
9. Start the audio unit
   
10. Rejoice
    

Here’s my code: I’m using both recording and playback. Use what applies to you!

Initialisation

Initialisation looks like this. We have a member variable of type AudioComponentInstance which will contain our audio unit.

The audio format described below uses SInt16 for samples (i.e. signed, 16 bits per sample)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

#define kOutputBus 0
#define kInputBus 1
 
// ...
 
 
OSStatus status;
AudioComponentInstance audioUnit;
 
// Describe audio component
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_RemoteIO;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
 
// Get component
AudioComponent inputComponent = AudioComponentFindNext(NULL, &desc);
 
// Get audio units
status = AudioComponentInstanceNew(inputComponent, &audioUnit);
checkStatus(status);
 
// Enable IO for recording
UInt32 flag = 1;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_EnableIO, 
                              kAudioUnitScope_Input, 
                              kInputBus,
                              &flag, 
                              sizeof(flag));
checkStatus(status);
 
// Enable IO for playback
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_EnableIO, 
                              kAudioUnitScope_Output, 
                              kOutputBus,
                              &flag, 
                              sizeof(flag));
checkStatus(status);
 
// Describe format
audioFormat.mSampleRate			= 44100.00;
audioFormat.mFormatID			= kAudioFormatLinearPCM;
audioFormat.mFormatFlags		= kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket	= 1;
audioFormat.mChannelsPerFrame	= 1;
audioFormat.mBitsPerChannel		= 16;
audioFormat.mBytesPerPacket		= 2;
audioFormat.mBytesPerFrame		= 2;
 
// Apply format
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_StreamFormat, 
                              kAudioUnitScope_Output, 
                              kInputBus, 
                              &audioFormat, 
                              sizeof(audioFormat));
checkStatus(status);
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_StreamFormat, 
                              kAudioUnitScope_Input, 
                              kOutputBus, 
                              &audioFormat, 
                              sizeof(audioFormat));
checkStatus(status);
 
 
// Set input callback
AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = recordingCallback;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioOutputUnitProperty_SetInputCallback, 
                              kAudioUnitScope_Global, 
                              kInputBus, 
                              &callbackStruct, 
                              sizeof(callbackStruct));
checkStatus(status);
 
// Set output callback
callbackStruct.inputProc = playbackCallback;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_SetRenderCallback, 
                              kAudioUnitScope_Global, 
                              kOutputBus,
                              &callbackStruct, 
                              sizeof(callbackStruct));
checkStatus(status);
 
// Disable buffer allocation for the recorder (optional - do this if we want to pass in our own)
flag = 0;
status = AudioUnitSetProperty(audioUnit, 
                              kAudioUnitProperty_ShouldAllocateBuffer,
                              kAudioUnitScope_Output, 
                              kInputBus,
                              &flag, 
                              sizeof(flag));
 
// TODO: Allocate our own buffers if we want
 
// Initialise
status = AudioUnitInitialize(audioUnit);
checkStatus(status);

Then, when you’re ready to start:

1
2

OSStatus status = AudioOutputUnitStart(audioUnit);
checkStatus(status);

And to stop:

1
2

OSStatus status = AudioOutputUnitStop(audioUnit);
checkStatus(status);

Then, when we’re finished:

1

AudioUnitUninitialize(audioUnit);

And now for our callbacks.

Recording

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

static OSStatus recordingCallback(void *inRefCon, 
                                  AudioUnitRenderActionFlags *ioActionFlags, 
                                  const AudioTimeStamp *inTimeStamp, 
                                  UInt32 inBusNumber, 
                                  UInt32 inNumberFrames, 
                                  AudioBufferList *ioData) {
 
    // TODO: Use inRefCon to access our interface object to do stuff
    // Then, use inNumberFrames to figure out how much data is available, and make
    // that much space available in buffers in an AudioBufferList.
 
    AudioBufferList *bufferList; // <- Fill this up with buffers (you will want to malloc it, as it's a dynamic-length list)
 
    // Then:
    // Obtain recorded samples
 
    OSStatus status;
 
    status = AudioUnitRender([audioInterface audioUnit], 
                             ioActionFlags, 
                             inTimeStamp, 
                             inBusNumber, 
                             inNumberFrames, 
                             bufferList);
    checkStatus(status);
 
    // Now, we have the samples we just read sitting in buffers in bufferList
    DoStuffWithTheRecordedAudio(bufferList);
    return noErr;
}

Playback

1
2
3
4
5
6
7
8
9
10
11

static OSStatus playbackCallback(void *inRefCon, 
                                  AudioUnitRenderActionFlags *ioActionFlags, 
                                  const AudioTimeStamp *inTimeStamp, 
                                  UInt32 inBusNumber, 
                                  UInt32 inNumberFrames, 
                                  AudioBufferList *ioData) {    
    // Notes: ioData contains buffers (may be more than one!)
    // Fill them up as much as you can. Remember to set the size value in each buffer to match how
    // much data is in the buffer.
    return noErr;
}

Finally, rejoice with me in this discovery ;)

Resources that helped

• http://pastie.org/pastes/219616
• http://developer.apple.com/samplecode/CAPlayThrough/listing8.html
• http://listas.apesol.org/pipermail/svn-libsdl.org/2008-July/000797.html

No thanks at all to Apple for their lack of accessible documentation on this topic – They really have a long way to go here! Also boo to them with their lack of search engine, and refusal to open up their docs to Google. It’s a jungle out there!

Update: You can adjust the latency of RemoteIO (and, in fact, any other audio framework) by setting the kAudioSessionProperty_PreferredHardwareIOBufferDuration property:

float aBufferLength = 0.005; // In seconds
AudioSessionSetProperty(kAudioSessionProperty_PreferredHardwareIOBufferDuration, 
                        sizeof(aBufferLength), &aBufferLength);

This adjusts the length of buffers that’re passed to you – if buffer length was originally, say, 1024 samples, then halving the number of samples halves the amount of time taken to process them.