Tuesday, June 30, 2009

Streaming and playing fixed-length MP3s using the iPhone SDK

The Problem

Lately, I've been doing a fair amount of coding using Objective-C and the iPhone SDK. Once I got used to the language I really started to like it. The more I used the SDK the more impressed I became with how much I could do with it. However, when a customer of ours at Episodic needed an iPhone App that could play a list of MP3s (RSS Feed to be exact) with "iPod-like" functionality, I was surprised to find that this is not something I can do with the embedded Quicktime Player.  In fact, you can't even just hook into the MPMoviePlayerController.  For example, there is no way to know when one songs ends so that I can start the next song since the MPMoviePlayerPlaybackDidFinishNotification notification is fired when the song ends or the user hits the done button.  So after a lot of searching I came across Matt Gallagher's great post "Revisiting an old post: Streaming and playing an MP3 stream".

Modifying the AudioStreamer Class to Play Fixed-Length MP3s

So Matt's post got me part of the way there.  It at least gave me an understanding of the tools I would need to use to accomplish my task.  The main difference between what he had and what I needed is that he was playing an MP3 stream and I wanted to play MP3 files.  The first change I made was I added a new constructor to the AudioStreamer class.  This allowed me to initialized the class with a URL to a file on the file system (I'll talk about how to get the file on your file system later).

- (id)initWithFileURL:(NSURL *)aURL {
[self initWithURL:aURL];
fixedLength = YES;
return self;
}


This constructor sets a flag to indicate that we are streaming fixed-length MP3s.  I then key off this flag in the openFileStream method.

if (fixedLength) {
stream = CFReadStreamCreateWithFile(
kCFAllocatorDefault, (CFURLRef)url);
} else {
CFHTTPMessageRef message = CFHTTPMessageCreateRequest(
NULL, (CFStringRef)@"GET", (CFURLRef)url,

kCFHTTPVersion1_1);
stream = CFReadStreamCreateForHTTPRequest(NULL, message);
CFRelease(message);
...


The above code simply creates the stream from the file system instead of a remote file. Because the result is still a CFReadStreamRef object the rest of the class mostly remains unchanged.

Downloading the MP3

In my controller I just need to download the MP3 file to a location on the file system.  This can be done very easily by using a combination of NSURLConnection and NSFileHandle.

audioFile = [[NSFileHandle
fileHandleForWritingAtPath:downloadFileName] retain];
NSURLRequest *downloadRequest = [NSURLRequest
requestWithURL:resourceURL
cachePolicy:NSURLRequestUseProtocolCachePolicy
timeoutInterval:60.0];
download = [[NSURLConnection alloc]
initWithRequest:downloadRequest delegate:self];


This code creates the file to write the data to and creates the connection to fetch the data.  I know just need to handle the data.

- (void)connection:(NSURLConnection *)connection
didReceiveData:(NSData *)data {
// Append the data to the file on the file system
[audioFile writeData:data];
downloadedLength += [data length];

// When we get enough of the file, then just start playing.
if (!streamer && downloadedLength > 

     DEFAULT_MIN_LENGTH_TO_PLAY) {
NSLog(@"start playback for %@", downloadFileName);
[self initAndStartStreamer];
}
}


The above code is the NSURLConnection delegate method for receiving the data and writing the data to the file. Once I have "enough" of the MP3 I can start playing it by create the streamer and passing it the URL to our audioFile.

NEW: Handling EOF

One bug that I ran into since my initial post is that if I actually put this on my phone and went outside, away from my wireless network, the song would sometimes end early.  What was happening is that the AudioStreamer was playing the file faster than it was being downloaded.  The AudioStreamer would hit the EOF and then stop.  To prevent this case I added some "throttling" to the playback loop that checks if we are too close to the end of the file and changes the state to buffering until we have enough to continue. The following code is in startInternal.

// Flag to indicate that we have gotten too close to the EOF
// before the entire file has downloaded so we need throttle
// the playback.
BOOL isThrottling = NO;{

//
// Process the run loop until playback is finished or failed.
//
BOOL isRunning = YES;
do
{
// If we are playing a fixed-length MP3 make sure we are 

// not too close to the end of the file. This prevents us
// from hitting the end of the file before it is fully 
// downloaded. Very useful when not on 3G since the song 
// may be played faster than it is downloaded.
if (!fixedLength || self.fileDownloadComplete ||
self.fileDownloadCurrentSize > (fileDownloadBytesRead +
(kAQBufSize * kNumAQBufs))) {

isRunning = [[NSRunLoop currentRunLoop
runMode:NSDefaultRunLoopMode
beforeDate:

  [NSDate dateWithTimeIntervalSinceNow:0.25]];

if (isThrottling) {
isThrottling = NO;
AudioQueueStart(audioQueue, NULL);
self.state = AS_PLAYING;
}

//
// If there are no queued buffers, we need to check here 

// since the handleBufferCompleteForQueue:buffer: should 
// not change the state(may not enter the synchronized 
// section).
//
if (buffersUsed == 0 && self.state == AS_PLAYING)
{
self.state = AS_BUFFERING;
}
} else if (!isThrottling && self.state == AS_PLAYING) {
NSLog(@"Throttle because we are too close to EOF.");

self.state = AS_BUFFERING;

AudioQueuePause(audioQueue);
isThrottling = YES;

// Sleep for a few seconds
[NSThread sleepForTimeInterval:3.0];
}

} while (isRunning && ![self runLoopShouldExit]);


Seeking?

I was unable to figure out a way to allow for seeking. This would also provide a way for playback to pick up where it left off when the connection during the download phase.  If anyone has ideas on how to support this I would love to hear about it.

Conclusion

This is still far from ideal and it really seems like this is something that should be a lot easier to do within the SDK.  In the meantime, hopefully this is helpful to others.

You can download my source code here.

32 comments:

  1. I haven't really given it another try.

    ReplyDelete
  2. Thanks for the feedback. Certainly my code is not without bugs. It is really just meant to be a reference to help people get started. Since I did this post I have updated my app that this code is ported from and fixed several small bugs like this. These both just sounds like I need to do a better job of handling events in playbackStateChanged. I'll try to look into these and update my code when I find some spare time.

    ReplyDelete
  3. Thank you! It's a great job you did here...but
    What about the progress slider? Did you find how to make it? When do you think you'll be able to post your new project? We could maybe help you ;o)

    Thanks for sharing your work

    ReplyDelete
  4. Thanks for posting about this, I would love to read more about this topic.

    ReplyDelete
  5. Good work. Curious if you've seen problems streaming some mp3 files? At first glance, the mp3 files seem to be > 7MB.

    ReplyDelete
  6. I did have some problems with a group of MP3s but it wasn't size related. What's the problem that you are seeing with large MP3 files? Do they start playing at all?

    ReplyDelete
  7. Hi, as far as I can tell, the code as written cannot be extended to allow seek because it uses CFReadStream and streams are non-seekable. So someone wanting to provide a seek feature would have to re-code all of the file i/o, for example to use NSFileHandle and potentially waitForDataInBackgroundAndNotify if they wanted asynchronous file i/o.

    ReplyDelete
  8. Dear Randy

    Thanks for your reply.

    I need to play a succession of files with absolutely no gap between --- as if it were a single audio file. I've just had a quick go at your method: it works! *but* there's audible noise between the files. So I can't use it :(

    I suppose an AudioFileStream is to stream a file.

    Perhaps I should concatenate my files together into a single file-like object before sending it to the streamer.

    The only other option I can think of is to avoid AudioFileStream and use AudioQueue directly, ..., if it is possible to send data from more than one file down a single queue.

    Any ideas? I'm working in straight C for now.

    Thanks and best wishes

    Ivan

    ReplyDelete
  9. Thanks, that's a good idea.

    My merging files idea has a brutal simplicity to it, so I'll try that first. The metadata is all in 44 bytes at the start of the file (for .wav files) and that will be easy to construct (I hope): everything will be constant apart from file length.

    I'll let you know how I get on.

    Best

    Ivan

    ReplyDelete
  10. Hmm my first post didn't show up.

    Anyways.. I'm having problem playing files based on this code and your application.

    The streams works perfectly when playing directly. But when I'm using this to cache the files or play with this player it crashes.
    This is very string.

    Can anyone help me?

    ReplyDelete
  11. Is seeking not possible with this code?

    ReplyDelete
  12. Seeking is not implemented in this code.

    ReplyDelete
  13. One other note. It looks like that code is just attempting to retrieve the bitrate property, but it doesn't look like that value is used anywhere (as far as i can tell). So, I tried commenting lines 1439 and 1440, and the first second or two of the song played before EXC_BAD_ACCESS occurred. It's failing in HP_IOCucleTelemetry_SaveTelmetryItem. Still stumped...

    ReplyDelete
  14. HI Shrads,

    No, I have not. I moved on to another task while awaiting a response here...but it doesn't look like I'm going to get one. I'll post the solution here if I find one.

    ReplyDelete
  15. I'm not sure. It has worked for just about any mp3 I have tried.

    ReplyDelete
  16. Looks like it won't work with anything below a 128kBPS MP3 file.

    ReplyDelete
  17. Nice post buddy thanks a lot :)

    ReplyDelete
  18. Randy - sorry to bug, I've been searching a ton and can't seem to find any rhyme/reason why the specific specs on these smaller, lower-quality mp3s don't seem to work with the stream player. I'm trying to get something to play smaller mp3's due to bandwidth contraints (and at&t's sucky data network!). I'm trying to keep the mp3s under 10k size to play in my app. Since the existing mp3's I have (about 1000) are all recorded at 48kBPS/16Hz they're nice and small. Any other suggestions/ideas? Sorry for the newbie questions, just trying to incorporate your and Matt's excellent work somehow w/o having to resort to larger mp3s. Thanks!

    ReplyDelete
  19. Thanks Randy I'll take a look and advise if I find a solution.

    ReplyDelete
  20. Nice post buddy thanks a lot :)

    ReplyDelete
  21. Thanks Randy I'll take a look and advise if I find a solution.

    ReplyDelete
  22. Thanks Randy I'll take a look and advise if I find a solution.

    ReplyDelete
  23. Nice post buddy thanks a lot :)

    ReplyDelete
  24. Nice post buddy thanks a lot :)

    ReplyDelete
  25. Nice post buddy thanks a lot :)

    ReplyDelete
  26. Nice post buddy thanks a lot :)

    ReplyDelete
  27. Thanks Randy I'll take a look and advise if I find a solution.

    ReplyDelete
  28. Thanks Randy I'll take a look and advise if I find a solution.

    ReplyDelete
  29. Randy - sorry to bug, I've been searching a ton and can't seem to find any rhyme/reason why the specific specs on these smaller, lower-quality mp3s don't seem to work with the stream player. I'm trying to get something to play smaller mp3's due to bandwidth contraints (and at&t's sucky data network!). I'm trying to keep the mp3s under 10k size to play in my app. Since the existing mp3's I have (about 1000) are all recorded at 48kBPS/16Hz they're nice and small. Any other suggestions/ideas? Sorry for the newbie questions, just trying to incorporate your and Matt's excellent work somehow w/o having to resort to larger mp3s. Thanks!

    ReplyDelete
  30. Nice post buddy thanks a lot :)

    ReplyDelete
  31. Hey Randy, I know this post is getting old, but if you're still trying to figure out seeking in a downloaded file, you should take a look at CFReadStreamSetProperty and kCFStreamPropertyFileCurrentOffset. I got that to work. Have you been able to fix the problem with the play progress not being accurate during the last few seconds of the file?

    ReplyDelete
  32. bryanmontz, How did you manage to use CFReadStreamSetProperty & FileCurrentOffset ? I've got silence after seek :(

    ReplyDelete