Jun
30

Streaming and playing fixed-length MP3s using the iPhone SDK

Randy Audio Queue Services, AudioStreamer, Objective-C, iPhone SDK 2009-06-30

The Problem

Lately, I’ve been doing a fair amount of coding using Objective-C and the iPhone SDK. Once I got used to the language I really started to like it. The more I used the SDK the more impressed I became with how much I could do with it. However, when a customer of ours at Episodic needed an iPhone App that could play a list of MP3s (RSS Feed to be exact) with “iPod-like” functionality, I was surprised to find that this is not something I can do with the embedded Quicktime Player.  In fact, you can’t even just hook into the MPMoviePlayerController.  For example, there is no way to know when one songs ends so that I can start the next song since the MPMoviePlayerPlaybackDidFinishNotification notification is fired when the song ends or the user hits the done button.  So after a lot of searching I came across Matt Gallagher’s great post “Revisiting an old post: Streaming and playing an MP3 stream“.

Modifying the AudioStreamer Class to Play Fixed-Length MP3s

So Matt’s post got me part of the way there.  It at least gave me an understanding of the tools I would need to use to accomplish my task.  The main difference between what he had and what I needed is that he was playing an MP3 stream and I wanted to play MP3 files.  The first change I made was I added a new constructor to the AudioStreamer class.  This allowed me to initialized the class with a URL to a file on the file system (I’ll talk about how to get the file on your file system later).

- (id)initWithFileURL:(NSURL *)aURL {
[self initWithURL:aURL];
fixedLength = YES;
return self;
}

This constructor sets a flag to indicate that we are streaming fixed-length MP3s.  I then key off this flag in the openFileStream method.

if (fixedLength) {
stream = CFReadStreamCreateWithFile(
kCFAllocatorDefault, (CFURLRef)url);
} else {
CFHTTPMessageRef message = CFHTTPMessageCreateRequest(
NULL, (CFStringRef)@"GET", (CFURLRef)url, kCFHTTPVersion1_1);
stream = CFReadStreamCreateForHTTPRequest(NULL, message);
CFRelease(message);
...

The above code simply creates the stream from the file system instead of a remote file. Because the result is still a CFReadStreamRef object the rest of the class mostly remains unchanged.

Downloading the MP3

In my controller I just need to download the MP3 file to a location on the file system.  This can be done very easily by using a combination of NSURLConnection and NSFileHandle.

audioFile = [[NSFileHandle
fileHandleForWritingAtPath:downloadFileName] retain];
NSURLRequest *downloadRequest = [NSURLRequest
requestWithURL:resourceURL
cachePolicy:NSURLRequestUseProtocolCachePolicy
timeoutInterval:60.0];
download = [[NSURLConnection alloc]
initWithRequest:downloadRequest delegate:self];

This code creates the file to write the data to and creates the connection to fetch the data.  I know just need to handle the data.

- (void)connection:(NSURLConnection *)connection
didReceiveData:(NSData *)data {
// Append the data to the file on the file system
[audioFile writeData:data];
downloadedLength += [data length];

// When we get enough of the file, then just start playing it.
if (!streamer && downloadedLength > DEFAULT_MIN_LENGTH_TO_PLAY) {
NSLog(@"start playback for %@", downloadFileName);
[self initAndStartStreamer];
}
}

The above code is the NSURLConnection delegate method for receiving the data and writing the data to the file. Once I have “enough” of the MP3 I can start playing it by create the streamer and passing it the URL to our audioFile.

NEW: Handling EOF

One bug that I ran into since my initial post is that if I actually put this on my phone and went outside, away from my wireless network, the song would sometimes end early.  What was happening is that the AudioStreamer was playing the file faster than it was being downloaded.  The AudioStreamer would hit the EOF and then stop.  To prevent this case I added some “throttling” to the playback loop that checks if we are too close to the end of the file and changes the state to buffering until we have enough to continue. The following code is in startInternal.

// Flag to indicate that we have gotten too close to the EOF before the
// entire file has downloaded so we need throttle the playback.
BOOL isThrottling = NO;{

//
// Process the run loop until playback is finished or failed.
//
BOOL isRunning = YES;
do
{
// If we are playing a fixed-length MP3 make sure we are not too close
// to the end of the file. This prevents us from hitting the end of
// the file. before it is fully downloaded. Very useful when not on
// 3G since the song may be played faster than it is downloaded.
if (!fixedLength || self.fileDownloadComplete ||
self.fileDownloadCurrentSize > (fileDownloadBytesRead +
(kAQBufSize * kNumAQBufs))) {

isRunning = [[NSRunLoop currentRunLoop
runMode:NSDefaultRunLoopMode
beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.25]];

if (isThrottling) {
isThrottling = NO;
AudioQueueStart(audioQueue, NULL);
self.state = AS_PLAYING;
}

//
// If there are no queued buffers, we need to check here since the
// handleBufferCompleteForQueue:buffer: should not change the state
// (may not enter the synchronized section).
//
if (buffersUsed == 0 && self.state == AS_PLAYING)
{
self.state = AS_BUFFERING;
}
} else if (!isThrottling && self.state == AS_PLAYING) {
NSLog(@"Start throttling because we are too close to EOF.");

self.state = AS_BUFFERING;

AudioQueuePause(audioQueue);
isThrottling = YES;

// Sleep for a few seconds
[NSThread sleepForTimeInterval:3.0];
}

} while (isRunning && ![self runLoopShouldExit]);

Seeking?

I was unable to figure out a way to allow for seeking. This would also provide a way for playback to pick up where it left off when the connection during the download phase.  If anyone has ideas on how to support this I would love to hear about it.

Conclusion

This is still far from ideal and it really seems like this is something that should be a lot easier to do within the SDK.  In the meantime, hopefully this is helpful to others.

You can download my source code here.

Address: http://www.atwofer.com/2009/06/30/streaming-and-playing-fixed-length-mp3s-using-the-iphone-sdk/
« Getting the errorID from an ErrorEvent in ActionScript
Handling redirects with initWithContentURL »
Trackback

37 comments untill now

  1. ncohen @ 2009-07-13 16:24

    Did you figure out how to implement the progress slider?

  2. Jarmod @ 2009-07-15 13:02

    Thanks for the work you’ve done here. A couple of minor problems I noticed with this:

    1. 3/4 of the way through playing a small MP3 file, the app put up the ‘Buffering…” message with the spinner. It continued playing the MP3 without any obvious audio gap and then finally stopped playing. It never cleared the “Buffering…” message, however, which left the app uncontrollable.

    2. it showed the “Buffering…” message when I gave it a bad URL, again leaving the app uncontrollable.

  3. admin @ 2009-07-15 13:07

    I haven’t really given it another try.

  4. admin @ 2009-07-15 13:13

    Thanks for the feedback. Certainly my code is not without bugs. It is really just meant to be a reference to help people get started. Since I did this post I have updated my app that this code is ported from and fixed several small bugs like this. These both just sounds like I need to do a better job of handling events in playbackStateChanged. I’ll try to look into these and update my code when I find some spare time.

  5. ncohen @ 2009-07-16 18:05

    Thank you! It’s a great job you did here…but
    What about the progress slider? Did you find how to make it? When do you think you’ll be able to post your new project? We could maybe help you ;o)

    Thanks for sharing your work

  6. MT @ 2009-07-25 12:47

    Good work. Curious if you’ve seen problems streaming some mp3 files? At first glance, the mp3 files seem to be > 7MB.

  7. Randy @ 2009-07-26 18:00

    I did have some problems with a group of MP3s but it wasn’t size related. What’s the problem that you are seeing with large MP3 files? Do they start playing at all?

  8. michael @ 2009-08-04 19:53

    i am trying the seeking problem too.
    \this packet of code is helpful, and I am told it works

    I think what it does, is depends on the data being read into NSData, and skipping over to the buffer.

    I have been trying to implment this,(for days almost)
    but no luck

    AudioQueueStop( audioQueue, true );
    UInt32 flags = 0;
    err = AudioFileStreamParseBytes(audioFileStream, length, bytes, kAudioFileStreamParseFlag_Discontinuity);

    OSStatus status = AudioFileStreamSeek( audioFileStream, framePacket.mPacket, &currentOffset, &flags );
    NSLog(@”Setting next byte offset to: %qi, flags: %d”, (long long)currentOffset, flags);

    // then read data from the new offset set by AudioFileStreamSeek
    [fileHandle seekToFileOffset:currentOffset];
    NSData * data = “” readDataOfLength:4096];

    flags = kAudioFileStreamParseFlag_Discontinuity;
    status = AudioFileStreamParseBytes( stream, [data length], [data bytes], flags );
    if( status != noErr )
    NSLog(@”Error parsing bytes: %d”, status);

  9. jarmod @ 2009-08-11 10:07

    Hi, I’ve been digging further into this code and there is a very significant issue related to the streamer’s knowledge of its state.

    The code does two things for fixed audio files:

    1) read audio data from a file, and
    2) feed that data into audio queue services.

    The streamer uses one state variable to cover both things. It needs two state variables. Bugs caused by the current approach include, but are not limited to:

    1) if the file being read is shorter than 32 buffers worth (about 4 seconds for a basic WAV file) then it will *never* play any audio. This is because the state will never transition from ‘waiting for data’ to ‘waiting for queue to start’ because it has already hit EOF and the state has changed to stopped/EOF.

    2) if you pause an audio file for long enough (and that might be just a few seconds) then the file streamer can hit EOF long before the audio ends in which case the state changes to stopped/EOF and you can never subsequently pause the audio again. This is because the pause code requires the streamer to be in playing state and yet it is in stopped/EOF state so the request to pause is ignored.

    So, the code needs two independent states to track:

    1. the state of the file streaming, and
    2. the state of the audio playback.

  10. Randy @ 2009-08-11 10:45

    So I looked into these issues you have listed here.

    For #1 I see the code you are talking about. The “handleReadFromStream” method should still try to queue up the buffer which just means that you could modify the “enqueueBuffer” to force it to start if the EOF was reached and we haven’t started playback yet. However, this is a bit of a guess since I don’t have an MP3 to test with here.

    For #2, I was unable to reproduce this issue. Playback should only stop when when all buffers have been played and not when all buffers have been filled. The new code I just posted includes a pause button and everything seems to be fine so let me know if you can reproduce this issue.

  11. infinity @ 2009-08-13 08:01

    hi
    how can i get or calculate the duration of the mp3 file

  12. jarmod @ 2009-08-14 10:45

    Hi, as far as I can tell, the code as written cannot be extended to allow seek because it uses CFReadStream and streams are non-seekable. So someone wanting to provide a seek feature would have to re-code all of the file i/o, for example to use NSFileHandle and potentially waitForDataInBackgroundAndNotify if they wanted asynchronous file i/o.

  13. Ricky @ 2009-10-25 15:33

    Any chance you could post the updated code with the fixed bugs (e.g. unable to leave the “Buffering” screen). Thanks!

  14. Randy @ 2009-10-26 07:50

    Hey Ricky,

    I posted updated code a while back which I thought addressed these types of problems. Can you give me a case where you get stuck on the buffering screen?

  15. Ivan @ 2009-10-27 09:27

    Dear Randy

    Thank you for this post. I found it very valuable.

    Is it possible to send more than one file into the stream?

    I’d like to send a number of audio files (two to start with) into an AudioFileStream. I’m using wav files with a stripped down version of your code, and using read() to read data from a file descriptor instead of recv() to receive data from a socket. My basic idea is just to enqueue the buffer and flush the queue, then run the while (!myData->failed) block again to read and parse the data from the new file descriptor:

    while (!myData->failed) {
    // read data from the file
    printf(”->reading file 2\n”);
    ssize_t bytesRecvd = read(fd, buf, kRecvBufSize);
    printf(”bytesRecvd %d\n”, bytesRecvd);
    if (bytesRecvd audioFileStream, bytesRecvd, buf, 0);
    if (err) { PRINTERROR(”AudioFileStreamParseBytes”); break; }
    }

    The read part is working, but AudioFileStreamParseBytes is not finding any Property data or Audio data in the second wav file.

    I can’t think what I need to do? Do I need to Close and reOpen the stream for each file? I’d appreciate any advise, and I’ll report back if/when I get it working.

    Best wishes

    Ivan

  16. Randy @ 2009-10-27 10:28

    Off the top of my head, I’m really not sure. I haven’t tried anything like this. The way I have used the AudioStreamer for multiple files is to stop and release the streamer object when the first audio file completes, then create a new AudioStreamer with the new audio file to play. Is there a reason you need to pass two files at a time?

  17. Ivan @ 2009-10-27 15:27

    Dear Randy

    Thanks for your reply.

    I need to play a succession of files with absolutely no gap between — as if it were a single audio file. I’ve just had a quick go at your method: it works! *but* there’s audible noise between the files. So I can’t use it :(

    I suppose an AudioFileStream is to stream a file.

    Perhaps I should concatenate my files together into a single file-like object before sending it to the streamer.

    The only other option I can think of is to avoid AudioFileStream and use AudioQueue directly, …, if it is possible to send data from more than one file down a single queue.

    Any ideas? I’m working in straight C for now.

    Thanks and best wishes

    Ivan

  18. Randy @ 2009-10-27 18:20

    I don’t think you can really merge files easily since the Audio Queue Services probably relies on some metadata in the files. What I would try next if I were you is to have two instances of the AudioStreamer class. One for the current file and one for the next. However, for the “next AudioStreamer” have it buffer until the first one finishes and it is told it can start playback. I’m not exactly sure on how to do this but I think the loop in startInternal is a good place to start. You can make it so that AudioQueueStart is not called until AudioStreamer gets the go ahead, but in the meantime it has buffered enough that it is ready to go.

    Hope this helps.

  19. Ivan @ 2009-10-28 01:15

    Thanks, that’s a good idea.

    My merging files idea has a brutal simplicity to it, so I’ll try that first. The metadata is all in 44 bytes at the start of the file (for .wav files) and that will be easy to construct (I hope): everything will be constant apart from file length.

    I’ll let you know how I get on.

    Best

    Ivan

  20. Mikael @ 2009-11-21 12:35

    Hi,

    I have a problem. Some of the files I stream/download from my server makes the app crash and I receive a “Program received signal: “EXC_BAD_ACCESS”.”

    What can cause this?

  21. Ivan @ 2009-11-23 06:47

    Randy

    Just a quick note to say that merging works — as long as you look after the header, so it requires some bit twiddling. What you end up with you can bung down the queue, it is identical to wav file content.

    Best

    Ivan

  22. Mikael @ 2009-11-25 07:31

    Hi again,

    When I play the stream from url it works as from the originating blog post.
    I’ve tried to buffer 150kb before writing to disk. But this doesn’t help.

    Any ide?

  23. Mikael @ 2009-11-26 05:23

    Hmm my first post didn’t show up.

    Anyways.. I’m having problem playing files based on this code and your application.

    The streams works perfectly when playing directly. But when I’m using this to cache the files or play with this player it crashes.
    This is very string.

    Can anyone help me?

  24. Mikael @ 2009-11-26 06:08

    You can try with this -> http://mikl.se/pod.mp3

  25. Damian @ 2009-12-21 15:03

    Is seeking not possible with this code?

  26. Randy @ 2009-12-21 15:24

    Seeking is not implemented in this code.

  27. Dustin Clark @ 2009-12-22 15:27

    Hi Randy,

    First of all, thanks for the post. It is very helpful…but I’m having a few issues. It doesn’t work for m4a files for some reason (i.e. iTunes previews). Here’s an example URL: http://a1.phobos.apple.com/us/r1000/047/Music/04/ba/02/mzm.qfcwiwxm.aac.p.m4a.

    It fails on line 1439 of AudioStreamer, and if I’m translating the error code correctly, the error is kAudioFileStreamError_DataUnavailable. In apple’s documentation (http://developer.apple.com/mac/library/documentation/MusicAudio/Reference/AudioStreamReference/Reference/reference.html), it mentions that this can happen “when you call the AudioFileStreamGetProperty function from inside the property listener, because of boundaries in the input data”. Matt Gallagher’s code doesn’t seem to have this issue, so it has to be something about retrieving the audio property from the local file. I’m stumped at the moment….any ideas?

    Thanks,
    Dustin

  28. Dustin Clark @ 2009-12-22 15:43

    One other note. It looks like that code is just attempting to retrieve the bitrate property, but it doesn’t look like that value is used anywhere (as far as i can tell). So, I tried commenting lines 1439 and 1440, and the first second or two of the song played before EXC_BAD_ACCESS occurred. It’s failing in HP_IOCucleTelemetry_SaveTelmetryItem. Still stumped…

  29. Mike @ 2010-01-01 08:08

    Ivan,

    You commented that merging wav files works… could you provide a sample of how you made it work? I have been working through CoreAudio for about a month to try and concat audio files together and have been unsuccessful. I am able to mix them, but not concat them.

    Thanks

    Mike

  30. manoranjan @ 2010-01-03 07:55

    Hi

    I am try to play Audio url.

    But i am getting error “EXC_BAD_ACCESS”.
    After that it fail due to HP_IOCucleTelemetry_SaveTelmetryItem.

    How i rectify that problem.

    May any one help me??

    Thanks

  31. Shrads @ 2010-01-04 03:14

    Hi Dustin,
    Were you able to resolve this issue?
    I am also facing this EXC_BAD_ACCESS crash.
    Is there some way one can avoid it?

  32. Dustin Clark @ 2010-01-05 11:47

    HI Shrads,

    No, I have not. I moved on to another task while awaiting a response here…but it doesn’t look like I’m going to get one. I’ll post the solution here if I find one.

  33. Hitesh @ 2010-01-08 03:01

    Hello,
    First of all, i would like to say Thanks for helping this code for my Application, But i want to know how it is possible for FastForwarding & Rewind functionality possible, please reply me a.s.a.p, it would be great help for me.

  34. QFDaniel @ 2010-01-11 09:24

    Actually,good post. thx

  35. Swap @ 2010-01-15 19:05

    Hi Randy,

    Thanks for this code base. Helps a lot.

    There is 1 critical issue we found in the throttling code. We have observed that under certain cases the code never enters the “Stop Throttling” (below) after the throttling has been started:

    if (isThrottling) {
    isThrottling = NO;
    AudioQueueStart(audioQueue, NULL);
    self.state = AS_PLAYING;
    }

    This results in an interrupting in the stream playback. Manually clicking pause/play starts the stream again.

    Any idea how this can be fixed? Would really appreciate your reply.

    Thanks a lot in advance.

    Swap

  36. Saurabh @ 2010-01-27 03:45

    Hi,

    Same problem as the others… The app crashes when trying to play certain files. Please can anyone enlighten more on this issue. I tried changing the bitrate of the file, but that doesn’t work.

    Thanks…

  37. Chris Hobbs @ 2010-02-04 19:45

    I’m not sure why you used the file system, perhaps there was an external reason. I was able to get fixed length mp3s to play from urls with out too much problem.

    My main goal was to get gapless playback of fixed length remote mp3s.
    Here’s the progress so far:
    http://github.com/ckhsponge/AudioStreamer

    It’s not perfect yet, I need to figure out how to ignore the mp3 padded silence, I believe.

Add your comment now