I'm currently making a swift app that records an aac file from an audiostream and uses shazamKit to identify the song. The stream itself plays back audio in the aac format which is why I download it as a .aac file (NOT of .m4a or mp3) but shazamKit reads in a .wav file which requires me to make a .aac to .wav converter function.
The recording part of my code works fine but every which way I try to open this .aac file (ExtAudioFileOpenURL or AVFile) I get the same 4 errors:
2023-07-22 09:58:55.795711+0900 KDVS[46452:2598649] ReadBytes Failed
2023-07-22 09:58:55.795818+0900 KDVS[46452:2598649] AACAudioFile::ParseAudioFile failed
2023-07-22 09:58:55.795911+0900 KDVS[46452:2598649] OpenFromDataSource failed
2023-07-22 09:58:55.795977+0900 KDVS[46452:2598649] Open failed
2023-07-22 09:58:55.796073+0900 KDVS[46452:2598649] [default] ExtAudioFile.cpp:210
I thought maybe the file is corrupted, but when I play the .aac file in any media player it works perfectly fine. Then I thought maybe the file URL isn't correct, so I wrote a print statement which prints the URL and T/F if the (fileExists(at: inputURL) at that returns true every time. The permissions should be fine too because the file is located in the Documents library
"file:///Users/johncarraher/Library/Developer/CoreSimulator/Devices/273C3EEA-823C-4A15-A67A-7DE5D5463AB5/data/Containers/Data/Application/A86A9BA4-F2EA-4D10-A93A-5C0F58690E8A/Documents/recording.aac".
I'm not too familiar with audiofile encodings so I'm not sure where to go from here, but I think either my file is a little corrupted or .aac files are not supported by most "AudioFile" readers. I have attached the class in my code that records the stream, and my aactowav converter function. Thank you in advance to anyone who can help.
//
// aactowav.swift
// KDVS
//
// Created by John Carraher on 7/21/23.
//
import Foundation
import AVFoundation
func convertAACtoWAV(inputURL: URL, outputURL: URL) {
var error: OSStatus = noErr
var destinationFile: ExtAudioFileRef? = nil
var sourceFile: ExtAudioFileRef? = nil
var srcFormat: AudioStreamBasicDescription = AudioStreamBasicDescription()
var dstFormat: AudioStreamBasicDescription = AudioStreamBasicDescription()
print("6 About to open \(inputURL) which has a status of \(fileExists(at: inputURL)) which looks like this: \(inputURL as CFURL) as a CFURL")
ExtAudioFileOpenURL(inputURL as CFURL, &sourceFile) //**Line where error comes from**
print("7")
var thePropertySize: UInt32 = UInt32(MemoryLayout.stride(ofValue: srcFormat))
ExtAudioFileGetProperty(sourceFile!,
kExtAudioFileProperty_FileDataFormat,
&thePropertySize, &srcFormat)
dstFormat.mSampleRate = 44100 // Set sample rate
dstFormat.mFormatID = kAudioFormatLinearPCM
dstFormat.mChannelsPerFrame = 1
dstFormat.mBitsPerChannel = 16
dstFormat.mBytesPerPacket = 2 * dstFormat.mChannelsPerFrame
dstFormat.mBytesPerFrame = 2 * dstFormat.mChannelsPerFrame
dstFormat.mFramesPerPacket = 1
dstFormat.mFormatFlags = kLinearPCMFormatFlagIsPacked | kAudioFormatFlagIsSignedInteger
// Create destination file
error = ExtAudioFileCreateWithURL(
outputURL as CFURL,
kAudioFileWAVEType,
&dstFormat,
nil,
AudioFileFlags.eraseFile.rawValue,
&destinationFile)
print("Error 1 in convertAACtoWAV: \(error.description)")
error = ExtAudioFileSetProperty(sourceFile!,
kExtAudioFileProperty_ClientDataFormat,
thePropertySize,
&dstFormat)
print("Error 2 in convertAACtoWAV: \(error.description)")
error = ExtAudioFileSetProperty(destinationFile!,
kExtAudioFileProperty_ClientDataFormat,
thePropertySize,
&dstFormat)
print("Error 3 in convertAACtoWAV: \(error.description)")
let bufferByteSize: UInt32 = 32768
var srcBuffer = [UInt8](repeating: 0, count: Int(bufferByteSize))
var sourceFrameOffset: ULONG = 0
while true {
var fillBufList = AudioBufferList(
mNumberBuffers: 1,
mBuffers: AudioBuffer(
mNumberChannels: 2,
mDataByteSize: bufferByteSize,
mData: &srcBuffer
)
)
var numFrames: UInt32 = 0
if dstFormat.mBytesPerFrame > 0 {
numFrames = bufferByteSize / dstFormat.mBytesPerFrame
}
error = ExtAudioFileRead(sourceFile!, &numFrames, &fillBufList)
print("Error 4 in convertAACtoWAV: \(error.description)")
if numFrames == 0 {
error = noErr
break
}
sourceFrameOffset += numFrames
error = ExtAudioFileWrite(destinationFile!, numFrames, &fillBufList)
print("Error 5 in convertAACtoWAV: \(error.description)")
}
error = ExtAudioFileDispose(destinationFile!)
print("Error 6 in convertAACtoWAV: \(error.description)")
error = ExtAudioFileDispose(sourceFile!)
print("Error 7 in convertAACtoWAV: \(error.description)")
}
func fileExists(at url: URL) -> Bool {
let fileManager = FileManager.default
return fileManager.fileExists(atPath: url.path)
}
//
// CachingPlayerItem.swift
// KDVS
//
// Created by John Carraher on 7/20/23.
//
import Foundation
import AVFoundation
fileprivate extension URL {
func withScheme(_ scheme: String) -> URL? {
var components = URLComponents(url: self, resolvingAgainstBaseURL: false)
components?.scheme = scheme
return components?.url
}
}
@objc protocol CachingPlayerItemDelegate {
/// Is called when the media file is fully downloaded.
@objc optional func playerItem(_ playerItem: CachingPlayerItem, didFinishDownloadingData data: Data)
/// Is called every time a new portion of data is received.
@objc optional func playerItem(_ playerItem: CachingPlayerItem, didDownloadBytesSoFar bytesDownloaded: Int, outOf bytesExpected: Int)
/// Is called after initial prebuffering is finished, means
/// we are ready to play.
@objc optional func playerItemReadyToPlay(_ playerItem: CachingPlayerItem)
/// Is called when the data being downloaded did not arrive in time to
/// continue playback.
@objc optional func playerItemPlaybackStalled(_ playerItem: CachingPlayerItem)
/// Is called on downloading error.
@objc optional func playerItem(_ playerItem: CachingPlayerItem, downloadingFailedWith error: Error)
}
open class CachingPlayerItem: AVPlayerItem {
class ResourceLoaderDelegate: NSObject, AVAssetResourceLoaderDelegate, URLSessionDelegate, URLSessionDataDelegate, URLSessionTaskDelegate {
var playingFromData = false
var mimeType: String? // is required when playing from Data
var session: URLSession?
var mediaData: Data?
var response: URLResponse?
var pendingRequests = Set<AVAssetResourceLoadingRequest>()
weak var owner: CachingPlayerItem?
var fileURL: URL!
var outputStream: OutputStream?
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
if playingFromData {
// Nothing to load.
} else if session == nil {
// If we're playing from a url, we need to download the file.
// We start loading the file on first request only.
guard let initialUrl = owner?.url else {
fatalError("internal inconsistency")
}
startDataRequest(with: initialUrl)
}
pendingRequests.insert(loadingRequest)
processPendingRequests()
return true
}
func startDataRequest(with url: URL) {
var recordingName = "record.mp3"
if let recording = owner?.recordingName{
recordingName = recording
}
//Find Documents Directory (If it don't exist, don't create it)
fileURL = try! FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false)
.appendingPathComponent(recordingName)
// Check if the file already exists
if FileManager.default.fileExists(atPath: fileURL.path) {
do {
// Clear the contents of the existing file
try Data().write(to: fileURL)
} catch {
print("Failed to clear existing file data: \(error)")
}
}
let configuration = URLSessionConfiguration.default
configuration.requestCachePolicy = .reloadIgnoringLocalAndRemoteCacheData
session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
session?.dataTask(with: url).resume()
outputStream = OutputStream(url: fileURL, append: true)
outputStream?.schedule(in: RunLoop.current, forMode: RunLoop.Mode.default)
outputStream?.open()
}
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, didCancel loadingRequest: AVAssetResourceLoadingRequest) {
pendingRequests.remove(loadingRequest)
}
// MARK: URLSession delegate
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {
let bytesWritten = data.withUnsafeBytes{outputStream?.write($0, maxLength: data.count)}
}
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) {
completionHandler(Foundation.URLSession.ResponseDisposition.allow)
mediaData = Data()
self.response = response
processPendingRequests()
}
func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {
if let errorUnwrapped = error {
owner?.delegate?.playerItem?(owner!, downloadingFailedWith: errorUnwrapped)
return
}
}
// MARK: -
func processPendingRequests() {
// get all fullfilled requests
let requestsFulfilled = Set<AVAssetResourceLoadingRequest>(pendingRequests.compactMap {
self.fillInContentInformationRequest($0.contentInformationRequest)
if self.haveEnoughDataToFulfillRequest($0.dataRequest!) {
$0.finishLoading()
return $0
}
return nil
})
// remove fulfilled requests from pending requests
_ = requestsFulfilled.map { self.pendingRequests.remove($0) }
}
func fillInContentInformationRequest(_ contentInformationRequest: AVAssetResourceLoadingContentInformationRequest?) {
if playingFromData {
contentInformationRequest?.contentType = self.mimeType
contentInformationRequest?.contentLength = Int64(mediaData!.count)
contentInformationRequest?.isByteRangeAccessSupported = true
return
}
guard let responseUnwrapped = response else {
// have no response from the server yet
return
}
contentInformationRequest?.contentType = responseUnwrapped.mimeType
contentInformationRequest?.contentLength = responseUnwrapped.expectedContentLength
contentInformationRequest?.isByteRangeAccessSupported = true
}
func haveEnoughDataToFulfillRequest(_ dataRequest: AVAssetResourceLoadingDataRequest) -> Bool {
let requestedOffset = Int(dataRequest.requestedOffset)
let requestedLength = dataRequest.requestedLength
let currentOffset = Int(dataRequest.currentOffset)
guard let songDataUnwrapped = mediaData,
songDataUnwrapped.count > currentOffset else {
return false
}
let bytesToRespond = min(songDataUnwrapped.count - currentOffset, requestedLength)
let dataToRespond = songDataUnwrapped.subdata(in: Range(uncheckedBounds: (currentOffset, currentOffset + bytesToRespond)))
dataRequest.respond(with: dataToRespond)
return songDataUnwrapped.count >= requestedLength + requestedOffset
}
deinit {
session?.invalidateAndCancel()
}
}
fileprivate let resourceLoaderDelegate = ResourceLoaderDelegate()
fileprivate let url: URL
fileprivate let initialScheme: String?
fileprivate var customFileExtension: String?
weak var delegate: CachingPlayerItemDelegate?
func stopDownloading(completion: @escaping () -> Void) {
resourceLoaderDelegate.session?.invalidateAndCancel()
completion()
}
// Function to get the URL of the downloaded file
func getDownloadedFileURL() -> URL? {
if resourceLoaderDelegate.playingFromData {
// If playing from Data, return the URL created for fake data
return resourceLoaderDelegate.fileURL
} else {
// If playing from URL, return the URL of the downloaded file
return try? FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false)
.appendingPathComponent(recordingName)
}
}
open func download() {
if resourceLoaderDelegate.session == nil {
resourceLoaderDelegate.startDataRequest(with: url)
}
}
private let cachingPlayerItemScheme = "cachingPlayerItemScheme"
var recordingName = "record.mp3"
/// Is used for playing remote files.
convenience init(url: URL, recordingName: String) {
self.init(url: url, customFileExtension: nil, recordingName: recordingName)
}
/// Override/append custom file extension to URL path.
/// This is required for the player to work correctly with the intended file type.
init(url: URL, customFileExtension: String?, recordingName: String) {
guard let components = URLComponents(url: url, resolvingAgainstBaseURL: false),
let scheme = components.scheme,
var urlWithCustomScheme = url.withScheme(cachingPlayerItemScheme) else {
fatalError("Urls without a scheme are not supported")
}
self.recordingName = recordingName
self.url = url
self.initialScheme = scheme
if let ext = customFileExtension {
urlWithCustomScheme.deletePathExtension()
urlWithCustomScheme.appendPathExtension(ext)
self.customFileExtension = ext
}
let asset = AVURLAsset(url: urlWithCustomScheme)
asset.resourceLoader.setDelegate(resourceLoaderDelegate, queue: DispatchQueue.main)
super.init(asset: asset, automaticallyLoadedAssetKeys: nil)
resourceLoaderDelegate.owner = self
addObserver(self, forKeyPath: "status", options: NSKeyValueObservingOptions.new, context: nil)
NotificationCenter.default.addObserver(self, selector: #selector(playbackStalledHandler), name:NSNotification.Name.AVPlayerItemPlaybackStalled, object: self)
}
/// Is used for playing from Data.
init(data: Data, mimeType: String, fileExtension: String) {
guard let fakeUrl = URL(string: cachingPlayerItemScheme + "://whatever/file.\(fileExtension)") else {
fatalError("internal inconsistency")
}
self.url = fakeUrl
self.initialScheme = nil
resourceLoaderDelegate.mediaData = data
resourceLoaderDelegate.playingFromData = true
resourceLoaderDelegate.mimeType = mimeType
let asset = AVURLAsset(url: fakeUrl)
asset.resourceLoader.setDelegate(resourceLoaderDelegate, queue: DispatchQueue.main)
super.init(asset: asset, automaticallyLoadedAssetKeys: nil)
resourceLoaderDelegate.owner = self
addObserver(self, forKeyPath: "status", options: NSKeyValueObservingOptions.new, context: nil)
NotificationCenter.default.addObserver(self, selector: #selector(playbackStalledHandler), name:NSNotification.Name.AVPlayerItemPlaybackStalled, object: self)
}
// MARK: KVO
override open func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
delegate?.playerItemReadyToPlay?(self)
}
// MARK: Notification hanlers
@objc func playbackStalledHandler() {
delegate?.playerItemPlaybackStalled?(self)
}
// MARK: -
override init(asset: AVAsset, automaticallyLoadedAssetKeys: [String]?) {
fatalError("not implemented")
}
deinit {
NotificationCenter.default.removeObserver(self)
removeObserver(self, forKeyPath: "status")
resourceLoaderDelegate.session?.invalidateAndCancel()
}
}
The problem was that my .aac file was incorrectly formatted. Since I downloaded from an audiostream, the program immediately started writing to the file, ignoring the structure of each frame and more importantly the header. As a result most of the metadata for the audio was in the file, but couldn't be read because it wasn't at the very front (my file in hex read EB3B, but had a FFF1 a couple hundred bits after). The fix was to make a function that deleted any data before the first header (signaled by FFF1). The function I used to re-format the .aac is below:
func deleteBeforeMarkerFFF1(inputURL: URL, completion: @escaping () -> Void) {
do {
// Read the AAC audio file as binary data
var inputData = try Data(contentsOf: inputURL)
// Find the position of the marker "FFF1"
guard let markerRange = inputData.range(of: Data([0xFF, 0xF1])) else {
completion()
return
}
// Remove all data before the marker "FFF1"
let trimmedData = inputData.subdata(in: markerRange.lowerBound..<inputData.endIndex)
// Replace the original binary data with the modified binary data
inputData = trimmedData
// Write the modified binary data back to the AAC audio file
try inputData.write(to: inputURL)
completion()
} catch {
// Handle the error here
print("Error: \(error)")
completion()
}
}