The Situation: I have a simple word game that uses an English language dictionary (the language dictionary) to provide definitions (in a dictionary data structure, the code-dictionary). Apple's APIs won't do what I need and I could not find a suitable 3rd party solution. The game has worked fine with a makeshift dictionary that was missing words and definitions. I took the Wiktionary database and parsed it way down but hit diminishing returns around 46MB. The dictionary is a simple text file, one word's definition(s) per line. There's also a separate wordlist file with one word per file. I do the parsing with a Python script so I can adjust the file(s) and formats.
The Problem: Reading the language dictionary to a code dictionary takes too long to do it on load. Making the language-dictionary a static const data structure causes XCode to use all my real memory and equivalent swap space eventually rendering my system unusable at around 15GB of swap. This happens whether I attempt a build or not and seems to be XCode scanning it for auto-completion.
The Question: Is there some way to work around this situation? Some kind of directive to not scan the static dictionary file? A simple precompiled library that won't be scanned? An alternative data-structure?
What I've tried: I did a bunch of profiling and sticking in print statements to determine that the language-dictionary loading was the slow part. The code was a simple get next line, add to code-dictionary loop. The static dictionary was my best attempt to fix the loading time so far.
What I haven't tried: A static array and splitting up the dictionary both require almost complete rewrites. I'm not against massive rewrites, if they're likely to resolve the problem. I looked at a plist, but it appears I would need to read that in as well but it would be much larger because of the XML. I considered learning Core Data, but it seemed overkill for a basic, if large lookup table.
This would have been a great excuse to learn Core Data if I was there already, but, I'm struggling with Swift forcing advanced concepts into areas of basic code.
I resolved the situation by changing the language dictionary builder to create a plist. I then loaded that directly into a String:String data dictionary. It was surprisingly fast in comparison.
Things I'm still considering as optimizations include breaking it into smaller pieces, say based on the first letter or two. If they're small enough, say a half second or less to load on the slowest device, they can be loaded into the dictionary as needed during play. Eventually, a background load might be better, but that's a whole 'nother thing.