My problem: JavaScript calls into the plugin, which forks a thread that opens a NSOpenPanel (or NSSavePanel) dialog. Often this works, but sometimes it crashes on "runModal", on both Firefox and Chrome. The crash seems to happen randomly and more frequently on some machines (10.7 maybe) than others. The stack location of the crash varies, but generally occurs on the thread that opens the dialog.
I use a POSIX thread because it's cross-platform code, but I do spawn a NSThread to let Cocoa know:
NSThread* t = [[NSThread alloc] init];
[t start];
[t release];
The code that opens the dialog:
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; // necessary?
NSOpenPanel* openPanel = [NSOpenPanel openPanel];
[openPanel setAllowsMultipleSelection:_allowMultipleFiles];
[openPanel setCanChooseFiles:YES];
[openPanel setCanChooseDirectories:YES];
NSInteger result = [openPanel runModal]; // crash
... handle result ...
[pool release];
A stack trace:
Process: plugin-container [25536]
Path: /Applications/Firefox.app/Contents/MacOS/plugin-container.app/Contents/MacOS/plugin-container
Identifier: org.mozilla.plugincontainer
Version: ??? (1.0)
Code Type: X86-64 (Native)
Parent Process: firefox-bin [25531]
Date/Time: 2011-12-09 17:12:04.271 -0800
OS Version: Mac OS X 10.6.8 (10K549)
Report Version: 6
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000000000000d
Crashed Thread: 14
Thread 14 Crashed:
0 XUL 0x0000000100d84ff4 mac_plugin_interposing_child_OnSetCursor + 38068
1 XUL 0x0000000100dd38bd mac_plugin_interposing_child_OnSetCursor + 359805
2 XUL 0x0000000100d7b247 mac_plugin_interposing_child_OnHideCursor + 1175
3 com.apple.AppKit 0x00007fff801cbfdb _NSCreateWindowWithOpaqueShape2 + 610
4 com.apple.AppKit 0x00007fff80160691 -[NSWindow _commonAwake] + 1214
5 com.apple.AppKit 0x00007fff803c282f -[NSWindow _setModal:] + 138
6 com.apple.AppKit 0x00007fff803c2183 -[NSApplication _orderFrontModalWindow:relativeToWindow:] + 482
7 com.apple.AppKit 0x00007fff803c1ce7 -[NSApplication _commonBeginModalSessionForWindow:relativeToWindow:modalDelegate:didEndSelector:contextInfo:] + 714
8 com.apple.AppKit 0x00007fff803c1a18 -[NSApplication beginModalSessionForWindow:] + 36
9 com.apple.AppKit 0x00007fff803c193a -[NSApplication runModalForWindow:] + 106
10 XUL 0x0000000100d7af01 mac_plugin_interposing_child_OnHideCursor + 337
11 com.apple.AppKit 0x00007fff80627112 -[NSSavePanel runModal] + 318
12 com.aspera.AsperaWeb 0x000000010509f9d4 Connect::Web::MacOpenFolderDialogImpl::doShow() + 596
13 com.aspera.AsperaWeb 0x00000001050a0fa3 Connect::Web::IDialogImpl::show() + 45
14 com.aspera.AsperaWeb 0x00000001050a1e07 Connect::Web::OpenFolderDialog::showHelper(bool) + 141
15 com.aspera.AsperaWeb 0x00000001050a1ef0 Connect::Web::OpenFolderDialog::run() + 42
16 com.aspera.AsperaWeb 0x000000010509eb91 Connect::Web::SingleJobWorker::run() + 63
17 com.aspera.AsperaWeb 0x0000000105098160 start_helper + 38
18 libSystem.B.dylib 0x00007fff8593afd6 _pthread_start + 331
19 libSystem.B.dylib 0x00007fff8593ae89 thread_start + 13
Thread 14 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x0000000102063840 rcx: 0x0000000000000000 rdx: 0x0000000101e27ee0
rdi: 0x0000000101e0bcc8 rsi: 0x0000000000000000 rbp: 0x0000000101e27ef0 rsp: 0x000000011d5805e0
r8: 0x0000000101e0bcc8 r9: 0x00000001040fc0a4 r10: 0x00007fffffe001a0 r11: 0x0000000000000202
r12: 0x0000000101e27ee0 r13: 0x0000000000000001 r14: 0x000000011d5805e0 r15: 0x000000011d580754
rip: 0x0000000100d84ff4 rfl: 0x0000000000010246 cr2: 0x000000000000000d
I'm guessing there's a problem with opening a dialog on a secondary thread. Maybe I need to open it on the main thread? However, I do want to keep the JavaScript call asynchronous. I'm looking at NSNotificationQueue or performSelectorOnMainThread.
I have a dialog abstraction I use in FireBreath here: https://gist.github.com/1368648
It turns out that you can't use runModal on a secondary thread; it has to be on the main thread of the application. This is a challenge in a NPAPI plugin because you also can't block the main thread during a NPAPI call or the browser (at least chrome and firefox 4+) is likely to kill you.
The solution is to have it run on the main thread, but not as part of the NPAPI call; you can do this by using a performSelectorOnMainThread and not waiting for it to complete. If you provide some sort of callback (an NPObject javascript function works great) you can invoke that when your dialog is done.
FireBreath uses this method for its BrowserHost::ScheduleOnMainThread function, which is what I now use in that gist I referenced. Someone (you?) posted a comment on the gist asking if it works for me; the gist at the time of the comment did not, but I have updated it with the fix (which I just figured out yesterday). Since we updated it to use the performSelectorOnMainThread to run the call later I haven't been able to reproduce the crash.
Looking back, you're already familiar with calling back on the main thread this way: How to callback plugin thread on Safari 5.1 on OSX?
You use the same technique; you'd think blocking the thread there would also be dangerous, but since during a modal dialog the system event loop continues to run you're okay; your code doesn't return, but other events still get dispatched. The browser isn't waiting for a NPAPI call to return so it doesn't seem to have any issues.