Search code examples
swiftmultithreadingthread-safetybcryptcommand-line-tool

Swift Command Line Tool utilizing Process() and multithreading crashes after a certain number of execution rounds (~3148)


I implemented a password generator script in Swift which utilizes Process() to execute Mac OS X command line tasks. The passwords themselves are just random Strings which then are encrypted (bcrypt) by the command line task as follows:

/usr/sbin/htpasswd -bnBC 10 '' this_is_the_password | /usr/bin/tr -d ':\n'

Also multiple threads are used to generate passwords with their hashes in parallel.

Note: Both the multithreading and the command line task (compared to serveral other native Swift libraries I tried) improved performance in terms of execution time drastically.

The Problem

The Programm runs fine for the first ~3148 rounds and always crashes around this number (problably correlated to the number of threads running). For example, if I configure 2000 passwords the code executes as expected terminates without any errors.

Common Error Messages

Setting a breakpoint in Process+Pipe.swift in the catch block of the execute(...) function at BREAKPOINT_1 results in:

Thread 1: signal SIGCHLD
po error.localizedDescription
"The operation couldn\\U2019t be completed. (NSPOSIXErrorDomain error 9 - Bad file descriptor)"

When uncommenting the four //return self.hash(string, cost: cost) code snippets to ignore the error the following errors finally crash the execution (again in execute(...), but not necessarily in the catch block):

Program stops ...
Thread 32: EXC_BAD_ACCESS (code=2, address=0x700003e6bfd4)
... on manual continue ...
Thread 2: EXC_BAD_ACCESS (code=2, address=0x700007e85fd4)
po process
error: Trying to put the stack in unreadable memory at: 0x700003e6bf40.

Code

The relevant code componets are the Main.swift which initialized and starts (and later stops) the PasswordGenerator and then loops n times to get passwords via nextPassword() from the PasswordGenerator. The PasswordGenerator itself utilized execute(...) from the Process extension to run commandline tasks which generate the hash.

Main.swift

class Main {
  private static func generate(...) {
    ...
    PasswordGenerator.start()
    for _ in 0..<n {
      let nextPassword = PasswordGenerator.nextPassword()
      let readablePassword = nextPassword.readable
      let password = nextPassword.hash
      ...
    }
    PasswordGenerator.stop()
    ...
  }
}

PasswordGenerator.swift

The PasswordGenerator runs multiple Threads in parallel. nextPassword() tries to get a password (if there is one in the passwords Array) or else waits for 100 seconds.

struct PasswordGenerator {
  typealias Password = (readable: String, hash: String)
  
  private static let semaphore = DispatchSemaphore(value: 1)
  private static var active = false
  private static var passwords: [Password] = []
  
  static func nextPassword() -> Password {
    self.semaphore.wait()
    if let password = self.passwords.popLast() {
      self.semaphore.signal()
      return password
    } else {
      self.semaphore.signal()
      sleep(100)
      return self.nextPassword()
    }
  }
  
  static func start(
      numberOfWorkers: UInt = 32,
      passwordLength: UInt = 10,
      cost: UInt = 10
  ) {
      self.active = true
      
      for id in 0..<numberOfWorkers {
          self.runWorker(id: id, passwordLength: passwordLength, cost: cost)
      }
  }
  
  static func stop() {
    self.semaphore.wait()
    self.active = false
    self.semaphore.signal()
  }
  
  private static func runWorker(
    id: UInt,
    passwordLength: UInt = 10,
    cost: UInt = 10
  ) {
    DispatchQueue.global().async {
      var active = true
      repeat {
        // Update active.
        self.semaphore.wait()
        active = self.active
        print("numberOfPasswords: \(self.passwords.count)")
        self.semaphore.signal()
        
        // Generate Password.
        // Important: The bycrypt(cost: ...) step must be done outside the Semaphore!
        let readable = String.random(length: Int(passwordLength))
        let password = Password(readable: readable, hash: Encryption.hash(readable, cost: cost))
        
        // Add Password.
        self.semaphore.wait()
        self.passwords.append(password)
        self.semaphore.signal()
      } while active
    }
  }
}

Encryption.swift

struct Encryption {
  static func hash(_ string: String, cost: UInt = 10) -> String {
    // /usr/sbin/htpasswd -bnBC 10 '' this_is_the_password | /usr/bin/tr -d ':\n'
    
    let command = "/usr/sbin/htpasswd"
    let arguments: [String] = "-bnBC \(cost) '' \(string)".split(separator: " ").map(String.init)
    
    let result1 = Process.execute(
      command: command,//"/usr/sbin/htpasswd",
      arguments: arguments//["-bnBC", "\(cost)", "''", string]
    )
    
    let errorString1 = String(
      data: result1?.error?.fileHandleForReading.readDataToEndOfFile() ?? Data(),
      encoding: String.Encoding.utf8
    ) ?? ""
    guard errorString1.isEmpty else {
//      return self.hash(string, cost: cost)
      fatalError("Error: Command \(command) \(arguments.joined(separator: " ")) failed with error: \(errorString1)")
    }
    
    guard let output1 = result1?.output else {
//    return self.hash(string, cost: cost)
      fatalError("Error: Command \(command) \(arguments.joined(separator: " ")) failed! No output.")
    }
      
    let command2 = "/usr/bin/tr"
    let arguments2: [String] = "-d ':\n'".split(separator: " ").map(String.init)
    
    let result2 = Process.execute(
      command: command2,
      arguments: arguments2,
      standardInput: output1
    )
    
    let errorString2 = String(
      data: result2?.error?.fileHandleForReading.readDataToEndOfFile() ?? Data(),
      encoding: String.Encoding.utf8
    ) ?? ""
    guard errorString2.isEmpty else {
//      return self.hash(string, cost: cost)
      fatalError("Error: Command \(command) \(arguments.joined(separator: " ")) failed with error: \(errorString2)")
    }
    
    guard let output2 = result2?.output else {
//      return self.hash(string, cost: cost)
      fatalError("Error: Command \(command) \(arguments.joined(separator: " ")) failed! No output.")
    }
    
    guard
      let hash = String(
        data: output2.fileHandleForReading.readDataToEndOfFile(),
        encoding: String.Encoding.utf8
      )?.replacingOccurrences(of: "$2y$", with: "$2a$")
    else {
      fatalError("Hash: String replacement failed!")
    }
    
    return hash
  }
}

Process+Pipe.swift

extension Process {
  static func execute(
    command: String,
    arguments: [String] = [],
    standardInput: Any? = nil
  ) -> (output: Pipe?, error: Pipe?)? {
    let process = Process()
    process.executableURL = URL(fileURLWithPath: command)
    process.arguments = arguments
    
    let outputPipe = Pipe()
    let errorPipe = Pipe()
    process.standardOutput = outputPipe
    process.standardError = errorPipe
    
    if let standardInput = standardInput {
      process.standardInput = standardInput
    }
    
    do {
      try process.run()
    } catch {
      print(error.localizedDescription)
      // BREAKPOINT_1
      return nil
    }
    
    process.waitUntilExit()
    
    return (output: outputPipe, error: errorPipe)
  }
}

Question(s)

  1. Why does the program crash?
  2. Why does it not crash for also huge numbers like 2000 passwords?
  3. Is the multithreading implemented correct?
  4. Is there a problem in the execute(...) code?

Solution

  • I have found a fix while researching this bug. It seems that, despite what the documentation claims, Pipe will not automatically close its reading filehandle.

    So if you add a try outputPipe.fileHandleForReading.close() after reading from it, that will fix the issue.