Search code examples
androidiosswiftandroid-tv

Voice input in Android TV Remote Control


I am developing a voice search feature using Android TV Remote Control v2 technology on iOS.
While I can successfully connect to the smart TV, send commands, and modify text in input fields, I am unable to trigger the voice assistant.
Based on my understanding, I need to send a RemoteVoiceBegin packet, which I am already implementing. However, the voice assistant does not activate, and the socket closes with a sessionID of -1.

Here is the code I am using.
Can you help identify what might be going wrong?

 private func createVoiceCommand() {
        // Create the voice message data
        guard let data = createVoiceMessage() else {
            print("Failed to create voice message.")
            return
        }
        
        // Send the data asynchronously in the background
                // Attempt to send the data
                self.remoteManager.sendVoiceData(data: data)
                print("Sending start voice command: \(data)")
        
    }

    func createVoiceMessage() -> Data? {
        
        // Create voice begin with config
        var message = Com_Multi_Tv_Utils_AndroidTvRemote_RemoteCommunicationTv_RemoteMessage()
        let voiceBegin = Com_Multi_Tv_Utils_AndroidTvRemote_RemoteCommunicationTv_RemoteVoiceBegin.with {
            
            $0.voiceConfig = Com_Multi_Tv_Utils_AndroidTvRemote_RemoteCommunicationTv_RemoteVoiceConfig.with {
                $0.sampleRate = Int32(RemoteXVoiceInput.sampleRate)
                $0.channelConfig = Int32(RemoteXVoiceInput.channelConfig)
                $0.audioFormat = Int32(RemoteXVoiceInput.audioFormat.rawValue)
            }
            $0.sessionID = -1
        }

        // Create final message
        message.remoteVoiceBegin = voiceBegin

        print("Voice message created: \(message)")

        // Serialize the message and handle any errors
        do {
            return try message.serializedData()
        } catch {
            print("Failed to serialize voice message: \(error)")
            return nil
        }
    }
func sendVoiceData(data: Data) {
            let varintData = Data(Encoder.encodeVarint(UInt(data.count)))
            print("Encoded length (varint): \(varintData)")
            
            // Combine varint length data and serialized message data
            var combinedData = Data()
            combinedData.append(varintData)
            combinedData.append(data)
            print("Combined data to send: \(combinedData)")
            
            // Now send the data using the provided send method
            self.remoteManager.send(varintData, data)
            print("Data sent successfully")
            print(Array(combinedData))
        
    }

Solution

  • I got Voice Input feature working.

    To handle VoiceBegin, you have to first listen for a RemoteVoiceBegin message from the TV device. This message is sent in response to sending KEYCODE_SEARCH (84) to initiate the search.

    The RemoteVoiceBegin message from the device contains a session_id and a package name, but only the session_id needs to be read, as it will be used in subsequent calls to VoiceBegin, VoicePayload, and VoiceEnd.

    Upon receiving the RemoteVoiceBegin, a RemoteVoiceBegin message must be sent back with just the session_id. This step is necessary to enable the reception of Payload messages afterward.