I am trying to use p5.SerialPort in google colab. Using p5(link ahead) everything works fine, I think my problem is that the libraries are not imported properly because the code doesn't run.
it's getting stuck during initaition of this line:
serial = new p5.SerialPort();
Google colab link
and the contents:
from google.colab import files
from IPython.display import HTML, Audio
from google.colab.output import eval_js
from base64 import b64decode
C_HTML = """
<script>
var test = document.createTextNode("TEST");
document.body.appendChild(test);
serial = new p5.SerialPort();
serial.open(serialPort);
serial.write('1');
var test = document.createTextNode("TEST2");
document.body.appendChild(test);
</script>
"""
def run():
display(HTML(C_HTML))
p5 Editor link
and the contents:
const serialPort = 'COM13';
let serial;
function setup() {
serial = new p5.SerialPort();
serial.open(serialPort);
serial.write('4');
}
function draw() {
}
You will need to include p5.js and p5.serialport.js first.
After that it will take a bit of time until the scripts are loaded and ready to use. Athough it's a bit hacky, checking with setInterval() should do the trick.
Here's a basic colab Python script to test:
from google.colab import files
from IPython.display import HTML, Audio
from google.colab.output import eval_js
from base64 import b64decode
C_HTML = """
<script language="javascript" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.1/p5.min.js"></script>
<script language="javascript" type="text/javascript" src="https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.serialport.js" onload="setupSerial();" onerror="console.warn(e)";></script>
<script>
const serialPort = 'COM13';
let serial;
let isOn = false;
function setupSerial(){
serial = new p5.SerialPort();
serial.open(serialPort);
setInterval(blink, 1000);
console.log("serial setup complete");
}
function blink(){
isOn = !isOn;
if(serial){
serial.write(isOn ? '1' : '0');
}
}
serialInterval = setInterval(checkSerial,500);
function checkSerial(){
console.log('p5.SerialPort',p5.SerialPort);
if(p5.SerialPort){
clearInterval(serialInterval);
setupSerial();
}
}
</script>
"""
def run():
display(HTML(C_HTML))
run()
This is the basic sketch used for testing:
bool ledState;
void setup() {
Serial.begin(9600);
pinMode(13, OUTPUT);
}
void loop() {
if(Serial.available()){
ledState = Serial.read() == '1';
}
digitalWrite(13, ledState);
}
This is a breakdown of the HTML/JS parts of the notebook:
<script language="javascript" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.1/p5.min.js"></script>
<script language="javascript" type="text/javascript" src="https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.serialport.js";></script>
This part includes the required js libraries (p5.js and p5.serial)
Regarding the rest of the JavaScript, here's a commented version:
from google.colab import files
from IPython.display import HTML, Audio
from google.colab.output import eval_js
from base64 import b64decode
C_HTML = """
<script language="javascript" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.1/p5.min.js"></script>
<script language="javascript" type="text/javascript" src="https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.serialport.js" onload="setupSerial();" onerror="console.warn(e)";></script>
<script>
// serial port name
const serialPort = 'COM13';
// reference to the serial port
let serial;
// boolean state of the Arduino onboard LED (at pin 13)
let isOn = false;
// open the serial connection and call blink every second
function setupSerial(){
serial = new p5.SerialPort();
serial.open(serialPort);
setInterval(blink, 1000);
console.log("serial setup complete");
}
// toggle the LED state and if serial is initialised send the data
function blink(){
isOn = !isOn;
if(serial){
serial.write(isOn ? '1' : '0');
}
}
// loading p5.js and p5.serialport.js will take some time load and initialise
// check every 0.5 seconds the p5.SerialPort class is available
serialInterval = setInterval(checkSerial,500);
function checkSerial(){
console.log('p5.SerialPort',p5.SerialPort);
// if p5.serial is ready, clear the interval and setup the port
if(p5.SerialPort){
clearInterval(serialInterval);
setupSerial();
}
}
For more advanced examples of Colab notebooks/HTML communication check out the advanced_outputs.ipynb notebook
(The above assumes you're already running https://github.com/p5-serial/p5.serialcontrol/releases)
Update Another option using Python in colab is to use the websocket-client module and ngrok:
ngrok tcp 8081 from Terminal/Command Prompt (note you may need to setup a free to use auth token for TCP)'{"method":"openserial","data":{"serialport":"COM13","serialoptions":{}}}' to open the serial port, '{"method":"write","data":"1"}' to write '1' to serial, etc.))To install websocket-client in colab you'd use:
!pip install -q websocket-client
and here's an example that turns on LED on for 1 second then off (using the above Arduino example):
from time import sleep
import websocket
# when the websocket is open send a serial open command (on port COM13) then send a '1' then a '0' with 1 second in between
def on_open(ws):
ws.send('{"method":"openserial","data":{"serialport":"COM13","serialoptions":{}}}')
sleep(1)
print('sending ON')
wsapp.send('{"method":"write","data":"1"}')
sleep(1)
print('sending OFF')
wsapp.send('{"method":"write","data":"0"}')
def on_message(ws, message):
print(message)
def on_error(wsapp, err):
print("Got a an error: ", err)
wsapp = websocket.WebSocketApp("ws://YOUR_NGROK_TCP_SERVER_HERE",
# YOUR_NGROK_TCP_SERVER_HERE example #.tcp.ngrok.io:#####,
on_message = on_message,
on_error=on_error)
wsapp.on_open = on_open
wsapp.run_forever()
(also note run_forever() is a blocking loop: based on your application you may want to manually run open and control a websocket connection (as opposed to using WebSocketApp) or use threading, depending on what makes)
The reason simply running pyserial on Colab won't work is that the Python Code executed there runs on a virtual machine on a server somewhere in Google Cloud's infrastructure. It's very unlikely you'd be able to get your arduino with USB cable connected somehow to that specific physical server (and to the VM running within it) to access the Serial port directly :). What you can do though is use a bridge: the serial port is on your machine therefore you'd need a way to for the Colab notebook to talk to it.
This is where the p5.serialcontrol utility comes in: it talks to serial locally but it's also a web socket server (which means both the client side and Python back end can connect to it. The caveat here is the client side code (as the name suggests), runs on your machine, but the Python backed is on Google Cloud's VM so for localhost means itself, not your machine (as it does on the client side), hence the need to use a reverse tunnel like ngrok. (This means Google Colab's VM talks to ngrok's server which is a reverse tunnel to your computer).
On the client side there could be another option: WebSerial. This could potentially remove the need for the p5.serialcontrol utility program, however it's still in early days and only supported in the modern versions of Edge, Chrome, Opera.