So, I used pre-trained alexnet model and only changed last layer output and weights of fully connected layers. I did it on c++ using pytorch. Now I want use this model to predict what objects are on webcam and i need to use DE1-SOC FPGA. Also, it should be only processed on FPGA itself.
My suggestion is to feed webcam images into this model when button is pressed, then model will give some number, and after there will some simple procedures on this number. So, the problem is how to import this model or use this c++ model on FPGA or how import it into VHDL language with less pain?
Also, I found this video, https://www.youtube.com/watch?v=iyEZOx1YwmM&t=73s . Would it be helpful? I am new on FPGAs and VHDL, so I would admire any suggestions or examples with code.
So, the problem is how to import this model or use this c++ model on FPGA or how import it into VHDL language with less pain?
"Less pain" makes the question already very subjective. However this is not the first time I see a request for converting'neural network' code to run on an FPGA.
Why an FPGA?
There seems to be the (false) perception that "On an FPGA things run much faster!"
An FPGA with moderate complex code can run at 100-200 MHz. Your CPU can have four cores running at 3.3GHz. So you loose about a factor 60 of speed.
In order for the FPGA to outperform your CPU you have to make up that speed loss factor.
Where an FPGA can outperform a CPU is in parallel processing an pipe-lining. Thus your algorithm, which was compiled to run (sequential) on four cores, must be re-written to be split in at least 60 parts and those must all run at the same time. So you need to build 60+ parallel engines, or 15 engines each consisting of four pipe line stages, or 4 engines with... (You get the gist)
To HDL
At the same time you have to convert your C++, Python or whatever code to HDL.
Some good progress is mode in that direction, but the input still has to adhere to many special rules and the results I have seen are not very good yet. (Compared to manual written code) Most of it is big and slow. It is suited where you are short of development time and have plenty of time/space resource on your FPGA.
Many articles have been written that "an FPGA is well suited to implement a neural network". I suspect this where the drive for generating FPGA code comes from. However that does not mean that your c++ pytorch code can be automatically converted to a neural network in HDL code.
I/O.
I/O is often overlooked. In your case you want to process images from a webcam. So how do you plan to put those images into the FPGA? Believe me making a camera interface on an FPGA is not a trivial matter. Been there, done that! Even with pre-defined and tested cores it takes several weeks to get things running. If you want to pass data from e.g. a PC to the FPGA you will need some other interface. PCI-e comes to mind but add time for writing drivers. Then you have to learn how to get your data from such an FPGA PCI-e IP core and pass it into your neural network nodes.
Back to your question:
Can your code be converted to HDL and run on an FPGA? Maybe, I don't know. But I very much suspect you will have lots and lots of "pain" trying to do so.
Just a last remark: The video link you give, shows how to run an C++ code on an ARM processor. That ARM processor happens to be embedded alongside a lot of programmable gates. However non of those programmable gates are used. It does NOT convert "Hello world" into HDL.