I would not recommend LocalAI. There documentation is somewhat lacking and it’s an all in one utility with many moving parts. The parts also tend to break, quite often.
I have a similar setup and it’s been a huge pain when I when I have to do the OS updates.
The Coral needs a dkms module, but the sources and Google’s own documentation for it are out of date. I would highly recommend using the iGPU for inference.
I personally would stick to *sense. I personally used OPNSense there’s a huge community backing, well documented, and actively maintained. I like to use the CLI, but using the Web GUI was a breeze and I mainly wanted to set it and forget it.
Much appreciated <3