Inference☁️
Inferenza cloud
The cloud platforms for models: alternatives when local hardware isn't enough. Replicate and Runpod in the catalog are the child resources.
Resources
Tools, frameworks, and models for local AI
3 resources found for "Go"
Tools, frameworks, and models for local AI
3 resources found for "Go"
The cloud platforms for models: alternatives when local hardware isn't enough. Replicate and Runpod in the catalog are the child resources.
Google's open-source (Apache 2.0) on-device GPU inference engine, successor to the TensorFlow Lite GPU delegate: tensor virtualization, stage-aware KV cache for prefill and decode, FP16/INT8/INT4. Backends: OpenCL (Android), Metal (Apple), WebGPU via Dawn (browser), OpenGL ES 3.1+. Integrates with LiteRT; in production on Chrome, YouTube, Photos and Meet.
Open-source inference engine in pure Go that embeds llama.cpp in-process via purego/ffi (no CGo). Reads GGUF models and runs on CUDA, Metal, Vulkan, OpenVINO and in-browser WebGPU. Covers roughly 96% of llama.cpp's functionality; optional server with an OpenAI-compatible API.