Tools, frameworks, and models for local AI
140 resources available
7B open-weight model (32-layer DiT) that unifies text-to-image generation up to 2048x2048, editing with up to 10 reference images, and native RGBA transparency. Designed to run locally on 16GB+ VRAM; supported by diffusers and ComfyUI.
Ultra-compact 0.5B-parameter model. Runs on any hardware, even embedded: ideal for lightweight summarization and low-power local devices.
Clear step up in reasoning and instruction quality over 8B models. Excellent for coding, agents and structured writing on 16GB GPUs or Macs with 32GB RAM.
Near-70B quality at a fraction of the size: the sweet spot for advanced use. Excellent in Italian, code and reasoning. Needs 24GB+ VRAM.
3B-parameter model with excellent multilingual support, including Italian. Good balance of quality and hardware requirements for summarization and drafting.
The top open dense model of its class, with first-class multilingual support for Italian, French and other European languages. Needs 48-80GB setups.
Mixture-of-Experts model: 30B total params but only 3.3B active per token. Small-model speed with mid-size quality; the best speed/quality ratio in its class.
Il modello dense 8B della famiglia Qwen3: 8,19B di parametri, contesto nativo di 40K token (espandibile con YaRN) e ragionamento in modalità think/non-think attivabile per prompt. Licenza Apache 2.0. È il punto di equilibrio per l'uso locale su GPU da 8-12GB: in Q4_K_M occupa circa 5GB, lasciando spazio per contesto lungo.
Il modello dense multimodale della generazione Qwen3.8 di Alibaba: 27,78B di parametri, input testo+immagini+video e contesto nativo di 262K token. Licenza Apache 2.0. L'attenzione ibrida (DeltaNet lineare + GQA) riduce la KV cache di circa il 75% rispetto a un transformer standard: in Q4_K_M sta su una singola GPU da 24GB.
0.6B safety-moderation model with a token-level classifier for real-time monitoring during incremental generation. Three severity tiers (safe, controversial, unsafe) and 119 languages: small and local, ideal for filtering outputs without the cloud.
State-of-the-art open video model at 720p: professional-level motion quality and frame coherence. Needs 24GB+ VRAM.
La Radeon con la maggiore capacità consumer: 24GB GDDR6 a 960 GB/s (bus 384-bit). Per l'IA locale la capacità batte la novità: mentre le schede da 16GB non caricano i modelli 27B, qui Qwen3.8-27B e i 32B in Q4 ci stanno con contesto ampio. 960 GB/s è il 96% della RTX 4090. ROCm ufficiale (gfx1100), 355W.
La GPU RDNA 4 di AMD: 16GB GDDR6 su bus 256-bit (640 GB/s), 64 CU, 304W. Con ROCm 7 la prima Radeon consumer con supporto ufficiale su Linux (gfx1201, senza workaround): ~92 tok/s su gpt-oss-20b, ~52 tok/s su Qwen 14B. Il 640 GB/s è il 43% in più della RTX 5060 Ti 16GB: la carta 16GB più veloce per modelli 9-20B, ma il 16GB non tiene un 27B dense.
La APU AMD con la massima memoria unificata su x86: fino a 128GB di LPDDR5X condivisa tra CPU (16 core Zen 5), GPU integrata (40 CU RDNA 3.5) e NPU. Fino a 96GB allocabili alla GPU: modelli fino a 100B+ a 4 bit senza GPU dedicata, su mini-PC o laptop. Bandwidth 256 GB/s.
Apple's top-tier chip: up to 256GB unified memory on Mac Studio. The simplest way to run 70B+ models, with CPU and GPU sharing the same memory.
Apple Silicon chip with exceptional performance for mobile development and edge inference.
Il chip Apple Silicon di punta per l'IA locale: 18 core CPU e 40 core GPU con memoria unificata fino a 128GB a 614 GB/s. Il Neural Accelerator su ogni core GPU raddoppia la velocità dei modelli MoE in decode rispetto alla generazione M4: ~88 tok/s su gpt-oss-120b, il più veloce della classe.
Il desktop compatto di Apple con il chip M6, primo processore Apple in 2nm: 12 core CPU, 12 core GPU con Neural Accelerator in ogni core e Neural Engine doppio da 16 core. 16GB di memoria unificata di serie (fino a 32GB, 170 GB/s), Wi-Fi 7, Thunderbolt 5. La piattaforma d'ingresso per IA locale: modelli fino a ~20B quantizzati a 4 bit girano interamente sul dispositivo, e la versione M5 Pro arriva a 64GB e 307 GB/s.
La workstation Apple di fascia alta: M5 Max con 18 core CPU, GPU fino a 40 core con Neural Accelerator e fino a 128GB di memoria unificata a 614 GB/s. Modelli fino a ~120B in 4 bit (e i MoE a bassa attivazione come gpt-oss-120b in tempo quasi reale), con MLX e llama.cpp. 10GbE di serie, Thunderbolt 5, fino a 8 display.
Il Mac più potente di Apple: M5 Ultra, primo chip a quattro die (due M5 Max uniti da UltraFusion), fino a 36 core CPU, 80 core GPU e 512GB di memoria unificata a 1,2 TB/s. Pensato per modelli frontier-class in locale: MoE da centinaia di miliardi di parametri e RAG su corpora enormi, senza cloud. Il clustering RDMA di più unità porta fino a 3× l'inferenza distribuita.
The most advanced open image model: excellent prompt adherence, solid anatomy, full fine-tuning. The standard for serious work.
Very few steps, high quality, and strong text rendering. The distilled FLUX variant, built for speed.
DeepSeek's MoE model with 552B total parameters and 8-16B active per token. 1M-token context, MIT license: free commercial use. Built for online serving, too large for local use on a single PC.
Google's open model family in 1B, 4B, 12B and 27B sizes. 128K-token context, 140+ supported languages and multimedia input (text and images). An excellent choice for RAG and long documents on consumer hardware.
La taglia compatta della famiglia Gemma 3 di Google: 4,32B di parametri, contesto di 128K token e supporto nativo a 140+ lingue (incluso l'italiano), con input testo+immagini. Licenza Gemma (uso commerciale con condizioni). In Q4_K_M occupa circa 2,9GB: gira su GPU da 8GB e su CPU moderne.
La taglia 12B della famiglia Gemma 4 di Google: 11,95B dense, contesto 256K e multimodalità integrata senza encoder separati (testo+immagini+audio). Licenza Apache 2.0. In Q4_K_M occupa circa 7,4GB: il punto di equilibrio tra qualità e requisiti hardware per l'uso locale su 8-16GB.
Il modello MoE della famiglia Gemma 4: 25,2B di parametri totali con 3,8B attivi per token. Contesto 256K, input testo+immagini, oltre 140 lingue, Apache 2.0. In Q4_K_M pesa circa 16,8GB: la scelta per chi ha 24GB di VRAM e vuole qualità da modello grande con velocità da modello piccolo.
Compact model from Google's Gemma 4 family: 8B total parameters, of which 4.5B effective thanks to per-layer embeddings (PLE). 131K-token context, 140+ languages, text+image and audio input. The entry size of the Gemma 4 family for local use on 8-16GB.
API platform to run ML models without managing infrastructure.
Serverless platform for deploying and auto-scaling AI models with GPUs.
La prima scheda Intel da prendere sul serio per l'IA locale: Xe2 Battlemage, 12GB GDDR6 a 456 GB/s, 190W. 12GB a prezzo d'ingresso: i modelli 8-14B in Q4 stanno interamente in VRAM. Il software è il punto debole: IPEX-LLM è stato archiviato a gennaio 2026, il percorso mantenuto oggi è Vulkan (Ollama nativo, llama.cpp) con ~40-42 tok/s su 8B Q4.
Meta's largest open model: closed-frontier quality but datacenter requirements. In practice it needs high-VRAM multi-GPU or the cloud.
Frontier-class open model with 131K-token context. Needs 48-80GB VRAM, a Mac with unified memory, or a multi-GPU setup.
The de-facto local workhorse: solid quality, 131K-token context and a huge GGUF quantization ecosystem. The standard choice for chat and assistants.
Small, fast model with up to 131K-token context. Ideal entry point for chat and summarization on modest hardware, great on phones and mini-PCs.
Il modello MoE grande della famiglia Llama 4 di Meta: 400B di parametri totali con 17B attivi (128 esperti). Contesto 1M di token, nativamente multimodale (testo+immagini), licenza comunitaria Meta. Richiede multi-GPU o macchine con memoria unificata 128GB+ anche quantizzato.
The compact MoE of the Llama 4 family: 17B active parameters out of 109B total (16 experts). Meta's community license allows free use under 700 million monthly users. One of the best entry points into the Llama 4 family.
Generates up to 30 seconds of music or sound effects from a text description. The open reference for music models: light and versatile.
Fast, efficient 7B model, well-tuned for multilingual European tasks. Excellent instruction quality at a fraction of the cost of 13B+ models.
Il primo 'modello unificato' di Mistral: dense da 128B che unisce instruction-following, reasoning e coding in un'unica serie di pesi, sostituendo Medium 3.1, Magistral e Devstral 2. Multimodale (testo+immagini), contesto 256K, ragionamento configurabile. Open weights con licenza Modified MIT: uso libero fino a una soglia di fatturato per le grandi aziende.
Il modello della linea Small di Mistral, ora MoE: 119B di parametri totali con 6,5B attivi per token (128 esperti, 4 attivi). Unifica instruct, reasoning e coding in un unico modello multimodale (testo+immagini) con contesto 256K. Licenza Apache 2.0.
Eight 7B experts with two active per token: 47B total params with large-model reasoning at a fraction of the compute cost. An open-world classic.
C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.
Sintesi vocale in locale a bassa latenza basata su ONNX: genera parlato naturale da testo in tempo quasi reale, anche su CPU. Multilingue (italiano incluso) e facile da integrare nei pipeline: download del modello, una riga di comando. MIT.
High-throughput LLM serving with PagedAttention and hardware optimizations.
Il port C/C++ ad alte prestazioni di Whisper: trascrizione vocale in locale ottimizzata per CPU e Apple Silicon, senza dipendenze da Python. Quantizzazione GGML (Q4_0–Q8_0) con riduzione fino al 65% a disco, streaming via whisper-stream, accelerazione Metal/Vulkan. MIT.
80GB HBM2e datacenter GPU: the standard for datacenter training and inference. Requires dedicated servers and enterprise power.
La workstation desktop NVIDIA per l'IA locale: superchip Grace Blackwell GB10 con 128GB di memoria unificata a 273 GB/s e circa 1 petaFLOP in FP4. CUDA completo, ~12 tok/s su modelli 27B a 4 bit, prefill veloce su contesti lunghi. Prezzo al lancio 3.999 USD, poi 4.699 USD per la carenza di memoria.
The top-tier datacenter GPU with 80GB HBM3: the top-of-the-line for large-model training and inference. Enterprise configurations only.
11B end-to-end full-duplex voice model: audio understanding and generation in a single architecture, without the cascaded ASR→LLM→TTS stack. First open full-duplex model with tool calling during conversation, ~450ms responses, interruptions, and natural turns.
The most accessible consumer GPU with 12GB VRAM: the entry point for local inference. Comfortably runs 7-8B quantized models.
24GB VRAM at very favorable used-market prices: the cheapest way to reach 24GB for 30B models. Discontinued.
16GB VRAM at an entry-level price: the right choice for quantized 13-14B models. Good headroom for personal use.
16GB VRAM with 672 GB/s bandwidth: high-end performance at a contained price. Among the best VRAM/price ratios.
The world's most powerful consumer GPU for local training and high-speed inference.
Il 16GB d'ingresso della generazione Blackwell: GB206, 16GB GDDR7 a 448 GB/s, 180W, PCIe 5.0 x8. Lancio a 429 USD: il modo più economico di far girare Qwen3.8-27B e i modelli 30B+ in Q4 su una singola scheda. Prezzo street 2026 più alto per la carenza di GDDR7.
Blackwell GB205 di fascia media: 12GB GDDR7 su bus 192-bit (672 GB/s), 6.144 CUDA core, 250W. Supporto completo ai formati FP4/FP8 per l'inferenza. Lancio a 549 USD, marzo 2025.
Blackwell GB203 da fascia alta: 16GB GDDR7 su bus 256-bit (896 GB/s), 8.960 CUDA core, 300W. Il 16GB rende possibile Qwen3.8-27B o i modelli 30B+ in Q4 con contesto ampio. Lancio a 749 USD, febbraio 2025.
Il modello di ragionamento open-weight di OpenAI: MoE con 21B di parametri totali e 3,6B attivi (32 esperti, top-4), contesto 128K e quantizzazione MXFP4 nativa. Licenza Apache 2.0: i pesi partono già quantizzati a 4 bit, senza passo di conversione. Sta in 16GB di VRAM e gira anche su RAM unificata o CPU.
Il modello di trascrizione di riferimento per l'IA locale: 809M di parametri, 99 lingue, licenza MIT. Versione distillata di Large-v3 con decoder ridotto da 32 a 4 layer: trascrive circa 8 volte più veloce con perdita di accuratezza trascurabile. Gira in tempo reale su una GPU consumer e anche solo su CPU.
Up to 47 seconds of text-to-audio: high-quality music and sound effects. Excellent for local audio production and sound design.
The image-generation classic: fast, light, and with the largest LoRA/ControlNet ecosystem of all. Perfect for 6-8GB GPUs.
The current standard for photoreal and artistic images at 1024px. Better prompt adherence and more coherent composition than SD 1.5.
Turns an image into a short ~2s video clip. The entry point for local video generation; 16GB VRAM recommended.
The node-based visual workflow for diffusion models: the standard for SDXL, FLUX, video and audio. Maximum flexibility, from simple to professional pipelines.
One-click SDXL image generation, Midjourney-style: zero parameters to manage. The simplest way to produce quality images.
Terminal-free desktop app for downloading and running GGUF models: GUI, built-in model search and a local server. The simplest way to get started.
Full ChatGPT-style web interface for Ollama and llama.cpp: multi-user accounts, organized conversations, document management for RAG, per-model system prompts, tools and plugins. Version 0.11.4 (September 2026) makes file metadata travel with the retrieved chunks and runs hybrid search natively in the database: large knowledge bases are much faster. The simplest way to turn your local AI into an everyday product.
Alibaba Group (Hangzhou, China): founded in 1999, its AI lab releases the Qwen models — among the world's most performant open-weight LLMs, under permissive licenses. The Qwen models in the catalog are the child resources.
| Headquarters | Hangzhou (China) |
|---|---|
| Founded | 1999 |
| Employees | ~128,000 |
| Revenue | 996 bn RMB (FY2025) |
| Ownership | Public company (NYSE: BABA) |
AMD (Santa Clara, California, USA): founded in 1969, its RDNA-architecture Radeon GPUs are the second consumer platform for local AI. Since ROCm 7, consumer cards (RX 7900 XTX, RX 9070 XT) are officially supported on Linux with no workarounds, and the Strix Halo APU brings up to 128GB of unified memory in a laptop. The GPUs and APUs in the catalog are the child resources.
| Headquarters | Santa Clara, California (USA) |
|---|---|
| Founded | 1969 |
| Employees | ~31,000 |
| Revenue | 34.6 bn USD (2025) |
| Ownership | Public company (Nasdaq: AMD) |
Apple (Cupertino, California, USA): founded in 1976, it makes Apple Silicon chips and unified-memory Macs — a reference for local AI on consumer hardware. Apple products in the catalog are the child resources.
| Headquarters | Cupertino, California (USA) |
|---|---|
| Founded | 1976 |
| Employees | ~166,000 |
| Revenue | 416 bn USD (FY2025) |
| Ownership | Public company (Nasdaq: AAPL) |
Black Forest Labs (Munich, Germany): founded in 2024 in Freiburg by Robin Rombach and the co-authors of Stable Diffusion, it produces the FLUX models — today's reference for high-quality open image generation. The FLUX models in the catalog are the child resources.
| Headquarters | Munich (Germany) |
|---|---|
| Founded | 2024 |
| Employees | < 100 (small research team) |
| Revenue | Not disclosed (private startup) |
| Ownership | Private company, 3.25 bn USD valuation (2025) |
DeepSeek (Hangzhou, China): founded in 2023 as a division of the brokerage High-Flyer, with ~200 employees it builds the most efficient open-weight LLMs in their class, focused on low-activation MoE. DeepSeek V4.1-Flash pushes 552B total parameters with ~8B active and a 1M context. The models in the catalog are the child resources.
| Headquarters | Hangzhou (China) |
|---|---|
| Founded | 2023 |
| Employees | ~200 |
| Revenue | Not disclosed |
| Ownership | Private company (a division of High-Flyer) |
Google (Mountain View, California, USA): founded in 1998, with DeepMind (2010) its Gemma models are Google's open-weight line — compact, multilingual and multimodal, under a permissive license. The Gemma 4 family (2026) reaches a 256K context. The models in the catalog are the child resources.
| Headquarters | Mountain View, California (USA) |
|---|---|
| Founded | 1998 (DeepMind: 2010) |
| Employees | ~185,000 |
| Revenue | 402.8 bn USD (2025) |
| Ownership | Public company (Nasdaq: GOOGL) |
Intel (Santa Clara, California, USA): founded in 1968, its Arc (Xe) GPUs are the third consumer platform for local AI, with 12-16GB of VRAM at entry prices. The maintained inference path is Vulkan (native Ollama, llama.cpp): the IPEX-LLM project was archived in January 2026. The GPUs in the catalog are the child resources.
| Headquarters | Santa Clara, California (USA) |
|---|---|
| Founded | 1968 |
| Employees | ~110,000 |
| Revenue | ~52 bn USD (2025) |
| Ownership | Public company (Nasdaq: INTC) |
Meta Platforms (Menlo Park, California, USA): founded in 2004 as Facebook, it publishes the Llama family — the world's most widely used open-weight LLM line, from 1B to 405B parameters, under a commercial-use license. The Llama models in the catalog are the child resources.
| Headquarters | Menlo Park, California (USA) |
|---|---|
| Founded | 2004 |
| Employees | ~72,000 |
| Revenue | 201 bn USD (2025) |
| Ownership | Public company (Nasdaq: META) |
Mistral AI (Paris, France): founded in 2023 by ex-Meta and ex-Google researchers, it builds the most efficient European open-weight LLMs, with permissive licenses and a focus on privacy and localization. The Mistral models in the catalog are the child resources.
| Headquarters | Paris (France) |
|---|---|
| Founded | 2023 |
| Employees | ~1,000 |
| Revenue | Not disclosed (private startup) |
| Ownership | Private company, >21 bn € valuation (2026) |
NVIDIA (Santa Clara, California, USA): founded in 1993, its CUDA GPUs are the de facto standard for local AI inference. VRAM determines which models run locally: 24GB (RTX 4090) covers ~30B quantized, 80GB (A100/H100) covers 70B+ models at full precision. The GPUs in the catalog are the child resources.
| Headquarters | Santa Clara, California (USA) |
|---|---|
| Founded | 1993 |
| Employees | ~45,000 |
| Revenue | 215.9 bn USD (FY2026) |
| Ownership | Public company (Nasdaq: NVDA) |
OpenAI (San Francisco, California, USA): founded in 2015, beyond its cloud models (GPT-5) the gpt-oss series (2025) made its reasoning models open-weight and free, in MXFP4 — the first large reasoning models runnable on a laptop. Whisper is the open standard for speech transcription. The models in the catalog are the child resources.
| Headquarters | San Francisco, California (USA) |
|---|---|
| Founded | 2015 |
| Employees | ~4,500 |
| Revenue | Not disclosed |
| Ownership | Private PBC (benefit corporation, 2025) |
Stability AI (London, UK): founded in 2019, it develops the Stable Diffusion models for image, video and audio generation — open-weight under permissive licenses, downloaded more than 150 million times. The Stable models in the catalog are the child resources.
| Headquarters | London (United Kingdom) |
|---|---|
| Founded | 2019 |
| Employees | 50-100 |
| Revenue | Not disclosed (private startup) |
| Ownership | Private company |
Il supercomputer desktop di NVIDIA basato su GB300 Grace Blackwell Ultra: fino a 748GB di memoria coerente, fino a 20 petaflop in FP4, CPU Grace a 72 core collegati via NVLink-C2C. Pensato per sviluppare ed eseguire agenti e modelli frontier-class al desk. Annunciato a GTC nel marzo 2025; versione per Windows annunciata il 1 giugno 2026 in arrivo a fine anno. Listino nell'ordine dei 99.000 USD.
Single-board computer with an 8-core ARM Cortex-A76 CPU and 2–16 GB of RAM: the reference for learning and running small local AI models in CPU inference.
NVIDIA's Rust CLI for AI-assisted Linux kernel development, focused on local AI: patch validation, builds, boot and tests, designed for backporting patches to older kernels. It supports local back-ends (Claude, OpenCode, Codex) or OpenAI-compatible servers. Apache 2.0, led by Andrea Righi.
Open-source tool that estimates local model inference speed across 250+ hardware configurations (GPU, Mac, CPU): measured estimates from community benchmarks or derived from similar setups. Use it to choose hardware before buying.
An online guide matching open-weights models to real hardware: for every GPU, Mac and mini-PC it shows what runs, how much memory it needs and at how many tokens per second, with guides on quantization and memory. The right tool for the own-hardware path.
Open-source proxy that intercepts and displays the contents of HTTPS requests in plaintext: the tool to verify what actually goes over the network from your local AI setup.
GTK4/libadwaita app for GNOME to chat with local models via Ollama, in Rust under GPLv3: it handles Ollama setup, models, and loading local documents and images.
The runtimes that run local models on your hardware: Ollama for simplicity, llama.cpp for portability, vLLM for production throughput. The engines in the catalog are the child resources.
A zero-dependency Python tool that recognizes and summarizes every file with a local LLM (Ollama, LM Studio or MLX-LM), fully offline. Detects type by content, extracts structure (CSV, email, scripts, logs) and reports files carrying instructions to the assistant. Works without a model (--no-llm). MIT license.
A Swift/MLX runtime for Kyutai's streaming speech recognition on Apple Silicon. The reference checkpoint is the STT-1B (English and French) in INT8/FP16, with a causal Mimi encoder (32 audio codes every 80ms). Supports INT4/INT8 exports. CC-BY-4.0 weights license.
The graphical frontends for using local models without accounts or cloud uploads: LM Studio, ComfyUI, Fooocus and Open WebUI. The studio in the catalog are the child resources.
100% local macOS dictation app: on-device transcription with Whisper and a rewrite pass with a local LLM (Qwen) on the M-series GPU. Text lands in the target app; no data touches the cloud.
The classic command-line tool for capturing network packets: the most direct way to see who is talking to the outside from your machine.
The graphical tool for capturing and analyzing network traffic: filters, colors, statistics and real-time visualization of every connection.
An 11.4MB (single-file) System-1 decision model built on an e5-small-v2 text encoder. 128MB resident-memory peak, runs on CPU. Given a state and typed questions, it returns a calibrated probability per option in one pass, without generating tokens. English-only.
The newest open music model with excellent control over lyrics and styles: generates complete songs with lyrics in many languages, including Italian.
An open decision model from Autotrust, 35B parameters (3B active), a fine-tune of Qwen3.5-35B-A3B for text classification and risk scoring. In NVFP4 quantization it occupies about 25.6 GB of VRAM: it runs on a 32 GB GPU with no frontier inference needed.
Bespoke Labs' 9B decision model: it runs entirely locally via Ollama 0.35 and answers typed questions (true/false, numeric scale, multiple choice) through the new systemone API. Built for fast classification in automated pipelines, no cloud.
27B reasoning model with ternary weights at 1.72 bits per weight, derived from Qwen3.8-27B with hybrid attention (~75% linear). In GGUF PTQ1_0 it takes 5.95 GB, retains 98.2% of the FP16 intelligence and reaches ~47 tok/s on an Apple M5 Max laptop, with a 262K-token context. Apache 2.0 license. Requires Prism ML's llama.cpp fork (ternary kernels are not in stock llama.cpp).
Cloudflare's 27B multimodal decision model (base Qwen3.8-27B): reads a state (text, JSON, images, video) and a schema of typed questions and returns a probability per option in a single pass, with no text generation. Jev and SystemOne API compatible.
A 5.6B open-weights image model for text-to-image, single-reference editing and multi-reference composition in one model, with placement boxes in the prompt. Qwen3-VL-4B text encoder. In BF16 (11.1GB), FP8 and NVFP4 (5.3GB). FLUX Kommunity license (non-commercial + qualified-user exception).
Modello MoE open-weight di inclusionAI: 560B parametri totali, 25B attivi per token, architettura ibrida (attenzione lineare KDA + Gated MLA), finestra di contesto fino a 1M di token, licenza MIT. Reasoning ibrido con modalità di pensiero on/off. A 4 bit entra in 512GB di memoria unificata.
Kandinsky Lab's foundation models for synchronized video and audio generation: from a prompt they generate video clips with aligned audio, released as open weights. Top paper of the day on Hugging Face (2026-10-06), with checkpoints and Spaces already available.
82-million-parameter speech synthesis model on a VITS architecture, Apache-2.0 for the weights, with voices in dozens of languages and surprisingly high quality for its size. Runs efficiently on CPU alone.
Aleph Alpha's MoE: 78.1B parameters (3.46B active per token), Apache 2.0, FP8 weights, context up to 1M tokens, reasoning mode and tool calling. Built for mission-critical workloads on local hardware.
Model for local agents and specialized applications, 1.73B parameters, tri-hybrid architecture (13 gated short convolutions, 5 sliding-window attention, 6 global attention). Explicit agentic post-training (harnesses like Hermes Agent, Pi, OpenClaw), 256K-token context, 16 languages including Italian, ~1GB of RAM at 4-bit. LFM Open License v1.0.
Inference API for Anthropic's Claude models (Claude 1 since 2023). Like OpenAI, you pay per token, input and output, with request limits per plan: the frontier path with no servers of your own.
Reimplementation of Whisper based on CTranslate2: up to 4 times faster than openai/whisper with the same accuracy and less memory. Supports 8-bit quantization on both CPU and GPU.
The cloud platforms for models: alternatives when local hardware isn't enough. Replicate and Runpod in the catalog are the child resources.
Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.
Google's open-source (Apache 2.0) on-device GPU inference engine, successor to the TensorFlow Lite GPU delegate: tensor virtualization, stage-aware KV cache for prefill and decode, FP16/INT8/INT4. Backends: OpenCL (Android), Metal (Apple), WebGPU via Dawn (browser), OpenGL ES 3.1+. Integrates with LiteRT; in production on Chrome, YouTube, Photos and Meet.
Serverless platform for running code and AI models on GPUs: you pay for real seconds of use, it scales to zero when idle, and you can run any open-weights model. One of the routes to inference on rented hardware without managing servers.
Inference API for OpenAI's frontier models (GPT series). You pay per token: a fixed price per million input and output tokens, no GPUs to manage, with rate limits per account tier.
Cloud gateway for LLM inference: a single API endpoint for hundreds of models (including open-weight MoE), with per-token pricing and automatic routing across providers. For Ling 3.1 Flash: $0.075/M input, $0.225/M output.
Open-source serving engine for LLMs with RadixAttention, continuous batching and first-class speculative decoding (MTP, EAGLE, draft models, ngram). Apache-2.0 license.
A marketplace for renting GPUs: GPU owners put their capacity up for sale and rent it out by the hour or by period, at prices that follow demand. A middle path between frontier cloud and your own hardware.
Hugging Face's Python library for running the pipelines of generative models: image, video, and audio. The standard way to run models like Qwen-Image locally, with CUDA support and CPU offload for lower-memory GPUs.
The standard library for NLP and multimodal models, with thousands of pre-trained models.
Framework for building LLM-based applications with chains, agents and tool use.
Open source framework for unified fine-tuning of 100+ LLMs and VLMs (Qwen, Llama, Mistral, Gemma): from full SFT to LoRA, QLoRA and DPO, with a built-in WebUI and startup from a single YAML file. The community's reference project (ACL 2024).
Apple's machine learning framework for Apple Silicon: lazy arrays computed across unified CPU and GPU, with unified memory as an asset. The mlx-lm library brings LLM training, fine-tuning (LoRA/QLoRA) and inference directly to the Mac.
Self-hosted workflow automation platform: a visual node-based canvas to connect apps, data and AI models (local or API), with AI Agent nodes and over 400 integrations.
Distributed platform for scaling ML and AI applications.
Optimized fine-tuning framework for NVIDIA GPUs and Apple Silicon: custom CUDA matrices that cut memory usage and double training speed versus standard methods. Supports LLMs and diffusion models.
