Large Language Models - LLMs

Frontier Models
- subscriptions, API charges

Provider Trials, Aggregators

Local LLMs
llama.cpp
Unsloth Studio
https://unsloth.ai/docs/new/studio/install#windows-1

Frontier Models

Minimax 3

DeepSeek
DeepSeek API
API Docs

Kimi K2

-------------

Grok (xAI)

Claude - Sonnet 4.6 - Opus 4.6 (Anthropic)

Gemini 3.1 (Google)

Codex - ChatGPT (OpenAI)

GLM-5

Qwen 3 (Alibaba)

Provider Trials

1. cerebras

https://inference-docs.cerebras.ai/introduction
Cerebras Models

gpt-oss-120b
zai-glm-4.7

endpoint: https://api.cerebras.ai/v1
https://api.cerebras.ai/v1/chat/completions

2. Groq

https://console.groq.com/docs/overview
https://console.groq.com/docs/api-reference#chat-create
Groq Models

endpoint: https://api.groq.com/openai/v1
https://api.groq.com/openai/v1/chat/completions

3. Mistral

https://admin.mistral.ai/organization
https://docs.mistral.ai/
Mistral Models

endpoint: https://api.mistral.ai/v1/chat/completions

4. Google Gemini

https://ai.google.dev/gemini-api/docs

Google Models

AIzaSyDwpd1g3kaMsU7OxcIIAyeajP3z0yAee2s
endpoint: https://generativelanguage.googleapis.com/v1beta/openai/chat/completions

Local LLMs

vllm-mlx on Apple Silicon

LM Studio 0.4 adds a mode for server-native deployment, parallel inference requests, and a stateful REST API that supports using local MCP servers.

Introducing llmster
Parallel Requests. Serve multiple concurrent inference requests. Requests to the same model can now be processed in parallel, instead of queued. Try this feature by putting two chats side-by-side in the new Split View feature.

https://huggingface.co/TeichAI/Qwen3-14B-Claude-4.5-Opus-High-Reasoning-Distill-GGUF/tree/main

Gemma 4 26B-A4B

Llama.cpp

.\llama-cli.exe -m qwen2.5-coder-7b-q8_0.gguf -ngl 60 --flash-attn on -c 32768 -b 2048 -ub 512 -t 6 --no-mmap --color on gguf models
tags
offloading
tokens/sec

.\llama-server.exe -m qwen2.5-coder-7b-q8_0.gguf -ngl 60 --flash-attn on -c 32768 -b 2048 -ub 512 -t 6 llama-server.exe -m "gemma-4-12b-it-Q4_K_M.gguf" -cmoe -c 248000 -ngl 99 --n-gpu-layers 99 -v gemma 4 github repo pls best version 28b? im using llama.cpp on hp i5 pc with 32 gig ram and quadro 8gig gpuOfficial Gemma repo: https://github.com/google-deepmind/gemma Best option for your hardware (i5 + 32GB RAM + Quadro 8GB GPU + llama.cpp) There isn't an exact 28B model. The closest (and very strong) is the Gemma 4 26B-A4B (Mixture-of-Experts, only ~4B active params per token). It's one of the best tradeoffs in the family. Recommended for your setup: Top pick if you want the biggest/best quality you can reasonably run: Gemma 4 26B-A4B-it (instruction-tuned) in a low quantization like Q4_K_M or IQ4_XS / Q3_K (try from highest quality that fits). Safer/faster daily driver: Gemma 4 12B-it or E4B — these will run much better on 8GB VRAM. Why 26B-A4B? MoE architecture makes it surprisingly efficient (behaves more like a 4B-8B model in compute while delivering near-31B quality on many tasks). GGUF Downloads (llama.cpp optimized) Main collection: https://huggingface.co/collections/ggml-org/gemma-4 Popular high-quality ones: ggml-org/gemma-4-26B-A4B-it-GGUF (best balance) unsloth/gemma-4-26B-A4B-it-GGUF (also excellent, often has good quants) ggml-org/gemma-4-12B-it-GGUF or E4B --hf flag in recent llama.cpp for easy download, e.g.: llama-server -hf ggml-org/gemma-4-26B-A4B-it-GGUF:Q4_K_M Offload as many layers as possible to GPU with -ngl 30-40 (or whatever fits without OOM — test incrementally). The rest goes to CPU/RAM. Use low quants + --cache-type-k q4_0 --cache-type-v q4_0 to save VRAM. --chat-template-file with the Gemma 4 template from llama.cpp. Your 32GB system RAM is excellent for CPU offloading and large context (up to 128k-256k on 26B with care).

Python API request

import urllib.request, json data = json.dumps({ "model": "qwen2.5-coder-7b-q8_0", "messages": [{"role": "user", "content": "What is a tsunami?"}], "temperature": 0.7, "max_tokens": 512 }).encode() req = urllib.request.Request( "http://127.0.0.1:8080/v1/chat/completions", data=data, headers={"Content-Type": "application/json"} ) with urllib.request.urlopen(req) as r: result = json.loads(r.read()) print(result["choices"][0]["message"]["content"])

Local Hardware

Mac Studio
Apple M4 Max chip
16-core CPU, a 40-core GPU, and a 16-core Neural Engine
128GB of integrated memory
1TB SSD storage

130k baht
-----------------------

Mac Studio
Apple M3 Ultra chip
28-core CPU, a 60-core GPU, and a 32-core Neural Engine
256GB of integrated memory
1TB SSD storage

200k++ baht

Cloud Servers

To RUN, EDIT, MANAGE a large model.
Lease hosted, managed server to use or modify Large LLMs

Distilling
Fine Tuning
Inference

Hostinger
digital Ocean, affiliate program
https://www.alibabacloud.com/en