# AI inference hosting
> Self-hosted AI inference means running open-weight models on GPU hardware you rent rather than calling a hosted API. Incognito VPS provides RTX 4090, RTX 5090, A100 and H100 servers with CUDA 13 and PyTorch preinstalled, unmetered transfer, and no logging of what you run.
**Source:** https://incognitovps.com/use-cases/ai-inference-hosting
**Updated:** 2026-08-08

## Recommended configuration

| Field | Value |
| --- | --- |
| Plan | FORGE-XL (gx-5090) |
| Specification | 12 vCPU, 96 GB, 2 TB |
| Jurisdiction | Amsterdam, Netherlands |
| Price | $219.00/mo |

32 GB of VRAM runs a 32B model at 4-bit or a 13B at full precision with room for context. Amsterdam keeps first-token latency low for European users.

## Resource guidance

| Scenario | Minimum |
| --- | --- |
| 7–8B, 4-bit | 1 × RTX 4090 (24 GB) |
| 13–14B, 8-bit | 1 × RTX 5090 (32 GB) |
| 32–34B, 4-bit | 1 × RTX 5090 (32 GB) |
| 70B, 4-bit | 1 × H100 (80 GB) |
| 70B full precision | 4 × H100 (320 GB) |

## When self-hosting beats an API

Hosted APIs are cheaper per token until they are not. The crossover is roughly continuous use: if a GPU is busy most of the day, a monthly rental costs less than the equivalent API spend, and the gap widens as volume grows.

The other reasons are structural rather than economic. Your prompts do not leave your server. There is no content policy applied to your requests. There is no rate limit, no deprecation of the model you built against, and no provider deciding your use case is no longer welcome. For some work these matter more than the price.

## VRAM is the only spec that matters at first

Model size in parameters times bytes per parameter gives you the floor, and you need headroom on top for the KV cache, which grows with context length and batch size. A 4-bit quantised 7B model is about 4 GB; a 4-bit 70B is about 40 GB.

The practical mapping: 24 GB runs 7B to 13B comfortably. 32 GB reaches 32B at 4-bit. 80 GB runs 70B at 4-bit with generous context. Beyond that you need multiple cards and tensor parallelism.

## The software stack

vLLM is the right default for serving — continuous batching and PagedAttention give it a large throughput advantage over naive implementations, and it speaks the OpenAI API format so most client code works unmodified. Ollama is simpler if you are experimenting and do not need throughput. TGI and SGLang are both worth a look for specific workloads.

All of them are one command away on our images, which ship with CUDA 13, cuDNN 9 and PyTorch 2.6 already installed.


## Frequently asked questions

### Which GPU for a 70B model?

A single H100 with 80 GB runs 70B at 4-bit quantisation with generous context. Full precision needs roughly 140 GB, so four H100s behind NVLink.

### Do you log what I run?

No. There is no inspection of GPU workloads, no prompt logging and no content policy applied to computation. Our AUP governs what you distribute from the server, not what you compute on it.

### Is the GPU shared with other customers?

No. The physical card is passed through and dedicated to your VM for the term — no time-slicing, no MIG partitioning, full VRAM and full throughput.

### Can I fine-tune, not just infer?

Yes. LoRA fine-tuning of a 7B model fits comfortably on a 4090. Full fine-tunes of larger models want an A100 or H100. Unmetered transfer means moving datasets in and checkpoints out costs nothing.


---

Incognito VPS — Offshore VPS, bare-metal and GPU servers in 16 locations, 8 of them chosen for their legal properties. No identity required, crypto-only payment, deployed in 55 seconds.
Operating since 2020. 16 locations, 8 of them privacy jurisdictions. AS204287.
Payment: BTC, XMR, ETH, USDT, USDC, LTC, TRX, SOL, BNB, DOGE, TON, DAI. No identity verification at any point.
Catalogue: https://incognitovps.com/pricing.md · API: https://incognitovps.com/openapi.json · Index: https://incognitovps.com/llms.txt