.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
Laya is an open-weights model that does what Jev by TypeSafe AI does: you send it a piece of text and a typed question (pick one of these options, rate this on a scale, yes or no) and it returns the answer with a probability for every option and a confidence score, in one forward pass, without generating a word. The weights are Apache 2.0 on Hugging Face, and since version 0.3.7 the package ships laya-serve, an HTTP server that speaks the same POST /v1/systemone protocol as TypeSafe's hosted API. So a client written against Jev moves to your own server by changing its base URL.
This guide installs Laya on an Ubuntu 24.04 GPU VPS, starts the server on the card, keeps it up with systemd and sends the first request.
The project is a week old as I write this (the repository was created on 18 September 2026 and had 110 open issues five days later), so pin the version. Every command below was checked against the 0.3.7 README; where the README is silent I say so.
What Laya AI is
Laya comes from Convai Innovations and is basically described as a "non-autoregressive System 1 decision engine". The code is at github.com/NandhaKishorM/laya and the weights sit under the convaiinnovations organisation on Hugging Face.
There are three checkpoints and a Router that picks one per request by looking at the script and language of the input before the forward pass:
| Checkpoint | Encoder | Parameters | Context | Picked for |
|---|---|---|---|---|
laya | ModernBERT-large | 421M | 512 | English text |
laya-multilingual | mmBERT-base | 322M | 1024 | 100+ languages, about twice as fast |
laya-typed-decisions | ModernBERT-large | 421M | 1024 | Typed-decision workflows, on request |
The three question types are the same as Jev's: choice, score and noul, their name for a yes/no question. Several questions ride in one request and are answered together. The reason to route rather than run the English checkpoint for everything is in the README's own numbers: on non-Latin scripts the English model collapses while staying confident (Khmer scores 0.000 accuracy at 0.952 confidence), so the confidence score can't save you and the decision has to be made before the model runs.
There is also a free hosted endpoint from impossibl that serves the same checkpoints. The reason to run your own is that your tickets, signups and log lines never leave your server, and a request path with no third party in it is the whole point of a self-hosted decision model.
Which GPU VPS Laya needs?
Honestly? Not much of one :)
The largest checkpoint is 421M parameters, so all three loaded at once fit in 16 GB of VRAM with most of it free. The README's benchmarks were measured on a Tesla T4: 32.8 ms for a single question on laya-multilingual (39.5 ms on laya), 7.2 ms per question with ten batched in one call and 103 to 332 questions per second on one card.
The 16 GB RTX PRO 2000 Blackwell at the bottom of our GPU VPS lineup has 1.7 times the CUDA cores of a T4 and covers this with room to spare; we haven't measured Laya on Blackwell ourselves, so the numbers in this article stay the README's.
Do you need the GPU at all? For a script that classifies a few hundred tickets a night, no. Laya runs on CPU, and the README puts a single question at 193 to 464 ms there.
However... the GPU case is an API that sits in a request path: at 33 ms a Laya call in front of a chat reply or an order form is invisible to the person waiting; at 400 ms it is not.
Install laya-serve on Ubuntu 24.04
Step 1: Check the driver and Python
Log in to the server and confirm the card is visible:
nvidia-smiYou want the table with the GPU name and the driver version. If the command is missing, install the driver first with sudo ubuntu-drivers install and reboot; nothing below works without it. Then check Python, because Laya's dependencies (transformers 5, torch 2.14, huggingface_hub 1) set a floor of 3.10:
python3 --version
sudo apt install -y python3-venvUbuntu 24.04 ships Python 3.12, which is fine. The python3-venv package is the one the README warns about on Debian and Ubuntu: without it, creating the virtual environment fails with a message about ensurepip.
Step 2: Install Laya with the serve extra
Give the server its own system user and a home under /opt, so the weights cache and the virtual environment live somewhere predictable and not in your login account:
sudo useradd -r -m -d /opt/laya -s /usr/sbin/nologin laya
sudo -u laya python3 -m venv /opt/laya/.venv
sudo -u laya /opt/laya/.venv/bin/python -m pip install "laya[serve]==0.3.7"The [serve] extra adds FastAPI and uvicorn; the plain package is the SDK only. The install pulls the default PyTorch wheel from PyPI, which on Linux ships with CUDA, so there is no separate CUDA toolkit to install. Confirm both halves before starting the server:
sudo -u laya /opt/laya/.venv/bin/python -I -c "import laya; print(laya.__version__)"
sudo -u laya /opt/laya/.venv/bin/python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"The first line prints 0.3.7 without loading a checkpoint (the -I flag is the README's trick for making sure a stray local copy of the source can't mask a broken install). The second prints True and the card's name. If it prints False while nvidia-smi works, the driver is older than the CUDA build in the wheel; the PyTorch install page lists the builds, and you would install a matching one into the same venv before Laya.
Step 3: Start laya-serve on the GPU
A first run in the foreground, so you can watch the weights download:
sudo -u laya env LAYA_DEVICE=cuda LAYA_PRELOAD=1 LAYA_API_KEY=change-me /opt/laya/.venv/bin/laya-serveThe first start fetches all three checkpoints from Hugging Face into /opt/laya/.cache/huggingface; the project's Docker notes say to allow several minutes for it. Once uvicorn reports it is listening on 0.0.0.0:8000, the server is up with every checkpoint resident in VRAM.
Everything is configured through environment variables. The full list from the README's self-hosting section:
- LAYA_DEVICE:
cudaorcpu - LAYA_PRELOAD:
1loads every checkpoint at startup instead of on first use - LAYA_MODELS: comma list of checkpoints to preload, if you don't want all three
- LAYA_HOST and LAYA_PORT: bind address and port, default
0.0.0.0and8000 - LAYA_API_KEY: when set, every request must carry
Authorization: Bearer <key> - LAYA_THREADS: caps torch's CPU threads, only relevant on CPU
- LAYA_AUTO_TASK: task auto-detection
Always set the key. The default bind is every interface, and a decision model with no auth on a public port is a free classifier for anyone who finds it.
Step 4: Send the first request
From a second terminal on the same box, the README's own example:
curl -s localhost:8000/v1/systemone \
-H "Authorization: Bearer change-me" \
-H "content-type: application/json" \
-d '{
"state": {"body": "billed twice, refund please or we cancel"},
"questions": {
"dept": {"type": "choice", "instructions": "which team?",
"criteria": {"billing": "refunds", "tech": "bugs"}}
}
}'The reply has an answers object keyed by your question names, each carrying the picked option, the per-option probabilities and a confidence number, plus a usage block with input and output token counts. That shape is what makes the swap possible: it is schema-identical to what Jev returns, so nothing downstream has to change.
A model field in the request is honoured when it names a Laya checkpoint (english, multilingual or typed-decisions). Leave it out and the router decides from the text.
Step 5: Keep it running with systemd
The repository ships a NixOS module with a hardened unit, and nothing for Ubuntu, so this unit is mine. Put the key in a root-only file first:
sudo mkdir -p /etc/laya
echo "LAYA_API_KEY=$(openssl rand -hex 32)" | sudo tee /etc/laya/serve.env
sudo chmod 600 /etc/laya/serve.envThen the unit at /etc/systemd/system/laya-serve.service:
[Unit]
Description=Laya System One server
After=network-online.target
Wants=network-online.target
[Service]
User=laya
WorkingDirectory=/opt/laya
Environment=LAYA_DEVICE=cuda
Environment=LAYA_PRELOAD=1
Environment=LAYA_HOST=0.0.0.0
Environment=LAYA_PORT=8000
EnvironmentFile=/etc/laya/serve.env
ExecStart=/opt/laya/.venv/bin/laya-serve
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.targetsudo systemctl daemon-reload
sudo systemctl enable --now laya-serve
journalctl -u laya-serve -fThe log shows the checkpoints loading and uvicorn binding the port; after a reboot the service comes back on its own, and the weights are already cached so it is up in seconds rather than minutes. If you only ever call it from the same machine, change LAYA_HOST to 127.0.0.1 and stop here. For an app on another server, keep the public bind and open port 8000 only to that address:
sudo ufw allow from 203.0.113.10 to any port 8000 proto tcpSwap in your app server's IP; the rule only means something if ufw is active with its default deny for incoming, which sudo ufw status confirms. TLS is not built in; if the client sits outside your network, put Caddy or nginx in front and point the client at that.
Point a Jev client at your server
Take whatever calls TypeSafe's hosted endpoint today and change two things: the base URL becomes http://YOUR_SERVER_IP:8000 and the bearer token becomes the key from /etc/laya/serve.env. The README names the hs-jev Haskell client as one that works with only the baseUrl repointed, and the same holds for a plain requests.post in Python, because the request and response bodies are the same JSON.
Batch where you can. The README's T4 table shows a single question at 32.8 ms and ten questions in one call at 72.3 ms, so five questions about one ticket in one request cost a fifth of five requests.
Where Laya is weaker than Jev
The README's comparison table is the vendor's, and it favours Laya: 7.8 times faster than Jev's independently measured 236 to 276 ms p50, and higher accuracy on typed-decisions (0.766 against 0.727), AG News and DAIR Emotion. Read the sysone-bench results before you believe the accuracy half. It is the first third-party head-to-head on byte-identical inputs, and in its September 21 run Jev wins most suites, including triage (0.888 against Laya's 0.800) and moderation (0.989 against 0.833), while Laya wins AG News and MNLI.
And yet... Laya was still faster, and it ran on a laptop CPU in that test. I think Laya is the cheaper and faster model and Jev is still the more accurate one on the messy classification a support queue produces. Which of those gaps matters is something you measure on your own tickets before trusting either - so try it yourself first.
Their own "Honest limits" section lists things the tutorial above does not fix. Choice questions with more than about 20 options degrade sharply, because all the option text shares a fixed token budget (Banking77 with 77 labels scores 0.425 against Jev's 0.870). noul on the English checkpoint can latch onto its own true/false labels and return a confident no for clearly positive text, tracked as issue #156; the workaround the maintainers give is to ask the same thing as a two-option choice with neutral keys. laya-multilingual rarely picks the first-listed level on score questions (#131). And the multilingual checkpoint ships with no fitted temperatures, so its probabilities are not calibrated until you fit them on your own data.
But I read one thing in the Readme that I think will bite a European deployment: the built-in language detector is a script check plus a function-word heuristic with no word list for some languages, and the README's own example shows analyse("Care este ora in Tokyo?") coming back as English. Short Spanish, Portuguese and Romanian messages go to the English checkpoint unless you tell the router otherwise.
So - If most of your traffic is not English, start the server with the multilingual checkpoint as the default, or pass lang_guess from a language-identification model you already run.
