
Verification of video2video using Cosmos3
This page has been translated by machine translation. View original
It's been a while. I'm Kent, an intern.
While working on Physical AI-related tasks, I had the opportunity to use NVIDIA's cloud environment Brev and verify video-to-video data synthesis with cosmos3, so I'd like to share what I found.
About NVIDIA Brev
For the GPU cloud environment this time, I used NVIDIA Brev.
This service is similar to vast.ai, which became a topic of conversation a while ago, and allows you to rent GPUs from providers around the world for however long you need.
The fact that it's a service provided by NVIDIA itself made me feel comfortable using it.
The advantages include being cheaper than hosting on AWS or Google Cloud and the ease of setting up the environment.
In practice, with AWS and similar services you need to establish connection routes after launching an instance, but with Brev, Jupyter Notebook is available from the start, making it easy to use even for those unfamiliar with the cloud.
Setting Up cosmos3 (vLLM-Omni)
This time, I set it up on an instance equipped with an H100 (driver 580.x), serving Cosmos3-Nano via vLLM-Omni in a .venv native environment. The steps are as follows. Basically, following the setup instructions from Nvidia/Cosmos on Github should be fine, but some additional steps are required. Also, please note that the cosmos framework provided by NVIDIA does not support video input as of the time of writing (July 15, 2026).
1. Preparing uv
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv --version
2. Creating a venv (Python 3.13)
cd /home/shadeform
uv venv --python 3.13 --seed --managed-python
3. Logging in to Hugging Face
Cosmos3-Nano requires a safeguard for video and image generation, and since that safeguard is a gated model, you need to agree to the license on Hugging Face in advance. The model used is here: Cosmon-1.0-Guardrail
After agreeing, log in using a token. Since this is a rented cloud rather than a personally owned PC, it would be safer to grant only READ permission for the token you generate.
uvx --from huggingface_hub hf auth login --token $HF_TOKEN
4. Installing vLLM-Omni from the main branch
uv pip install --torch-backend=cu130 \
"vllm-omni @ git+https://github.com/vllm-project/vllm-omni.git@main"
5. Installing vLLM itself
Since vLLM itself is not added to the runtime environment in step 4, you need to install it here. This time, we're using version 0.25.0 to match the vLLM-Omni version.
uv pip install --torch-backend=cu130 "vllm==0.25.0"
6. Installing the Guardrail (Safety Checker)
If you haven't agreed to the model's terms of use on Hugging Face, an error will occur in a later step, so please refer to this page if necessary.
uv pip install --torch-backend=cu130 cosmos-guardrail
7. Verification
.venv/bin/vllm serve --help 2>&1 | grep -icE "Failed to load plugin|Traceback"
.venv/bin/python -c "import torch,vllm,vllm_omni; \
print('torch',torch.__version__,'cuda',torch.cuda.is_available()); \
print('vllm',vllm.__version__,'omni',vllm_omni.__version__)"
The expected output is as follows.
0
torch 2.11.0+cu130 cuda True
vllm 0.25.0 omni 0.25.0rc2.dev10+...
The 0 on the first line indicates that no plugin load errors occurred, and lines 2–3 confirm that torch recognizes the GPU with a CUDA 13 build and that the vllm and omni versions are aligned.
8. Starting the Server
source .venv/bin/activate
export HF_HUB_DISABLE_XET=1
vllm serve nvidia/Cosmos3-Nano \
--omni \
--model-class-name Cosmos3OmniDiffusersPipeline \
--allowed-local-media-path / \
--port 8000 \
--init-timeout 1800
Once Application startup complete. is displayed, the API will be available at http://localhost:8000.
9. Calling video-to-video
Use POST /v1/videos/sync (synchronously returns an MP4 directly) or POST /v1/videos (asynchronously returns a job ID). The video_reference is not passed as a file upload, but as a JSON string in the form {"video_url": "data:video/mp4;base64,..."}.
Converting video to a base64 data URL
python -c 'import base64,json; b=open("sample_input.mp4","rb").read(); \
open("vref.json","w").write(json.dumps({"video_url":"data:video/mp4;base64,"+base64.b64encode(b).decode()}))'
Running video-to-video
curl -X POST http://localhost:8000/v1/videos/sync \
-F "model=nvidia/Cosmos3-Nano" \
-F "prompt=Replace the background with a snowy mountain landscape at sunset, keep the main subject unchanged, cinematic lighting" \
-F "negative_prompt=blurry, distorted, low quality" \
-F "video_reference=<vref.json" \
-F "num_frames=17" \
-F "num_inference_steps=20" \
-F "fps=8" \
-F "seed=42" \
-o sample_output.mp4
No description of video-to-video could be found in the official documentation, but I was able to call it using the method above.