Apache-2.0 · real-time · edge-ready
D-FINE-seg is a family of vision models that find objects, outline them, label every pixel of a scene and place keypoints such as human pose, fast enough to run on a robot, a car or a camera.
The same model family, switched by a single setting. Pick a task.
Every pixel gets a class: road, sidewalk, car, person, tree, sky. It is how a car knows where it can drive.
The night sky, mines, reefs, cities and building sites. Trained once per domain. Click any tile to explore.
Jets from a black hole glow in radio (amber). The galaxy that launched them is a faint infrared dot (gray), often far from the brightest radio. The model boxes each radio galaxy, names its type and pins its host: 70.5 host AP vs 48.2, ahead of every published model.
A hammer and a folding tool hidden in iron ore at full belt speed. The ore is mapped, and each foreign object is flagged before it can jam the crusher.
A reef model trained on Red Sea surveys, run on a swim over the other side of the world. Live coral by type, sand, rock and fish.
Five dancers, 17 joints each, through every move. Hollow joints are ones the model knows are hidden.
Around 190 people in every frame, down to the far side of the crossing. A model trained on drone footage, looking down from above.
The truck and its driver outlined as it crosses the site. The general model has never seen an excavator; a site-specific model is a small fine-tune away.
The rider outlined at speed from an FPV drone, through motion blur and banking turns.
Glare hides the oncoming car from a regular camera, and the far car on the right barely shows in thermal. The model reads both cameras at once and finds all seven cars and both people. Switch views in the viewer.
Measured end-to-end on one RTX 5070 Ti with TensorRT, the way it would run in a product.
S model · 640×640 input · end-to-end latency per image
Each dot is a model size. Up and to the left is better.
| Model | mIoU | Latency (ms) | Params (M) |
|---|---|---|---|
| D-FINE-seg-X | 84.84 | 14.64 | 56.30 |
| D-FINE-seg-L | 83.80 | 9.77 | 24.97 |
| YOLO26x-sem | 83.58 | 13.58 | 40.13 |
| Mask2Former-Swin-L | 83.33 | 94.07 | 215.46 |
| Mask2Former-Swin-B | 83.29 | 79.85 | 106.89 |
| EfficientViT-L2 | 83.20 | 19.87 | 52.61 |
| EfficientViT-B3 | 83.01 | 21.58 | 39.60 |
| YOLO26l-sem | 82.95 | 8.77 | 17.85 |
| EfficientViT-L1 | 82.64 | 17.78 | 40.24 |
| Mask2Former-Swin-S | 82.43 | 73.32 | 68.72 |
| Mask2Former-Swin-T | 82.19 | 69.33 | 47.41 |
| EfficientViT-B2 | 82.06 | 16.25 | 15.32 |
| YOLO26m-sem | 82.02 | 7.54 | 14.30 |
| D-FINE-seg-M | 81.41 | 8.26 | 15.95 |
| SegFormer-B5 | 81.41 | 42.12 | 84.61 |
| SegFormer-B4 | 81.15 | 36.78 | 64.01 |
| D-FINE-seg-S | 81.12 | 6.37 | 7.99 |
| YOLO26s-sem | 80.80 | 5.52 | 6.49 |
| SegFormer-B3 | 80.77 | 29.43 | 47.24 |
| EfficientViT-B1 | 80.53 | 13.35 | 4.80 |
| SegFormer-B2 | 79.94 | 23.46 | 27.36 |
| Mask2Former-R101 | 79.72 | 68.93 | 63.00 |
| Mask2Former-R50 | 79.30 | 66.51 | 44.01 |
| YOLO26n-sem | 78.25 | 4.71 | 1.63 |
| D-FINE-seg-N | 77.99 | 4.21 | 3.36 |
| SegFormer-B1 | 77.42 | 12.32 | 13.68 |
| EfficientViT-B0 | 75.63 | 11.90 | 0.71 |
| SegFormer-B0 | 74.43 | 8.78 | 3.72 |
Cityscapes val, 500 images, one forward pass per frame. D-FINE-seg sizes trained by us; the other 23 are the official checkpoints, not retrained. End-to-end latency (each framework's own pre- and postprocessing), TensorRT FP16, batch 1, RTX 5070 Ti. Protocol and per-class IoU: cityscapes-benchmark ↗
| Model | F1 | Precision | Recall | Latency (ms) | Params (M) |
|---|---|---|---|---|---|
| D-FINE-seg S | 0.703 | 0.817 | 0.617 | 2.0 | 10.3 |
| YOLO26-M | 0.691 | 0.792 | 0.613 | 3.0 | 21.8 |
| RF-DETR-medium | 0.673 | 0.769 | 0.599 | 10.2 | 33.4 |
Every model fine-tuned by us on Cityscapes, scored on the 500 val images, confidence threshold tuned per framework for the best F1. D-FINE-seg and YOLO26 run at 640×640, RF-DETR at its own default resolution (576×576). End-to-end latency (each framework's own pre- and postprocessing), TensorRT FP16, batch 1, RTX 5070 Ti. Protocol: cityscapes-benchmark ↗
| Model | F1 | Precision | Recall | Latency (ms) | Params (M) |
|---|---|---|---|---|---|
| D-FINE-seg S | 0.661 | 0.748 | 0.592 | 3.1 | 11.9 |
| RF-DETR-seg-medium | 0.620 | 0.789 | 0.510 | 16.8 | 35.4 |
| YOLO26-M | 0.599 | 0.688 | 0.530 | 5.3 | 27.0 |
Same protocol as detection; RF-DETR-seg-medium runs at its own default resolution (432×432). End-to-end latency (each framework's own pre- and postprocessing), TensorRT FP16, batch 1, RTX 5070 Ti. Protocol: cityscapes-benchmark ↗
| Model | mIoU | Latency (ms) | Params (M) |
|---|---|---|---|
| D-FINE-seg M | 75.3 | 2.24 | 16.0 |
| YOLO26-L | 73.9 | 3.56 | 17.9 |
| YOLO26-M | 73.3 | 3.08 | 14.3 |
| D-FINE-seg S | 72.8 | 1.79 | 8.0 |
Every model fine-tuned by us on Cityscapes at 640×640, scored on the 500 val images at original resolution. Superseded by the full-resolution table above; the S row is the 1.8 ms semantic segmentation figure in the speed tiles. End-to-end latency (each framework's own pre- and postprocessing), TensorRT FP16, batch 1, RTX 5070 Ti. Protocol: cityscapes-benchmark ↗
| Model | Keypoint AP | AP50 | AP75 | Person box AP | Latency (ms) |
|---|---|---|---|---|---|
| D-FINE-seg S | 65.4 | 88.0 | 71.1 | 64.2 | 2.10 |
| YOLO26s-pose | 62.7 | 85.8 | 68.8 | 54.3 | 2.16 |
COCO val2017 at 640, official COCO AP. D-FINE-seg S trained for 40 epochs; YOLO26s-pose is the released checkpoint (300 epochs, Objects365 init). End-to-end latency (each framework's own pre- and postprocessing), TensorRT FP16, batch 1, RTX 5070 Ti. Details: D-FINE-seg changelog ↗
| Model | Box AP | Host keypoint AP | Latency (ms) | Params (M) | Input |
|---|---|---|---|---|---|
| D-FINE-seg S | 67.1 | 70.5 | 1.8 | 13.4 | 640 |
| Gal-DINO (published model) | 53.8 | 48.2 | 11.6 | 47 | 800 |
Test split, 420 images, 611 sources, both models scored with Gal-DINO's own evaluation (paper: 53.7 box AP, 48.1 host AP). Over three training seeds D-FINE-seg S scores 66.4 ± 0.7 box AP and 70.9 ± 0.4 host AP. The best published box AP is 54.5 (Gal-YOLOv8, same paper, no hosts). End-to-end latency (each framework's own pre- and postprocessing), TensorRT FP16, batch 1, RTX 5070 Ti. Code and raw predictions: radiogalaxynet-dfine-vs-galdino ↗
Off-road vehicles, construction equipment, underwater robots and planetary rovers need perception that is accurate, fast and small enough to run on board. D-FINE-seg trains on your own data, exports to TensorRT, ONNX, OpenVINO and CoreML, and ships with a C++ runtime for embedded GPUs.
Coming next
An open-source framework and model family for real-time object detection, keypoint estimation, instance and semantic segmentation. One codebase, one config setting per task (task: detect | segment | sem_seg | kpts), five model sizes from N to X, from training to export and deployment. The detection core follows D-FINE, a real-time detector from the DETR (detection transformer) family; the segmentation and keypoint heads and the rest of the pipeline are written from scratch.
Yes. The code and the released COCO weights are Apache-2.0, not AGPL-3.0 like Ultralytics YOLO, and models you train on your own data are yours to ship. Only the Cityscapes semantic segmentation weights are non-commercial, inherited from the dataset.
On Cityscapes semantic segmentation at full 2048×1024, D-FINE-seg-X reaches 84.84 mIoU, the highest of 28 models, and D-FINE-seg-L matches YOLO26x-sem at lower latency. On Cityscapes detection and instance segmentation, the S model beats the larger YOLO26-M and RF-DETR-medium on F1 and is faster. On COCO person keypoints, S scores 65.4 AP vs 62.7 for YOLO26s-pose at the same latency. All end to end with TensorRT FP16 on one RTX 5070 Ti; tables under "All the numbers".
Yes. Each detected object gets K points with a presence and a visibility score, so the model learns whatever points your labels define, like the host galaxy of a radio source in the demo above. Labels can be YOLO-pose or COCO keypoints. The released weights are trained on COCO human pose and are a good starting point.
Yes. Train on RGB plus one extra channel (thermal, depth or near-infrared); pretrained RGB weights still initialize the model. The "Blinded by headlights" demo reads RGB and thermal together: 93.1 mAP50 on M3FD.
pip install dfine-seg, put images and labels in YOLO or COCO format, set the task and model size in the config, train. Export and benchmarking are one command each. See the quick start, the video tutorial or the Colab notebook.