Releases: roboflow/inference
v0.15.0
What's Changed
- Fix broken NMS function by @PawelPeczek-Roboflow in #535
- Usage Tracking by @grzegorz-roboflow in #476
- Add workflow benchmark by @grzegorz-roboflow in #536
- Adjust usage collector by @grzegorz-roboflow in #537
- Add python code block to
workflows
by @PawelPeczek-Roboflow in #509 - Supervision Annotator Blocks by @yeldarby in #533
Full Changelog: v0.14.1...v0.15.0
v0.14.1
🔨 Fixed
We've not removed usage of @deprecated
elements of supervision
in release v0.14.0
which happened just a moment before supervision v0.22.0
. We are sorry for that problem. Fixing it with v0.14.1
.
Thanks @probicheaux for spotting a problem and providing PR with fix.
What to do if you cannot migrate to inference>=0.14.1
?
In script that resolve your environment (or in your requirements definition) enforce supervision<=0.21.0
Full Changelog: v0.14.0...v0.14.1
v0.14.0
🚀 Added
inference
is ready for Florence-2
🤩
Thanks to @probicheaux we have inference
package ready for Florence-2
. It is Large Multimodal Model capable of processing both image and text input handling wide range of generic vision and language-vision tasks.
We are excited to add it to the collection of models offered by inference
. Due to the complexity of build, model is shipped only within
docker image 🐋 . Everything within our official inference
server build for GPU 🤯 . To fully utilise the new models you need to wait on the release in Roboflow platform.
You should be able to spin up your container via inference-cli
:
inference server start
❗ What is required to run the container and what has changed in the build?
We've needed to bump required CUDA version in docker build for GPU server from 11.7
to 11.8
. That is why now, you may not be able to run
the container on servers having older CUDA. We've run the server experimentally on machine with CUDA 11.6 and it worked, but we cannot guarantee that to work on older builds.
🤔 How to run new model?
import requests
payload = {
"api_key": "<YOUR-ROBOFLOW-API-KEY>,
"image": {
"type": "url",
"value": "https://media.roboflow.com/dog.jpeg",
},
"prompt": "<CAPTION>",
"model_id": "<model-id-available-when-roboflow-platform-starts-the-support>"
}
response = requests.post(
f"{server_url}/infer/lmm",
json=payload,
)
print(response.json())
New blocks in workflows
🥹
We have added the following block to workflows
ecosystem:
Property Definition
which let you to use specific attribute of data as an input for next step or as outputDetections Classes Replacement
to replace classes of bounding boxes in scenario when you first run general object-detection model, then crop image based on predictions and you apply secondary classification model. Results of secondary model replaces originally predicted classes- and few others - explore our collection of blocks ✨
Blocks that were added are still in refinement - we may want to improve them over time - so stay tuned!
🌱 Changed
🔐 Mitigation for security vulnerabilities ❗ BREAKING 🚧
To two mitigate security vulnerabilities:
- unsafe deserialisation of pickled inputs enabled by default for self-hosted
inference
- Server-side request forgery (SSRF)
we needed to add couple of changes, among which one is breaking. From now on default value for env variable: ALLOW_NUMPY_INPUT
is False
.
Implications:
- if you rely on pickled numpy images passed to
inference
Python package or sent toinference
server - you need to set this env variable explicitly intoALLOW_NUMPY_INPUT=true
in your environment or start a server with this env variable (see how) - there are also other changes which you can optionally tune to run
inference
server safer - see our docs 📖
🔨 Fixed
❗ Removed bug in inference post-processing
Some models trained at Roboflow platform experienced problem with predictions post-processing when there was padding as
the option selected while creating dataset. Thanks to @grzegorz-roboflow it was fixed in #495
Other minor fixes
- fixed malformed
workflow
outputs in #499 - replace match statement with if-else for Python 3.9 compatibility by @natserract in #488
- InferencePipeline: allow it to run offline even if Active Learning enabled by @sberan in #491
- Import
sky
only when required because it is slow by @iurisilvio in #494 - Change GPT-4 default model into GPT-4o by @PawelPeczek-Roboflow in #500
- Monitoring improvements @robiscoding in #490, @robiscoding in #492
- Extend perspective correction to warp image by @grzegorz-roboflow in #503
- Show block name in error message thrown by steps_initialiser by @grzegorz-roboflow in #504
- Fix issue with workflows blocks after adding request id to response by @PawelPeczek-Roboflow in #505
- Follow config to import core models by @iurisilvio in #508
- Rename workflow blocks by @EmilyGavrilenko in #511
- Update upload weights list by @capjamesg in #512
- Default to Local Workflows Execution by @yeldarby in #515
🏅 New Contributors
- @natserract made their first contribution in #488
- @EmilyGavrilenko made their first contribution in #511
Full Changelog: v0.13.0...v0.14.0
v0.13.0
🚀 Added
🤯 Next-level workflows
Better integration with Roboflow platform
From now on, we have much better alignment with UI workflow creator available in Roboflow app
. Just take a look how nice it presents itself thanks to @hansent @EmilyGavrilenko @casmwenger @kresetar @jchens
But great look is not the only feature, the team has added tons of functionalities, including:
- operations on processed by
workflow
Execution Engine - including filtering and conditions are now possible to be build with UI creators - Roboflow models and projects available to be used are suggested automatically
- Preview option to run workflow that is under development is now available
- ... and much more - check out yourself!
workflows
Universal Query Language (UQL)
We've added Universal Query Language as extension to workflows
eco-system. We've discovered that it would be extremely helpful for users to be able to build chains of transformations (like filtering, selecting only specific bounding boxes, aggregating results etc) or expressions evaluating into booleans. UQL powers UI extensions like the one presented below:
Yes, we know that UQL
is not the best name, but as majority engineers we are struggling to find names for things we create. Please help us in that regards!
workflows
🤝 sv.Detections
From now on, the default representation of predictions from object-detection
, instance-segmentation
and keypoint-detection
models is sv.Detections
. That has a lot of practical implications for blocks creators. Take a look how easy it is to add a block that makes prediction from your custom model. This was mainly possible thanks to @grzegorz-roboflow
👉 Code snippet with your custom model block fitting our eco-system
from typing import Literal, Type
import supervision as sv
from inference.core.workflows.entities.base import (
Batch,
OutputDefinition,
WorkflowImageData,
)
from inference.core.workflows.entities.types import (
BATCH_OF_OBJECT_DETECTION_PREDICTION_KIND,
ImageInputField,
StepOutputImageSelector,
WorkflowImageSelector,
)
from inference.core.workflows.prototypes.block import (
BlockResult,
WorkflowBlock,
WorkflowBlockManifest,
)
class BlockManifest(WorkflowBlockManifest):
type: Literal["MyModel"]
images: Union[WorkflowImageSelector, StepOutputImageSelector] = ImageInputField
@classmethod
def describe_outputs(cls) -> List[OutputDefinition]:
return [
OutputDefinition(
name="predictions", kind=[BATCH_OF_OBJECT_DETECTION_PREDICTION_KIND]
)
]
class MyModelBlock(WorkflowBlock):
def __init__(self):
self._model = load_my_model(...)
@classmethod
def get_manifest(cls) -> Type[WorkflowBlockManifest]:
return BlockManifest
async def run(self, image: WorkflowImageData) -> BlockResult:
result = self._model(image)
detections = sv.Detections(...) # here you need to convert results into sv.Detections - there is a need to add couple of keys into .data property - docs covering that will come soon, in questions - do not hesitate to ask
return {"predictions": detections}
True conditional branching for SIMD operations in workflows
We had a serious technical limitation in previous iterations of workflows
Execution Engine - lack of ability to simulate different execution branches for each element of data processed`. This is no longer the case! Now it is possible to detect high-level objects, make crops based on detections and then for each cropped image independently decide whether or not to save in Roboflow project - based on condition stated in UQL 🤯
But this is not everything! As technical preview we prepared rock-paper-scissor game in workflows
. Check it out here
Advancements in video processing with workflows
This feature is still experimental, but we are making progress - now it is possible to process multiple videos at once with InferencePipeline
and workflows
:
Screen.Recording.2024-06-27.at.13.22.37.mov
👉 Code snippet
from typing import List, Optional
import cv2
import supervision as sv
from inference import InferencePipeline
from inference.core.interfaces.camera.entities import VideoFrame
from inference.core.utils.drawing import create_tiles
STOP = False
ANNOTATOR = sv.BoundingBoxAnnotator()
def main() -> None:
workflow_specification = {
"version": "1.0",
"inputs": [
{"type": "WorkflowImage", "name": "image"},
],
"steps": [
{
"type": "ObjectDetectionModel",
"name": "step_1",
"image": "$inputs.image",
"model_id": "yolov8n-640",
"confidence": 0.5,
}
],
"outputs": [
{"type": "JsonField", "name": "predictions", "selector": "$steps.step_1.predictions"},
],
}
pipeline = InferencePipeline.init_with_workflow(
video_reference=[
"<YOUR-VIDEO>",
"<YOUR-VIDEO>",
],
workflow_specification=workflow_specification,
on_prediction=workflows_sink,
)
pipeline.start()
pipeline.join()
def workflows_sink(
predictions: List[Optional[dict]],
video_frames: List[Optional[VideoFrame]],
) -> None:
images_to_show = []
for prediction, frame in zip(predictions, video_frames):
if prediction is None or frame is None:
continue
detections: sv.Detections = prediction["predictions"]
visualised = ANNOTATOR.annotate(frame.image.copy(), detections)
images_to_show.append(visualised)
tiles = create_tiles(images=images_to_show)
cv2.imshow(f"Predictions", tiles)
cv2.waitKey(1)
if __name__ == '__main__':
main()
Other changes:
- Step Name Property Copy Changes by @yeldarby in #444
- Abstract ImageInputField and RoboflowModelField + Copy Changes by @yeldarby in #445
- Allow CORS by default by @yeldarby in #485
- Add PerspectiveCorrectionBlock and PolygonSimplificationBlock by @grzegorz-roboflow in #441
List of contributors: @EmilyGavrilenko, @casmwenger, @kresetar, @jchens, @yeldarby, @grzegorz-roboflow, @hansent, @SkalskiP, @PawelPeczek-Roboflow
Predictions JSON ➕ visualisation @ Roboflow hosted platform
Previously clients needed to choose between visualisation of predictions and Predictions JSON returned from inference
server running at Roboflow hosted platform. This is no longer the case thanks to @SolomonLake and #467
from inference_sdk import InferenceHTTPClient, InferenceConfiguration
CLIENT = InferenceHTTPClient(
api_url="https://detect.roboflow.com/",
api_key="<YOUR-API-KEY>"
).configure(InferenceConfiguration(
format="image_and_json",
))
response = CLIENT.infer("<your_image>.jpg", model_id="yolov8n-640")
# check out
response["predictions"]
# and
response["visualisation"]
🌱 Changed
- Fixing yolov10 documentation by @nathan-marraccini in #480
- Supervision updates for Predict on a Video, Webcam or RTSP Stream Page by @nathan-marraccini in #477
- Add paligemma aliases for newly uploaded models by @probicheaux in #463
- Add PaliGemma LoRA by @probicheaux in #464
- Bump braces from 3.0.2 to 3.0.3 in /inference/landing by @dependabot in #466
- Fix security vulnerabilities by @PawelPeczek-Roboflow in #483
🥇 New Contributors
- @nathan-marraccini made their first contribution in #480
Full Changelog: v0.12.1...v0.13.0
v0.12.1
🔨 Fixed
Incompatibility of opencv-python
with numpy>=2.0.0
⚔️
Jun 16, there was release of numpy 2.0
making old builds of opencv-python
incompatible with new numpy
.
@grzegorz-roboflow investigated the issue and discovered that inference
users can be impacted if package inference-sdk
was used standalone, due to lack of upper-bound limit on numpy
dependency in that library.
To support impacted community members and Roboflow clients, we've prepared release with bug-fix.
Symptoms of the problem:
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead [...]
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
To solve the problem choose one of the following solutions:
👉 Install inference>=0.12.1
pip install "inference>=0.12.1"
# or
pip install "inference-cli>=0.12.1"
# or
pip install "inference-sdk>=0.12.1"
👉 Downgrade numpy
# in your Python environment hosting inference library
pip install "numpy<2.0.0"
We are sorry for inconvenience.
❗ Planned deprecations
np_image_to_base64(...)
to be replaced withencode_image_to_jpeg_bytes(...)
in the future - @grzegorz-roboflow in #469
🌱 Changed
- Remove sv.FPSMonitor deprecation warnings by @grzegorz-roboflow in #461
- Loose boto3 requirements by @iurisilvio in #457 -
inference
should install faster now 🤗 - Fix paligemma generation bug by @probicheaux in #459
- Add support for a tunnel to expose inference server to remote calls by @iurisilvio in #451
- Workflow documentation additions, add YOLOv10 docs by @capjamesg in #475
- fix Docker Getting Started link in docs returns 404 by @grzegorz-roboflow in #478
🏅 New Contributors
- @iurisilvio made their first contribution in #457
Full Changelog: v0.12.0...v0.12.1
v0.12.0
🔨 Fixed
🔥 YOLOv10
in inference
now has pre- and post-processing issues solved
Thanks to @jameslahm we have inconsistencies in results from YOLOv10
model in inference
package sorted out. PR #437
🌱 Changed
❗breaking change
❗Inference from PaliGemma
PaliGemma models changes model category from foundation one into Roboflow model. That implies the following change in a way how it is exposed by inference server
:
Before:
def do_gemma_request(prompt: str, image_path: str):
infer_payload = {
"image": {
"type": "base64",
"value": encode_bas64(image_path),
},
"api_key": "<ROBOFLOW-API-KEY>",
"prompt": prompt,
}
response = requests.post(
f'http://localhost:{PORT}/llm/paligemma',
json=infer_payload,
)
resp = response.json()
Now:
def do_gemma_request(prompt: str, image_path: str):
infer_payload = {
"image": {
"type": "base64",
"value": encode_bas64(image_path),
},
"prompt": prompt,
"model_id": "paligemma-3b-mix-224",
}
response = requests.post(
f'http://localhost:{PORT}/infer/lmm',
json=infer_payload,
)
resp = response.json()
PR #436
Other changes
- Replaced
sv.BoxAnnotator
withsv.BoundingBoxAnnotator
combined withsv.LabelAnnotator
to be prepare forsv.BoxAnnotator
deprecation by @grzegorz-roboflow in #434 - Add PaliGemma documentation, update table of contents by @capjamesg in #429
- Add http get support for legacy model inference by @PacificDou in #449
- Fix dead supported blocks link by @LinasKo in #448
- Docs: Remove banner saying Sv Keypoint annotators are experimental by @LinasKo in #450
🥇 New Contributors
- @jameslahm made their first contribution in #437
Full Changelog: v0.11.2...v0.12.0
v0.11.2
What's Changed
- Add YOLOv10 Object Detection Support by @NickHerrig and @probicheaux in #431
New Contributors
- @NickHerrig made their first contribution in #431
Full Changelog: v0.11.1...v0.11.2
v0.11.1
🔨 Fixed
❗ setuptools>=70.0.0
breaks CLIP
and YoloWorld
models in inference
Using setuptools
in version 70.0.0
and above breaks usage of Clip and YoloWorld models. That impacts historical version of inference package installed in python environments with newest setuptools
. Problem may affect clients using inference
as Python package in their environments, docker builds are not impacted.
Symptoms of the problem:
ImportError
while attemptingfrom inference.models import YOLOWorld
, despite previouspip install inference[yolo-world]
ImportError
while attemptingfrom inference.models import Clip
We release change pinning setuptools
version into compatible ones. This is not the ultimate solution for that problem (as some time in the future it may be needed to unblock setuptools
), that's why we will need to take actions in the future releases - stay tuned.
As a solution for now, we recommend enforcing setuptools<70.0.0
in all environments using inference
, so if you are impacted restrict setuptools
in your build:
pip install setuptools>=65.5.1,<70.0.0
🏗️ docker image for Jetson with Jetpack 4.5 is now fixed
We had issues with builds on Jetpack 4.5 which should be solved now. Details: #393
🌱 Changed
- In
workflows
, one can now define selectors to runtime inputs ($inputs.<name>
) in outputs definitions, making it possible to pass input data through theworkflow
.
Full Changelog: v0.11.0...v0.11.1
v0.11.0
🚀 Added
🎉 PaliGemma in inference
! 🎉
You've probably heard about new PaliGemma model, right? We have it supported in new release of inference
thanks to @probicheaux.
To run the model, you need to build and inference
server your GPU machine using the following commands:
# clone the inference repo
git clone https://github.com/roboflow/inference.git
# navigate into repository root
cd inference
# build inference server with PaliGemma dependencies
docker build -t roboflow/roboflow-inference-server-paligemma -f docker/dockerfiles/Dockerfile.paligemma .
# run server
docker run -p 9001:9001 roboflow/roboflow-inference-server-paligemma
👉 To prompt the model visit our examples 📖 or use the following code snippet:
import base64
import requests
import os
PORT = 9001
API_KEY = os.environ["ROBOFLOW_API_KEY"]
IMAGE_PATH = "<PATH-TO-YOUR>/image.jpg"
def encode_bas64(image_path: str):
with open(image_path, "rb") as image:
x = image.read()
image_string = base64.b64encode(x)
return image_string.decode("ascii")
def do_gemma_request(image_path: str, prompt: str):
infer_payload = {
"image": {
"type": "base64",
"value": encode_bas64(image_path),
},
"api_key": API_KEY,
"prompt": prompt
}
response = requests.post(
f'http://localhost:{PORT}/llm/paligemma',
json=infer_payload,
)
return response.json()
print(do_gemma_request(
image_path=IMAGE_PATH,
prompt="Describe the image"
))
🌱 Changed
- documentations updates:
- document source_id parameter of VideoFrame by @sberan in #395
- fix workflows specification URL and other docs updates by @SolomonLake in #398
- add link to Roboflow licensing by @capjamesg in #403
🔨 Fixed
- Bug introduced into
InferencePipeline.init_with_workflow(...)
inv0.10.0
causing import errors yielding misleading error message informing about broken dependencies:
inference.core.exceptions.CannotInitialiseModelError: Could not initialise workflow processing due to lack of dependencies required. Please provide an issue report under https://github.com/roboflow/inference/issues
Fixed with this PR #407
Full Changelog: v0.10.0...v0.11.0
v0.10.0
🚀 Added
🎊 Core modules of workflows
are Apache-2.0
now
We're excited to announce that the core of workflows
is now open-source under the Apache-2.0 license! We invite the community to explore the workflows
ecosystem and contribute to its growth. We have plenty of ideas for improvements and would love to hear your feedback.
Feel free to check out our examples and docs 📖 .
🏗️ Roboflow workflows
are changing before our eyes
We've undergone a major refactor of the workflows
Execution Engine to make it more robust:
blocks
can now be stand-alone modules - what makes them separated from Execution Enginebocks
now expose OpenAPI manifests for automatic parsing and validation- custom
plugins
withblocks
can be created, installed via pip, and integrated with our core libraryblocks
.
Thanks to @SkalskiP and @stellasphere we've made the documentation much better. Relying on new blocks self-describing capabilities we can now automatically generate workflows
docs - you can now see exactly how to connect different blocks and how JSON definitions should look like.
Visit our docs 📖 to discover more
❗ There are minor breaking changes in manifests of some steps (DetectionsFilter
, DetectionsConsensus
, ActiveLearningDataCollector
) as we needed to fix shortcuts made in initial version. Migration would require plugging output of another step
into fields image_metadata
, prediction_type
of mentioned blocks.
🔧 inference --version
Thanks to @Griffin-Sullivan we have now a new command in inference-cli
available to show details on what version of inference*
packages are installed.
inference --version
🌱 Changed
- Huge general docs upgrade by @LinasKo (#385, #378, #372) fixing broken links, general structure and aliases for keypoints coco-models
🔨 Fixed
- Inconsistency in builds due to release of
fastapi
package by @grzegorz-roboflow #374 - Middleware error in
inference server
- making every response not gettingHTTP 2xx
intoHTTP 500
😢 - introduced in v0.9.23 - thanks @probicheaux for taking the effort to fix it - bug that was present in post-processing of all
instance-segmentation
models making batch inference faulty when some image yields zero predictions - huge kudos to @grzegorz-roboflow for spotting the problem and fixing it.
🏅 New Contributors
- @Griffin-Sullivan made their first contribution in #339
Full Changelog: v0.9.23...v0.10.0