CrewAI Tool Error · AI Developer Accelerator

Jun '24 • CrewAI

CrewAI Tool Error

I'm using crewAI to create a simple agentic system to detect an object in an uploaded image. I create a custom tool but it is not working as expected.

The tool has been created as following:

@tool

def ImageObjectDetectorTool(arguments: dict) -> str:

"""A tool for identifying objects in images. The input will be a dictionary with two keys:

- The first [IMAGE_PATH] will be the full path of an image file.

- The second [TEXT_PROMPT] will be replaced by the string query describing the object/s to identify in the image.

To use this tool just call ImageObjectDetectorTool(dictionary) replacing dictionary by {"IMAGE_PATH": value, "TEXT_PROMPT": value}

variables by their values.

The output will the a path with an image file with the identified objects overlayed in the image.

"""

print("Tool activated:", arguments) # just to see if the agent achieves to enter into this tool

BOX_TRESHOLD = 0.35

TEXT_TRESHOLD = 0.25

IMAGE_PATH = arguments["IMAGE_PATH"]

image_source, image = load_image()

boxes, logits, phrases = predict(

model=model,

image=image,

caption=arguments["TEXT_PROMPT"],

box_threshold=BOX_TRESHOLD,

text_threshold=TEXT_TRESHOLD

)

annotated_frame = annotate(image_source=image_source, boxes=boxes, logits=logits, phrases=phrases)

output_file = IMAGE_PATH.split('.')[0] + "_annotated_image.jpg"

cv2.imwrite(output_file, annotated_frame)

return output_file

And the Agent, Task and Crew have been created as:

### AGENT ###

obj_detector_agent = Agent(

role="Object Detector",

goal="""Find and detect an object into a image.

The result will be the path where is saved the image with the object located. Just give to the user the final answer with the path.

""",

backstory="""As an Object Detector, you are responsible for detecting an object in the image

""",

verbose=True,

llm=llm,

tools=[ImageObjectDetectorTool]

)

### TASK ###

obj_detector_task = Task(

description="""Detect a {object} in the following image {image_path} and show the annoted image

returned by ImageObjectDetectorTool as a result""",

expected_output="""Show the annotated image from the annotated image path received from

ImageObjectDetectorTool output with the founded object remarked in the image""",

agent=obj_detector_agent,

)

### CREW ###

crew = Crew(

agents=[obj_detector_agent],

tasks=[obj_detector_task],

verbose=2,

human_input=True

)

result = crew.kickoff(inputs={"image_path": IMAGE_PATH, "object": TEXT_PROMPT})

When I execute this code (attached image), it seems the agent is not entering to the tool because the response it is not the output of the tool nor the image is created as it does the "cv2.imwrite(output_file, annotated_frame)" code of the tool.

What am I doing wrong?

2 comments