Cloudflare Docs
Workers AI
Cloudflare Docs
Workers AI
Edit this page
Report an issue with this page
Log into the Cloudflare dashboard
Set theme to dark (⇧+D)
  1. Products
  2. Workers AI
  3. ...
  4. Embedded
  5. Troubleshooting

Troubleshooting

This section will describe tools for troubleshooting and address common errors.

​​ Logging

General logging capabilities for Workers also apply to embedded function calling.

​​ Function invocations

The invocations of tools can be logged as in any Worker using console.log():

Logging tool invocations
export default {
	async fetch(request, env, ctx) {
		const sum = (args: { a: number; b: number }): Promise<string> => {
			const { a, b } = args;
      // Logging from within embedded function invocations
      console.log(`The sum function has been invoked with the arguments a: ${a} and b: ${b}`)
			return Promise.resolve((a + b).toString());
		};
    ...
  }

}

​​ Logging within runWithTools

The runWithTools function has a verbose mode that emits helpful logs for debugging of function calls as well input and output statistics.

Enabled verbose mode
const response = await runWithTools(
  env.AI,
  '@hf/nousresearch/hermes-2-pro-mistral-7b',
  {
    messages: [
      ...
    ],
    tools: [
      ...
    ],
  },
  // Enable verbose mode
  { verbose: true }

);

​​ Performance

To respond to a LLM prompt with embedded function, potentially multiple AI inference requests and function invocations are needed, which can have an impact on user experience.

Consider the following to improve performance:

  • Shorten prompts (to reduce time for input processing)
  • Reduce number of tools provided
  • Stream the final response to the end user (to minimize the time to interaction). See example below:
Streamed response example
async fetch(request, env, ctx) {
  const response = (await runWithTools(
    env.AI,
    '@hf/nousresearch/hermes-2-pro-mistral-7b',
    {
      messages: [
        ...
      ],
      tools: [
        ...
      ],
    },
    {
      // Enable response streaming
      streamFinalResponse: true,
    }
  )) as ReadableStream;


  // Set response headers for streaming
  return new Response(response, {
    headers: {
      'content-type': 'text/event-stream',
    },
  });

}

​​ Common Errors

If you are getting a BadInput error, your inputs may exceed our current context window for our models. Try reducing input tokens to resolve this error.