Closed
Description
Tracer Version(s)
3.8.0
Python Version(s)
Python 3.13
Pip Version(s)
pip 22.0.4
Bug Report
Hi I'm the litellm maintainer. This is now the 2nd time we've seen a critical issue caused by dd-trace. We have numerous user reports of the latest version of dd-trace leading to failed llm api calls.
While DataDog is a great product, it's unacceptable that a logging integration has led to failing llm calls (not just once but this is now the 2nd time). Once this is resolved, I'd really appreciate a root cause analysis so that we can have trust in recommending dd-trace to our users.
Here's the full stack trace
Traceback (most recent call last):
File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3104, in async_data_generator
async for chunk in proxy_logging_obj.async_post_call_streaming_iterator_hook(
...<18 lines>...
yield f"data: {str(e)}\n\n"
File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 291, in async_post_call_streaming_iterator_hook
async for item in response:
yield item
File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 291, in async_post_call_streaming_iterator_hook
async for item in response:
yield item
File "/usr/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 291, in async_post_call_streaming_iterator_hook
async for item in response:
yield item
[Previous line repeated 4 more times]
File "/usr/lib/python3.13/site-packages/ddtrace/contrib/internal/litellm/utils.py", line 75, in __aiter__
_loop_handler(chunk, self._streamed_chunks)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/ddtrace/contrib/internal/litellm/utils.py", line 108, in _loop_handler
streamed_chunks[choice.index].append(choice)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
IndexError: list index out of range
Reproduction Code
No response
Error Logs
No response
Libraries in Use
No response
Operating System
Mac