Skip to content

fix(openai): handle null choices from model_dump() for vLLM compatibility#34791

Closed
Vedant Madane (VedantMadane) wants to merge 1 commit into
langchain-ai:masterfrom
VedantMadane:fix/vllm-null-choices-robust-handling
Closed

fix(openai): handle null choices from model_dump() for vLLM compatibility#34791
Vedant Madane (VedantMadane) wants to merge 1 commit into
langchain-ai:masterfrom
VedantMadane:fix/vllm-null-choices-robust-handling

Conversation

@VedantMadane

Copy link
Copy Markdown

Summary

Fixes #32252

When using OpenAI-compatible APIs like vLLM, model_dump() can return null for choices even though the response object has valid choices accessible via direct attribute access. This occurs because the OpenAI SDK's Pydantic model may serialize differently when handling non-OpenAI API responses.

Changes

  • Added fallback logic in _create_chat_result that extracts choices directly from the response object when model_dump() returns null
  • Added unit test to verify the fix

Test plan

  • Added unit test test_create_chat_result_with_null_choices_in_model_dump that simulates the vLLM scenario
  • Manual testing with actual vLLM endpoint (not available in my environment)

@github-actions github-actions Bot added integration PR made that is related to a provider partner package integration openai `langchain-openai` package issues & PRs fix For PRs that implement a fix labels Jan 17, 2026
@codspeed-hq

codspeed-hq Bot commented Jan 17, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 6 untouched benchmarks
⏩ 30 skipped benchmarks1


Comparing VedantMadane:fix/vllm-null-choices-robust-handling (1f06e9b) with master (fbfe4b8)

Open in CodSpeed

Footnotes

  1. 30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@VedantMadane

Copy link
Copy Markdown
Author

vLLM requires NVIDIA GPU. The Docker image cannot run on CPU-only systems. I had one provisioned on GCP from work when I ran into this issue but that instance has since been decommissioned after project completion.

@VedantMadane Vedant Madane (VedantMadane) force-pushed the fix/vllm-null-choices-robust-handling branch from 91ca677 to 1f06e9b Compare March 8, 2026 18:49
@github-actions github-actions Bot added the size: S 50-199 LOC label Mar 9, 2026
@VedantMadane

Copy link
Copy Markdown
Author

Closing -- I can no longer reproduce/test this since the GCP instance with vLLM has been decommissioned. If someone else encounters null choices from model_dump() with vLLM, feel free to re-open or reference this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external fix For PRs that implement a fix integration PR made that is related to a provider partner package integration openai `langchain-openai` package issues & PRs size: S 50-199 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LangChain-OpenAI raises error due to null choices when using vLLM OpenAI-compatible API

2 participants