Skip to content

fix(ai): 법률 RAG 답변 LLM 호출 연결 - #31

Merged
HOKAGO-MEMORIES merged 2 commits into
developfrom
fix/f3-live-legal-rag-answer
Jun 22, 2026
Merged

fix(ai): 법률 RAG 답변 LLM 호출 연결#31
HOKAGO-MEMORIES merged 2 commits into
developfrom
fix/f3-live-legal-rag-answer

Conversation

@HOKAGO-MEMORIES

@HOKAGO-MEMORIES HOKAGO-MEMORIES commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

변경 내용

  • backend-ai LLMClient가 법률 RAG 카드가 있을 때 설정된 OpenAI-compatible /chat/completions 엔드포인트를 호출하도록 연결했습니다.
  • GMS_API_KEY, LLM_MODEL, 선택적 LLM_BASE_URL 설정을 사용합니다.
  • LLM 키가 없거나 호출 실패 시 기존 deterministic fallback을 유지합니다.
  • 외부 네트워크 없이 검증 가능한 fake HTTP 테스트를 추가했습니다.

원인

  • 기존 구현은 LLMClient라는 경계만 있고 실제 LLM 호출 없이 템플릿 답변을 반환해서, 사용자가 챗봇을 테스트하면 정해진 답변처럼 보였습니다.

테스트

  • cd backend-ai && C:\Users\SSAFY\AppData\Local\Temp='C:\salmanhae\.tmp\pytest'; C:\Users\SSAFY\AppData\Local\Temp='C:\salmanhae\.tmp\pytest'; .\.venv\Scripts\python.exe -m pytest tests -p no:cacheprovider

closes #30

Summary by CodeRabbit

  • New Features
    • Added live legal consultation via an OpenAI-compatible chat-completions endpoint, with automatic fallback to predefined responses when live results aren’t available.
    • Supports additional request handling for property search, pricing analysis, safety assessment, and calculation-related intents with deterministic outputs.
  • Chores
    • Introduced configurable llm_base_url setting (with environment override) for controlling the LLM API endpoint.
  • Tests
    • Added/updated tests to verify live request behavior and robust extraction of assistant text from chat-completions responses.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b033c06f-f360-4d4c-8430-82f23582df8b

📥 Commits

Reviewing files that changed from the base of the PR and between dc68f97 and 409a0fe.

📒 Files selected for processing (2)
  • backend-ai/app/clients/llm_client.py
  • backend-ai/tests/test_legal_answer_generation.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • backend-ai/tests/test_legal_answer_generation.py
  • backend-ai/app/clients/llm_client.py

📝 Walkthrough

Walkthrough

LLMClient gains a configurable constructor with injectable HTTP, a live _generate_live_legal_answer method that POSTs a RAG prompt to an OpenAI-compatible endpoint, and a new extract_chat_completion_text helper. generate_answer expands intent routing and falls back to deterministic behavior when the live call is unconfigured or fails. A new llm_base_url config field and matching tests are added.

Changes

Live Legal LLM Integration

Layer / File(s) Summary
Config field and LLMClient constructor
backend-ai/app/core/config.py, backend-ai/app/clients/llm_client.py
Adds llm_base_url field with LLM_BASE_URL env alias (default https://api.openai.com/v1) to Settings. Introduces LLMClient.__init__ accepting api_key, model, base_url (normalized), timeout_seconds, and injectable http_post. Expands generate_answer to attempt a live legal path first, then fall back, and adds branches for PRICE_ANALYSIS, SAFETY_ANALYSIS, HUG_CALC, and a default clarification message.
Live answer generation and content extraction
backend-ai/app/clients/llm_client.py
_generate_live_legal_answer validates config and inputs, builds a RAG prompt from state["message"] and legal_cards, POSTs to {base_url}/chat/completions with bearer auth and timeout, raises on HTTP errors, and returns None on missing config or caught exceptions. extract_chat_completion_text parses choices[0].message.content as either a plain string or structured list of text/output_text parts, joining results with newlines.
Tests: live path and extraction
backend-ai/tests/conftest.py, backend-ai/tests/test_legal_answer_generation.py
conftest.py sets GMS_API_KEY="" at import. Existing tests updated to pass api_key="" explicitly. New test_llm_client_uses_live_chat_completion_when_configured injects a fake_post to assert URL, bearer auth header, model, and message content, then validates the returned answer. New test_extract_chat_completion_text_supports_text_parts asserts newline-joined extraction of mixed content parts. New test_extract_chat_completion_text_handles_malformed_payloads verifies None for unexpected shapes.

Sequence Diagram

sequenceDiagram
    participant Client
    participant LLMClient
    participant _generate_live
    participant HTTPPost
    participant LLMEndpoint as OpenAI /chat/completions
    participant extract

    Client->>LLMClient: generate_answer(state, Intent.LEGAL_CONSULT)
    LLMClient->>_generate_live: state
    alt Config present (base_url, model, api_key)
        _generate_live->>HTTPPost: POST {base_url}/chat/completions<br/>Bearer auth, RAG prompt
        HTTPPost->>LLMEndpoint: HTTP request with model/message
        LLMEndpoint-->>HTTPPost: JSON choices[0].message
        HTTPPost-->>_generate_live: Response
        _generate_live->>extract: response payload
        extract-->>_generate_live: text or None
        _generate_live-->>LLMClient: text or None
    else Missing config or HTTP error
        _generate_live-->>LLMClient: None
    end
    alt Live result present
        LLMClient-->>Client: text answer
    else Fallback path
        LLMClient->>LLMClient: generate_legal_answer(state)
        LLMClient-->>Client: fallback answer
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • ssafy-salman/salmanhae#23: Directly modifies the same LLMClient legal-consult path, adding the grounded generate_legal_answer from legal_cards that this PR now wraps as the deterministic fallback.

Poem

🐇 A rabbit once knew only scripts by rote,
But now calls the cloud for a legal quote!
With bearer tokens and RAG in tow,
It POSTs the prompt and watches answers flow.
If the LLM sleeps, the fallback speaks—
Deterministic wisdom for legal geeks! 📜✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title 'fix(ai): 법률 RAG 답변 LLM 호출 연결' clearly summarizes the main change—connecting LLM call integration for legal RAG answer generation.
Description check ✅ Passed The PR description covers the key changes, root cause, and testing approach following the template structure, though the test command formatting is unconventional and includes local paths.
Linked Issues check ✅ Passed Code changes fully implement the three objectives from issue #30: LLM API call integration [backend-ai/app/clients/llm_client.py], deterministic fallback preservation, and testable verification without external calls.
Out of Scope Changes check ✅ Passed All changes are in scope: configuration field addition in settings, import/environment variable setup in conftest, and test additions are all supporting the LLM integration objective.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/f3-live-legal-rag-answer

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
backend-ai/tests/test_legal_answer_generation.py (1)

121-138: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Add a malformed-payload regression test for extractor fallback

Current coverage validates text-part extraction but not malformed payload shapes. Add a non-dict payload case to lock in None behavior and prevent future fallback regressions.

Suggested test addition
 def test_extract_chat_completion_text_supports_text_parts() -> None:
     assert (
         extract_chat_completion_text(
             {
                 "choices": [
                     {
                         "message": {
                             "content": [
                                 {"type": "text", "text": "첫 문장"},
                                 {"type": "output_text", "text": "둘째 문장"},
                             ]
                         }
                     }
                 ]
             }
         )
         == "첫 문장\n둘째 문장"
     )
+
+
+def test_extract_chat_completion_text_returns_none_for_non_object_payload() -> None:
+    assert extract_chat_completion_text([]) is None
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend-ai/tests/test_legal_answer_generation.py` around lines 121 - 138, Add
a regression test for malformed payload handling in the
extract_chat_completion_text function to ensure the fallback behavior is locked
in. Create a new test function (e.g.,
test_extract_chat_completion_text_handles_malformed_payload) that passes various
malformed payload shapes to extract_chat_completion_text (such as non-dict
payloads, missing required fields, or other invalid structures) and asserts that
the function returns None gracefully to prevent future regressions in the
extractor fallback logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend-ai/app/clients/llm_client.py`:
- Around line 92-95: The function `extract_chat_completion_text` calls `.get()`
on the `payload` parameter without first verifying it is a dictionary. Add a
guard clause at the beginning of the function to check if `payload` is an
instance of dict using `isinstance(payload, dict)`, and return None if it is
not, before attempting to call `.get("choices")` on it. This will prevent
`AttributeError` when the provider returns a non-dict JSON payload.

---

Nitpick comments:
In `@backend-ai/tests/test_legal_answer_generation.py`:
- Around line 121-138: Add a regression test for malformed payload handling in
the extract_chat_completion_text function to ensure the fallback behavior is
locked in. Create a new test function (e.g.,
test_extract_chat_completion_text_handles_malformed_payload) that passes various
malformed payload shapes to extract_chat_completion_text (such as non-dict
payloads, missing required fields, or other invalid structures) and asserts that
the function returns None gracefully to prevent future regressions in the
extractor fallback logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 65983b29-7586-4883-9735-c2e9f1c52e11

📥 Commits

Reviewing files that changed from the base of the PR and between a739415 and dc68f97.

📒 Files selected for processing (4)
  • backend-ai/app/clients/llm_client.py
  • backend-ai/app/core/config.py
  • backend-ai/tests/conftest.py
  • backend-ai/tests/test_legal_answer_generation.py

Comment thread backend-ai/app/clients/llm_client.py Outdated
@HOKAGO-MEMORIES
HOKAGO-MEMORIES merged commit b9594f6 into develop Jun 22, 2026
1 check passed
@HOKAGO-MEMORIES
HOKAGO-MEMORIES deleted the fix/f3-live-legal-rag-answer branch June 25, 2026 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG][F-3] 법률 RAG 챗봇 답변 템플릿 fallback 개선

1 participant