Skip to content

[Phase 4] feat(ai): 법률 RAG 근거 답변 생성 - #23

Merged
HOKAGO-MEMORIES merged 1 commit into
developfrom
phase/4-grounded-legal-answer
Jun 22, 2026
Merged

[Phase 4] feat(ai): 법률 RAG 근거 답변 생성#23
HOKAGO-MEMORIES merged 1 commit into
developfrom
phase/4-grounded-legal-answer

Conversation

@HOKAGO-MEMORIES

@HOKAGO-MEMORIES HOKAGO-MEMORIES commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

변경 사항

  • legalCards 기반 grounded legal answer 생성 추가
  • 법률 RAG prompt/context formatting helper 추가
  • 근거가 없을 때 조문 citation을 지어내지 않는 fallback 추가
  • chat integration에서 답변이 legalCards의 법령/조문을 포함하는지 검증
  • Phase 4 하네스 문서 추가

테스트

  • cd backend-ai && .\.venv\Scripts\python.exe -m pytest tests
  • git diff --check

Closes #22

Summary by CodeRabbit

  • New Features

    • Added grounded legal consultation responses that cite specific laws and articles from retrieved legal references.
    • Displays appropriate fallback messaging when no legal references are available to answer queries.
  • Tests

    • Expanded test coverage for legal answer generation, verifying citation accuracy and preventing hallucinated references.
  • Documentation

    • Added phase documentation for the legal RAG pipeline implementation and acceptance criteria.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a8c4da9-e5b5-44a5-8478-45fd562e14cc

📥 Commits

Reviewing files that changed from the base of the PR and between fd871ff and 3f49798.

📒 Files selected for processing (5)
  • backend-ai/app/clients/llm_client.py
  • backend-ai/app/rag/prompts.py
  • backend-ai/tests/test_agent_chat.py
  • backend-ai/tests/test_legal_answer_generation.py
  • phases/ai-legal-rag/phase4-grounded-legal-answer.md

📝 Walkthrough

Walkthrough

The PR implements Phase 4 of the legal RAG pipeline by adding format_legal_context and build_legal_rag_prompt helpers in prompts.py, routing the LEGAL_CONSULT intent in LLMClient.generate_answer to a new generate_legal_answer function that either returns a no-references fallback or assembles a grounded response, and covering all behaviors with new unit tests and an updated chat integration test.

Changes

Grounded Legal Answer Pipeline

Layer / File(s) Summary
Legal context formatting and RAG prompt helpers
backend-ai/app/rag/prompts.py
Adds format_legal_context (numbers and truncates legal cards into a context string with empty fallback) and build_legal_rag_prompt (composes full RAG prompt from system prompt, user question, formatted context, and Korean citation/professional-review instructions).
LLMClient LEGAL_CONSULT routing and generate_legal_answer
backend-ai/app/clients/llm_client.py
Imports format_legal_context, reworks generate_answer branching to delegate Intent.LEGAL_CONSULT to generate_legal_answer, and implements generate_legal_answer which returns a no-references fallback when legal_cards is empty or builds the grounded response using format_legal_context.
Unit/integration tests and phase spec
backend-ai/tests/test_legal_answer_generation.py, backend-ai/tests/test_agent_chat.py, phases/ai-legal-rag/phase4-grounded-legal-answer.md
Adds LEGAL_CARDS fixture and four tests covering context formatting, prompt composition, grounded citation accuracy, and empty-cards no-invention behavior. Updates chat integration test with card_text_in_answer helper asserting lawName/articleNo presence. Adds Phase 4 spec with acceptance criteria and scope constraints.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • ssafy-salman/salmanhae#17: Established the LEGAL_CONSULT contract and legalCards field shape (lawName, articleNo, title, content, score) that generate_legal_answer now consumes via format_legal_context.
  • ssafy-salman/salmanhae#21: Implemented the Phase 3 pgvector retriever that populates state["legal_cards"] with the normalized fields read by this PR's grounded answer generation and tests.

Poem

🐇 Hippity-hop through the law books I went,
With cards full of articles, numbered and bent,
No citations invented when the shelf is bare—
Just "전문가" advice floating softly in air.
Grounded in context, each answer rings true,
Phase Four is delivered—now what shall we do? 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title '[Phase 4] feat(ai): 법률 RAG 근거 답변 생성' clearly describes the main change—implementing grounded legal answer generation for Phase 4 of the legal RAG system—and is directly related to the changeset.
Description check ✅ Passed The PR description covers key changes (grounded answer generation, prompt/context helpers, fallback mechanism, chat integration validation, and Phase 4 documentation), testing instructions, and links to issue #22, though the template suggests listing specific sections more formally.
Linked Issues check ✅ Passed The PR successfully implements all coding objectives from issue #22: legal RAG prompt/context formatting [test_legal_answer_generation.py], legalCards-based answer generation [llm_client.py], citation prevention fallback [generate_legal_answer], and chat integration validation [test_agent_chat.py].
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #22's objectives: prompt/context formatting, answer generation logic, fallback mechanisms, chat integration tests, and Phase 4 documentation. No out-of-scope modifications detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch phase/4-grounded-legal-answer

Comment @coderabbitai help to get the list of available commands and usage tips.

@HOKAGO-MEMORIES
HOKAGO-MEMORIES merged commit 1231680 into develop Jun 22, 2026
1 check passed
@HOKAGO-MEMORIES
HOKAGO-MEMORIES deleted the phase/4-grounded-legal-answer branch June 22, 2026 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Phase 4] grounded legal answer

1 participant