diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..30c7082 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,88 @@ +# AGENTS.md + +This file provides guidance to AI agents when working with code in this repository. + +## Project Overview + +This is a chat function connecting students to an AI educational chatbot that is integrated with the **Lambda-Feedback** educational platform. It deploys as an AWS Lambda function (containerized via Docker) that receives student chat messages with educational context and returns LLM-powered chatbot responses. + +## Commands + +**Testing:** +```bash +pytest # Run all unit tests +python tests/manual_agent_run.py # Test agent locally with example inputs +python tests/manual_agent_requests.py # Test running Docker container +``` + +**Docker:** +```bash +docker build -t llm_chat . +docker run --env-file .env -p 8080:8080 llm_chat +``` + +**Manual API test (while Docker is running):** +```bash +curl -X POST http://localhost:8080/2015-03-31/functions/function/invocations \ + -H 'Content-Type: application/json' \ + -d '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}' +``` + +**Run a single test:** +```bash +pytest tests/test_module.py # Run specific test file +pytest tests/test_index.py::test_function_name # Run specific test +``` + +## Architecture + +### Request Flow + +``` +Lambda event → index.py (handler) + → validates via lf_toolkit ChatRequest schema + → src/module.py (chat_module) + → extracts muEd API context (messages, conversationId, question context, user type) + → parses educational context to prompt text via src/agent/context.py + → src/agent/agent.py (BaseAgent / LangGraph) + → routes to call_llm or summarize_conversation node + → calls LLM provider (OpenAI / Google / Azure / Ollama) + → returns ChatResponse (output, summary, conversationalStyle, processingTime) +``` + +### Key Files + +| File | Role | +|------|------| +| `index.py` | AWS Lambda entry point; parses event body, validates schema | +| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse | +| `src/agent/agent.py` | LangGraph stateful graph; manages message history and summarization | +| `src/agent/prompts.py` | System prompts for tutor behavior, summarization, style detection | +| `src/agent/llm_factory.py` | Factory classes for each LLM provider (OpenAI, Google, Azure, Ollama) | +| `src/agent/context.py` | Converts muEd question/submission context dicts to LLM prompt text | +| `tests/utils.py` | Shared test helpers: `assert_valid_chat_request`, `assert_valid_chat_response` | +| `tests/example_inputs/` | Real muEd payloads used for end-to-end tests | + +### Agent Logic (LangGraph) + +`BaseAgent` maintains a state graph with two nodes: +- **`call_llm`**: Invokes the LLM with system prompt + conversation summary + conversational style preference +- **`summarize_conversation`**: Triggered when message count exceeds ~11; summarizes history and also extracts the student's preferred conversational style + +Messages are trimmed after summarization to keep context window manageable. The `summary` and `conversationalStyle` fields persist across calls via the `ChatRequest` metadata. + +### muEd API Format + +`src/module.py` handles the muEd request format (https://mued.org/). The `context` field in `ChatRequest` contains nested educational data (question parts, student submissions, task info) that gets parsed into a tutoring prompt via `src/agent/context.py`. + +### LLM Configuration + +LLM provider and model are set via environment variables (see `.env.example`). The `llm_factory.py` selects the provider at runtime. The Lambda function name/identity is set in `config.json`. + +The agent uses **two separate LLM instances** — `self.llm` for chat responses and `self.summarisation_llm` for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper model for summarisation) by changing the class in `agent.py`. + +## Deployment + +- Pushing to `dev` branch triggers the dev deployment GitHub Actions workflow +- Pushing to `main` triggers staging deployment, with manual approval required for production +- All environment variables (API keys, model names) are injected via GitHub Actions secrets/variables — do not hardcode them diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..b2f1d63 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,88 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +This is a chat function connecting students to an AI educational chatbot that is integrated with the **Lambda-Feedback** educational platform. It deploys as an AWS Lambda function (containerized via Docker) that receives student chat messages with educational context and returns LLM-powered chatbot responses. + +## Commands + +**Testing:** +```bash +pytest # Run all unit tests +python tests/manual_agent_run.py # Test agent locally with example inputs +python tests/manual_agent_requests.py # Test running Docker container +``` + +**Docker:** +```bash +docker build -t llm_chat . +docker run --env-file .env -p 8080:8080 llm_chat +``` + +**Manual API test (while Docker is running):** +```bash +curl -X POST http://localhost:8080/2015-03-31/functions/function/invocations \ + -H 'Content-Type: application/json' \ + -d '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}' +``` + +**Run a single test:** +```bash +pytest tests/test_module.py # Run specific test file +pytest tests/test_index.py::test_function_name # Run specific test +``` + +## Architecture + +### Request Flow + +``` +Lambda event → index.py (handler) + → validates via lf_toolkit ChatRequest schema + → src/module.py (chat_module) + → extracts muEd API context (messages, conversationId, question context, user type) + → parses educational context to prompt text via src/agent/context.py + → src/agent/agent.py (BaseAgent / LangGraph) + → routes to call_llm or summarize_conversation node + → calls LLM provider (OpenAI / Google / Azure / Ollama) + → returns ChatResponse (output, summary, conversationalStyle, processingTime) +``` + +### Key Files + +| File | Role | +|------|------| +| `index.py` | AWS Lambda entry point; parses event body, validates schema | +| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse | +| `src/agent/agent.py` | LangGraph stateful graph; manages message history and summarization | +| `src/agent/prompts.py` | System prompts for tutor behavior, summarization, style detection | +| `src/agent/llm_factory.py` | Factory classes for each LLM provider (OpenAI, Google, Azure, Ollama) | +| `src/agent/context.py` | Converts muEd question/submission context dicts to LLM prompt text | +| `tests/utils.py` | Shared test helpers: `assert_valid_chat_request`, `assert_valid_chat_response` | +| `tests/example_inputs/` | Real muEd payloads used for end-to-end tests | + +### Agent Logic (LangGraph) + +`BaseAgent` maintains a state graph with two nodes: +- **`call_llm`**: Invokes the LLM with system prompt + conversation summary + conversational style preference +- **`summarize_conversation`**: Triggered when message count exceeds ~11; summarizes history and also extracts the student's preferred conversational style + +Messages are trimmed after summarization to keep context window manageable. The `summary` and `conversationalStyle` fields persist across calls via the `ChatRequest` metadata. + +### muEd API Format + +`src/module.py` handles the muEd request format (https://mued.org/). The `context` field in `ChatRequest` contains nested educational data (question parts, student submissions, task info) and the `user` field contains user-specific information (e.g., user type, preferences, task progress) that gets parsed into a tutoring prompt via `src/agent/context.py`. + +### LLM Configuration + +LLM provider and model are set via environment variables (see `.env.example`). The `llm_factory.py` selects the provider at runtime. The Lambda function name/identity is set in `config.json`. + +The agent uses **two separate LLM instances** — `self.llm` for chat responses and `self.summarisation_llm` for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper model for summarisation) by changing the class in `agent.py`. + +## Deployment + +- Pushing to `dev` branch triggers the dev deployment GitHub Actions workflow +- Pushing to `main` triggers staging deployment, with manual approval required for production +- All environment variables (API keys, model names) are injected via GitHub Actions secrets/variables — do not hardcode them diff --git a/README.md b/README.md index 1bf3503..fb03792 100755 --- a/README.md +++ b/README.md @@ -81,7 +81,9 @@ Also, don't forget to update or delete the Quickstart chapter from the `README.m You can create your own invocation to your own agents hosted anywhere. Copy or update the `agent.py` from `src/agent/` and edit it to match your LLM agent requirements. Import the new invocation in the `module.py` file. -You agent can be based on an LLM hosted anywhere, you have available currently OpenAI, AzureOpenAI, and Ollama models but you can introduce your own API call in the `src/agent/utils/llm_factory.py`. +Your agent can be based on an LLM hosted anywhere. OpenAI, Google AI, Azure OpenAI, and Ollama are available out of the box via `src/agent/llm_factory.py`, and you can add your own provider there too. + +The agent uses **two separate LLM instances** — `self.llm` for chat responses and `self.summarisation_llm` for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper or faster model for summarisation) by changing the class in `agent.py`. ### Prerequisites @@ -99,13 +101,17 @@ You agent can be based on an LLM hosted anywhere, you have available currently O ├── docs/ # docs for devs and users ├── src/ │ ├── agent/ -│ │ ├── utils/ # utils for the agent, including the llm_factory -│ │ ├── agent.py # the agent logic -│ │ └── prompts.py # the system prompts defining the behaviour of the chatbot -│ └── module.py +│ │ ├── agent.py # LangGraph stateful agent logic +│ │ ├── context.py # converts muEd context dicts to LLM prompt text +│ │ ├── llm_factory.py # factory classes for each LLM provider +│ │ └── prompts.py # system prompts defining the behaviour of the chatbot +│ └── module.py └── tests/ # contains all tests for the chat function + ├── example_inputs/ # muEd example payloads for end-to-end tests ├── manual_agent_requests.py # allows testing of the docker container through API requests ├── manual_agent_run.py # allows testing of any LLM agent on a couple of example inputs + ├── utils.py # shared test helpers + ├── test_example_inputs.py # pytests for the example input files ├── test_index.py # pytests └── test_module.py # pytests ``` @@ -165,7 +171,7 @@ This will start the chat function and expose it on port `8080` and it will be op ```bash curl --location 'http://localhost:8080/2015-03-31/functions/function/invocations' \ --header 'Content-Type: application/json' \ ---data '{"body":"{\"message\": \"hi\", \"params\": {\"conversation_id\": \"12345Test\", \"conversation_history\": [{\"type\": \"user\", +--data '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}' ``` #### Call Docker Container @@ -184,21 +190,98 @@ http://localhost:8080/2015-03-31/functions/function/invocations Body (stringified within body for API request): ```JSON -{"body":"{\"message\": \"hi\", \"params\": {\"conversation_id\": \"12345Test\", \"conversation_history\": [{\"type\": \"user\", \"content\": \"hi\"}]}}"} +{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"} ``` -Body with optional Params: -```JSON +Body with optional fields: +```json { - "message":"hi", - "params":{ - "conversation_id":"12345Test", - "conversation_history":[{"type":"user","content":"hi"}], - "summary":" ", - "conversational_style":" ", - "question_response_details": "", - "include_test_data": true, + "conversationId": "", + "messages": [ + { "role": "USER", "content": "" }, + { "role": "ASSISTANT", "content": "" }, + { "role": "USER", "content": "" } + ], + "user": { + "type": "LEARNER", + "preference": { + "conversationalStyle": "" + }, + "taskProgress": { + "timeSpentOnQuestion": "30 minutes", + "accessStatus": "a good amount of time spent on this question today.", + "markedDone": "This question is still being worked on.", + "currentPart": { + "position": 0, + "timeSpentOnPart": "10 minutes", + "markedDone": "This part is not marked done.", + "responseAreas": [ + { + "responseType": "EXPRESSION", + "totalSubmissions": 3, + "wrongSubmissions": 2, + "latestSubmission": { + "submission": "", + "feedback": "", + "answer": "" + } + } + ] + } } + }, + "context": { + "summary": "", + "set": { + "title": "Fundamentals", + "number": 2, + "description": "" + }, + "question": { + "title": "Understanding Polymorphism", + "number": 3, + "guidance": "", + "content": "", + "estimatedTime": "15-25 minutes", + "parts": [ + { + "position": 0, + "content": "", + "answerContent": "", + "workedSolutionSections": [ + { "position": 0, "title": "Step 1", "content": "..." } + ], + "structuredTutorialSections": [ + { "position": 0, "title": "Hint", "content": "..." } + ], + "responseAreas": [ + { + "position": 0, + "responseType": "EXPRESSION", + "answer": "", + "preResponseText": "