Spaces:
Sleeping
Sleeping
| error_type_pdf_file_path = './Defined_Error_Types.pdf' | |
| error_type_workflow_file_path = './Defined_Error_Types_Along_WorkFlow.pdf' | |
| ET_VECTOR_STORE_ID = "vs_6965956609088191adca0763c0d6f9df" #PDF file for error type definition only | |
| ET_WORKFLOW_VECTOR_STORE_ID = "vs_696595699b4c81918652784b1f2e409f" #PDF file for error type definition along workflow | |
| ET_MD_DEF_VECTOR_STORE_ID = "vs_698bf1e8e93c8191a0da7d212a614425" #MD file for error type definition | |
| ET_MD_DEF_WORKFLOW_VECTOR_STORE_ID = "vs_698bf1ed00648191a836a5c281b4f245" #MD file for error type definition along workflow | |
| basic_prompt_wout_image = """ | |
| You are analyzing a student's number-line fraction homework. | |
| You have access to: | |
| 1) A Knowledge Base that defines the valid Error Types. | |
| 2) Object-detection results in JSON (labels, bboxes, fraction values, and relationship links). | |
| "relationship" links connect one tick index and one fraction/whole number index (0-indexed). | |
| Both formats may appear: "T#-F#" or "T#-Zero#". | |
| Example: "T5-F1" means tick[5] is associated with fraction[1]. | |
| Example: "T5-Zero0" means tick[5] is associated with whole number Zero[0]. | |
| Always consult the Knowledge Base criteria before deciding any error type. | |
| Use the Knowledge Base definitions to determine which error types are present, and use the JSON as evidence. | |
| The detection may contain minor mistakes or missing fraction values. | |
| You MAY make LIMITED, evidence-based inferences to fill missing fraction values ONLY when the inference is strongly supported and unambiguous. | |
| If you infer any missing fraction value, explicitly mark it as INFERRED in your analysis. | |
| Do NOT invent new objects/ticks/fractions not present in the JSON. | |
| If multiple completions are plausible, do NOT infer; state that evidence is insufficient. | |
| """ | |
| advanced_prompt_wout_image = """ | |
| You are analyzing a student's number-line fraction homework. | |
| You have access to: | |
| 1) A Knowledge Base that defines the valid Error Types along with the decision workflow. | |
| 2) Object-detection results in JSON (labels, bboxes, fraction values, and relationship links). | |
| Use the Knowledge Base definitions to determine which error types are present, and use the JSON as evidence. | |
| "relationship" links connect one tick index and one fraction/whole number index (0-indexed). | |
| Both formats may appear: "T#-F#" or "F#-T#" or "T#-Zero#". | |
| Example: "T5-F1" means tick[5] is associated with fraction[1]. | |
| Example: "T5-Zero0" means tick[5] is associated with whole number Zero[0]. | |
| Always consult the Knowledge Base criteria before deciding any error type. | |
| Use the Knowledge Base definitions to determine which error types are present, and use the JSON as evidence. | |
| The detection may contain minor mistakes or missing fraction values. | |
| You MAY make LIMITED, evidence-based inferences to fill missing fraction values ONLY when the inference is strongly supported and unambiguous. | |
| If you infer any missing fraction value, explicitly mark it as INFERRED in your analysis. | |
| Do NOT invent new objects/ticks/fractions not present in the JSON. | |
| If multiple completions are plausible, do NOT infer; state that evidence is insufficient. | |
| """ | |
| basic_prompt = "Please analyze the error type of the student's work based on the definition in the knowledge base." | |
| advanced_prompt = "Please analyze the error type of the student's work based on the definition in the knowledge base and the decision workflow to identify a specific error type." | |
| with_analysis_prompt = """ | |
| The student's work may contain zero, one, or multiple error types. | |
| If the student's work is totally correct, then there is NO error. | |
| Output ONLY a valid JSON object in the following format. | |
| Case 1: One or more errors are present | |
| { | |
| "error_types": [ | |
| { | |
| "type": "Error Type X", | |
| "analysis": "Detailed reason explaining why this error occurs in the student's work" | |
| } | |
| ] | |
| } | |
| Case 2: No error is present | |
| { | |
| "error_types": [] | |
| } | |
| Rules: | |
| - Valid Error Types (use EXACT strings only): | |
| [ | |
| "Error 1: Unequal segmentation", | |
| "Error 2: Wrong number of segments (denominator)", | |
| "Error 3: Wrong number of segments chosen (numerator)", | |
| "Error 4: Incorrect tick labels (confused with whole numbers)", | |
| "Error 5: Incorrect tick labels (careless)", | |
| "Error 6: Incorrect 0 or 1s", | |
| "Error 7: Unit of 1 and subunits" | |
| ] | |
| - "error_types" MUST be a JSON array. | |
| - The array may contain zero, one, or multiple error types. | |
| - Each object MUST contain both "type" and "analysis". | |
| - The "type" field MUST be an EXACT string match from the Valid Error Types list. | |
| - Do NOT include any text outside the JSON. | |
| """ | |
| without_analysis_prompt = """ | |
| The student's work may contain zero, one, or multiple error types. | |
| If the student's work is totally correct, then there is NO error. | |
| Output ONLY a valid JSON object in the following format. | |
| Case 1: One or more errors are present | |
| { | |
| "error_types": ["Error Type X", "Error Type Y"] | |
| } | |
| Case 2: No error is present | |
| { | |
| "error_types": [] | |
| } | |
| Rules: | |
| - Valid Error Types (use EXACT strings only): | |
| [ | |
| "Error 1: Unequal segmentation", | |
| "Error 2: Wrong number of segments (denominator)", | |
| "Error 3: Wrong number of segments chosen (numerator)", | |
| "Error 4: Incorrect tick labels (confused with whole numbers)", | |
| "Error 5: Incorrect tick labels (careless)", | |
| "Error 6: Incorrect 0 or 1s", | |
| "Error 7: Unit of 1 and subunits" | |
| ] | |
| - "error_types" MUST be a JSON array. | |
| - The array may contain zero, one, or multiple error types. | |
| - The "type" field MUST be an EXACT string match from the Valid Error Types list. | |
| - Do NOT include any text outside the JSON. | |
| """ | |
| with_analysis_one_error_prompt = """ | |
| The student's work may contain at most one error type. | |
| If the student's work is totally correct, then there is NO error. | |
| Output ONLY a valid JSON object in the following format. | |
| Case 1: One error is present | |
| { | |
| "error_type": [ | |
| { | |
| "type": "Error Type X", | |
| "analysis": "Detailed reason explaining why this error occurs in the student's work" | |
| } | |
| ] | |
| } | |
| Case 2: No error is present | |
| { | |
| "error_type": [] | |
| } | |
| Rules: | |
| -There is only one valid Error Type (use EXACT strings only): | |
| [ | |
| "Error1: Unequal segmentation", | |
| ] | |
| - "error_type" MUST be a JSON array. | |
| - The array may contain at most one error type. | |
| - Each object MUST contain both "type" and "analysis". | |
| - The "type" field MUST be an EXACT string match from the Valid Error Types list. | |
| - Do NOT include any text outside the JSON. | |
| """ | |
| without_analysis_one_error_prompt = """ | |
| The student's work may contain at most one error type. | |
| If the student's work is totally correct, then there is NO error. | |
| Output ONLY a valid JSON object in the following format. | |
| Case 1: One error is present | |
| { | |
| "error_type": "Error1: Unequal segmentation" | |
| } | |
| Case 2: No error is present | |
| { | |
| "error_type": [] | |
| } | |
| Rules: | |
| -There is only one valid Error Type (use EXACT strings only): | |
| [ | |
| "Error1: Unequal segmentation", | |
| ] | |
| - "error_type" MUST be a JSON array. | |
| - The array may contain at most one error type. | |
| - The "type" field MUST be an EXACT string match from the Valid Error Types list. | |
| - Do NOT include any text outside the JSON. | |
| """ | |
| gemini_without_analysis_one_error_prompt = """ | |
| Your ONLY task is to check for "Error1: Unequal segmentation" based on the definitions in the ATTACHED | |
| Knowledge Base file Ignore all other potential error types. | |
| CONSTRAINTS (Strict Adherence Required): | |
| 1. NO ANALYSIS: Do not provide any reasoning, thoughts, or step-by-step calculations. | |
| 2. ONLY JSON: Your entire response must be a single JSON object. No preamble, no postscript. | |
| 3. SCOPE LENS: If the data shows any error OTHER than "Error1: Unequal segmentation", you must treat it as NO ERROR for this specific task. | |
| 4. KNOWLEDGE BASE OVERRIDE: Use the attached Knowledge Base ONLY to understand the definition of Error 1. Disregard Error 2 through Error 7. | |
| OUTPUT FORMAT: | |
| - If Error 1 is detected: {{"error_type": ["Error1: Unequal segmentation"]}} | |
| - If Error 1 is NOT detected (even if other errors exist): {{"error_type": []}} | |
| """ | |
| gemini_with_analysis_one_error_prompt = """ | |
| Your ONLY task is to check for "Error1: Unequal segmentation" based on the definitions in the ATTACHED | |
| Knowledge Base file Ignore all other potential error types. | |
| CONSTRAINTS (Strict Adherence Required): | |
| 1. CHAIN OF THOUGHT: You must ALWAYS provide a step-by-step reasoning process in the "analysis" field FIRST, before deciding on the error type. | |
| 2. ONLY JSON: Your entire response must be a single JSON object. Do not output markdown code blocks (```json) or any text outside the JSON. | |
| 3. SCOPE LENS: If the data shows any error OTHER than "Error1: Unequal segmentation", you must treat it as NO ERROR for this specific task. | |
| 4. EXACT MATCH: The "type" field must ONLY contain the exact string "Error1: Unequal segmentation". | |
| OUTPUT FORMAT (Must exactly match one of these structures): | |
| If Error 1 is present: | |
| {{ | |
| "analysis": "Step 1: Check the segments. Step 2: The ratio of max segment to min segment is X. Step 3: According to the KB, this qualifies as Error 1.", | |
| "error_type": [ | |
| {{ | |
| "type": "Error1: Unequal segmentation" | |
| }} | |
| ] | |
| }} | |
| If Error 1 is NOT present (or if other out-of-scope errors are present): | |
| {{ | |
| "analysis": "Step 1: Check the segments. Step 2: The ratio is Y, which does not meet the threshold for Error 1. Step 3: The labels might be reversed, but that is out of scope.", | |
| "error_type": [] | |
| }} | |
| """ | |
| gemini_with_analysis_multiple_error_prompt = """ | |
| SYSTEM ROLE: | |
| You are a deterministic grading validator. Your ONLY task is to classify the student's work based strictly on the definitions in the ATTACHED Knowledge Base file. Ignore all other potential error types not listed below. | |
| CONSTRAINTS (Strict Adherence Required): | |
| 1. CHAIN OF THOUGHT: You must ALWAYS provide your reasoning in the "analysis" field FIRST, before concluding the error type. | |
| 2. ONLY JSON: Your entire response must be a single valid JSON object. Do not output markdown code blocks (like ```json) or any preamble/postscript. | |
| 3. AT MOST ONE ERROR: The student's work contains AT MOST ONE error type. If multiple errors seem to exist, identify the most fundamental one based on the KB. | |
| 4. KNOWLEDGE BASE GROUNDING: Use the attached Knowledge Base ONLY to understand the definitions of the errors. | |
| OUTPUT FORMAT EXPECTATION: | |
| { | |
| "analysis": "Your step-by-step reasoning based on the evidence.", | |
| "error_type": ["<Exact String Match>"] | |
| } | |
| RULES FOR JSON: | |
| - "error_type" MUST be a JSON array. | |
| - The array MUST contain AT MOST ONE error type. | |
| - If the work is totally correct (NO error is present), output an empty array: [] | |
| - If an error is present, the string inside "error_type" MUST exactly match ONE of the following Valid Error Types: | |
| [ | |
| "Error 1: Unequal segmentation", | |
| "Error 2: Wrong number of segments (denominator)", | |
| "Error 3: Wrong number of segments chosen (numerator)", | |
| "Error 4: Incorrect tick labels (confused with whole numbers)", | |
| "Error 5: Incorrect tick labels (careless)", | |
| "Error 6: Incorrect 0 or 1s", | |
| "Error 7: Unit of 1 and subunits" | |
| ] | |
| """ | |
| gemini_without_analysis_multiple_error_prompt = """ | |
| SYSTEM ROLE: | |
| You are a deterministic grading validator. Your ONLY task is to classify the student's work based strictly on the definitions in the ATTACHED Knowledge Base file. Ignore all other potential error types not listed below. | |
| CONSTRAINTS (Strict Adherence Required): | |
| 1. NO ANALYSIS: Do not provide any reasoning, thoughts, or step-by-step calculations. | |
| 2. ONLY JSON: Your entire response must be a single valid JSON object. Do not output markdown code blocks (like ```json) or any preamble/postscript. | |
| 3. AT MOST ONE ERROR: The student's work contains AT MOST ONE error type. If multiple errors seem to exist, identify the most fundamental one based on the KB. | |
| 4. KNOWLEDGE BASE GROUNDING: Use the attached Knowledge Base ONLY to understand the definitions of the errors. | |
| OUTPUT FORMAT EXPECTATION: | |
| { | |
| "error_type": ["<Exact String Match>"] | |
| } | |
| RULES FOR JSON: | |
| - "error_type" MUST be a JSON array. | |
| - The array MUST contain AT MOST ONE error type. | |
| - If the work is totally correct (NO error is present), output an empty array: [] | |
| - If an error is present, the string inside "error_type" MUST exactly match ONE of the following Valid Error Types: | |
| [ | |
| "Error 1: Unequal segmentation", | |
| "Error 2: Wrong number of segments (denominator)", | |
| "Error 3: Wrong number of segments chosen (numerator)", | |
| "Error 4: Incorrect tick labels (confused with whole numbers)", | |
| "Error 5: Incorrect tick labels (careless)", | |
| "Error 6: Incorrect 0 or 1s", | |
| "Error 7: Unit of 1 and subunits" | |
| ] | |
| """ | |
| def get_gemini_one_error_prompt_with_analysis(target_error: str) -> str: | |
| """ | |
| 传入你要测试的特定错误类型，例如："Error 1: Unequal segmentation" | |
| """ | |
| return f""" | |
| SYSTEM ROLE: | |
| You are a deterministic grading validator. Your ONLY task is to check for "{target_error}" based on its definition in the ATTACHED Knowledge Base file. Ignore all other potential error types. | |
| CONSTRAINTS (Strict Adherence Required): | |
| 1. CHAIN OF THOUGHT: You must ALWAYS provide a step-by-step reasoning process in the "analysis" field FIRST, before deciding on the error type. | |
| 2. ONLY JSON: Your entire response must be a single valid JSON object. Do not output markdown code blocks (like ```json) or any text outside the JSON. | |
| 3. SCOPE LENS: If the data shows any error OTHER than "{target_error}", you must treat it as NO ERROR for this specific task. | |
| 4. EXACT MATCH: If the error is present, the "error_type" array must ONLY contain the exact string "{target_error}". | |
| OUTPUT FORMAT EXPECTATION (Must exactly match one of these structures): | |
| If {target_error} is PRESENT: | |
| {{ | |
| "analysis": "Step 1: Examine the relevant visual evidence in the student's work. Step 2: Compare the evidence against the specific criteria for {target_error} defined in the Knowledge Base. Step 3: The criteria are met, confirming the error.", | |
| "error_type": [ | |
| "{target_error}" | |
| ] | |
| }} | |
| If {target_error} is NOT PRESENT (or if only out-of-scope errors are present): | |
| {{ | |
| "analysis": "Step 1: Examine the relevant visual evidence in the student's work. Step 2: Compare the evidence against the specific criteria for {target_error} defined in the Knowledge Base. Step 3: The criteria are NOT met. Any other anomalies are out of scope.", | |
| "error_type": [] | |
| }} | |
| """ | |
| def get_gemini_one_error_prompt_without_analysis(target_error: str) -> str: | |
| """ | |
| 传入你要测试的特定错误类型，例如："Error 1: Unequal segmentation" | |
| """ | |
| return f""" | |
| SYSTEM ROLE: | |
| You are a deterministic grading validator. Your ONLY task is to check for "{target_error}" based on its definition in the ATTACHED Knowledge Base file. Ignore all other potential error types. | |
| CONSTRAINTS (Strict Adherence Required): | |
| 1. NO ANALYSIS: Do not provide any reasoning, thoughts, or step-by-step calculations. | |
| 2. ONLY JSON: Your entire response must be a single valid JSON object. Do not output markdown code blocks (like ```json) or any preamble/postscript. | |
| 3. SCOPE LENS: If the data shows any error OTHER than "{target_error}", you must treat it as NO ERROR for this specific task. | |
| 4. EXACT MATCH: If the error is present, the array must ONLY contain the exact string "{target_error}". | |
| OUTPUT FORMAT EXPECTATION: | |
| If {target_error} is PRESENT: | |
| {{ | |
| "error_type": [ | |
| "{target_error}" | |
| ] | |
| }} | |
| If {target_error} is NOT PRESENT (or if only out-of-scope errors are present): | |
| {{ | |
| "error_type": [] | |
| }} | |
| """ | |
| ERROR_MAPS = {} | |
| ERROR_MAPS["Uneven segment only"] = "Error 1: Unequal segmentation" | |
| ERROR_MAPS["Wrong number of segments (denom"] = "Error 2: Wrong number of segments (denominator)" | |
| ERROR_MAPS["Wrong number of segments chosen"] = "Error 3: Wrong number of segments chosen (numerator)" | |
| ERROR_MAPS["Unit of 1 and subunits only"] = "Error 7: Unit of 1 and subunits" | |
| ERROR_MAPS[" Wrong tick labels (confuse who"] = "Error 4: Incorrect tick labels (confused with whole numbers)" | |
| ERROR_MAPS["Incorrect 0s or 1s"] = "Error 6: Incorrect 0 or 1s" | |
| ERROR_MAPS["Careless lable only"] = "Error 5: Incorrect tick labels (careless)" | |