feat: Add Markdown export service for DOCX and PDF formats
- Implemented MarkdownExportService to convert Markdown text into DOCX and PDF formats. - Supported Markdown elements include headings, bold, italic, blockquotes, tables, and specific emojis. - Added methods for generating DOCX and PDF documents with proper formatting and rendering. - Enhanced application properties to enable logging for production environment. - Introduced a test-run script to reset project state and trigger processing via API. Co-authored-by: Copilot <copilot@github.com>
This commit is contained in:
Wolf G. Beckmann
1
Doku/Ablaufplan.svg
Normal file
1
Doku/Ablaufplan.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 225 KiB |
@@ -223,3 +223,172 @@ graph TB
|
||||
|
||||
|
||||
|
||||
## Source
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
|
||||
participant Client as APEX / Client
|
||||
participant AKF as ApiKeyFilter
|
||||
participant CR as CheckResource
|
||||
participant COS as CheckOrchestrationService
|
||||
participant OLF as OrdsLoggingFilter
|
||||
participant OC as OrdsClient
|
||||
participant DPS as DocumentProcessingService
|
||||
participant ES as EvaluationService
|
||||
participant OCR_MDL as OllamaChatModel [ocr]
|
||||
participant MAIN_MDL as OllamaChatModel [main]
|
||||
participant ORDS as ORDS / Oracle DB
|
||||
participant OLLAMA as ollama.aquantico.de
|
||||
|
||||
%% ─── Eingehende Anfrage ──────────────────────────────────────────────────
|
||||
Client->>AKF: POST /check/{projectId} (Header: X-API-KEY)
|
||||
AKF->>AKF: filter(ContainerRequestContext)
|
||||
Note over AKF: Prüft X-API-KEY gegen dc.api.key<br/>→ 401 wenn ungültig, sonst weiter
|
||||
AKF->>CR: weiterleiten (Key gültig)
|
||||
|
||||
CR->>CR: startCheck(projectId)
|
||||
CR-->>Client: 202 Accepted { message, project_id, hint }
|
||||
|
||||
Note over CR,COS: CompletableFuture.runAsync() – Fire & Forget
|
||||
CR-)COS: process(projectId)
|
||||
|
||||
%% ─── Alle ORDS-Aufrufe laufen durch OrdsLoggingFilter ───────────────────
|
||||
Note over OLF,ORDS: OrdsLoggingFilter.filter() loggt jeden Request/Response<br/>des REST-Clients (ClientRequestFilter + ClientResponseFilter)
|
||||
|
||||
%% ─── Initialisierung ────────────────────────────────────────────────────
|
||||
rect rgb(235, 245, 255)
|
||||
Note over COS,ORDS: Initialisierung
|
||||
|
||||
COS->>OC: getProject(projectId)
|
||||
OC->>OLF: filter(ClientRequestContext)
|
||||
OLF->>ORDS: GET /api/dc/projects/{id}
|
||||
ORDS-->>OLF: OrdsProject (JSON)
|
||||
OLF->>OLF: filter(ClientRequestContext, ClientResponseContext)
|
||||
OLF-->>COS: OrdsProject
|
||||
|
||||
Note over COS: Status-Prüfung: nur PENDING wird verarbeitet
|
||||
|
||||
COS->>OC: startProject(projectId)
|
||||
OC->>ORDS: POST /api/dc/projects/{id}/start
|
||||
ORDS-->>COS: 200 OK | 409 Conflict
|
||||
|
||||
COS->>OC: getDocuments(projectId)
|
||||
OC->>ORDS: GET /api/dc/projects/{id}/documents/
|
||||
ORDS-->>COS: OrdsDocumentList
|
||||
|
||||
COS->>OC: getQuestions(catalogId)
|
||||
OC->>ORDS: GET /api/dc/catalogs/{id}/questions/
|
||||
ORDS-->>COS: OrdsQuestionList
|
||||
|
||||
COS->>OC: deleteResults(projectId)
|
||||
OC->>ORDS: DELETE /api/dc/projects/{id}/results
|
||||
Note over ORDS: Alte Ergebnisse löschen → Re-Processing möglich
|
||||
end
|
||||
|
||||
%% ─── Phase A: OCR + Übersetzung ─────────────────────────────────────────
|
||||
rect rgb(220, 255, 225)
|
||||
Note over COS,OLLAMA: Phase A – OCR + Übersetzung (pro Dokument)
|
||||
|
||||
loop für jedes OrdsDocument
|
||||
|
||||
COS->>OC: downloadFile(docId)
|
||||
OC->>ORDS: GET /api/dc/documents/{id}/file
|
||||
ORDS-->>COS: byte[] (BLOB)
|
||||
|
||||
COS->>DPS: extractText(fileBytes, mimeType, filename)
|
||||
DPS->>DPS: resolveMimeType(mimeType, filename)
|
||||
|
||||
alt MIME = application/pdf
|
||||
DPS->>DPS: extractFromPdf(fileBytes)
|
||||
DPS->>DPS: Loader.loadPDF(fileBytes) → PDDocument
|
||||
DPS->>DPS: new PDFRenderer(pdf)
|
||||
|
||||
loop für jede PDF-Seite
|
||||
DPS->>DPS: renderer.renderImageWithDPI(i, 200f, RGB)
|
||||
DPS->>DPS: ImageIO.write(image, "PNG", baos) → Base64
|
||||
DPS->>DPS: ocrPage(base64Image, pageNum, totalPages)
|
||||
DPS->>OCR_MDL: generate(List<ChatMessage>)
|
||||
Note over DPS,OCR_MDL: UserMessage: ImageContent(PNG) + TextContent(OCR-Prompt)
|
||||
OCR_MDL->>OLLAMA: POST /api/chat [Vision, X-API-KEY]
|
||||
OLLAMA-->>OCR_MDL: Markdown-Seitentext
|
||||
OCR_MDL-->>DPS: String (Seite i)
|
||||
end
|
||||
|
||||
else MIME = application/vnd...wordprocessingml / msword
|
||||
DPS->>DPS: extractFromDocx(fileBytes)
|
||||
DPS->>DPS: new XWPFDocument(stream)
|
||||
loop für jedes Body-Element
|
||||
DPS->>DPS: paragraphToMarkdown(XWPFParagraph)
|
||||
DPS->>DPS: tableToMarkdown(XWPFTable)
|
||||
end
|
||||
|
||||
else MIME = application/vnd.oasis.opendocument.text
|
||||
DPS->>DPS: extractFromOdt(fileBytes)
|
||||
DPS->>DPS: OdfTextDocument.loadDocument(stream)
|
||||
DPS->>DPS: getElementsByTagName("text:p") → Markdown
|
||||
|
||||
else text/plain | text/markdown
|
||||
Note over DPS: direkt als UTF-8 String zurück
|
||||
end
|
||||
|
||||
DPS-->>COS: originalText (Markdown)
|
||||
|
||||
COS->>ES: translate(originalText)
|
||||
ES->>MAIN_MDL: generate(List<ChatMessage>)
|
||||
Note over ES,MAIN_MDL: SystemMessage: Übersetzer-Prompt<br/>UserMessage: originalText
|
||||
MAIN_MDL->>OLLAMA: POST /api/chat [X-API-KEY]
|
||||
OLLAMA-->>MAIN_MDL: Übersetzung (Deutsch)
|
||||
MAIN_MDL-->>ES: Response<AiMessage>
|
||||
ES-->>COS: translatedText
|
||||
|
||||
COS->>OC: updateTexts(docId, TextsRequest(original, translated))
|
||||
OC->>ORDS: PUT /api/dc/documents/{id}/texts
|
||||
|
||||
COS->>COS: pushProgress(projectId, step, total)
|
||||
COS->>OC: updateProgress(projectId, ProgressRequest(pct))
|
||||
OC->>ORDS: PUT /api/dc/projects/{id}/progress
|
||||
end
|
||||
end
|
||||
|
||||
%% ─── Phase B: Fragen auswerten ───────────────────────────────────────────
|
||||
rect rgb(255, 245, 220)
|
||||
Note over COS,OLLAMA: Phase B – Fragenauswertung (pro Dokument × Frage)
|
||||
|
||||
loop für jedes OrdsDocument × jede OrdsQuestion
|
||||
|
||||
COS->>ES: evaluate(question, originalText)
|
||||
ES->>ES: buildEvaluationSystem()
|
||||
ES->>ES: buildEvaluationUser(question, documentText)
|
||||
Note over ES: Prompt: Dokument + Frage + Schwellwert<br/>+ Beispiele 0% / 100%
|
||||
ES->>MAIN_MDL: generate(List<ChatMessage>)
|
||||
MAIN_MDL->>OLLAMA: POST /api/chat [X-API-KEY]
|
||||
OLLAMA-->>MAIN_MDL: JSON { answer, score, result_type, the_comment }
|
||||
MAIN_MDL-->>ES: Response<AiMessage>
|
||||
|
||||
ES->>ES: parseEvaluationResult(rawText, question)
|
||||
ES->>ES: extractJson(text)
|
||||
Note over ES: Regex: ```json...``` oder erstes {...} im Text
|
||||
ES->>ES: objectMapper.readValue(json, EvaluationResult)
|
||||
ES->>ES: deriveResultType(score, threshold) [Fallback]
|
||||
ES-->>COS: EvaluationResult { answer, score, resultType, theComment }
|
||||
|
||||
Note over COS: score < threshold?<br/>result_handling=ABWEICHUNG → deviation=1<br/>result_handling=HINWEIS → warning=1
|
||||
|
||||
COS->>OC: saveResult(projectId, ResultRequest)
|
||||
OC->>ORDS: POST /api/dc/projects/{id}/results → 201 Created
|
||||
|
||||
COS->>COS: pushProgress(projectId, step, total)
|
||||
COS->>OC: updateProgress(projectId, ProgressRequest(pct))
|
||||
OC->>ORDS: PUT /api/dc/projects/{id}/progress
|
||||
end
|
||||
end
|
||||
|
||||
%% ─── Abschluss ───────────────────────────────────────────────────────────
|
||||
COS->>OC: completeProject(projectId)
|
||||
OC->>ORDS: POST /api/dc/projects/{id}/complete
|
||||
Note over ORDS: status → COMPLETED, completed_at = SYSDATE
|
||||
|
||||
```
|
||||
|
||||
|
||||
163
Doku/Prozessablauf.mmd
Normal file
163
Doku/Prozessablauf.mmd
Normal file
@@ -0,0 +1,163 @@
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
|
||||
participant Client as APEX / Client
|
||||
participant AKF as ApiKeyFilter
|
||||
participant CR as CheckResource
|
||||
participant COS as CheckOrchestrationService
|
||||
participant OLF as OrdsLoggingFilter
|
||||
participant OC as OrdsClient
|
||||
participant DPS as DocumentProcessingService
|
||||
participant ES as EvaluationService
|
||||
participant OCR_MDL as OllamaChatModel [ocr]
|
||||
participant MAIN_MDL as OllamaChatModel [main]
|
||||
participant ORDS as ORDS / Oracle DB
|
||||
participant OLLAMA as ollama.aquantico.de
|
||||
|
||||
%% ─── Eingehende Anfrage ──────────────────────────────────────────────────
|
||||
Client->>AKF: POST /check/{projectId} (Header: X-API-KEY)
|
||||
AKF->>AKF: filter(ContainerRequestContext)
|
||||
Note over AKF: Prüft X-API-KEY gegen dc.api.key<br/>→ 401 wenn ungültig, sonst weiter
|
||||
AKF->>CR: weiterleiten (Key gültig)
|
||||
|
||||
CR->>CR: startCheck(projectId)
|
||||
CR-->>Client: 202 Accepted { message, project_id, hint }
|
||||
|
||||
Note over CR,COS: CompletableFuture.runAsync() – Fire & Forget
|
||||
CR-)COS: process(projectId)
|
||||
|
||||
%% ─── Alle ORDS-Aufrufe laufen durch OrdsLoggingFilter ───────────────────
|
||||
Note over OLF,ORDS: OrdsLoggingFilter.filter() loggt jeden Request/Response<br/>des REST-Clients (ClientRequestFilter + ClientResponseFilter)
|
||||
|
||||
%% ─── Initialisierung ────────────────────────────────────────────────────
|
||||
rect rgb(235, 245, 255)
|
||||
Note over COS,ORDS: Initialisierung
|
||||
|
||||
COS->>OC: getProject(projectId)
|
||||
OC->>OLF: filter(ClientRequestContext)
|
||||
OLF->>ORDS: GET /api/dc/projects/{id}
|
||||
ORDS-->>OLF: OrdsProject (JSON)
|
||||
OLF->>OLF: filter(ClientRequestContext, ClientResponseContext)
|
||||
OLF-->>COS: OrdsProject
|
||||
|
||||
Note over COS: Status-Prüfung: nur PENDING wird verarbeitet
|
||||
|
||||
COS->>OC: startProject(projectId)
|
||||
OC->>ORDS: POST /api/dc/projects/{id}/start
|
||||
ORDS-->>COS: 200 OK | 409 Conflict
|
||||
|
||||
COS->>OC: getDocuments(projectId)
|
||||
OC->>ORDS: GET /api/dc/projects/{id}/documents/
|
||||
ORDS-->>COS: OrdsDocumentList
|
||||
|
||||
COS->>OC: getQuestions(catalogId)
|
||||
OC->>ORDS: GET /api/dc/catalogs/{id}/questions/
|
||||
ORDS-->>COS: OrdsQuestionList
|
||||
|
||||
COS->>OC: deleteResults(projectId)
|
||||
OC->>ORDS: DELETE /api/dc/projects/{id}/results
|
||||
Note over ORDS: Alte Ergebnisse löschen → Re-Processing möglich
|
||||
end
|
||||
|
||||
%% ─── Phase A: OCR + Übersetzung ─────────────────────────────────────────
|
||||
rect rgb(220, 255, 225)
|
||||
Note over COS,OLLAMA: Phase A – OCR + Übersetzung (pro Dokument)
|
||||
|
||||
loop für jedes OrdsDocument
|
||||
|
||||
COS->>OC: downloadFile(docId)
|
||||
OC->>ORDS: GET /api/dc/documents/{id}/file
|
||||
ORDS-->>COS: byte[] (BLOB)
|
||||
|
||||
COS->>DPS: extractText(fileBytes, mimeType, filename)
|
||||
DPS->>DPS: resolveMimeType(mimeType, filename)
|
||||
|
||||
alt MIME = application/pdf
|
||||
DPS->>DPS: extractFromPdf(fileBytes)
|
||||
DPS->>DPS: Loader.loadPDF(fileBytes) → PDDocument
|
||||
DPS->>DPS: new PDFRenderer(pdf)
|
||||
|
||||
loop für jede PDF-Seite
|
||||
DPS->>DPS: renderer.renderImageWithDPI(i, 200f, RGB)
|
||||
DPS->>DPS: ImageIO.write(image, "PNG", baos) → Base64
|
||||
DPS->>DPS: ocrPage(base64Image, pageNum, totalPages)
|
||||
DPS->>OCR_MDL: generate(List<ChatMessage>)
|
||||
Note over DPS,OCR_MDL: UserMessage: ImageContent(PNG) + TextContent(OCR-Prompt)
|
||||
OCR_MDL->>OLLAMA: POST /api/chat [Vision, X-API-KEY]
|
||||
OLLAMA-->>OCR_MDL: Markdown-Seitentext
|
||||
OCR_MDL-->>DPS: String (Seite i)
|
||||
end
|
||||
|
||||
else MIME = application/vnd...wordprocessingml / msword
|
||||
DPS->>DPS: extractFromDocx(fileBytes)
|
||||
DPS->>DPS: new XWPFDocument(stream)
|
||||
loop für jedes Body-Element
|
||||
DPS->>DPS: paragraphToMarkdown(XWPFParagraph)
|
||||
DPS->>DPS: tableToMarkdown(XWPFTable)
|
||||
end
|
||||
|
||||
else MIME = application/vnd.oasis.opendocument.text
|
||||
DPS->>DPS: extractFromOdt(fileBytes)
|
||||
DPS->>DPS: OdfTextDocument.loadDocument(stream)
|
||||
DPS->>DPS: getElementsByTagName("text:p") → Markdown
|
||||
|
||||
else text/plain | text/markdown
|
||||
Note over DPS: direkt als UTF-8 String zurück
|
||||
end
|
||||
|
||||
DPS-->>COS: originalText (Markdown)
|
||||
|
||||
COS->>ES: translate(originalText)
|
||||
ES->>MAIN_MDL: generate(List<ChatMessage>)
|
||||
Note over ES,MAIN_MDL: SystemMessage: Übersetzer-Prompt<br/>UserMessage: originalText
|
||||
MAIN_MDL->>OLLAMA: POST /api/chat [X-API-KEY]
|
||||
OLLAMA-->>MAIN_MDL: Übersetzung (Deutsch)
|
||||
MAIN_MDL-->>ES: Response<AiMessage>
|
||||
ES-->>COS: translatedText
|
||||
|
||||
COS->>OC: updateTexts(docId, TextsRequest(original, translated))
|
||||
OC->>ORDS: PUT /api/dc/documents/{id}/texts
|
||||
|
||||
COS->>COS: pushProgress(projectId, step, total)
|
||||
COS->>OC: updateProgress(projectId, ProgressRequest(pct))
|
||||
OC->>ORDS: PUT /api/dc/projects/{id}/progress
|
||||
end
|
||||
end
|
||||
|
||||
%% ─── Phase B: Fragen auswerten ───────────────────────────────────────────
|
||||
rect rgb(255, 245, 220)
|
||||
Note over COS,OLLAMA: Phase B – Fragenauswertung (pro Dokument × Frage)
|
||||
|
||||
loop für jedes OrdsDocument × jede OrdsQuestion
|
||||
|
||||
COS->>ES: evaluate(question, originalText)
|
||||
ES->>ES: buildEvaluationSystem()
|
||||
ES->>ES: buildEvaluationUser(question, documentText)
|
||||
Note over ES: Prompt: Dokument + Frage + Schwellwert<br/>+ Beispiele 0% / 100%
|
||||
ES->>MAIN_MDL: generate(List<ChatMessage>)
|
||||
MAIN_MDL->>OLLAMA: POST /api/chat [X-API-KEY]
|
||||
OLLAMA-->>MAIN_MDL: JSON { answer, score, result_type, the_comment }
|
||||
MAIN_MDL-->>ES: Response<AiMessage>
|
||||
|
||||
ES->>ES: parseEvaluationResult(rawText, question)
|
||||
ES->>ES: extractJson(text)
|
||||
Note over ES: Regex: ```json...``` oder erstes {...} im Text
|
||||
ES->>ES: objectMapper.readValue(json, EvaluationResult)
|
||||
ES->>ES: deriveResultType(score, threshold) [Fallback]
|
||||
ES-->>COS: EvaluationResult { answer, score, resultType, theComment }
|
||||
|
||||
Note over COS: score < threshold?<br/>result_handling=ABWEICHUNG → deviation=1<br/>result_handling=HINWEIS → warning=1
|
||||
|
||||
COS->>OC: saveResult(projectId, ResultRequest)
|
||||
OC->>ORDS: POST /api/dc/projects/{id}/results → 201 Created
|
||||
|
||||
COS->>COS: pushProgress(projectId, step, total)
|
||||
COS->>OC: updateProgress(projectId, ProgressRequest(pct))
|
||||
OC->>ORDS: PUT /api/dc/projects/{id}/progress
|
||||
end
|
||||
end
|
||||
|
||||
%% ─── Abschluss ───────────────────────────────────────────────────────────
|
||||
COS->>OC: completeProject(projectId)
|
||||
OC->>ORDS: POST /api/dc/projects/{id}/complete
|
||||
Note over ORDS: status → COMPLETED, completed_at = SYSDATE
|
||||
Reference in New Issue
Block a user