po/Dokumenten-Check
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Externe Dokument-Referenzen erkennen, herunterladen & mitbewerten

Browse Source 
Nach der Transkription eines Dokuments werden referenzierte externe
Dokumente (AGB, Verhaltenskodex) erkannt, sicher heruntergeladen und ihr
Text dem Hauptdokument fuer die "Summen"-Bewertung angehaengt (Option D).
Die Referenzen werden zusaetzlich als eigene, herunterladbare Zeilen
gespeichert.

- DB: dc_project_documents um parent_document_id, is_reference, source_url
  erweitert (DC_REFERENCES_ALTER.sql)
- ORDS: Dokumentenliste um die Felder erweitert; neue Endpunkte
  POST /documents/:id/references und DELETE /projects/:id/references;
  q-Quote-Parserfehler im ai-costs-Handler ('[]') behoben
- Backend: ReferenceDownloadService (SSRF-sicher via DNS-/Public-IP-
  Validierung, Redirect-Pruefung, Groessen-/Timeout-Limits, jsoup fuer HTML),
  EvaluationService.extractReferenceUrls (LLM-basierte URL-Erkennung),
  Orchestrierung (Discovery in Phase A, Referenz-Skip + combinedText in Phase B)
- Config: dc.references.* in application.properties

Verifiziert an Projekt 144: AGB-PDF/-HTML heruntergeladen, verknuepft,
"Summen"-Bewertung zitiert AGB-Klauseln aus dem Referenzdokument.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Philipp Ostmeyer
2026-06-22 12:45:55 +02:00
parent d00525c15f
commit c75f460ec5
91 changed files with 3949 additions and 255 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
dc-backend/Dockerfile
View File

@@ -1,7 +1,7 @@
# =============================================================================
# Stage 1: Build
# =============================================================================
FROM maven:3.9-eclipse-temurin-21 AS build
FROM docker.io/library/maven:3.9-eclipse-temurin-21 AS build
WORKDIR /build
@@ -15,7 +15,7 @@ RUN mvn package -DskipTests -q
# =============================================================================
# Stage 2: Runtime
# =============================================================================
FROM eclipse-temurin:21-jre
FROM docker.io/library/eclipse-temurin:21-jre
WORKDIR /app

0
dc-backend/build-and-push.sh Normal file → Executable file
View File

20
dc-backend/follow_logs.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(pwd)"
export KUBECONFIG="$SCRIPT_DIR/../env/frigo-dev.yaml"
POD=$(kubectl get pod -n ai-env -l app=dc-backend \
--field-selector=status.phase=Running \
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)
if [ -z "$POD" ]; then
echo "Kein laufender dc-backend Pod gefunden. Alle Pods:"
kubectl get pods -n ai-env -l app=dc-backend
exit 1
fi
echo "=== Logs: $POD ==="
kubectl logs "$POD" -n ai-env --tail $1
while true; do kubectl logs -f "$POD" -n ai-env --tail 0 && break || sleep 2; done
kubectl logs -f "$POD" -n ai-env --tail $1

0
dc-backend/getPods.sh Normal file → Executable file
View File

4
dc-backend/k8s/deployment.yaml
View File

@@ -66,10 +66,10 @@ spec:
periodSeconds: 10
resources:
requests:
memory: "256Mi"
memory: "512Mi"
cpu: "100m"
limits:
memory: "512Mi"
memory: "2Gi"
cpu: "500m"
---
apiVersion: v1

0
dc-backend/logs.sh Normal file → Executable file
View File

174
dc-backend/pod-ram-usage.sh Executable file
View File

@@ -0,0 +1,174 @@
#!/usr/bin/env bash
# pod-ram-usage.sh — show RAM usage by pod, with optional namespace / sorting flags
#
# Usage:
# ./pod-ram-usage.sh # all namespaces
# ./pod-ram-usage.sh -n my-namespace # specific namespace
# ./pod-ram-usage.sh -s # sort by usage (highest first)
# ./pod-ram-usage.sh -n kube-system -s # combine flags
# ./pod-ram-usage.sh -w 5 # watch mode, refresh every 5 seconds
# ./pod-ram-usage.sh -h # show help
set -euo pipefail
# ── defaults ────────────────────────────────────────────────────────────────
NAMESPACE=""
SORT=false
WATCH_INTERVAL=0 # 0 = run once
# ── helpers ─────────────────────────────────────────────────────────────────
usage() {
grep '^#' "$0" | grep -v '#!/' | sed 's/^# \{0,2\}//'
exit 0
}
die() { echo "ERROR: $*" >&2; exit 1; }
check_deps() {
command -v kubectl >/dev/null 2>&1 || die "kubectl not found in PATH"
# metrics-server check (soft): warn rather than abort
if ! kubectl top pod --all-namespaces >/dev/null 2>&1; then
echo "WARNING: 'kubectl top' failed — is metrics-server installed and ready?" >&2
echo " Install it with:" >&2
echo " kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml" >&2
exit 1
fi
}
# Convert Mi/Ki/Gi strings to MiB float for sorting
to_mib() {
local val="$1"
if [[ "$val" =~ ^([0-9]+(\.[0-9]+)?)Gi$ ]]; then echo "${BASH_REMATCH[1]} * 1024" | bc
elif [[ "$val" =~ ^([0-9]+(\.[0-9]+)?)Mi$ ]]; then echo "${BASH_REMATCH[1]}"
elif [[ "$val" =~ ^([0-9]+(\.[0-9]+)?)Ki$ ]]; then echo "scale=2; ${BASH_REMATCH[1]} / 1024" | bc
elif [[ "$val" =~ ^([0-9]+(\.[0-9]+)?)m$ ]]; then echo "0" # millicores — shouldn't appear for RAM
else echo "0"
fi
}
# ── parse args ───────────────────────────────────────────────────────────────
while getopts ":n:sw:h" opt; do
case $opt in
n) NAMESPACE="$OPTARG" ;;
s) SORT=true ;;
w) WATCH_INTERVAL="$OPTARG" ;;
h) usage ;;
:) die "Option -$OPTARG requires an argument." ;;
\?) die "Unknown option -$OPTARG" ;;
esac
done
# ── main logic ───────────────────────────────────────────────────────────────
run_once() {
# build kubectl top args
local top_args=("top" "pod")
if [[ -n "$NAMESPACE" ]]; then
top_args+=("-n" "$NAMESPACE")
else
top_args+=("--all-namespaces")
fi
top_args+=("--no-headers")
# fetch raw data
local raw
raw=$(kubectl "${top_args[@]}" 2>/dev/null) || {
echo "ERROR: kubectl top pod failed." >&2; return 1
}
# also get resource requests/limits for context
local req_args=("get" "pods")
if [[ -n "$NAMESPACE" ]]; then
req_args+=("-n" "$NAMESPACE")
else
req_args+=("--all-namespaces")
fi
req_args+=("-o" "custom-columns=\
NS:.metadata.namespace,\
NAME:.metadata.name,\
MEM_REQ:.spec.containers[*].resources.requests.memory,\
MEM_LIM:.spec.containers[*].resources.limits.memory" \
"--no-headers")
local limits
limits=$(kubectl "${req_args[@]}" 2>/dev/null) || limits=""
# ── header ──────────────────────────────────────────────────────────────
local ns_label="ALL NAMESPACES"
[[ -n "$NAMESPACE" ]] && ns_label="$NAMESPACE"
echo ""
echo "═══════════════════════════════════════════════════════════════════════"
printf " RAM Usage by Pod — namespace: %-35s\n" "$ns_label"
printf " %s\n" "$(date '+%Y-%m-%d %H:%M:%S')"
echo "═══════════════════════════════════════════════════════════════════════"
printf "%-20s %-42s %10s %10s %10s\n" \
"NAMESPACE" "POD" "MEMORY" "REQUEST" "LIMIT"
echo "───────────────────────────────────────────────────────────────────────"
# ── build output rows ────────────────────────────────────────────────────
declare -a rows
while IFS= read -r line; do
[[ -z "$line" ]] && continue
# kubectl top --all-namespaces: NAMESPACE POD CPU MEM
# kubectl top -n NS: POD CPU MEM
if [[ -n "$NAMESPACE" ]]; then
local pod cpu mem
read -r pod cpu mem <<< "$line"
local ns="$NAMESPACE"
else
local ns pod cpu mem
read -r ns pod cpu mem <<< "$line"
fi
# look up request / limit from the limits table
local req lim
req=$(echo "$limits" | awk -v n="$ns" -v p="$pod" '$1==n && $2==p {print $3}' | head -1)
lim=$(echo "$limits" | awk -v n="$ns" -v p="$pod" '$1==n && $2==p {print $4}' | head -1)
[[ -z "$req" || "$req" == "<none>" ]] && req="—"
[[ -z "$lim" || "$lim" == "<none>" ]] && lim="—"
# for sort key convert mem to MiB numeric
local sort_key
sort_key=$(to_mib "$mem")
rows+=("$(printf '%020.4f\t%-20s\t%-42s\t%10s\t%10s\t%10s' \
"$sort_key" "$ns" "$pod" "$mem" "$req" "$lim")")
done <<< "$raw"
# ── sort if requested ────────────────────────────────────────────────────
if $SORT; then
IFS=$'\n' sorted=($(printf '%s\n' "${rows[@]}" | sort -t$'\t' -k1 -rn))
else
IFS=$'\n' sorted=("${rows[@]}")
fi
# ── print rows ───────────────────────────────────────────────────────────
local total_mib=0
local count=0
for row in "${sorted[@]}"; do
local sort_key ns pod mem req lim
IFS=$'\t' read -r sort_key ns pod mem req lim <<< "$row"
printf "%-20s %-42s %10s %10s %10s\n" "$ns" "$pod" "$mem" "$req" "$lim"
total_mib=$(echo "$total_mib + $sort_key" | bc)
(( count++ )) || true
done
echo "───────────────────────────────────────────────────────────────────────"
printf " %d pod(s) Total usage: %.0f MiB\n" "$count" "$total_mib"
echo "═══════════════════════════════════════════════════════════════════════"
echo ""
}
# ── entry point ──────────────────────────────────────────────────────────────
check_deps
if [[ "$WATCH_INTERVAL" -gt 0 ]]; then
echo "Watch mode — refreshing every ${WATCH_INTERVAL}s (Ctrl-C to stop)"
while true; do
clear
run_once
sleep "$WATCH_INTERVAL"
done
else
run_once
fi

14
dc-backend/pom.xml
View File

@@ -90,6 +90,13 @@
<version>3.0.3</version>
</dependency>
<!-- Markdown → PDF Export -->
<dependency>
<groupId>com.github.librepdf</groupId>
<artifactId>openpdf</artifactId>
<version>1.3.43</version>
</dependency>
<!-- DOCX → Markdown -->
<dependency>
<groupId>org.apache.poi</groupId>
@@ -104,6 +111,13 @@
<version>0.12.0</version>
</dependency>
<!-- HTML → Text (heruntergeladene Referenz-Webseiten) -->
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.18.1</version>
</dependency>
<!-- ================================================================ -->
<!-- Tests -->
<!-- ================================================================ -->

19
dc-backend/src/main/java/de/frigosped/dc/client/MistralOcrClient.java Normal file
View File

@@ -0,0 +1,19 @@
package de.frigosped.dc.client;
import de.frigosped.dc.model.MistralOcrRequest;
import de.frigosped.dc.model.MistralOcrResponse;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.MediaType;
import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;
@RegisterRestClient(configKey = "mistral-ocr")
@Path("/v1")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public interface MistralOcrClient {
@POST
@Path("/ocr")
MistralOcrResponse ocr(@HeaderParam("Authorization") String authorization,
MistralOcrRequest request);
}

29
dc-backend/src/main/java/de/frigosped/dc/client/OrdsClient.java
View File

@@ -116,10 +116,12 @@ public interface OrdsClient {
/**
* POST /api/dc/projects/:projectId/complete
* Setzt Status auf COMPLETED + completed_at = SYSDATE.
* Body enthält optional das vorberechnete report_markdown.
*/
@POST
@Path("/projects/{projectId}/complete")
Response completeProject(@PathParam("projectId") long projectId);
Response completeProject(@PathParam("projectId") long projectId,
CompleteProjectRequest body);
// =========================================================================
// Dokumente
@@ -159,6 +161,31 @@ public interface OrdsClient {
@Path("/documents/{docId}/texts")
Response updateTexts(@PathParam("docId") long docId, TextsRequest body);
/**
* PUT /api/dc/documents/:docId/progress
* Setzt status und/oder progress (0-100) eines Dokuments.
*/
@PUT
@Path("/documents/{docId}/progress")
Response updateDocumentProgress(@PathParam("docId") long docId, DocumentProgressRequest body);
/**
* POST /api/dc/documents/:parentId/references
* Legt eine heruntergeladene externe Referenz als neue dc_project_documents-Zeile an.
* Antwort: 201 Created, Body {"id": <new_id>, ...}
*/
@POST
@Path("/documents/{parentId}/references")
Response insertReference(@PathParam("parentId") long parentId, ReferenceCreateRequest body);
/**
* DELETE /api/dc/projects/:projectId/references
* Löscht alle heruntergeladenen Referenzen eines Projekts (vor Re-Processing).
*/
@DELETE
@Path("/projects/{projectId}/references")
Response deleteReferences(@PathParam("projectId") long projectId);
// =========================================================================
// Ergebnisse
// =========================================================================

8
dc-backend/src/main/java/de/frigosped/dc/client/OrdsLoggingFilter.java
View File

@@ -47,6 +47,8 @@ public class OrdsLoggingFilter implements ClientRequestFilter, ClientResponseFil
LOG.infof("=== ORDS %s %s END ===", requestCtx.getMethod(), requestCtx.getUri());
}
private static final int MAX_BODY_LOG = 500;
private String readRequest(ClientRequestContext ctx) {
Object entity = ctx.getEntity();
if (entity == null) return null;
@@ -63,7 +65,11 @@ public class OrdsLoggingFilter implements ClientRequestFilter, ClientResponseFil
} else {
body = entity.toString();
}
return body.isBlank() ? null : body;
if (body.isBlank()) return null;
if (body.length() > MAX_BODY_LOG) {
return body.substring(0, MAX_BODY_LOG) + String.format(" ... [%d Zeichen gesamt]", body.length());
}
return body;
}
private String readResponse(ClientResponseContext ctx) throws IOException {

22
dc-backend/src/main/java/de/frigosped/dc/model/CompleteProjectRequest.java Normal file
View File

@@ -0,0 +1,22 @@
package de.frigosped.dc.model;
import java.util.List;
public class CompleteProjectRequest {
private String reportMarkdown;
private List<DocumentPdfEntry> documentPdfs;
public CompleteProjectRequest() {}
public CompleteProjectRequest(String reportMarkdown, List<DocumentPdfEntry> documentPdfs) {
this.reportMarkdown = reportMarkdown;
this.documentPdfs = documentPdfs;
}
public String getReportMarkdown() { return reportMarkdown; }
public void setReportMarkdown(String v) { this.reportMarkdown = v; }
public List<DocumentPdfEntry> getDocumentPdfs() { return documentPdfs; }
public void setDocumentPdfs(List<DocumentPdfEntry> v) { this.documentPdfs = v; }
}

20
dc-backend/src/main/java/de/frigosped/dc/model/DocumentPdfEntry.java Normal file
View File

@@ -0,0 +1,20 @@
package de.frigosped.dc.model;
public class DocumentPdfEntry {
private String filename;
private String pdfBase64;
public DocumentPdfEntry() {}
public DocumentPdfEntry(String filename, String pdfBase64) {
this.filename = filename;
this.pdfBase64 = pdfBase64;
}
public String getFilename() { return filename; }
public void setFilename(String v) { this.filename = v; }
public String getPdfBase64() { return pdfBase64; }
public void setPdfBase64(String v) { this.pdfBase64 = v; }
}

18
dc-backend/src/main/java/de/frigosped/dc/model/DocumentProgressRequest.java Normal file
View File

@@ -0,0 +1,18 @@
package de.frigosped.dc.model;
import com.fasterxml.jackson.annotation.JsonInclude;
@JsonInclude(JsonInclude.Include.NON_NULL)
public class DocumentProgressRequest {
private Integer progress;
private String status;
public DocumentProgressRequest(Integer progress, String status) {
this.progress = progress;
this.status = status;
}
public Integer getProgress() { return progress; }
public String getStatus() { return status; }
}

28
dc-backend/src/main/java/de/frigosped/dc/model/MistralOcrRequest.java Normal file
View File

@@ -0,0 +1,28 @@
package de.frigosped.dc.model;
public class MistralOcrRequest {
private String model;
private Document document;
public MistralOcrRequest(String model, Document document) {
this.model = model;
this.document = document;
}
public String getModel() { return model; }
public Document getDocument() { return document; }
public static class Document {
private String type;
private String documentUrl;
public Document(String type, String documentUrl) {
this.type = type;
this.documentUrl = documentUrl;
}
public String getType() { return type; }
public String getDocumentUrl() { return documentUrl; }
}
}

37
dc-backend/src/main/java/de/frigosped/dc/model/MistralOcrResponse.java Normal file
View File

@@ -0,0 +1,37 @@
package de.frigosped.dc.model;
import java.util.List;
public class MistralOcrResponse {
private List<Page> pages;
private UsageInfo usageInfo;
public List<Page> getPages() { return pages; }
public void setPages(List<Page> pages) { this.pages = pages; }
public UsageInfo getUsageInfo() { return usageInfo; }
public void setUsageInfo(UsageInfo usageInfo) { this.usageInfo = usageInfo; }
public static class Page {
private int index;
private String markdown;
public int getIndex() { return index; }
public void setIndex(int index) { this.index = index; }
public String getMarkdown() { return markdown; }
public void setMarkdown(String markdown) { this.markdown = markdown; }
}
public static class UsageInfo {
private int pagesProcessed;
private Long docSizeBytes;
public int getPagesProcessed() { return pagesProcessed; }
public void setPagesProcessed(int pagesProcessed) { this.pagesProcessed = pagesProcessed; }
public Long getDocSizeBytes() { return docSizeBytes; }
public void setDocSizeBytes(Long docSizeBytes) { this.docSizeBytes = docSizeBytes; }
}
}

23
dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocument.java
View File

@@ -14,8 +14,13 @@ public class OrdsDocument {
private Long documentTypeId;
private Long catalogId;
private LocalDateTime uploadedAt;
private String status;
private Integer progress;
private Integer hasOriginalText; // 0 oder 1
private Integer hasTranslatedText; // 0 oder 1
private Long parentDocumentId; // gesetzt bei heruntergeladenen Referenzen
private Integer isReference; // 0 = Originaldokument, 1 = Referenz
private String sourceUrl; // Ursprungs-URL der Referenz
private Integer rowVersion;
private LocalDateTime created;
private LocalDateTime updated;
@@ -40,12 +45,30 @@ public class OrdsDocument {
public LocalDateTime getUploadedAt() { return uploadedAt; }
public void setUploadedAt(LocalDateTime v) { this.uploadedAt = v; }
public String getStatus() { return status; }
public void setStatus(String status) { this.status = status; }
public Integer getProgress() { return progress; }
public void setProgress(Integer progress) { this.progress = progress; }
public Integer getHasOriginalText() { return hasOriginalText; }
public void setHasOriginalText(Integer v) { this.hasOriginalText = v; }
public Integer getHasTranslatedText() { return hasTranslatedText; }
public void setHasTranslatedText(Integer v) { this.hasTranslatedText = v; }
public Long getParentDocumentId() { return parentDocumentId; }
public void setParentDocumentId(Long v) { this.parentDocumentId = v; }
public Integer getIsReference() { return isReference; }
public void setIsReference(Integer v) { this.isReference = v; }
/** true, wenn dieses Dokument eine heruntergeladene Referenz ist. */
public boolean isReference() { return Integer.valueOf(1).equals(isReference); }
public String getSourceUrl() { return sourceUrl; }
public void setSourceUrl(String v) { this.sourceUrl = v; }
public Integer getRowVersion() { return rowVersion; }
public void setRowVersion(Integer v) { this.rowVersion = v; }

12
dc-backend/src/main/java/de/frigosped/dc/model/OrdsProject.java
View File

@@ -17,6 +17,9 @@ public class OrdsProject {
private Integer progress; // 0–100
private LocalDateTime completedAt;
private String notificationEmail;
private String notifyOk; // 'Y' | 'N'
private String notifyHinweis; // 'Y' | 'N'
private String notifyAbweichend; // 'Y' | 'N'
private Integer rowVersion;
private LocalDateTime created;
private LocalDateTime updated;
@@ -50,6 +53,15 @@ public class OrdsProject {
public String getNotificationEmail() { return notificationEmail; }
public void setNotificationEmail(String v) { this.notificationEmail = v; }
public String getNotifyOk() { return notifyOk; }
public void setNotifyOk(String v) { this.notifyOk = v; }
public String getNotifyHinweis() { return notifyHinweis; }
public void setNotifyHinweis(String v) { this.notifyHinweis = v; }
public String getNotifyAbweichend() { return notifyAbweichend; }
public void setNotifyAbweichend(String v) { this.notifyAbweichend = v; }
public Integer getRowVersion() { return rowVersion; }
public void setRowVersion(Integer v) { this.rowVersion = v; }

38
dc-backend/src/main/java/de/frigosped/dc/model/ReferenceCreateRequest.java Normal file
View File

@@ -0,0 +1,38 @@
package de.frigosped.dc.model;
/**
* Request-Body für POST /api/dc/documents/:parent_id/references
* {"source_url":"https://...","filename":"AGB.pdf",
* "mime_type":"application/pdf","file_base64":"JVBERi0x..."}
*
* Legt eine heruntergeladene externe Referenz als neue dc_project_documents-Zeile an.
*/
public class ReferenceCreateRequest {
private String sourceUrl;
private String filename;
private String mimeType;
private String fileBase64;
public ReferenceCreateRequest() {}
public ReferenceCreateRequest(String sourceUrl, String filename,
String mimeType, String fileBase64) {
this.sourceUrl = sourceUrl;
this.filename = filename;
this.mimeType = mimeType;
this.fileBase64 = fileBase64;
}
public String getSourceUrl() { return sourceUrl; }
public void setSourceUrl(String v) { this.sourceUrl = v; }
public String getFilename() { return filename; }
public void setFilename(String v) { this.filename = v; }
public String getMimeType() { return mimeType; }
public void setMimeType(String v) { this.mimeType = v; }
public String getFileBase64() { return fileBase64; }
public void setFileBase64(String v) { this.fileBase64 = v; }
}

222
dc-backend/src/main/java/de/frigosped/dc/service/CheckOrchestrationService.java
View File

@@ -14,6 +14,7 @@ import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
@@ -48,6 +49,22 @@ public class CheckOrchestrationService {
@Inject
EvaluationService evaluationService;
@Inject
MarkdownReportBuilder markdownReportBuilder;
@Inject
MarkdownExportService markdownExportService;
@Inject
ReferenceDownloadService referenceDownloadService;
@Inject
com.fasterxml.jackson.databind.ObjectMapper objectMapper;
@org.eclipse.microprofile.config.inject.ConfigProperty(
name = "dc.references.max-per-document", defaultValue = "5")
int maxReferencesPerDocument;
// =========================================================================
// Haupt-Einstiegspunkt
// =========================================================================
@@ -82,7 +99,7 @@ public class CheckOrchestrationService {
if (documents.isEmpty()) {
LOG.warnf("Projekt %d hat keine Dokumente – markiere als abgeschlossen", projectId);
ordsClient.completeProject(projectId);
ordsClient.completeProject(projectId, new CompleteProjectRequest("", java.util.Collections.emptyList()));
return;
}
@@ -115,28 +132,39 @@ public class CheckOrchestrationService {
}
int[] stepHolder = {0};
// 6. Alte Ergebnisse löschen (ermöglicht Re-Processing)
// 6. Alte Ergebnisse + heruntergeladene Referenzen löschen (ermöglicht Re-Processing)
Response delResp = ordsClient.deleteResults(projectId);
LOG.debugf("Alte Ergebnisse gelöscht: HTTP %d", delResp.getStatus());
try {
Response delRefResp = ordsClient.deleteReferences(projectId);
LOG.debugf("Alte Referenzen gelöscht: HTTP %d", delRefResp.getStatus());
} catch (Exception e) {
LOG.debugf("Referenzen löschen fehlgeschlagen: %s", e.getMessage());
}
// Texte sammeln (in-memory: vermeidet Re-Fetch aus ORDS)
// Texte und Ergebnisse sammeln (in-memory für Markdown-Report)
Map<Long, String> originalTexts = new HashMap<>();
Map<Long, List<ResultRequest>> resultsByDoc = new HashMap<>();
// Original-Texte heruntergeladener Referenzen je Hauptdokument-ID
Map<Long, List<String>> referenceTextsByDoc = new HashMap<>();
// ─── PHASE A: OCR + Übersetzung (wird übersprungen wenn Text bereits vorhanden) ──
for (OrdsDocument doc : documents) {
// Vor dem Schritt: Status auf "OCR für dieses Dokument" setzen
if (doc.isReference()) continue; // Referenzen werden während Phase A erzeugt
pushProgress(projectId, stepHolder[0], totalSteps, "OCR",
doc.getId(), null, startedAt);
pushDocumentProgress(doc.getId(), 0, "IN_PROGRESS");
String originalText = "";
if (Integer.valueOf(1).equals(doc.getHasOriginalText())) {
// Resume: Text aus vorherigem Lauf wiederverwenden
LOG.infof("Dokument %d: Text bereits vorhanden – überspringe OCR/Übersetzung",
doc.getId());
try {
OrdsDocumentTexts texts = ordsClient.getTexts(doc.getId());
originalText = texts.getOriginalText() != null ? texts.getOriginalText() : "";
pushDocumentProgress(doc.getId(), 50, null);
} catch (Exception e) {
LOG.warnf("Text-Laden für Dokument %d fehlgeschlagen: %s", doc.getId(), e.getMessage());
}
@@ -148,6 +176,7 @@ public class CheckOrchestrationService {
if (fileResp.getStatus() != 200) {
LOG.warnf("Download Dokument %d fehlgeschlagen (HTTP %d)",
doc.getId(), fileResp.getStatus());
pushDocumentProgress(doc.getId(), null, "ERROR");
} else {
byte[] fileBytes = fileResp.readEntity(byte[].class);
@@ -155,22 +184,34 @@ public class CheckOrchestrationService {
fileBytes, doc.getMimeType(), doc.getFilename(), projectId);
LOG.debugf("Dokument %d: %d Zeichen extrahiert",
(long) doc.getId(), (long) originalText.length());
pushDocumentProgress(doc.getId(), 25, null);
translatedText = evaluationService.translate(originalText, projectId);
LOG.debugf("Dokument %d: Übersetzung fertig (%d Zeichen)",
(long) doc.getId(), (long) translatedText.length());
pushDocumentProgress(doc.getId(), 50, null);
ordsClient.updateTexts(doc.getId(),
new TextsRequest(originalText, translatedText));
}
} catch (Exception e) {
LOG.errorf(e, "Fehler bei Dokumentenverarbeitung %d", doc.getId());
pushDocumentProgress(doc.getId(), null, "ERROR");
}
}
originalTexts.put(doc.getId(), originalText);
// Nach dem Schritt: Fortschritt hochzählen + ETA aktualisieren
// Externe Referenzen erkennen, herunterladen und als eigene Zeilen speichern
if (!originalText.isBlank() && referenceDownloadService.isEnabled()) {
try {
processReferences(doc, originalText, projectId, referenceTextsByDoc);
} catch (Exception e) {
LOG.warnf("Referenz-Verarbeitung für Dokument %d fehlgeschlagen: %s",
doc.getId(), e.getMessage());
}
}
stepHolder[0]++;
pushProgress(projectId, stepHolder[0], totalSteps, "OCR",
doc.getId(), null, startedAt);
@@ -178,7 +219,12 @@ public class CheckOrchestrationService {
// ─── PHASE B: Fragen auswerten ────────────────────────────────
for (OrdsDocument doc : documents) {
String originalText = originalTexts.getOrDefault(doc.getId(), "");
if (doc.isReference()) continue; // Referenzen werden nur „in Summe" mitbewertet
// Auswertung über Haupt- + Referenztext (Option D: „Summe")
String originalText = buildCombinedText(
originalTexts.getOrDefault(doc.getId(), ""),
referenceTextsByDoc.get(doc.getId()));
Long catId = doc.getCatalogId();
if (catId == null || !questionsByCatalog.containsKey(catId)) {
@@ -197,6 +243,7 @@ public class CheckOrchestrationService {
}
List<List<OrdsQuestion>> batches = partition(questions, 4);
int batchesDone = 0;
for (List<OrdsQuestion> batch : batches) {
OrdsQuestion first = batch.get(0);
@@ -217,17 +264,64 @@ public class CheckOrchestrationService {
for (OrdsQuestion question : batch) {
EvaluationResult result = batchResults.get(question.getQuestionId());
if (result == null) continue;
saveResult(projectId, doc.getId(), question, result);
ResultRequest rr = saveResult(projectId, doc.getId(), question, result);
if (rr != null) {
resultsByDoc.computeIfAbsent(doc.getId(), k -> new ArrayList<>()).add(rr);
}
}
stepHolder[0] += batch.size();
batchesDone += batch.size();
int docProgress = 50 + (int) Math.min(batchesDone * 50L / questions.size(), 49);
pushDocumentProgress(doc.getId(), docProgress, null);
pushProgress(projectId, stepHolder[0], totalSteps, "QUESTIONS",
doc.getId(), batch.get(batch.size() - 1).getQuestionId(), startedAt);
}
pushDocumentProgress(doc.getId(), 100, "COMPLETED");
}
// 7. Abschließen
ordsClient.completeProject(projectId);
// 7. Kombinierten Markdown-Bericht für die DB generieren
String reportMarkdown = "";
try {
reportMarkdown = markdownReportBuilder.build(
project, documents, questionsByCatalog, resultsByDoc);
LOG.infof("Projekt %d: Markdown-Bericht generiert (%d Zeichen)",
projectId, reportMarkdown.length());
} catch (Exception e) {
LOG.warnf(e, "Markdown-Generierung fehlgeschlagen – sende leeren Body");
}
// 8. Pro Dokument ein eigenes PDF generieren
Map<Long, OrdsQuestion> questionById = new HashMap<>();
for (List<OrdsQuestion> qs : questionsByCatalog.values()) {
for (OrdsQuestion q : qs) questionById.put(q.getQuestionId(), q);
}
List<DocumentPdfEntry> documentPdfs = new ArrayList<>();
for (OrdsDocument doc : documents) {
List<ResultRequest> docResults =
resultsByDoc.getOrDefault(doc.getId(), java.util.Collections.emptyList());
if (docResults.isEmpty()) continue;
try {
String docMd = markdownReportBuilder.buildForDocument(
project, doc, questionById, docResults);
byte[] pdfBytes = markdownExportService.toPdf(docMd);
String baseName = doc.getFilename().replaceAll("\\.[^.]+$", "");
String filename = "Pruefbericht_"
+ baseName.replaceAll("[^A-Za-z0-9_äöüÄÖÜß.-]", "_")
+ ".pdf";
documentPdfs.add(new DocumentPdfEntry(filename,
Base64.getEncoder().encodeToString(pdfBytes)));
LOG.infof("Projekt %d: Dok %d – PDF generiert (%d Bytes)",
projectId, doc.getId(), pdfBytes.length);
} catch (Exception e) {
LOG.warnf(e, "PDF-Generierung für Dok %d fehlgeschlagen", doc.getId());
}
}
ordsClient.completeProject(projectId,
new CompleteProjectRequest(reportMarkdown, documentPdfs));
LOG.infof("=== Projekt %d erfolgreich abgeschlossen ===", projectId);
} catch (Exception e) {
@@ -257,6 +351,98 @@ public class CheckOrchestrationService {
}
}
/** Setzt progress und/oder status eines Dokuments. null-Werte werden ignoriert. */
private void pushDocumentProgress(long docId, Integer progress, String status) {
try {
jakarta.ws.rs.core.Response resp =
ordsClient.updateDocumentProgress(docId, new DocumentProgressRequest(progress, status));
if (resp.getStatus() < 200 || resp.getStatus() >= 300) {
LOG.warnf("Dokument-Fortschritt: HTTP %d (Dok %d, progress=%s, status=%s)",
resp.getStatus(), docId, progress, status);
}
} catch (Exception e) {
LOG.warnf("Dokument-Fortschritt konnte nicht gesetzt werden (Dok %d): %s", docId, e.getMessage());
}
}
/**
* Erkennt externe Referenzen im Dokumenttext, lädt sie sicher herunter,
* legt sie als eigene dc_project_documents-Zeilen an (is_reference=1) und
* sammelt ihren Original-Text für die spätere „Summen"-Auswertung.
*/
private void processReferences(OrdsDocument doc, String originalText, long projectId,
Map<Long, List<String>> referenceTextsByDoc) {
List<String> urls = evaluationService.extractReferenceUrls(originalText, projectId);
if (urls.isEmpty()) return;
LOG.infof("Dokument %d: %d Referenz-URL(s) vom Modell erkannt", doc.getId(), urls.size());
int processed = 0;
for (String url : urls) {
if (processed >= maxReferencesPerDocument) {
LOG.infof("Dokument %d: Referenz-Limit (%d) erreicht – Rest ignoriert",
doc.getId(), maxReferencesPerDocument);
break;
}
var fetched = referenceDownloadService.fetch(url, projectId);
if (fetched.isEmpty()) continue;
ReferenceDownloadService.DownloadedReference ref = fetched.get();
try {
// 1. Referenz als neue Dokument-Zeile (mit BLOB) anlegen
String base64 = Base64.getEncoder().encodeToString(ref.bytes());
Response insResp = ordsClient.insertReference(doc.getId(),
new ReferenceCreateRequest(ref.sourceUrl(), ref.filename(),
ref.mimeType(), base64));
String body = insResp.readEntity(String.class);
if (insResp.getStatus() != 201) {
LOG.warnf("Referenz-Insert fehlgeschlagen (HTTP %d): %s", insResp.getStatus(), body);
continue;
}
long refId = extractId(body);
// 2. Texte der Referenz speichern (Original + Übersetzung) – für separate Ansicht
String refTranslated = evaluationService.translate(ref.text(), projectId);
ordsClient.updateTexts(refId, new TextsRequest(ref.text(), refTranslated));
pushDocumentProgress(refId, 100, "COMPLETED");
// 3. Original-Text für die „Summen"-Auswertung des Hauptdokuments merken
referenceTextsByDoc.computeIfAbsent(doc.getId(), k -> new ArrayList<>())
.add("<!-- Referenz: " + ref.sourceUrl() + " -->\n" + ref.text());
processed++;
LOG.infof("Dokument %d: Referenz gespeichert (id=%d, %s)",
doc.getId(), refId, ref.sourceUrl());
} catch (Exception e) {
LOG.warnf("Referenz '%s' konnte nicht gespeichert werden: %s", url, e.getMessage());
}
}
}
/** Liest das Feld "id" aus einer ORDS-Insert-Antwort. */
private long extractId(String responseBody) throws Exception {
Map<?, ?> map = objectMapper.readValue(responseBody, Map.class);
Object id = map.get("id");
if (id instanceof Number n) return n.longValue();
throw new IllegalStateException("Kein 'id'-Feld in ORDS-Antwort: " + responseBody);
}
/**
* Fügt Haupttext und Referenztexte zu einem Bewertungstext zusammen (Option D).
* Die Referenzen werden klar abgegrenzt angehängt.
*/
private String buildCombinedText(String mainText, List<String> referenceTexts) {
if (referenceTexts == null || referenceTexts.isEmpty()) {
return mainText != null ? mainText : "";
}
StringBuilder sb = new StringBuilder(mainText != null ? mainText : "");
for (String refText : referenceTexts) {
sb.append("\n\n--- Referenzdokument ---\n\n").append(refText);
}
return sb.toString();
}
/** Teilt eine Liste in Untergruppen der gewünschten Größe auf. */
private static <T> List<List<T>> partition(List<T> list, int size) {
List<List<T>> parts = new ArrayList<>();
@@ -282,14 +468,20 @@ public class CheckOrchestrationService {
return map;
}
private void saveResult(long projectId, long docId,
OrdsQuestion question, EvaluationResult result) {
private ResultRequest saveResult(long projectId, long docId,
OrdsQuestion question, EvaluationResult result) {
try {
int deviation = 0, warning = 0;
if (result.getScore() != null && question.getThreshold() != null
&& result.getScore() < question.getThreshold()) {
if ("ABWEICHUNG".equalsIgnoreCase(question.getResultHandling())) deviation = 1;
else if ("HINWEIS".equalsIgnoreCase(question.getResultHandling())) warning = 1;
if ("UNKLAR".equalsIgnoreCase(result.getResultType())) {
// Teilweise Erfüllung (falscher Paragraph, gleiches Ziel, o.ä.) → immer Hinweis
warning = 1;
} else if ("ABWEICHUNG".equalsIgnoreCase(question.getResultHandling())) {
deviation = 1;
} else if ("HINWEIS".equalsIgnoreCase(question.getResultHandling())) {
warning = 1;
}
}
ResultRequest rr = new ResultRequest(
question.getQuestionId(), docId,
@@ -301,9 +493,11 @@ public class CheckOrchestrationService {
LOG.warnf("Ergebnis speichern: HTTP %d (Frage %d, Dok %d)",
saveResp.getStatus(), question.getQuestionId(), docId);
}
return rr;
} catch (Exception e) {
LOG.errorf(e, "Ergebnis-Speichern fehlgeschlagen: Frage %d / Dokument %d",
question.getQuestionId(), docId);
return null;
}
}

68
dc-backend/src/main/java/de/frigosped/dc/service/DocumentProcessingService.java
View File

@@ -1,5 +1,8 @@
package de.frigosped.dc.service;
import de.frigosped.dc.client.MistralOcrClient;
import de.frigosped.dc.model.MistralOcrRequest;
import de.frigosped.dc.model.MistralOcrResponse;
import dev.langchain4j.data.message.ImageContent;
import dev.langchain4j.data.message.TextContent;
import dev.langchain4j.data.message.UserMessage;
@@ -11,7 +14,10 @@ import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.rendering.ImageType;
import org.apache.pdfbox.rendering.PDFRenderer;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.poi.xwpf.usermodel.*;
import org.eclipse.microprofile.config.inject.ConfigProperty;
import org.eclipse.microprofile.rest.client.inject.RestClient;
import org.jboss.logging.Logger;
import org.odftoolkit.odfdom.doc.OdfTextDocument;
import org.w3c.dom.Node;
@@ -42,13 +48,26 @@ public class DocumentProcessingService {
/** 200 DPI: gutes Gleichgewicht zwischen Qualität und Dateigröße für OCR */
private static final float OCR_DPI = 200f;
/** Mindest-Zeichen pro Seite – darunter gilt das PDF als gescannt */
private static final int MIN_CHARS_PER_PAGE = 50;
@Inject
@Named("ocr")
ChatLanguageModel ocrModel;
@Inject
@RestClient
MistralOcrClient mistralOcrClient;
@Inject
AiCostTrackerService costTracker;
@ConfigProperty(name = "dc.ai.provider", defaultValue = "ollama")
String aiProvider;
@ConfigProperty(name = "dc.ai.mistral.api-key", defaultValue = "")
String mistralApiKey;
// =========================================================================
// Öffentliche API
// =========================================================================
@@ -96,28 +115,32 @@ public class DocumentProcessingService {
// =========================================================================
private String extractFromPdf(byte[] fileBytes, Long projectId) throws Exception {
LOG.debug("Starte PDF-OCR...");
StringBuilder result = new StringBuilder();
try (PDDocument pdf = Loader.loadPDF(fileBytes)) {
PDFRenderer renderer = new PDFRenderer(pdf);
int pageCount = pdf.getNumberOfPages();
LOG.debugf("PDF hat %d Seite(n)", pageCount);
String extracted = new PDFTextStripper().getText(pdf).trim();
if (extracted.length() >= pageCount * MIN_CHARS_PER_PAGE) {
LOG.debugf("PDF-Textextraktion: %d Zeichen", extracted.length());
return extracted;
}
LOG.debugf("PDF hat wenig eingebetteten Text (%d Zeichen) – verwende Vision-OCR", extracted.length());
if ("mistral".equalsIgnoreCase(aiProvider)) {
return mistralOcr(fileBytes);
}
PDFRenderer renderer = new PDFRenderer(pdf);
StringBuilder result = new StringBuilder();
for (int i = 0; i < pageCount; i++) {
LOG.debugf("OCR Seite %d/%d", i + 1, pageCount);
BufferedImage image = renderer.renderImageWithDPI(i, OCR_DPI, ImageType.RGB);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(image, "PNG", baos);
String base64Image = Base64.getEncoder().encodeToString(baos.toByteArray());
String pageText = ocrPage(base64Image, i + 1, pageCount, projectId);
result.append(pageText).append("\n\n");
result.append(ocrPage(base64Image, i + 1, pageCount, projectId)).append("\n\n");
}
return result.toString().trim();
}
return result.toString().trim();
}
private String ocrPage(String base64Image, int pageNum, int totalPages, Long projectId) {
@@ -143,6 +166,31 @@ public class DocumentProcessingService {
return response.content().text();
}
private String mistralOcr(byte[] fileBytes) {
LOG.debug("Mistral OCR API – sende PDF direkt...");
String base64 = Base64.getEncoder().encodeToString(fileBytes);
String dataUrl = "data:application/pdf;base64," + base64;
MistralOcrRequest request = new MistralOcrRequest(
"mistral-ocr-latest",
new MistralOcrRequest.Document("document_url", dataUrl)
);
MistralOcrResponse response = mistralOcrClient.ocr("Bearer " + mistralApiKey, request);
StringBuilder result = new StringBuilder();
for (MistralOcrResponse.Page page : response.getPages().stream()
.sorted(java.util.Comparator.comparingInt(MistralOcrResponse.Page::getIndex))
.toList()) {
if (page.getMarkdown() != null && !page.getMarkdown().isBlank()) {
result.append(page.getMarkdown()).append("\n\n");
}
}
LOG.debugf("Mistral OCR abgeschlossen: %d Seite(n), %d Zeichen",
response.getPages().size(), result.length());
return result.toString().trim();
}
// =========================================================================
// DOCX → Markdown (Apache POI)
// =========================================================================

85
dc-backend/src/main/java/de/frigosped/dc/service/EvaluationService.java
View File

@@ -19,6 +19,7 @@ import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
/**
* KI-gestützte Auswertung:
@@ -78,7 +79,79 @@ public class EvaluationService {
Response<AiMessage> response = mainModel.generate(messages);
costTracker.track("main", "TRANSLATE", projectId, null, response.tokenUsage());
return response.content().text().trim();
return stripCodeFence(response.content().text().trim());
}
/**
* Entfernt eine umschließende ```-Markierung (z. B. ```markdown), die das
* Modell der Übersetzung manchmal trotz Anweisung hinzufügt. So wird der
* Markdown-Text einheitlich ohne Code-Fence gespeichert.
*/
private String stripCodeFence(String text) {
if (text == null) return "";
String t = text.trim();
if (t.startsWith("```")) {
int firstNewline = t.indexOf('\n');
if (firstNewline > 0) {
t = t.substring(firstNewline + 1); // erste Zeile (```markdown) entfernen
}
if (t.endsWith("```")) {
t = t.substring(0, t.length() - 3); // schließendes ``` entfernen
}
}
return t.trim();
}
// =========================================================================
// Referenz-Erkennung
// =========================================================================
/**
* Lässt das Hauptmodell die relevanten externen Dokument-Referenz-URLs direkt
* aus dem Dokumenttext ausgeben (kein Regex). Login-, Video-, Impressum- und
* Portalseiten werden anhand des Kontexts herausgefiltert.
*
* @return Liste der vom Modell gelieferten URLs (kann leer sein)
*/
public List<String> extractReferenceUrls(String documentText, long projectId) {
if (documentText == null || documentText.isBlank()) return List.of();
List<ChatMessage> messages = List.of(
SystemMessage.from(
"Du analysierst Transport- und Speditionsdokumente und findest Verweise auf\n" +
"EXTERNE Dokumente, die per Link erreichbar sind (z. B. AGB, Allgemeine\n" +
"Geschäftsbedingungen, Verhaltenskodex/Code of Conduct, Vertragsbedingungen,\n" +
"Sicherheits- oder Compliance-Richtlinien).\n\n" +
"Gib NUR Links zurück, die tatsächlich im Text stehen und auf ein solches\n" +
"Dokument verweisen. Schließe ausdrücklich AUS: Login-/Portalseiten,\n" +
"Video-Links (z. B. YouTube), Impressum, Datenschutz-Übersichtsseiten,\n" +
"Hinweisgeber-/Whistleblower-Portale, Social-Media- und reine Startseiten.\n\n" +
"Verändere die URLs nicht – gib sie exakt so zurück, wie sie im Text stehen.\n\n" +
"Antworte AUSSCHLIESSLICH mit einem gültigen JSON-Array von Strings (URLs).\n" +
"Wenn es keine passenden Links gibt, antworte mit []."
),
UserMessage.from("## Dokument\n\n" + documentText + "\n\n---\n\nAntworte mit dem JSON-Array der relevanten URLs:")
);
try {
Response<AiMessage> response = mainModel.generate(messages);
costTracker.track("main", "REF_EXTRACT", projectId, null, response.tokenUsage());
String json = extractJsonArray(response.content().text());
List<String> urls = objectMapper.readValue(
json,
objectMapper.getTypeFactory().constructCollectionType(List.class, String.class)
);
// Duplikate entfernen, leere Einträge filtern, Reihenfolge erhalten
return urls.stream()
.filter(u -> u != null && !u.isBlank())
.map(String::trim)
.distinct()
.collect(Collectors.toList());
} catch (Exception e) {
LOG.warnf("Referenz-URL-Extraktion fehlgeschlagen: %s", e.getMessage());
return List.of();
}
}
// =========================================================================
@@ -163,6 +236,11 @@ public class EvaluationService {
Du bist ein Experte für Transportrecht, AGB-Recht und Vertragsanalyse.
Du prüfst Transport- und Speditionsdokumente gegen einen Fragenkatalog.
Wichtige Suchregel: Suche im Dokument ZUERST nach expliziten, wörtlichen Formulierungen,
die die Anforderung direkt bestätigen oder verneinen. Zitiere immer die stärkste,
direkteste Textstelle. Nutze indirekte oder implizite Belege NUR dann, wenn keine
direkte Formulierung im Dokument existiert.
Antworte AUSSCHLIESSLICH mit einem gültigen JSON-Array – kein Text davor oder danach.
Das Array enthält für JEDE Frage genau ein Objekt:
@@ -275,6 +353,11 @@ public class EvaluationService {
Du bist ein Experte für Transportrecht, AGB-Recht und Vertragsanalyse.
Du prüfst Transport- und Speditionsdokumente gegen einen Fragenkatalog.
Wichtige Suchregel: Suche im Dokument ZUERST nach expliziten, wörtlichen Formulierungen,
die die Anforderung direkt bestätigen oder verneinen. Zitiere immer die stärkste,
direkteste Textstelle. Nutze indirekte oder implizite Belege NUR dann, wenn keine
direkte Formulierung im Dokument existiert.
Antworte AUSSCHLIESSLICH mit einem gültigen JSON-Objekt – kein Text davor oder danach.
Das JSON-Objekt muss folgende Felder enthalten:

376
dc-backend/src/main/java/de/frigosped/dc/service/MarkdownExportService.java
View File

@@ -20,6 +20,12 @@ import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
/**
* Konvertiert Markdown-Text zu DOCX (Apache POI) oder PDF (PDFBox).
@@ -66,41 +72,90 @@ public class MarkdownExportService {
|| cp == 0x1F4C4; // 📄
}
/** Findet das schließende einzelne * ab Position from; überspringt ** -Paare. */
private static int findItalicClose(String text, int from) {
int end = text.indexOf('*', from);
while (end >= 0) {
if (end + 1 >= text.length() || text.charAt(end + 1) != '*') return end;
end = text.indexOf('*', end + 2);
}
return -1;
}
/**
* Zerlegt eine Zeile in typisierte Inline-Segmente.
* Zerlegt text in typisierte Inline-Segmente mit Zeilenübergreifender Format-Fortsetzung.
*
* Erkennungsreihenfolge pro Position:
* 1. **text** → BOLD
* 2. *text* → ITALIC (nur wenn nicht Teil von **)
* 3. isReportEmoji() → EMOJI (inkl. nachfolgende Variation-Selektoren U+FE0F / ZWJ U+200D)
* 4. alles andere → PLAIN (gesammelt bis zum nächsten Marker oder Emoji)
* openFormatRef[0] gibt das ggf. bereits offene Format vom vorherigen Aufruf an
* (null = keines, BOLD = innerhalb **, ITALIC = innerhalb *).
* Nach dem Aufruf enthält openFormatRef[0] das weiterhin offene Format.
*
* Beispiel: "✅ **Frage** ist *kursiv*"
* → [EMOJI "✅"], [PLAIN " "], [BOLD "Frage"], [PLAIN " ist "], [ITALIC "kursiv"]
* Wird ** oder * geöffnet aber auf dieser Zeile nicht geschlossen, erstreckt sich
* die Formatierung bis ans Zeilenende und openFormatRef[0] wird entsprechend gesetzt,
* damit der nächste Aufruf die Formatierung nahtlos fortsetzt.
*/
static List<Seg> parseSegments(String text) {
static List<Seg> parseSegments(String text, SegType[] openFormatRef) {
List<Seg> out = new ArrayList<>();
if (text == null || text.isEmpty()) return out;
SegType open = openFormatRef[0];
int pos = 0;
// Format-Fortsetzung von der Vorgängerzeile aufnehmen
if (open == SegType.BOLD) {
int end = text.indexOf("**");
if (end >= 0) {
if (end > 0) out.add(new Seg(SegType.BOLD, text.substring(0, end)));
pos = end + 2;
open = null;
} else {
out.add(new Seg(SegType.BOLD, text));
openFormatRef[0] = SegType.BOLD;
return out;
}
} else if (open == SegType.ITALIC) {
int end = findItalicClose(text, 0);
if (end >= 0) {
if (end > 0) out.add(new Seg(SegType.ITALIC, text.substring(0, end)));
pos = end + 1;
open = null;
} else {
out.add(new Seg(SegType.ITALIC, text));
openFormatRef[0] = SegType.ITALIC;
return out;
}
}
while (pos < text.length()) {
// **bold**
if (text.startsWith("**", pos)) {
int end = text.indexOf("**", pos + 2);
if (end > pos + 1) {
if (end >= 0) {
out.add(new Seg(SegType.BOLD, text.substring(pos + 2, end)));
pos = end + 2;
continue;
} else {
// kein schließendes ** auf dieser Zeile → bis Zeilenende + Folgezeile
String boldText = text.substring(pos + 2);
if (!boldText.isEmpty()) out.add(new Seg(SegType.BOLD, boldText));
open = SegType.BOLD;
pos = text.length();
}
continue;
}
// *italic* (not part of **)
if (text.charAt(pos) == '*'
&& (pos + 1 >= text.length() || text.charAt(pos + 1) != '*')) {
int end = text.indexOf('*', pos + 1);
if (end > pos && (end + 1 >= text.length() || text.charAt(end + 1) != '*')) {
int end = findItalicClose(text, pos + 1);
if (end >= 0) {
out.add(new Seg(SegType.ITALIC, text.substring(pos + 1, end)));
pos = end + 1;
continue;
} else {
// kein schließendes * auf dieser Zeile → bis Zeilenende + Folgezeile
String italicText = text.substring(pos + 1);
if (!italicText.isEmpty()) out.add(new Seg(SegType.ITALIC, italicText));
open = SegType.ITALIC;
pos = text.length();
}
continue;
}
// Bekanntes Report-Emoji
int cp = text.codePointAt(pos);
@@ -124,9 +179,16 @@ public class MarkdownExportService {
}
if (pos > start) out.add(new Seg(SegType.PLAIN, text.substring(start, pos)));
}
openFormatRef[0] = open;
return out;
}
/** Überladung ohne Zustandsverfolgung (für Tabellenzellen und isolierte Aufrufe). */
static List<Seg> parseSegments(String text) {
return parseSegments(text, new SegType[]{null});
}
// =========================================================================
// DOCX-Export via Apache POI
// =========================================================================
@@ -154,22 +216,30 @@ public class MarkdownExportService {
try (XWPFDocument doc = new XWPFDocument()) {
String[] lines = markdown.replace("\r", "").split("\n", -1);
int i = 0;
SegType[] inlineState = {null}; // zeilenübergreifendes Bold/Italic-State
while (i < lines.length) {
String line = lines[i];
if (line.startsWith("#### ")) {
inlineState[0] = null;
addHeading(doc, line.substring(5).trim(), 4);
} else if (line.startsWith("### ")) {
inlineState[0] = null;
addHeading(doc, line.substring(4).trim(), 3);
} else if (line.startsWith("## ")) {
inlineState[0] = null;
addHeading(doc, line.substring(3).trim(), 2);
} else if (line.startsWith("# ")) {
inlineState[0] = null;
addHeading(doc, line.substring(2).trim(), 1);
} else if (line.equals("---")) {
inlineState[0] = null;
addHorizontalRule(doc);
} else if (line.startsWith("> ")) {
inlineState[0] = null;
addBlockquote(doc, line.substring(2).trim());
} else if (line.startsWith("|")) {
inlineState[0] = null;
// Tabellenzeilen sammeln bis die Pipe-Sequenz endet
List<String[]> rows = new ArrayList<>();
while (i < lines.length && lines[i].startsWith("|")) {
@@ -185,9 +255,10 @@ public class MarkdownExportService {
if (!rows.isEmpty()) addTable(doc, rows);
continue; // i wurde schon inkrementiert
} else if (!line.isBlank()) {
addParagraph(doc, line);
addParagraph(doc, line, inlineState);
} else {
inlineState[0] = null; // Leerzeile = Absatzgrenze → Format zurücksetzen
}
// Leerzeile erzeugt implizit Abstand durch fehlenden Paragraph-Inhalt
i++;
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
@@ -215,21 +286,21 @@ public class MarkdownExportService {
run.setFontSize(switch (level) { case 1 -> 22; case 2 -> 18; case 3 -> 14; default -> 12; });
}
/**
* Fügt einen normalen Absatz mit Inline-Formatierung hinzu.
* Delegiert an renderInline(), das Bold/Italic/Emoji korrekt als separate Runs setzt.
*/
/** Fügt einen normalen Absatz hinzu; state wird für zeilenübergreifende Formatierung mitgeführt. */
private void addParagraph(XWPFDocument doc, String text, SegType[] state) {
renderInline(doc.createParagraph(), text, state);
}
private void addParagraph(XWPFDocument doc, String text) {
renderInline(doc.createParagraph(), text);
}
/**
* Rendert Inline-Formatierung in einen vorhandenen Paragraph.
* parseSegments() liefert die Segmente; jedes bekommt einen eigenen XWPFRun
* mit setBold()/setItalic(). Emoji-Codepoints landen als Rohtext (Word rendert sie nativ).
* state wird an parseSegments() weitergereicht und hält das über Zeilen offene Format.
*/
private void renderInline(XWPFParagraph p, String text) {
for (Seg seg : parseSegments(text)) {
private void renderInline(XWPFParagraph p, String text, SegType[] state) {
for (Seg seg : parseSegments(text, state)) {
XWPFRun run = p.createRun();
run.setText(seg.text());
if (seg.type() == SegType.BOLD) run.setBold(true);
@@ -237,6 +308,11 @@ public class MarkdownExportService {
}
}
/** Überladung ohne Zustandsverfolgung (für Tabellenzellen). */
private void renderInline(XWPFParagraph p, String text) {
renderInline(p, text, new SegType[]{null});
}
/**
* Fügt eine visuelle Trennlinie ein (Dashes-Zeichen, kein echtes OOXML-Border).
*/
@@ -317,13 +393,14 @@ public class MarkdownExportService {
private static final float TEXT_WIDTH = PAGE_WIDTH - 2 * MARGIN;
/**
* Erzeugt ein PDF aus dem Markdown-String.
* \r wird vorab entfernt (Oracle CLOBs liefern \r\n; PDType1Font akzeptiert
* nur Latin-1 0x20–0xFF, \r würde eine Exception werfen).
* Erzeugt ein PDF aus dem Markdown-String via PDFBox.
* Emoji werden als farbige Java2D-Icons eingebettet (via PdfWriter.emojiImage()).
* \r wird vorab entfernt (Oracle CLOBs liefern \r\n-Zeilenenden).
*/
public byte[] toPdf(String markdown) throws Exception {
try (PDDocument doc = new PDDocument()) {
new PdfWriter(doc).write(markdown.replace("\r", ""));
PdfWriter writer = new PdfWriter(doc);
writer.write(markdown.replace("\r", ""));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
return baos.toByteArray();
@@ -337,6 +414,12 @@ public class MarkdownExportService {
PDPageContentStream cs; // aktiver Content-Stream der aktuellen Seite
float y; // aktuelle y-Schreibposition (oben nach unten)
final PDType1Font fNorm, fBold, fItalic;
// Statischer Cache: "codepoint@px" → BufferedImage (JVM-weit, überlebt mehrere PDFs)
private static final java.util.concurrent.ConcurrentHashMap<String, BufferedImage>
BUFFERED_IMAGE_CACHE = new java.util.concurrent.ConcurrentHashMap<>();
// Pro-Dokument-Cache: "codepoint@px" → PDImageXObject (PDDocument-gebunden)
private final java.util.Map<String, PDImageXObject> emojiCache = new java.util.HashMap<>();
int emojiCacheMisses = 0;
PdfWriter(PDDocument doc) throws Exception {
this.doc = doc;
@@ -356,6 +439,7 @@ public class MarkdownExportService {
doc.addPage(p);
cs = new PDPageContentStream(doc, p);
y = PAGE_HEIGHT - MARGIN;
LOG.infof("PDF: neue Seite %d", doc.getPages().getCount());
}
/**
@@ -390,88 +474,122 @@ public class MarkdownExportService {
PDImageXObject emojiImage(String emoji, float sizePt) throws Exception {
int px = Math.max(24, (int)(sizePt * 3));
int pad = Math.max(1, px / 16);
BufferedImage img = new BufferedImage(px + pad * 2, px + pad * 2,
BufferedImage.TYPE_INT_ARGB);
Graphics2D g = img.createGraphics();
g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
g.setRenderingHint(RenderingHints.KEY_TEXT_ANTIALIASING,
RenderingHints.VALUE_TEXT_ANTIALIAS_ON);
g.setRenderingHint(RenderingHints.KEY_STROKE_CONTROL,
RenderingHints.VALUE_STROKE_PURE);
String cacheKey = emoji.codePointAt(0) + "@" + px;
float cx = pad + px / 2f;
float cy = pad + px / 2f;
float r = px / 2f - 0.5f;
// Level 1: per-document PDImageXObject cache (PDDocument-gebunden)
PDImageXObject cached = emojiCache.get(cacheKey);
if (cached != null) return cached;
long tEmoji = System.currentTimeMillis();
emojiCacheMisses++;
// Level 2: JVM-weiter BufferedImage-Cache (überlebt mehrere Dokumente)
// Lambda darf keine checked Exceptions werfen → alles per Java2D-Shapes, kein AWT-Font
int cp = emoji.isEmpty() ? 0 : emoji.codePointAt(0);
BufferedImage img = BUFFERED_IMAGE_CACHE.computeIfAbsent(cacheKey, k -> {
BufferedImage bi = new BufferedImage(px + pad * 2, px + pad * 2,
BufferedImage.TYPE_INT_ARGB);
Graphics2D g = bi.createGraphics();
g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
g.setRenderingHint(RenderingHints.KEY_STROKE_CONTROL, RenderingHints.VALUE_STROKE_PURE);
switch (cp) {
case 0x2705 -> { // ✅ grüner Kreis + weißes Häkchen
g.setColor(new Color(52, 168, 83));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
g.setColor(Color.WHITE);
g.setStroke(new BasicStroke(px / 6f,
BasicStroke.CAP_ROUND, BasicStroke.JOIN_ROUND));
g.drawPolyline(
new int[]{(int)(cx - r*.42f), (int)(cx - r*.08f), (int)(cx + r*.42f)},
new int[]{(int)(cy + r*.05f), (int)(cy + r*.42f), (int)(cy - r*.32f)},
3);
float cx = pad + px / 2f;
float cy = pad + px / 2f;
float r = px / 2f - 0.5f;
switch (cp) {
case 0x2705 -> { // ✅ grüner Kreis + weißes Häkchen
g.setColor(new Color(52, 168, 83));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
g.setColor(Color.WHITE);
g.setStroke(new BasicStroke(px / 6f,
BasicStroke.CAP_ROUND, BasicStroke.JOIN_ROUND));
g.drawPolyline(
new int[]{(int)(cx - r*.42f), (int)(cx - r*.08f), (int)(cx + r*.42f)},
new int[]{(int)(cy + r*.05f), (int)(cy + r*.42f), (int)(cy - r*.32f)},
3);
}
case 0x274C -> { // ❌ roter Kreis + weißes X
g.setColor(new Color(234, 67, 53));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
g.setColor(Color.WHITE);
g.setStroke(new BasicStroke(px / 5.5f,
BasicStroke.CAP_ROUND, BasicStroke.JOIN_ROUND));
float d = r * 0.34f;
g.drawLine((int)(cx-d),(int)(cy-d),(int)(cx+d),(int)(cy+d));
g.drawLine((int)(cx+d),(int)(cy-d),(int)(cx-d),(int)(cy+d));
}
case 0x26A0 -> { // ⚠️ gelbes Dreieck + ! (Shape, kein AWT-Font)
g.setColor(new Color(251, 188, 4));
g.fillPolygon(
new int[]{(int)cx, pad + px, pad},
new int[]{pad, pad + px, pad + px}, 3);
g.setColor(new Color(50, 50, 50));
// "!" als Shape: Balken (fillRect) + Punkt (Ellipse)
float barW = Math.max(2f, px * 0.10f);
float barH = px * 0.32f;
float dotD = Math.max(2f, px * 0.11f);
g.fill(new java.awt.geom.RoundRectangle2D.Float(
cx - barW / 2f, pad + px * 0.37f, barW, barH, barW / 2, barW / 2));
g.fill(new Ellipse2D.Float(
cx - dotD / 2f, pad + px * 0.77f, dotD, dotD));
}
case 0x2753 -> { // ❓ roter Kreis + weißes ? (Shape, kein AWT-Font)
g.setColor(new Color(234, 67, 53));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
g.setColor(Color.WHITE);
float sw = Math.max(1.5f, px / 7.5f);
g.setStroke(new BasicStroke(sw, BasicStroke.CAP_ROUND, BasicStroke.JOIN_ROUND));
// "?" Haken via quadratische Bézierkurven
float x0 = pad, y0 = pad;
java.awt.geom.Path2D.Float path = new java.awt.geom.Path2D.Float();
path.moveTo(x0 + px * 0.28f, y0 + px * 0.26f); // linkes offenes Ende
path.quadTo(x0 + px * 0.26f, y0 + px * 0.12f,
x0 + px * 0.50f, y0 + px * 0.12f); // nach oben-mitte
path.quadTo(x0 + px * 0.74f, y0 + px * 0.12f,
x0 + px * 0.74f, y0 + px * 0.30f); // über die Spitze rechts
path.quadTo(x0 + px * 0.74f, y0 + px * 0.46f,
x0 + px * 0.56f, y0 + px * 0.50f); // rechte Seite einwärts
path.quadTo(x0 + px * 0.50f, y0 + px * 0.52f,
x0 + px * 0.50f, y0 + px * 0.62f); // Schaft nach unten
g.draw(path);
float dotD2 = Math.max(2f, px * 0.10f);
g.fill(new Ellipse2D.Float(
x0 + px * 0.50f - dotD2 / 2f, y0 + px * 0.78f, dotD2, dotD2));
}
case 0x1F4C4 -> { // 📄 blaues Dokument
int dw = (int)(px * .70f), dh = (int)(px * .90f);
int dx = (int)(cx - dw / 2f), dy = pad + (int)(px * .05f);
int fold = (int)(px * .22f);
g.setColor(new Color(66, 133, 244));
g.fillPolygon(
new int[]{dx, dx + dw - fold, dx + dw, dx + dw, dx},
new int[]{dy, dy, dy + fold, dy + dh, dy + dh}, 5);
g.setColor(new Color(30, 90, 200));
g.fillPolygon(
new int[]{dx + dw - fold, dx + dw - fold, dx + dw},
new int[]{dy, dy + fold, dy + fold}, 3);
g.setColor(Color.WHITE);
int lw = (int)(dw * .56f), lx = dx + (int)(dw * .18f);
int lh = Math.max(1, (int)(dh * .08f)), gap = (int)(dh * .13f);
int ly = dy + (int)(dh * .38f);
for (int l = 0; l < 3; l++) g.fillRect(lx, ly + l * gap, lw, lh);
}
default -> { // Unbekannter Codepoint: grauer Kreis (Fallback)
g.setColor(new Color(158, 158, 158));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
}
}
case 0x274C -> { // ❌ roter Kreis + weißes X
g.setColor(new Color(234, 67, 53));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
g.setColor(Color.WHITE);
g.setStroke(new BasicStroke(px / 5.5f,
BasicStroke.CAP_ROUND, BasicStroke.JOIN_ROUND));
float d = r * 0.34f;
g.drawLine((int)(cx-d),(int)(cy-d),(int)(cx+d),(int)(cy+d));
g.drawLine((int)(cx+d),(int)(cy-d),(int)(cx-d),(int)(cy+d));
}
case 0x26A0 -> { // ⚠️ gelbes Dreieck + !
g.setColor(new Color(251, 188, 4));
g.fillPolygon(
new int[]{(int)cx, pad + px, pad},
new int[]{pad, pad + px, pad + px}, 3);
g.setColor(new Color(50, 50, 50));
g.setFont(new java.awt.Font("Dialog", java.awt.Font.BOLD, (int)(px * .42f)));
FontMetrics fm = g.getFontMetrics();
g.drawString("!", (int)(cx - fm.stringWidth("!") / 2f),
(int)(pad + px * .82f));
}
case 0x2753 -> { // ❓ roter Kreis + weißes ?
g.setColor(new Color(234, 67, 53));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
g.setColor(Color.WHITE);
g.setFont(new java.awt.Font("Dialog", java.awt.Font.BOLD, (int)(px * .55f)));
FontMetrics fm = g.getFontMetrics();
g.drawString("?", (int)(cx - fm.stringWidth("?") / 2f),
(int)(cy + px * .20f));
}
case 0x1F4C4 -> { // 📄 blaues Dokument
int dw = (int)(px * .70f), dh = (int)(px * .90f);
int dx = (int)(cx - dw / 2f), dy = pad + (int)(px * .05f);
int fold = (int)(px * .22f);
g.setColor(new Color(66, 133, 244));
g.fillPolygon(
new int[]{dx, dx + dw - fold, dx + dw, dx + dw, dx},
new int[]{dy, dy, dy + fold, dy + dh, dy + dh}, 5);
g.setColor(new Color(30, 90, 200)); // gefaltete Ecke dunkler
g.fillPolygon(
new int[]{dx + dw - fold, dx + dw - fold, dx + dw},
new int[]{dy, dy + fold, dy + fold}, 3);
g.setColor(Color.WHITE); // drei weiße Textzeilen
int lw = (int)(dw * .56f), lx = dx + (int)(dw * .18f);
int lh = Math.max(1, (int)(dh * .08f)), gap = (int)(dh * .13f);
int ly = dy + (int)(dh * .38f);
for (int l = 0; l < 3; l++) g.fillRect(lx, ly + l * gap, lw, lh);
}
default -> { // Unbekannter Codepoint: grauer Kreis (Fallback)
g.setColor(new Color(158, 158, 158));
g.fill(new Ellipse2D.Float(pad, pad, px, px));
}
}
g.dispose();
// LosslessFactory = verlustfreie PNG-Einbettung ins PDF
return LosslessFactory.createFromImage(doc, img);
g.dispose();
return bi;
});
// BufferedImage → PDImageXObject (verlustfreie PNG-Einbettung)
PDImageXObject xobj = LosslessFactory.createFromImage(doc, img);
emojiCache.put(cacheKey, xobj);
LOG.infof("PDF: emojiImage('%s' U+%04X, %dpx) neu gerendert in %dms",
cacheKey, emoji.codePointAt(0), px, System.currentTimeMillis() - tEmoji);
return xobj;
}
/**
@@ -551,9 +669,9 @@ public class MarkdownExportService {
*
* indent: horizontaler Einzug in Punkten (z.B. 20 für Blockquotes).
*/
void renderLine(String text, PDType1Font defaultFont, float size, float indent)
throws Exception {
List<Seg> segs = parseSegments(text);
void renderLine(String text, PDType1Font defaultFont, float size, float indent,
SegType[] state) throws Exception {
List<Seg> segs = parseSegments(text, state);
boolean onlyPlain = segs.stream().allMatch(s -> s.type() == SegType.PLAIN);
if (onlyPlain) {
writeWrappedText(defaultFont, size, indent, text);
@@ -585,6 +703,12 @@ public class MarkdownExportService {
if (!line.isEmpty()) flushInlineLine(line, size, emojiPt, indent);
}
/** Überladung ohne Zustandsverfolgung (für Blockquotes, Tabellenzeilen). */
void renderLine(String text, PDType1Font defaultFont, float size, float indent)
throws Exception {
renderLine(text, defaultFont, size, indent, new SegType[]{null});
}
/**
* Schreibt eine fertig zusammengestellte Inline-Zeile auf den PDF-Stream.
* Berechnet die Zeilenhöhe als Maximum aus LINE_HEIGHT und Emoji-Größe,
@@ -728,8 +852,21 @@ public class MarkdownExportService {
void write(String markdown) throws Exception {
String[] lines = markdown.split("\n", -1);
List<String[]> tableAccum = new ArrayList<>();
int lineNr = 0;
long tWrite = System.currentTimeMillis();
SegType[] inlineState = {null}; // zeilenübergreifendes Bold/Italic-State
for (String line : lines) {
lineNr++;
if (lineNr % 10 == 0) {
LOG.infof("PDF: write() Zeile %d/%d – %d Seiten – %dms bisher",
lineNr, lines.length, doc.getPages().getCount(),
System.currentTimeMillis() - tWrite);
}
if (lineNr >= lines.length - 20) {
LOG.infof("PDF: Zeile %d: [%s]", lineNr,
line.length() > 120 ? line.substring(0, 120) + "…" : line);
}
// Tabellenpuffer flushen sobald eine Nicht-Pipe-Zeile kommt
if (!line.startsWith("|") && !tableAccum.isEmpty()) {
renderTable(tableAccum);
@@ -737,14 +874,19 @@ public class MarkdownExportService {
}
if (line.startsWith("#### ")) {
inlineState[0] = null;
skip(10); renderLine(line.substring(5).trim(), fBold, 11, 0); skip(4);
} else if (line.startsWith("### ")) {
inlineState[0] = null;
skip(20); renderLine(line.substring(4).trim(), fBold, 13, 0); skip(6);
} else if (line.startsWith("## ")) {
inlineState[0] = null;
skip(28); renderLine(line.substring(3).trim(), fBold, 15, 0); skip(8);
} else if (line.startsWith("# ")) {
inlineState[0] = null;
skip(20); renderLine(line.substring(2).trim(), fBold, 18, 0); skip(1);
} else if (line.equals("---")) {
inlineState[0] = null;
skip(4);
ensureSpace(2);
cs.setLineWidth(0.5f);
@@ -753,8 +895,10 @@ public class MarkdownExportService {
cs.stroke();
y -= 6;
} else if (line.startsWith("> ")) {
inlineState[0] = null;
renderLine(line.substring(2).trim(), fItalic, 10, 20);
} else if (line.startsWith("|")) {
inlineState[0] = null;
if (!isTableSeparator(line)) {
String[] parts = line.split("\\|", -1);
List<String> cells = new ArrayList<>();
@@ -763,13 +907,23 @@ public class MarkdownExportService {
if (!cells.isEmpty()) tableAccum.add(cells.toArray(new String[0]));
}
} else if (line.isBlank()) {
inlineState[0] = null; // Leerzeile = Absatzgrenze → Format zurücksetzen
skip(4);
} else {
renderLine(line, fNorm, 11, 0);
renderLine(line, fNorm, 11, 0, inlineState);
}
}
if (!tableAccum.isEmpty()) renderTable(tableAccum);
if (!tableAccum.isEmpty()) {
LOG.infof("PDF: renderTable() wird aufgerufen (%d Zeilen)", tableAccum.size());
renderTable(tableAccum);
LOG.infof("PDF: renderTable() fertig");
}
LOG.infof("PDF: cs.close() wird aufgerufen (Seite %d)", doc.getPages().getCount());
cs.close();
LOG.infof("PDF: cs.close() fertig");
LOG.infof("PDF: write() fertig – %d Zeilen, %d Seiten, %dms",
lines.length, doc.getPages().getCount(),
System.currentTimeMillis() - tWrite);
}
}
}

260
dc-backend/src/main/java/de/frigosped/dc/service/MarkdownReportBuilder.java Normal file
View File

@@ -0,0 +1,260 @@
package de.frigosped.dc.service;
import de.frigosped.dc.model.*;
import jakarta.enterprise.context.ApplicationScoped;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.*;
import java.util.stream.Collectors;
@ApplicationScoped
public class MarkdownReportBuilder {
private static final String ICON_OK = "✅"; // ✅
private static final String ICON_NOK = "❌"; // ❌
private static final String ICON_WARN = "⚠️"; // ⚠️
private static final String ICON_UNKL = "❓"; // ❓
private static final String ICON_DOC = "📄"; // 📄
private static final DateTimeFormatter DATE_FMT =
DateTimeFormatter.ofPattern("dd.MM.yyyy HH:mm");
/**
* Generates a Markdown report equivalent to DC_UTILS_PKG.generate_report().
* @param project project meta (name, notify flags, completedAt)
* @param documents ordered list of project documents
* @param questionsByCatalog map catalogId → sorted questions
* @param resultsByDoc map docId → list of saved results
*/
public String build(OrdsProject project,
List<OrdsDocument> documents,
Map<Long, List<OrdsQuestion>> questionsByCatalog,
Map<Long, List<ResultRequest>> resultsByDoc) {
boolean showOk = !"N".equals(project.getNotifyOk());
boolean showWarn = !"N".equals(project.getNotifyHinweis());
boolean showDev = !"N".equals(project.getNotifyAbweichend());
// Flat lookup: questionId → question
Map<Long, OrdsQuestion> questionById = new HashMap<>();
for (List<OrdsQuestion> qs : questionsByCatalog.values()) {
for (OrdsQuestion q : qs) {
questionById.put(q.getQuestionId(), q);
}
}
// Global counts (total_all unfiltered, filtered for summary section)
int totalAll = 0, devCnt = 0, warnCnt = 0, unklarCnt = 0;
for (List<ResultRequest> list : resultsByDoc.values()) {
for (ResultRequest r : list) {
totalAll++;
if (isDeviation(r)) devCnt++;
else if (isWarning(r)) warnCnt++;
else if (isUnklar(r)) unklarCnt++;
}
}
int okCnt = totalAll - devCnt - warnCnt - unklarCnt;
// Filtered counts matching notify flags (same logic as PL/SQL)
int filtTotal = (showDev ? devCnt : 0)
+ (showWarn ? warnCnt : 0)
+ (showOk ? (okCnt + unklarCnt) : 0);
int filtDevCnt = showDev ? devCnt : 0;
int filtWarnCnt = showWarn ? warnCnt : 0;
int filtUnklCnt = showOk ? unklarCnt : 0;
int filtOkCnt = filtTotal - filtDevCnt - filtWarnCnt - filtUnklCnt;
StringBuilder sb = new StringBuilder();
// Header
sb.append("# Prüfbericht: ").append(project.getName()).append("\n\n");
LocalDateTime ts = project.getCompletedAt() != null
? project.getCompletedAt() : LocalDateTime.now();
sb.append("**Abgeschlossen:** ").append(ts.format(DATE_FMT)).append("\n\n");
sb.append("---\n\n");
// Summary
sb.append("## Zusammenfassung\n\n");
if (showOk) {
sb.append(ICON_OK).append(" OK: ").append(filtOkCnt).append("\n");
if (filtUnklCnt > 0) {
sb.append(ICON_UNKL).append(" Unklar: ").append(filtUnklCnt).append("\n");
}
}
if (showWarn) sb.append(ICON_WARN).append(" Hinweise: ").append(filtWarnCnt).append("\n");
if (showDev) sb.append(ICON_NOK).append(" Abweichungen: ").append(filtDevCnt).append("\n");
sb.append("**Gesamt: ").append(totalAll).append("**\n\n");
sb.append("---\n\n");
// Results per document
sb.append("## Ergebnisse\n\n");
for (OrdsDocument doc : documents) {
sb.append("## ").append(ICON_DOC).append(" ").append(doc.getFilename()).append("\n\n");
List<ResultRequest> docResults = resultsByDoc.getOrDefault(doc.getId(), Collections.emptyList());
// Per-doc counts
int dDev = 0, dWarn = 0, dUnkl = 0;
for (ResultRequest r : docResults) {
if (isDeviation(r)) dDev++;
else if (isWarning(r)) dWarn++;
else if (isUnklar(r)) dUnkl++;
}
int dAll = docResults.size();
int dOk = dAll - dDev - dWarn - dUnkl;
if (showOk) {
sb.append(ICON_OK).append(" OK: ").append(dOk).append("\n");
if (dUnkl > 0) sb.append(ICON_UNKL).append(" Unklar: ").append(dUnkl).append("\n");
}
if (showWarn) sb.append(ICON_WARN).append(" Hinweise: ").append(dWarn).append("\n");
if (showDev) sb.append(ICON_NOK).append(" Abweichungen: ").append(dDev).append("\n");
sb.append("**Gesamt: ").append(dAll).append("**\n\n");
sb.append("---\n\n");
// Results filtered + sorted by category order, then question order
List<ResultRequest> visible = docResults.stream()
.filter(r -> isVisible(r, showOk, showWarn, showDev))
.sorted(Comparator
.comparingInt((ResultRequest r) -> catOrder(r, questionById))
.thenComparingInt(r -> qOrder(r, questionById)))
.collect(Collectors.toList());
Long currentCatId = null;
for (ResultRequest r : visible) {
OrdsQuestion q = questionById.get(r.getQuestionId());
if (q == null) continue;
if (!Objects.equals(q.getCategoryId(), currentCatId)) {
currentCatId = q.getCategoryId();
String catName = q.getCategoryName() != null ? q.getCategoryName() : "Allgemein";
sb.append("\n### ").append(catName).append("\n\n");
}
String icon = icon(r);
sb.append("\n\n").append(icon).append(" **").append(q.getQuestionText()).append("**\n\n");
if (r.getAnswer() != null && !r.getAnswer().isBlank()) {
sb.append(r.getAnswer()).append("\n\n");
}
if (r.getScore() != null) {
sb.append("*Score: ").append(r.getScore()).append("%*\n");
}
if (r.getTheComment() != null && !r.getTheComment().isBlank()) {
sb.append("\n> ").append(r.getTheComment()).append("\n");
}
sb.append("\n");
}
sb.append("---\n\n");
}
return sb.toString();
}
/**
* Generates a Markdown report for a single document.
*/
public String buildForDocument(OrdsProject project,
OrdsDocument doc,
Map<Long, OrdsQuestion> questionById,
List<ResultRequest> docResults) {
boolean showOk = !"N".equals(project.getNotifyOk());
boolean showWarn = !"N".equals(project.getNotifyHinweis());
boolean showDev = !"N".equals(project.getNotifyAbweichend());
int dDev = 0, dWarn = 0, dUnkl = 0;
for (ResultRequest r : docResults) {
if (isDeviation(r)) dDev++;
else if (isWarning(r)) dWarn++;
else if (isUnklar(r)) dUnkl++;
}
int dAll = docResults.size();
int dOk = dAll - dDev - dWarn - dUnkl;
StringBuilder sb = new StringBuilder();
sb.append("# Prüfbericht: ").append(project.getName()).append("\n\n");
LocalDateTime ts = project.getCompletedAt() != null
? project.getCompletedAt() : LocalDateTime.now();
sb.append("**Abgeschlossen:** ").append(ts.format(DATE_FMT)).append("\n\n");
sb.append("---\n\n");
sb.append("## ").append(ICON_DOC).append(" ").append(doc.getFilename()).append("\n\n");
if (showOk) {
sb.append(ICON_OK).append(" OK: ").append(dOk).append("\n");
if (dUnkl > 0) sb.append(ICON_UNKL).append(" Unklar: ").append(dUnkl).append("\n");
}
if (showWarn) sb.append(ICON_WARN).append(" Hinweise: ").append(dWarn).append("\n");
if (showDev) sb.append(ICON_NOK).append(" Abweichungen: ").append(dDev).append("\n");
sb.append("**Gesamt: ").append(dAll).append("**\n\n");
sb.append("---\n\n");
List<ResultRequest> visible = docResults.stream()
.filter(r -> isVisible(r, showOk, showWarn, showDev))
.sorted(Comparator
.comparingInt((ResultRequest r) -> catOrder(r, questionById))
.thenComparingInt(r -> qOrder(r, questionById)))
.collect(Collectors.toList());
Long currentCatId = null;
for (ResultRequest r : visible) {
OrdsQuestion q = questionById.get(r.getQuestionId());
if (q == null) continue;
if (!Objects.equals(q.getCategoryId(), currentCatId)) {
currentCatId = q.getCategoryId();
String catName = q.getCategoryName() != null ? q.getCategoryName() : "Allgemein";
sb.append("\n### ").append(catName).append("\n\n");
}
String icon = icon(r);
sb.append("\n\n").append(icon).append(" **").append(q.getQuestionText()).append("**\n\n");
if (r.getAnswer() != null && !r.getAnswer().isBlank()) {
sb.append(r.getAnswer()).append("\n\n");
}
if (r.getScore() != null) {
sb.append("*Score: ").append(r.getScore()).append("%*\n");
}
if (r.getTheComment() != null && !r.getTheComment().isBlank()) {
sb.append("\n> ").append(r.getTheComment()).append("\n");
}
sb.append("\n");
}
sb.append("---\n\n");
return sb.toString();
}
private boolean isDeviation(ResultRequest r) { return Integer.valueOf(1).equals(r.getDeviation()); }
private boolean isWarning(ResultRequest r) { return Integer.valueOf(1).equals(r.getWarning()); }
private boolean isUnklar(ResultRequest r) { return "UNKLAR".equals(r.getResultType()) && !isDeviation(r) && !isWarning(r); }
private boolean isVisible(ResultRequest r, boolean showOk, boolean showWarn, boolean showDev) {
if (isDeviation(r)) return showDev;
if (isWarning(r)) return showWarn;
return showOk;
}
private String icon(ResultRequest r) {
if (isDeviation(r)) return ICON_NOK;
if (isWarning(r)) return ICON_WARN;
if (isUnklar(r)) return ICON_UNKL;
return ICON_OK;
}
private int catOrder(ResultRequest r, Map<Long, OrdsQuestion> byId) {
OrdsQuestion q = byId.get(r.getQuestionId());
return (q != null && q.getCategoryOrderNr() != null) ? q.getCategoryOrderNr() : Integer.MAX_VALUE;
}
private int qOrder(ResultRequest r, Map<Long, OrdsQuestion> byId) {
OrdsQuestion q = byId.get(r.getQuestionId());
return (q != null && q.getOrderNr() != null) ? q.getOrderNr() : Integer.MAX_VALUE;
}
}

281
dc-backend/src/main/java/de/frigosped/dc/service/ReferenceDownloadService.java Normal file
View File

@@ -0,0 +1,281 @@
package de.frigosped.dc.service;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import org.eclipse.microprofile.config.inject.ConfigProperty;
import org.jboss.logging.Logger;
import org.jsoup.Jsoup;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
/**
* Lädt externe Referenz-Dokumente sicher herunter und extrahiert deren Text.
*
* Sicherheit (SSRF): Es werden ausschließlich öffentlich auflösbare Domains
* akzeptiert. Der Hostname wird per DNS aufgelöst und jede resultierende IP
* gegen private/Loopback/Link-Local/CGNAT/ULA-Bereiche geprüft. Redirects
* werden manuell verfolgt und bei jedem Hop neu validiert. Größe und Timeout
* sind begrenzt.
*
* Text-Extraktion:
* - PDF/DOCX/ODT/Plain → {@link DocumentProcessingService#extractText}
* - text/html → jsoup → Text
*/
@ApplicationScoped
public class ReferenceDownloadService {
private static final Logger LOG = Logger.getLogger(ReferenceDownloadService.class);
private static final int MAX_REDIRECTS = 5;
@Inject
DocumentProcessingService documentProcessingService;
@ConfigProperty(name = "dc.references.enabled", defaultValue = "true")
boolean enabled;
@ConfigProperty(name = "dc.references.max-size-bytes", defaultValue = "15000000")
long maxSizeBytes;
@ConfigProperty(name = "dc.references.timeout", defaultValue = "30s")
Duration timeout;
@ConfigProperty(name = "dc.references.allowed-content-types",
defaultValue = "application/pdf,"
+ "application/vnd.openxmlformats-officedocument.wordprocessingml.document,"
+ "application/msword,"
+ "application/vnd.oasis.opendocument.text,"
+ "text/plain,text/markdown,text/html")
String allowedContentTypes;
/** Ergebnis eines erfolgreichen Referenz-Downloads. */
public record DownloadedReference(
byte[] bytes, String mimeType, String text, String filename, String sourceUrl) {}
public boolean isEnabled() {
return enabled;
}
/**
* Lädt eine URL herunter und extrahiert den Text. Bei jedem Fehler (ungültige
* URL, SSRF-Blockade, Timeout, zu groß, nicht erlaubter Typ, leerer Text) wird
* {@link Optional#empty()} zurückgegeben – die Verarbeitung läuft weiter.
*/
public Optional<DownloadedReference> fetch(String rawUrl, Long projectId) {
if (!enabled) return Optional.empty();
String url = normalize(rawUrl);
if (url == null) {
LOG.debugf("Referenz übersprungen – ungültige URL: %s", rawUrl);
return Optional.empty();
}
try {
HttpClient client = HttpClient.newBuilder()
.followRedirects(HttpClient.Redirect.NEVER)
.connectTimeout(timeout)
.build();
URI current = URI.create(url);
HttpResponse<InputStream> response = null;
for (int hop = 0; hop <= MAX_REDIRECTS; hop++) {
if (!isPublicHost(current.getHost())) {
LOG.warnf("Referenz blockiert (SSRF-Schutz): %s", current);
return Optional.empty();
}
HttpRequest request = HttpRequest.newBuilder(current)
.timeout(timeout)
.header("User-Agent", "Frigosped-DocumentCheck/1.0")
.GET()
.build();
response = client.send(request, HttpResponse.BodyHandlers.ofInputStream());
int status = response.statusCode();
if (status >= 300 && status < 400) {
Optional<String> loc = response.headers().firstValue("location");
if (loc.isEmpty()) break;
current = current.resolve(loc.get()); // relativ oder absolut
response.body().close();
continue;
}
break;
}
if (response == null || response.statusCode() != 200) {
LOG.debugf("Referenz übersprungen – HTTP %s: %s",
response == null ? "?" : response.statusCode(), url);
return Optional.empty();
}
String contentType = response.headers().firstValue("content-type")
.map(ct -> ct.split(";")[0].trim().toLowerCase())
.orElse("application/octet-stream");
if (!isAllowedType(contentType)) {
LOG.debugf("Referenz übersprungen – Content-Type %s nicht erlaubt: %s",
contentType, url);
response.body().close();
return Optional.empty();
}
byte[] bytes = readLimited(response.body(), maxSizeBytes);
if (bytes == null) {
LOG.warnf("Referenz übersprungen – größer als %d Bytes: %s", maxSizeBytes, url);
return Optional.empty();
}
String filename = deriveFilename(current, contentType);
String text = extract(bytes, contentType, filename, current.toString(), projectId);
if (text == null || text.isBlank()) {
LOG.debugf("Referenz übersprungen – kein Text extrahierbar: %s", url);
return Optional.empty();
}
LOG.infof("Referenz geladen: %s (%s, %d Bytes, %d Zeichen)",
url, contentType, bytes.length, text.length());
return Optional.of(new DownloadedReference(bytes, contentType, text, filename, url));
} catch (Exception e) {
LOG.debugf("Referenz-Download fehlgeschlagen (%s): %s", url, e.getMessage());
return Optional.empty();
}
}
// =========================================================================
// Text-Extraktion
// =========================================================================
private String extract(byte[] bytes, String contentType, String filename,
String url, Long projectId) {
if (contentType.equals("text/html")) {
String html = new String(bytes, StandardCharsets.UTF_8);
return Jsoup.parse(html, url).text();
}
return documentProcessingService.extractText(bytes, contentType, filename, projectId);
}
// =========================================================================
// SSRF-Schutz
// =========================================================================
/**
* true, wenn der Host öffentlich auflösbar ist und KEINE der aufgelösten
* IP-Adressen in einem privaten/internen Bereich liegt.
*/
boolean isPublicHost(String host) {
if (host == null || host.isBlank()) return false;
try {
InetAddress[] addresses = InetAddress.getAllByName(host);
if (addresses.length == 0) return false;
for (InetAddress addr : addresses) {
if (!isPublicAddress(addr)) return false;
}
return true;
} catch (Exception e) {
return false; // nicht auflösbar → blockieren
}
}
private boolean isPublicAddress(InetAddress addr) {
if (addr.isLoopbackAddress() || addr.isAnyLocalAddress()
|| addr.isLinkLocalAddress() || addr.isSiteLocalAddress()
|| addr.isMulticastAddress()) {
return false;
}
byte[] b = addr.getAddress();
if (b.length == 4) {
int o0 = b[0] & 0xFF, o1 = b[1] & 0xFF;
if (o0 == 0) return false; // 0.0.0.0/8
if (o0 == 127) return false; // Loopback (Sicherheit)
if (o0 == 100 && o1 >= 64 && o1 <= 127) return false; // CGNAT 100.64.0.0/10
if (o0 == 169 && o1 == 254) return false; // Link-Local
} else if (b.length == 16) {
if ((b[0] & 0xFE) == 0xFC) return false; // IPv6 ULA fc00::/7
}
return true;
}
// =========================================================================
// Hilfsmethoden
// =========================================================================
/** Normalisiert eine vom LLM gelieferte URL (trimmt Markdown/Satzzeichen). */
private String normalize(String raw) {
if (raw == null) return null;
String u = raw.trim();
if (u.startsWith("<") && u.endsWith(">")) u = u.substring(1, u.length() - 1).trim();
// typische nachgestellte Satzzeichen entfernen
while (!u.isEmpty() && ").,;\"'".indexOf(u.charAt(u.length() - 1)) >= 0) {
u = u.substring(0, u.length() - 1);
}
if (!u.regionMatches(true, 0, "http://", 0, 7)
&& !u.regionMatches(true, 0, "https://", 0, 8)) {
return null;
}
return u;
}
/** Liest den Stream bis zum Limit; gibt null zurück wenn das Limit überschritten wird. */
private byte[] readLimited(InputStream in, long limit) throws Exception {
try (in) {
java.io.ByteArrayOutputStream out = new java.io.ByteArrayOutputStream();
byte[] buf = new byte[8192];
long total = 0;
int n;
while ((n = in.read(buf)) != -1) {
total += n;
if (total > limit) return null;
out.write(buf, 0, n);
}
return out.toByteArray();
}
}
private boolean isAllowedType(String contentType) {
List<String> allowed = Arrays.stream(allowedContentTypes.split(","))
.map(String::trim).map(String::toLowerCase).toList();
return allowed.contains(contentType);
}
private String deriveFilename(URI uri, String contentType) {
String path = uri.getPath();
String name = null;
if (path != null && !path.isBlank()) {
int slash = path.lastIndexOf('/');
String last = slash >= 0 ? path.substring(slash + 1) : path;
if (!last.isBlank()) name = last;
}
if (name == null || name.isBlank()) {
name = (uri.getHost() != null ? uri.getHost() : "referenz") + extensionFor(contentType);
} else if (!name.contains(".")) {
name = name + extensionFor(contentType);
}
// Dateinamen säubern
return name.replaceAll("[^A-Za-z0-9_äöüÄÖÜß.-]", "_");
}
private String extensionFor(String contentType) {
return switch (contentType) {
case "application/pdf" -> ".pdf";
case "application/vnd.openxmlformats-officedocument.wordprocessingml.document" -> ".docx";
case "application/msword" -> ".doc";
case "application/vnd.oasis.opendocument.text" -> ".odt";
case "text/html" -> ".html";
case "text/markdown" -> ".md";
default -> ".txt";
};
}
}

24
dc-backend/src/main/resources/application.properties
View File

@@ -64,6 +64,14 @@ dc.ai.catalog.temperature=0.1
dc.ai.ocr.timeout=120s
dc.ai.ocr.max-retries=2
# =============================================================================
# Mistral OCR REST Client
# =============================================================================
quarkus.rest-client.mistral-ocr.url=https://api.mistral.ai
quarkus.rest-client.mistral-ocr.scope=jakarta.inject.Singleton
quarkus.rest-client.mistral-ocr.connect-timeout=10000
quarkus.rest-client.mistral-ocr.read-timeout=120000
# =============================================================================
# ORDS REST Client
# =============================================================================
@@ -71,11 +79,25 @@ dc.ai.ocr.max-retries=2
quarkus.rest-client.ords-api.url=http://ords:8080/ords/ai_dev
quarkus.rest-client.ords-api.scope=jakarta.inject.Singleton
quarkus.rest-client.ords-api.connect-timeout=10000
quarkus.rest-client.ords-api.read-timeout=60000
quarkus.rest-client.ords-api.read-timeout=600000
# ORDS-Authentifizierung (OAuth2 Bearer Token – leer = kein Auth in Dev)
dc.ords.bearer-token=
# =============================================================================
# Externe Referenz-Dokumente (Links im Dokument → herunterladen & mitbewerten)
# =============================================================================
# Feature an/aus
dc.references.enabled=true
# Max. Anzahl heruntergeladener Referenzen pro Hauptdokument
dc.references.max-per-document=5
# Max. Dateigröße pro Referenz (Bytes); größere Downloads werden verworfen
dc.references.max-size-bytes=15000000
# Verbindungs-/Lese-Timeout pro Download
dc.references.timeout=30s
# Erlaubte Content-Types (alles andere wird verworfen)
dc.references.allowed-content-types=application/pdf,application/vnd.openxmlformats-officedocument.wordprocessingml.document,application/msword,application/vnd.oasis.opendocument.text,text/plain,text/markdown,text/html
# =============================================================================
# Logging
# =============================================================================

1
dc-backend/target/build-metrics.json
View File

File diff suppressed because one or more lines are too long

24
dc-backend/target/classes/application.properties
View File

@@ -64,6 +64,14 @@ dc.ai.catalog.temperature=0.1
dc.ai.ocr.timeout=120s
dc.ai.ocr.max-retries=2
# =============================================================================
# Mistral OCR REST Client
# =============================================================================
quarkus.rest-client.mistral-ocr.url=https://api.mistral.ai
quarkus.rest-client.mistral-ocr.scope=jakarta.inject.Singleton
quarkus.rest-client.mistral-ocr.connect-timeout=10000
quarkus.rest-client.mistral-ocr.read-timeout=120000
# =============================================================================
# ORDS REST Client
# =============================================================================
@@ -71,11 +79,25 @@ dc.ai.ocr.max-retries=2
quarkus.rest-client.ords-api.url=http://ords:8080/ords/ai_dev
quarkus.rest-client.ords-api.scope=jakarta.inject.Singleton
quarkus.rest-client.ords-api.connect-timeout=10000
quarkus.rest-client.ords-api.read-timeout=60000
quarkus.rest-client.ords-api.read-timeout=600000
# ORDS-Authentifizierung (OAuth2 Bearer Token – leer = kein Auth in Dev)
dc.ords.bearer-token=
# =============================================================================
# Externe Referenz-Dokumente (Links im Dokument → herunterladen & mitbewerten)
# =============================================================================
# Feature an/aus
dc.references.enabled=true
# Max. Anzahl heruntergeladener Referenzen pro Hauptdokument
dc.references.max-per-document=5
# Max. Dateigröße pro Referenz (Bytes); größere Downloads werden verworfen
dc.references.max-size-bytes=15000000
# Verbindungs-/Lese-Timeout pro Download
dc.references.timeout=30s
# Erlaubte Content-Types (alles andere wird verworfen)
dc.references.allowed-content-types=application/pdf,application/vnd.openxmlformats-officedocument.wordprocessingml.document,application/msword,application/vnd.oasis.opendocument.text,text/plain,text/markdown,text/html
# =============================================================================
# Logging
# =============================================================================

BIN
dc-backend/target/classes/de/frigosped/dc/client/OrdsClient.class
View File

Binary file not shown.

BIN
dc-backend/target/classes/de/frigosped/dc/model/OrdsDocument.class
View File

Binary file not shown.

BIN
dc-backend/target/classes/de/frigosped/dc/model/OrdsProject.class
View File

Binary file not shown.

BIN
dc-backend/target/classes/de/frigosped/dc/service/CheckOrchestrationService.class
View File

Binary file not shown.

BIN
dc-backend/target/classes/de/frigosped/dc/service/DocumentProcessingService.class
View File

Binary file not shown.

BIN
dc-backend/target/classes/de/frigosped/dc/service/EvaluationService.class
View File

Binary file not shown.

43
dc-backend/target/maven-status/maven-compiler-plugin/compile/default-compile/createdFiles.lst
View File

@@ -1,25 +1,50 @@
de/frigosped/dc/model/GeneratedCatalogStructure$GeneratedCategory.class
de/frigosped/dc/model/MistralOcrRequest$Document.class
de/frigosped/dc/resource/ExportResource.class
de/frigosped/dc/model/OrdsQuestionCreateRequest.class
de/frigosped/dc/model/DocumentProgressRequest.class
de/frigosped/dc/model/MistralOcrRequest.class
de/frigosped/dc/model/CompleteProjectRequest.class
de/frigosped/dc/model/OrdsCategoryCreateRequest.class
de/frigosped/dc/ai/AiModelProducer.class
de/frigosped/dc/model/OrdsCatalogCreateRequest.class
de/frigosped/dc/model/OrdsCatalogStatusUpdateRequest.class
de/frigosped/dc/service/EvaluationService.class
de/frigosped/dc/service/MarkdownExportService.class
de/frigosped/dc/model/OrdsDocumentList.class
de/frigosped/dc/service/DocumentProcessingService.class
de/frigosped/dc/model/CatalogGenerateResponse.class
de/frigosped/dc/resource/CheckResource.class
de/frigosped/dc/service/MarkdownExportService$Seg.class
de/frigosped/dc/model/AiCostRequest.class
de/frigosped/dc/model/OrdsCatalogStatusResponse.class
de/frigosped/dc/client/OrdsClient.class
de/frigosped/dc/service/CheckOrchestrationService.class
de/frigosped/dc/model/GeneratedCatalogStructure.class
de/frigosped/dc/client/OrdsLoggingFilter.class
de/frigosped/dc/model/ResultRequest.class
de/frigosped/dc/service/ReferenceDownloadService.class
de/frigosped/dc/service/MarkdownReportBuilder.class
de/frigosped/dc/model/OrdsQuestion.class
de/frigosped/dc/client/MistralOcrClient.class
de/frigosped/dc/model/OrdsQuestionList.class
de/frigosped/dc/model/OrdsQuestionCreateRequest.class
de/frigosped/dc/model/ProgressRequest.class
de/frigosped/dc/model/OrdsDocument.class
de/frigosped/dc/model/OrdsCatalogStatusUpdateRequest.class
de/frigosped/dc/model/MistralOcrResponse.class
de/frigosped/dc/service/EvaluationService.class
de/frigosped/dc/model/OrdsProject.class
de/frigosped/dc/service/MarkdownExportService.class
de/frigosped/dc/model/CatalogGenerateResponse.class
de/frigosped/dc/model/TextsRequest.class
de/frigosped/dc/model/EvaluationResult.class
de/frigosped/dc/model/AiCostRequest.class
de/frigosped/dc/model/OrdsDocumentTexts.class
de/frigosped/dc/model/MistralOcrResponse$UsageInfo.class
de/frigosped/dc/resource/CatalogResource.class
de/frigosped/dc/model/ExportRequest.class
de/frigosped/dc/service/CatalogGenerationService.class
de/frigosped/dc/service/CheckOrchestrationService.class
de/frigosped/dc/model/DocumentPdfEntry.class
de/frigosped/dc/service/MarkdownExportService$SegType.class
de/frigosped/dc/model/GeneratedCatalogStructure.class
de/frigosped/dc/client/OrdsLoggingFilter.class
de/frigosped/dc/service/MarkdownExportService$PdfWriter.class
de/frigosped/dc/security/ApiKeyFilter.class
de/frigosped/dc/service/ReferenceDownloadService$DownloadedReference.class
de/frigosped/dc/model/GeneratedCatalogStructure$GeneratedQuestion.class
de/frigosped/dc/service/AiCostTrackerService.class
de/frigosped/dc/model/MistralOcrResponse$Page.class
de/frigosped/dc/model/ReferenceCreateRequest.class

73
dc-backend/target/maven-status/maven-compiler-plugin/compile/default-compile/inputFiles.lst
View File

@@ -1,32 +1,41 @@
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/ai/AiModelProducer.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/client/OrdsClient.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/client/OrdsLoggingFilter.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/AiCostRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/CatalogGenerateResponse.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/EvaluationResult.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ExportRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/GeneratedCatalogStructure.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCatalogCreateRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCatalogStatusResponse.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCatalogStatusUpdateRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCategoryCreateRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocument.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocumentList.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocumentTexts.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsProject.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestion.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestionCreateRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestionList.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ProgressRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ResultRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/TextsRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/CatalogResource.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/CheckResource.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/ExportResource.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/security/ApiKeyFilter.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/AiCostTrackerService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/CatalogGenerationService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/CheckOrchestrationService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/DocumentProcessingService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/EvaluationService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/MarkdownExportService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/ai/AiModelProducer.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/client/MistralOcrClient.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/client/OrdsClient.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/client/OrdsLoggingFilter.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/AiCostRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/CatalogGenerateResponse.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/CompleteProjectRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/DocumentPdfEntry.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/DocumentProgressRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/EvaluationResult.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ExportRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/GeneratedCatalogStructure.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/MistralOcrRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/MistralOcrResponse.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCatalogCreateRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCatalogStatusResponse.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCatalogStatusUpdateRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsCategoryCreateRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocument.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocumentList.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocumentTexts.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsProject.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestion.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestionCreateRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestionList.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ProgressRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ReferenceCreateRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ResultRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/TextsRequest.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/CatalogResource.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/CheckResource.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/ExportResource.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/security/ApiKeyFilter.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/AiCostTrackerService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/CatalogGenerationService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/CheckOrchestrationService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/DocumentProcessingService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/EvaluationService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/MarkdownExportService.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/MarkdownReportBuilder.java
/home/philipp/git/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/ReferenceDownloadService.java

12
dc-backend/target/maven-status/maven-compiler-plugin/compile/null/createdFiles.lst
View File

@@ -1,12 +0,0 @@
de/frigosped/dc/model/OrdsQuestion.class
de/frigosped/dc/model/OrdsQuestionList.class
de/frigosped/dc/model/ProgressRequest.class
de/frigosped/dc/client/OrdsClient.class
de/frigosped/dc/ai/AiModelProducer.class
de/frigosped/dc/model/OrdsDocument.class
de/frigosped/dc/model/OrdsProject.class
de/frigosped/dc/model/OrdsDocumentList.class
de/frigosped/dc/model/TextsRequest.class
de/frigosped/dc/model/ResultRequest.class
de/frigosped/dc/model/EvaluationResult.class
de/frigosped/dc/resource/CheckResource.class

15
dc-backend/target/maven-status/maven-compiler-plugin/compile/null/inputFiles.lst
View File

@@ -1,15 +0,0 @@
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/ai/AiModelProducer.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/client/OrdsClient.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/EvaluationResult.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocument.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsDocumentList.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsProject.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestion.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/OrdsQuestionList.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ProgressRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/ResultRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/model/TextsRequest.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/resource/CheckResource.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/CheckOrchestrationService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/DocumentProcessingService.java
/mnt/c/source/Frigosped/Dokumenten-Check/dc-backend/src/main/java/de/frigosped/dc/service/EvaluationService.java

21
dc-backend/update.sh Normal file → Executable file
View File

@@ -6,21 +6,36 @@ export KUBECONFIG="$SCRIPT_DIR/../env/frigo-dev.yaml"
TAG="${1:-latest}"
DB_USER="wksp_ai"
DB_PASS="s!)löyx209aasgtrasdasiudkash5235FDSXljLJ"
DB_HOST="10.75.10.171"
DB_PORT="32710"
DB_SID="freepdb1"
DB_CONNECT="${DB_USER}/${DB_PASS}@${DB_HOST}:${DB_PORT}/${DB_SID}"
echo "=== dc-backend Update ==="
echo ""
# --- DB Migration ---
echo "→ [1/4] DB Migration..."
sql -s "$DB_CONNECT" @"$SCRIPT_DIR/../Scripts/DC_DOCUMENT_PROGRESS_ALTER.sql"
sql -s "$DB_CONNECT" @"$SCRIPT_DIR/../Scripts/DC_DOCUMENT_PROGRESS_ORDS.sql"
sql -s "$DB_CONNECT" @"$SCRIPT_DIR/../Scripts/DC_REFERENCES_ALTER.sql"
echo "✓ DB Migration abgeschlossen."
# --- Build & Push ---
echo "→ [1/3] Build & Push Image (Tag: $TAG)..."
echo ""
echo "→ [2/4] Build & Push Image (Tag: $TAG)..."
bash "$SCRIPT_DIR/build-and-push.sh" "$TAG"
# --- Apply Kubernetes Manifests ---
echo ""
echo "→ [2/3] Kubernetes Manifests anwenden..."
echo "→ [3/4] Kubernetes Manifests anwenden..."
kubectl apply -f "$SCRIPT_DIR/k8s/deployment.yaml"
# --- Rolling Restart (zieht neues 'latest'-Image) ---
echo ""
echo "→ [3/3] Rolling Restart..."
echo "→ [4/4] Rolling Restart..."
kubectl rollout restart deployment/dc-backend -n ai-env
echo ""