Module htmltext 

Text of HTML reports (the report generators of FEA/CFD tools) for the full-text search, the assistant (read_file_text, propose_result_values) and the extraction templates; see docs/src/research/html-report-text.md.

An HTML file is untrusted input. Like a PDF it is read in the hidden teamcraft text-extract --format html child process with the time and memory limits of pdftext (in tests, where there is no such binary, on a blocking thread). The reader is a small own tokenizer, not a browser:

• a table row becomes one line, its cells are separated by | (Maximum von Mises stress | 152.4 | MPa); block elements end a line;
• entities are decoded (°, °, °,   …), numbers and units stay exactly as written;
• script, style, noscript, template, iframe, svg, canvas, comments and hidden elements (hidden, aria-hidden="true", an inline display:none / visibility:hidden) are skipped – hidden text is a prompt-injection vector, not part of the report;
• attribute values are not kept, except alt of images and the few that decide about hiding: a data: URI of an embedded image is passed over without being copied, and nothing is ever fetched (src, href, @import, <base>, <meta http-equiv=refresh> are not followed);
• the charset comes from a byte order mark, a <meta charset> in the first 1024 bytes, valid (or mostly valid) UTF-8, else Windows-1252 – a bad byte never fails;
• bounded: at most MAX_HTML_BYTES are read (more is cut, not refused), MAX_DEPTH open elements are tracked (deeper ones are flattened) and the text stops at crate::pdftext::MAX_CHARS characters.

Constants

MAX_DEPTH

Open elements that are tracked; deeper content is flattened.

MAX_HTML_BYTES

Bytes of an HTML file that are read (the beginning; more is cut).

Functions

decode

Bytes as text: byte order mark, then (for HTML, meta = true) a <meta charset> / http-equiv declaration in the first 1024 bytes, then UTF-8 if valid (a sequence cut at the very end does not count) or mostly valid (a stray bad byte in a solver log becomes one U+FFFD, the µ and ° around it stay), else Windows-1252. An unknown label is ignored. Never fails.

extract

The text of an HTML file held in memory (charset detection, then to_text, then the line clean-up of pdftext). A binary file named .html has no text. A panic is an error, not a crash.

extract_file

The text of a stored HTML file; empty if it has none or cannot be read. Read in the text-extract child like a PDF (time, memory, size limits).

is_html

Whether a file name is an HTML file.

run_job

One job of the child process: the text of the HTML file read from input (stdin) is written to output (stdout). At most max_input_bytes are read; the rest is cut (a big report keeps its first tables). Returns the number of characters.

to_text

The readable text of an HTML document (see the module documentation).