Skip to main content

Module pdftext

Module pdftext 

Source
Expand description

Text of PDF reports for the full-text search and the assistant (read_file_text); see docs/src/research/pdf-text-extraction.md.

A PDF is untrusted input, so it is never parsed in the server process: the hidden teamcraft text-extract child process does it, with a time limit, the memory limit of the 3D previews and a size cap before the file is opened (in tests, where there is no such binary, on a blocking thread). The result is bounded as well – MAX_PAGES pages, MAX_CHARS characters. There is no OCR: a scanned PDF has no text.

The same child reads HTML reports (--format html, see crate::htmltext); extract_isolated runs either.

Enums§

TextFormat
What the text-extract child reads: a PDF or an HTML report.

Constants§

MAX_CHARS
Characters kept (the cap of every extracted text).
MAX_PAGES
Pages read.
MAX_PDF_BYTES
A PDF above this size is not opened (CFD reports are small; big PDFs are scans).
TIME_LIMIT
Time limit of one extraction.

Functions§

configure
Takes the memory limit of the child process (0 = unlimited).
extract
The text of a PDF held in memory: at most MAX_PAGES pages and MAX_CHARS characters; pages that cannot be read are skipped. A parser panic is an error, not a crash.
extract_file
The text of a stored PDF; empty if it has none or cannot be read (a scanned or damaged file never fails an upload). The opened file is handed to the child on its standard input and the text comes back on its standard output, so no path travels anywhere.
is_pdf
Whether a file name is a PDF.
run_job
One job of the child process: the text of the PDF read from input (stdin) is written to output (stdout). At most max_input_bytes are read; more is refused. Returns the number of characters.
run_job_as
One job of the child process in the given format (see run_job and crate::htmltext::run_job).