Optional text-file content
Enable text extraction separately after installing the optional search migrations:studio.files.process permission before reading and again before saving. Custom permission mappings remain supported. It does not grant access by accepting an arbitrary PHP model class. A configured tenant resolver must resolve the intended tenant context for CLI use.
The source must have valid UTF-8 encoding and match its registered content checksum. Supported MIME types are text/plain, text/markdown, text/csv and text/html. HTML scripts/styles/embedded active content are removed without execution or remote fetching. Plain text and Markdown are not interpreted as HTML. External source processing is not supported by this extractor yet.
Source bytes default to1MiB with a hard maximum4MiB. Extracted text defaults to12000 Unicode characters with the same hard maximum, keeping each stored projection below the SQL TEXT byte limit. Longer text gets the explicit partial state; this is not full-document coverage. Unsupported/oversized files do not become successful extracted documents. Invalid encodings and changed bytes fail without publication. Original files remain unchanged.
Run media-studio:search-index --configure again to add content to searchable fields, then run scheduled synchronization. Content uses a version-bound fingerprint independent of the file name. Updating the content projection therefore causes reindexing even if the display name did not change. Search never fetches originals or parses files. It loads only projection hashes to verify current hits, not the stored text body. Text and raw engine payloads are not returned in result cards.
Disabling content immediately rejects hits carrying old content fingerprints; name results may temporarily disappear until synchronization removes content from those indexed documents. Stored projections are retained when disabled, not automatically destroyed. Deleting an entry cascades its SQL text projection; its engine document is removed by synchronization. Optional upload-triggered extraction is described below. Existing/imported files require explicit extraction; there is no automatic full-library content scan.
PDF with an existing text layer
Native PDF extraction uses optional Poppler command-line tools already installed on your server. Without OCR enabled, it requires no AI API or page rendering. Enable it separately:media-studio:search-extract command as for text files. Paths depend on the server installation. Missing executables, encrypted documents, invalid PDFs and parser failures do not become successfully indexed content. Both PDF and ordinary content extraction remain off by default.
Each parser process has a timeout (maximum30seconds), page selection (maximum100pages), bounded captured output and no shell interpolation. Metadata output is limited to64KiB and extracted text output to256KiB; exceeding these bounds fails rather than silently publishing incomplete parser output. Final indexed text remains capped at12000Unicode characters. Source files use private temporary staging cleaned in a finally block. For untrusted documents, run processing workers with OS/container memory, CPU and filesystem restrictions; a subprocess timeout is not an OS memory sandbox.
A processed document receives:
ready: text was extracted within configured coverage limits;partial: page/character limits were reached, or selected pages include empty text;no_text: the selected pages yielded no text. This may be a scan, a blank document or unsupported text encoding; it is not an OCR result.
Markdown, TSV and oversized files
Markdown files (.md) are supported when identified as text/plain,
text/markdown or text/x-markdown; optional charset parameters are normalized.
TSV (text/tab-separated-values) and application/csv are also supported.
A binary file renamed .md is not accepted as text. RTF and Office formats need
separate parsers and are not plain-text formats in this extractor.
Oversized sources persist skipped_size before opening the storage stream. This is
distinct from unsupported. The current implementation does not index10000-page
documents fully or resume from a page/byte checkpoint. Character-limited output
is explicitly partial. Full large-document background chunking remains planned;
searching never scans original document content on request.
Large-file policy
skip excludes files larger than the soft threshold before downloading any
original bytes. bounded permits extraction above the soft threshold, but never
above max_source_bytes (hard maximum4MiB). It retains existing page/character
limits and explicit partial coverage; it is not unlimited or resumable processing.
The soft threshold must be positive and no larger than the hard cap.
Skipped files persist skipped_size with an empty projection for their current
version. An earlier extracted body is cleared and index synchronization removes
its searchable text. Names can remain searchable. The state is an extraction
status, not an upload rejection: the original is retained unchanged. After
changing limits, explicitly rerun extraction for skipped files. No automatic
retry loop repeatedly downloads oversized originals.