The fastest non-VLM parser that preserves document structure: tables, headings, lists is OpenDataLoader PDF.
š The developers found room to improve on latency, so we profiled. We initially expected the sorting algorithm (XY-Cut++) to be the bottleneck, but it turned out to be less than **1% **of the tota...
Source: DEV Community
š The developers found room to improve on latency, so we profiled. We initially expected the sorting algorithm (XY-Cut++) to be the bottleneck, but it turned out to be less than **1% **of the total time. The real cost was hiding in content filtering (55%) and preprocessing (25%). šļø3 fixes applied š„Page-level parallel processing š„Hidden text detection ā opt-in š„Text-only fast path š¢Output is byte-for-byte identical before and after optimization. Only the speed changed results stay the same. šļøOpenDataLoader PDF highlights š#1 in latency š„(585 pages in 1.10s) šļø#1 in memory efficiency š„(7.4MB) š¢Java Ā· Python Ā· Node.js SDK š¢Multiple output formats (text, markdown, HTML, JSON, PDF) Check out the benchmark below for latency and memory usage results. See the PR for full details on what changed and how we got here. We'd love your feedback if you try it out! GitHub: http://github.com/opendataloader-project/opendataloader-pdf?utm_source=x&utm_medium=social&utm_campaign=per
