Repository navigation
Expand file tree
/
Copy pathNOTICE
More file actions
128 lines (98 loc) · 5.95 KB
/
Copy pathNOTICE
File metadata and controls
128 lines (98 loc) · 5.95 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
Markkit
Copyright 2024-2026 Everruns
Licensed under the MIT License.
=== Design Inspirations ===
Markkit is an independent implementation. The following projects
inspired aspects of its design:
* Readability (Apache 2.0) – Mozilla's approach to content extraction
informed our HTML-to-markdown pipeline design.
* Pandoc (GPL-2.0) – The universal document converter's format
coverage and conversion philosophy inspired our multi-format
architecture.
* turndown (MIT) – The JavaScript HTML-to-markdown converter's
rule-based approach influenced our conversion engine design.
Markkit's parser, converter, and extraction pipeline are original
Rust implementations. This project is not affiliated with, endorsed
by, or derived from any of the above projects.
=== Key Dependencies ===
Markkit relies on permissively licensed crates, including:
* scraper (ISC), html5ever (MIT/Apache-2.0) – HTML parsing
* zip (MIT), quick-xml (MIT), calamine (MIT), cfb (MIT) – Office document containers and XML
* hayro-syntax / hayro (MIT/Apache-2.0) – PDF parsing and page rendering
* ocrs, rten (MIT/Apache-2.0) – optional Rust OCR backend
* reqwest (MIT/Apache-2.0), rustls (Apache-2.0/ISC/MIT) – optional HTTP client
* tokio (MIT) – async runtime
* serde / serde_json (MIT/Apache-2.0) – serialization
* pyo3 (MIT/Apache-2.0), napi (MIT), wasm-bindgen (MIT/Apache-2.0) – bindings
The allowed license set is enforced by `cargo deny check licenses` (deny.toml).
Binary packages (CLI, Python wheels, Node native addon, WebAssembly) statically
link these crates; their license texts and copyright notices are in each
crate's source on crates.io, pinned by Cargo.lock. cssparser and selectors
(via scraper) are MPL-2.0; their unmodified source is available from crates.io.
=== Bundled Third-Party Data ===
The `pdf-ocr` feature renders PDF pages with hayro, which compiles the
following data into the binary. It is enabled in the CLI, the Python package
and the native Node package (not in the default crate build or WebAssembly).
--- Standard PDF fonts (hayro-interpret, assets/*.pfb) ---
Extracted from PDFium.
Copyright 2014 PDFium Authors. All rights reserved.
Original code copyright 2014 Foxit Software Inc. http://www.foxitsoftware.com
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Google Inc. nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
--- Adobe CMap resources (hayro-cmap, assets) ---
Copyright 1990-2023 Adobe. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Adobe nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
--- ICC color profiles (hayro-interpret, assets/*.icc) ---
CGATS001Compat-v2-micro.icc from saucecontrol/Compact-ICC-Profiles, CC0-1.0.
LAB.icc generated with LCMS2 by the hayro authors.
=== Not Bundled ===
OCR model weights (CC-BY-SA 4.0), Tesseract and its language data are not
included in any markkit package. `just ocr-models` downloads the pinned models
and writes their license and attribution to target/ocr-models/NOTICE.json.
=== Public Benchmark Tooling ===
scripts/public_benchmarks/vendor includes the Mira Python SDK (Everruns, MIT),
the AgentDocBench scorer (SylphxAI, MIT), and the WCXB scorer (Murrough Foley,
CC-BY-4.0), each with its license. Input documents retain the individual
licenses and attribution in scripts/public_benchmarks/CORPUS-NOTICE and
agentdocbench-corpus.json. The downloaded corpora are not included in packages.