Commit Graph

225 Commits (main)

Author SHA1 Message Date
Olivier 'reivilibre' ff0126bac4 Use published fork of Cylon 2022-04-01 22:53:05 +01:00
Olivier 'reivilibre' 00f05256e5 Add debug line 2022-04-01 22:47:54 +01:00
Olivier 'reivilibre' e2c2adefa2 Publish and use fancy_mdbx and metrics-process-promstyle 2022-04-01 22:47:52 +01:00
Olivier 'reivilibre' f31c2bba1e Add MIT/Apache2 licence 2022-04-01 22:25:06 +01:00
Olivier 'reivilibre' 6a68757e30 Use a non-readabilitised copy of the document for reference extraction
continuous-integration/drone the build failed Details
Fixes #7.
2022-03-29 22:43:31 +01:00
Olivier 'reivilibre' e6a402af19 Use trace! for cosmetic filter logging
continuous-integration/drone the build failed Details
2022-03-28 23:43:10 +01:00
Olivier 'reivilibre' 68b7c76d1e Support network filter checking 2022-03-28 23:43:01 +01:00
Olivier 'reivilibre' de610e5aab Fix the seed dump script 2022-03-28 23:17:45 +01:00
Olivier 'reivilibre' 61398f6153 Don't store images whilst raking, for now
Fixes #5
2022-03-28 23:17:40 +01:00
Olivier 'reivilibre' 5f93b68b4e Display datetime metadata in qp-rake1 2022-03-28 23:17:32 +01:00
Olivier 'reivilibre' ea0b08a3e1 Add some raking metrics
continuous-integration/drone the build failed Details
2022-03-28 22:43:53 +01:00
Olivier 'reivilibre' 98c05f59b5 Support dumping seeds as files 2022-03-28 22:40:21 +01:00
Olivier 'reivilibre' 507459b4ee Fix comment association 2022-03-28 22:26:20 +01:00
Olivier 'reivilibre' 3bfd192c28 Store dates in the document head 2022-03-28 22:24:36 +01:00
Olivier 'reivilibre' 7dc2369dd2 Some groundwork for pulling out JSON-LD dates 2022-03-28 19:45:39 +01:00
Olivier 'reivilibre' 2f68d4d825 Fix antifeature tag push 2022-03-28 19:31:47 +01:00
Olivier 'reivilibre' 4231bec80a Push antifeatures as tags for now (a temporary solution)
continuous-integration/drone the build failed Details
2022-03-27 22:58:57 +01:00
Olivier 'reivilibre' 53db706cc0 Stop eprintlning the readability metadata
continuous-integration/drone the build failed Details
2022-03-27 22:51:08 +01:00
Olivier 'reivilibre' fac9f2ebdb Make cosmetic filter output use the logging system 2022-03-27 22:47:15 +01:00
Olivier 'reivilibre' 791fa8c4ee Don't stop raking due to errors from processing a domain 2022-03-27 22:44:57 +01:00
Olivier 'reivilibre' 5719d0b402 Use the redirect-following client for robots.txt 2022-03-27 22:43:16 +01:00
Olivier 'reivilibre' ab25e0bb9a Retrieve icons from the icon store 2022-03-27 22:38:56 +01:00
Olivier 'reivilibre' 25db9fdb24 Store icons in the icon store 2022-03-27 22:06:06 +01:00
Olivier 'reivilibre' eb899ac9a5 Simplify textification of indexed pages 2022-03-27 21:36:21 +01:00
Olivier 'reivilibre' 8ec8003dbb Store favicon BLAKE3 hashes in the doc store 2022-03-27 21:30:38 +01:00
Olivier 'reivilibre' a271f83805 Add semicolon 2022-03-27 21:11:09 +01:00
Olivier 'reivilibre' f2354c5bf7 Display tags in the search UI 2022-03-27 21:04:24 +01:00
Olivier 'reivilibre' 945a1504ca Load tags and index them as necessary 2022-03-27 21:03:55 +01:00
Olivier 'reivilibre' 0a955bb2f5 Build seed lookup table 2022-03-27 20:53:29 +01:00
Olivier 'reivilibre' 6f596b54dc Move get_reduced_domain 2022-03-27 20:49:20 +01:00
Olivier 'reivilibre' ad10b9eb38 Bump up size cutoff to 4 GiB (uncompressed)
continuous-integration/drone the build failed Details
2022-03-27 19:45:44 +01:00
Olivier 'reivilibre' 27c3218097 Emit icons from the raker
continuous-integration/drone the build failed Details
2022-03-27 19:40:12 +01:00
Olivier 'reivilibre' bdfacc643e Increate the quality of packed WebP favicons 2022-03-27 19:31:52 +01:00
Olivier 'reivilibre' 4e0352220d Show favicons in qp-rake1 2022-03-27 19:31:24 +01:00
Olivier 'reivilibre' e401377db5 Restrict the size of raked images 2022-03-27 19:18:21 +01:00
Olivier 'reivilibre' 6d6e3c52e3 Allow redirects for favicon rakes 2022-03-27 19:15:24 +01:00
Olivier 'reivilibre' 7bdd7d4fc6 Allow specifying an intent with the qp-rake1 CLI 2022-03-27 19:03:21 +01:00
Olivier 'reivilibre' a31566a89f Support raking icons 2022-03-27 19:01:42 +01:00
Olivier 'reivilibre' 653acf68a9 Don't HTML escape the HTML excerpts 2022-03-27 17:39:31 +01:00
Olivier 'reivilibre' 4665bfd3a3 Load seeds in the indexer
continuous-integration/drone the build failed Details
2022-03-26 17:55:19 +00:00
Olivier 'reivilibre' f884324648 Factor out recursive loader 2022-03-26 17:45:31 +00:00
Olivier 'reivilibre' f4672181aa Pass through excerpts to the frontend 2022-03-26 17:31:50 +00:00
Olivier 'reivilibre' 39ffdd1f1f Pass through excerpts from the Tantivy backend 2022-03-26 17:25:43 +00:00
Olivier 'reivilibre' eedf2ed183 Add structure to support raking icons 2022-03-26 17:15:00 +00:00
Olivier 'reivilibre' 8714f0ec80 Enqueue favicons to be raked 2022-03-26 17:04:39 +00:00
Olivier 'reivilibre' d26b4271ce Add proof-of-concept search UI page
continuous-integration/drone the build failed Details
2022-03-25 23:39:27 +00:00
Olivier 'reivilibre' 9c7dfb93f1 Fix search and make some minor improvements 2022-03-25 19:32:45 +00:00
Olivier 'reivilibre' 3a6f2fdf7a Add a way to search for documents at the CLI (proof of concept) 2022-03-25 19:28:57 +00:00
Olivier 'reivilibre' c01740113f Open a fancy_mdbx database to store documents in for the Tantivy backend 2022-03-25 19:17:34 +00:00
Olivier 'reivilibre' cc1ba7c85f Remove .cargo config (set it per-computer instead) 2022-03-25 19:14:17 +00:00