Commit Graph

37 Commits

Author SHA1 Message Date
8ec8003dbb Store favicon BLAKE3 hashes in the doc store 2022-03-27 21:30:38 +01:00
0a955bb2f5 Build seed lookup table 2022-03-27 20:53:29 +01:00
6f596b54dc Move get_reduced_domain 2022-03-27 20:49:20 +01:00
a31566a89f Support raking icons 2022-03-27 19:01:42 +01:00
4665bfd3a3 Load seeds in the indexer 2022-03-26 17:55:19 +00:00
f884324648 Factor out recursive loader 2022-03-26 17:45:31 +00:00
d26b4271ce Add proof-of-concept search UI page 2022-03-25 23:39:27 +00:00
9c7dfb93f1 Fix search and make some minor improvements 2022-03-25 19:32:45 +00:00
c01740113f Open a fancy_mdbx database to store documents in for the Tantivy backend 2022-03-25 19:17:34 +00:00
9866da2d16 Load documents and index them! 2022-03-24 23:45:36 +00:00
4f4c3e36d1 Prepare indexer 2022-03-24 23:19:32 +00:00
73154e7e34 Move the indexer around 2022-03-24 19:50:20 +00:00
1773ba4f44 Start fleshing out the indexer 2022-03-23 20:11:12 +00:00
0060ec0764 Add seed sorting tool in order to approach first proof of concept 2022-03-22 23:54:28 +00:00
05ebfc8998 Add process metrics 2022-03-21 20:19:59 +00:00
9ef4fef858 Shut down gently on SIGINT or SIGTERM (supposedly) 2022-03-21 19:11:07 +00:00
71c22daf0d Emit rakepacks from the raker 2022-03-21 19:07:35 +00:00
8df430c7f1 Load and parse seeds 2022-03-20 20:50:31 +00:00
39aa4eb9b7 Add seed file parser 2022-03-20 20:29:32 +00:00
5e61386a83 Use ArcIntern and CompactString as needed 2022-03-20 15:49:00 +00:00
f9aac34104 Add support for Prometheus metrics 2022-03-20 12:24:35 +00:00
ea4f2d1332 Some partial progress towards raking pages 2022-03-19 22:57:36 +00:00
5bab279cc2 STASH 2022-03-19 15:39:59 +00:00
ab0b1e84ee STASH work on Raking 2022-03-19 21:04:12 +00:00
932fd74dd2 Flesh out more libMDBX structure 2022-03-19 00:50:02 +00:00
f6acb2959a Add baseline libMDBX structure 2022-03-18 20:09:08 +00:00
2d44cc351b Update the Cargo lock to fix stale 'Cylon' package 2022-03-17 23:55:41 +00:00
d09cfd4659 Initial working version of the seed collection service 2022-03-16 19:53:08 +00:00
2763ff2481 Add support for a Raker config 2022-03-14 23:53:55 +00:00
1f681164c9 Rearrange quickpeep crates 2022-03-14 23:36:37 +00:00
601ec553b5 Clean-ups and support pulling out references 2022-03-14 23:01:19 +00:00
5a94c825d7 Lazily textify the document tree only if it's needed 2022-03-14 20:33:12 +00:00
04b94b16ed Determine the language for raked HTML pages 2022-03-14 20:33:09 +00:00
a1097ef183 Add a lot more foundational work for raking 2022-03-13 21:33:03 +00:00
210e8ef10a Add more work towards raking all the different formats 2022-03-13 12:40:04 +00:00
db5524eb52 First step towards minimum usability 2022-03-12 17:52:01 +00:00
4cd259d0ac Initial commit 2022-03-12 15:01:40 +00:00