sugoidogo/modrinth
1
0
Fork 0
mirror of https://github.com/modrinth/code.git synced 2026-08-28 18:45:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: search indexing performance and batching (#6521)

Browse Source 
* Add consume batching delay

* maybe fix

* max batch size

* more logging

* parallelize remove tasks

* delete/upsert project/version messages

* prepare

* more logging

* log number of docs

* try more targeted, homogenous version change ops

* ensure only necessary fields are serialized into typesense

* disable index background task

* wip: script changes

* don't facet by project id

* fix

* fix clippy

* batch by document and dedup loaders

* fix test

* wip: projects/versions collections

* clean up SearchBackend interface

* cleanup pass

* cleanup pass 2

* standardise fn names

* cleanup pass

* fix compile

* factor out filter rewriting

* query perf

* put categories into the project doc

* wip: search filter AST

* doc comment

* more AST normalization

* (temp) convert search request errors into internal errors

* implement unary NOT

* tombi fmt

* fix tests

* revert request error

* try increase stack size
This commit is contained in:
aecsocket
2026-07-26 14:55:00 +00:00
committed by GitHub
parent 54cc1898fa
commit 158de019a6
32 changed files with 2787 additions and 720 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/labrinth/src/lib.rs
-24
View File
@@ -131,30 +131,6 @@ pub fn app_setup(
));
if enable_background_tasks {
// The interval in seconds at which the local database is indexed
// for searching. Defaults to 1 hour if unset.
let local_index_interval =
Duration::from_secs(ENV.LOCAL_INDEX_INTERVAL);
let pool_ref = pool.clone();
let redis_pool_ref = redis_pool.clone();
let search_backend_ref = search_backend.clone();
scheduler.run(local_index_interval, move || {
let pool_ref = pool_ref.clone();
let redis_pool_ref = redis_pool_ref.clone();
let search_backend = search_backend_ref.clone();
async move {
if let Err(err) = background_task::index_search(
pool_ref,
redis_pool_ref,
search_backend,
)
.await
{
warn!("Failed to index search: {err:?}");
}
}
});
// Changes statuses of scheduled projects/versions
let pool_ref = pool.clone();
// TODO: Clear cache when these are run