Commit Graph
32 Commits
Author SHA1 Message Date
aecsocketandGitHub 158de019a6 fix: search indexing performance and batching (#6521)
* Add consume batching delay

* maybe fix

* max batch size

* more logging

* parallelize remove tasks

* delete/upsert project/version messages

* prepare

* more logging

* log number of docs

* try more targeted, homogenous version change ops

* ensure only necessary fields are serialized into typesense

* disable index background task

* wip: script changes

* don't facet by project id

* fix

* fix clippy

* batch by document and dedup loaders

* fix test

* wip: projects/versions collections

* clean up SearchBackend interface

* cleanup pass

* cleanup pass 2

* standardise fn names

* cleanup pass

* fix compile

* factor out filter rewriting

* query perf

* put categories into the project doc

* wip: search filter AST

* doc comment

* more AST normalization

* (temp) convert search request errors into internal errors

* implement unary NOT

* tombi fmt

* fix tests

* revert request error

* try increase stack size
2026-07-26 14:55:00 +00:00
François-Xavier TalbotandGitHub 2cf71a53f2 chore(ci): increase rust stack size (#6874) 2026-07-26 00:11:08 +00:00
Michael H. 907db9e64a fix: remove sccache from turbo-ci 2026-07-25 00:01:27 +02:00
Michael H. 5a80df3cd4 build: change runner profiles 2026-07-24 21:32:06 +02:00
Michael H.andGitHub 1e85ec8649 chore: switch to namespace runners (#6851)
* chore: switch to namespace runners

* fix: sudo

* fix: move to medium runners for frontend

* chore: bump all actions, remove old turbo cache

* fix: bump frontend runners to large

* fix: wrangler

* fix: wrangler version, again

* chore: namespace windows and mac runners

* fix: revert tombi bump

* chore: tweak caching
2026-07-24 21:27:52 +02:00
Michael H. 054f1b42fd build: disable incremental compilation for sccache compat 2026-07-23 21:42:44 +02:00
François-Xavier TalbotandGitHub b4d681e713 feat(labrinth): Redis Cluster (#6771)
* chore(labrinth): bump to redis 1.4.1

* feat(labrinth): redis cluster

* chore: cleanup

* feat(labrinth): cache locking

* fix(labrinth): clippy

* chore(labrinth): cleanup env, remove postcard support

* chore(ci): fix test env for labrinth

* chore(labrinth): bump all key versions

* chore(labrinth): improve redis key identities handling

* chore(labrinth): simplify deadline handling

* chore(labrinth): remove unused lease tracking

* chore(labrinth): remove distributed cache locking for now

* chore(labrinth): improve redis backend init error

* feat(labrinth): expose redis read replica strategy

* chore(ci): remove other connection mode tests

* chore: split xredis crate

* feat(xredis): primaries routing

* chore: tombi fmt

* chore: clippy

* chore: update query cache
2026-07-23 11:35:02 +02:00
Michael H. b667b3ae9e fix: remove sccache s3 2026-06-30 11:41:08 +02:00
Michael H. 5fe07de187 chore: bump mold and add mold to other jobs 2026-06-26 17:22:46 +02:00
aecsocketandGitHub c6ef10e798 Incremental search indexing (#6453)
* wip: initial kafka support

* wip: kafka search queue

* wip: kafka consumer

* wip: create/delete docs

* reindexing works

* reindex in more places

* also incrementally update server pings

* cleanup

* tombi

* fix ci

* update tombi

* prepare

* fix ci

* fix

* fix PLEASE

* 🙏

* add route to force reindex project
2026-06-21 10:03:20 +00:00
Calum H.andGitHub 047b8c3bf7 fix: i18n string problems (#6131)
* fix: apply non-json i18n fixes

* fix: pruning

* fix: prepr

* fix: run.mjs

* fix: lint
2026-05-29 15:55:39 +00:00
Michael H. 99ac6b87b1 ci: make blacksmith usage internal only 2026-05-06 03:00:20 +02:00
Michael H. 6ed56a4756 ci: skip merge queue actions if clean 2026-05-06 00:11:11 +02:00
5b59e39a8a chore: improve actions performance and security practices (#5970)
* chore: bump actions and pin versions

* build: switch to blacksmith

* fix: use rust-toolchain stable

* build: improve pnpm store caching

* chore: remove emoji from workflows

* fix: run prepare job on blacksmith

* chore: kebab case id

* build: add concurrency groups to limit duplicate jobs

* build: switch around node setup and pnpm setup task

* chore: bump to nodejs 24, fix pnpm caching

* fix: enable corepack

* fix: concurrency deadlock in frontend preview

* fix: approve build scripts

* fix: just don't cancel concurrent previews

* build: remove pnpm setup action everywhere

* build: cache apt packages

* build: yet another attempt at fixing concurrency

* build: lower runner type for frontend deploy

* fix: eslint not existing

* build: add sccache to turbo-ci

* fix: correct nextest pkg

* fix: turbo ignoring sccache

* revert me: test labrinth tests

* Revert "revert me: test labrinth tests"

This reverts commit def5cc19183d5c0fe3b6f3c03635d73bb59bd312.

* build: compile app before docker build

* build: lower runner types

* build: remove docker inline caching

* build: try mold on labrinth

* build: tweak labrinth prod build profile

* fix: app windows builds and caching

* fix: tombi format cargo.toml

* fix: swap ping test to cubecraft to avoid CI flakiness

* typos fix

---------

Co-authored-by: aecsocket <aecsocket@tutanota.com>
2026-05-03 14:18:31 +02:00
Calum H.andGitHub 61d4a34f0f devex: skip services + env + db steps in CI if not needed (#4965) 2026-01-06 00:23:22 +00:00
Calum H.andGitHub 3adee66899 devex: prepr:web and app with proper caching (#4957)
* devex: prepr:web and app with proper caching

* fix: add tooling config to turbo global deps

* fix: exclude turbo + node modules for tooling-config

* feat: prepr:frontend

* fix: ci issue
2025-12-24 21:39:59 +00:00
b23d3e674f Update Rust & Java dependencies (#4540)
* Update Java dependencies

* Baselint lint fixes

* Update Rust version

* Update actix-files 0.6.6 -> 0.6.8

* Update actix-http 3.11.0 -> 3.11.2

* Update actix-rt 2.10.0 -> 2.11.0

* Update async_zip 0.0.17 -> 0.0.18

* Update async-compression 0.4.27 -> 0.4.32

* Update async-trait 0.1.88 -> 0.1.89

* Update async-tungstenite 0.30.0 -> 0.31.0

* Update const_format 0.2.34 -> 0.2.35

* Update bitflags 2.9.1 -> 2.9.4

* Update bytemuck 1.23.1 -> 1.24.0

* Update typed-path 0.11.0 -> 0.12.0

* Update chrono 0.4.41 -> 0.4.42

* Update cidre 0.11.2 -> 0.11.3

* Update clap 4.5.43 -> 4.5.48

* Update data-url 0.3.1 -> 0.3.2

* Update discord-rich-presence 0.2.5 -> 1.0.0

* Update enumset 1.1.7 -> 1.1.10

* Update flate2 1.1.2 -> 1.1.4

* Update hyper 1.6.0 -> 1.7.0

* Update hyper-util 0.1.16 -> 0.1.17

* Update iana-time-zone 0.1.63 -> 0.1.64

* Update image 0.25.6 -> 0.25.8

* Update indexmap 2.10.0 -> 2.11.4

* Update json-patch 4.0.0 -> 4.1.0

* Update meilisearch-sdk 0.29.1 -> 0.30.0

* Update clickhouse 0.13.3 -> 0.14.0

* Fix some prettier things

* Update lettre 0.11.18 -> 0.11.19

* Update phf 0.12.1 -> 0.13.1

* Update png 0.17.16 -> 0.18.0

* Update quick-xml 0.38.1 -> 0.38.3

* Update redis 0.32.4 -> 0.32.7

* Update regex 1.11.1 -> 1.11.3

* Update reqwest 0.12.22 -> 0.12.23

* Update rust_decimal 1.37.2 -> 1.38.0

* Update rust-s3 0.35.1 -> 0.37.0

* Update serde 1.0.219 -> 1.0.228

* Update serde_bytes 0.11.17 -> 0.11.19

* Update serde_json 1.0.142 -> 1.0.145

* Update serde_with 3.14.0 -> 3.15.0

* Update sentry 0.42.0 -> 0.45.0 and sentry-actix 0.42.0 -> 0.45.0

* Update spdx 0.10.9 -> 0.12.0

* Update sysinfo 0.36.1 -> 0.37.2

* Update tauri 2.7.0 -> 2.8.5

* Update tauri-build 2.3.1 -> 2.4.1

* Update tauri-plugin-deep-link 2.4.1 -> 2.4.3

* Update tauri-plugin-dialog 2.3.2 -> 2.4.0

* Update tauri-plugin-http 2.5.1 -> 2.5.2

* Update tauri-plugin-opener 2.4.0 -> 2.5.0

* Update tauri-plugin-os 2.3.0 -> 2.3.1

* Update tauri-plugin-single-instance 2.3.2 -> 2.3.4

* Update tempfile 3.20.0 -> 3.23.0

* Update thiserror 2.0.12 -> 2.0.17

* Update tracing-subscriber 0.3.19 -> 0.3.20

* Update url 2.5.4 -> 2.5.7

* Update uuid 1.17.0 -> 1.18.1

* Update webp 0.3.0 -> 0.3.1

* Update whoami 1.6.0 -> 1.6.1

* Note that windows and windows-core can't be updated yet

* Update zbus 5.9.0 -> 5.11.0

* Update zip 4.3.0 -> 6.0.0

* Fix build

* Enforce rustls crypto provider

* Refresh Cargo.lock

* Update transitive dependencies

* Bump Gradle usage to Java 17

* Use ubuntu-latest consistently across workflows

* Fix lint

* Fix lint in Rust

* Update native-dialog 0.9.0 -> 0.9.2

* Update regex 1.11.3 -> 1.12.2

* Update reqwest 0.12.23 -> 0.12.24

* Update rust_decimal 1.38.0 -> 1.39.0

* Remaining lock-only updates

* chore: move TLS impl of some other dependencies to aws-lc-rs

The AWS bloatware "virus" expands by sheer force of widespread adoption
by the ecosystem... 🫣

* chore(fmt): run Tombi

---------

Co-authored-by: Alejandro González <me@alegon.dev>
2025-10-15 20:45:47 +00:00
aecsocketandGitHub 36d0760a3e Use Nightly + Cranelift for dev, only fail on warnings in CI (#4388)
* Switch to nightly + cranelift

* Fail on warnings only in CI

* Fix check errors

* Don't use mold on Linux to fix CI

* Pin nightly toolchain and add default rustup components

* Fix another CI thing

* PR comment
2025-09-18 18:20:19 +00:00
Alejandro GonzálezandGitHub 3b8cd661bc feat(labrinth): database seed data fixtures for better installation and testing (#4132)
* feat(labrinth): database seed data fixtures for better installation and testing

* chore(labrinth): simplify and fixup seed data fixture

* docs(contributing/labrinth): enable all useful features for `sqlx-cli` install

* chore(docs/labrinth): fix typo

* chore(docs/labrinth): fix `cargo fmt` parameter typo

* chore: replace Labrinth -> labrinth
2025-08-09 14:51:04 +00:00
Alejandro GonzálezandGitHub 0e0ca1971a chore(ci): switch back to upstream cache-cargo-install-action (#4047) 2025-07-22 22:43:04 +00:00
Alejandro GonzálezandGitHub d4516d3527 feat(app): configurable Modrinth endpoints through .env files (#4015) 2025-07-21 22:55:57 +00:00
Josiah GlossonandGitHub 526561f2de Add --color to intl:extract verification (#4023) 2025-07-19 12:42:17 +00:00
Josiah GlossonandGitHub b943638afb Verify that intl:extract has been run (#3979) 2025-07-11 21:48:07 +00:00
Alejandro GonzálezandGitHub c9b98a6154 Small CI flakiness fix and performance tweak (#3780)
* perf(ci): use Turbo to schedule both `lint` and `test` tasks at once

* fix(ci): wait until service containers are initialized for tests

This is achieved by adding a health check to the containers, and
instructing the CI workflow to wait until the containers are healthy.
Not doing this wait risks spurious CI failures due to DB migrations
being applied before the DB even starts.

* chore(turbo): use locally installed schema in new Turbo override file

On the latest versions of Turbo, this ensures that the used schema is
always in sync with what's available in the installed Turbo version,
which is something that has already caused confusion to me before.
2025-06-13 21:34:40 +00:00
Alejandro GonzálezandGitHub 4e3bd4e282 enh(ci): optimize Turbo CI check workflow, track Rust and Node toolchain versions in well-known files (#3776)
* enh(ci): optimize Turbo CI check workflow, track Rust and Node toolchain versions in well-known files

* fix(ci): build `sqlx-cli` with `rustls` to fix Postgres TLS failures
2025-06-12 16:47:28 -07:00
c39bb78e38 App redesign (#2946)
* Start of app redesign

* format

* continue progress

* Content page nearly done

* Fix recursion issues with content page

* Fix update all alignment

* Discover page progress

* Settings progress

* Removed unlocked-size hack that breaks web

* Revamp project page, refactor web project page to share code with app, fixed loading bar, misc UI/UX enhancements, update ko-fi logo, update arrow icons, fix web issues caused by floating-vue migration, fix tooltip issues, update web tooltips, clean up web hydration issues

* Ads + run prettier

* Begin auth refactor, move common messages to ui lib, add i18n extraction to all apps, begin Library refactor

* fix ads not hiding when plus log in

* rev lockfile changes/conflicts

* Fix sign in page

* Add generated

* (mostly) Data driven search

* Fix search mobile issue

* profile fixes

* Project versions page, fix typescript on UI lib and misc fixes

* Remove unused gallery component

* Fix linkfunction err

* Search filter controls at top, localization for locked filters

* Fix provided filter names

* Fix navigating from instance browse to main browse

* Friends frontend (#2995)

* Friends system frontend

* (almost) finish frontend

* finish friends, fix lint

* Fix lint

---------

Signed-off-by: Geometrically <18202329+Geometrically@users.noreply.github.com>

* Refresh macOS app icon

* Update web search UI more

* Fix link opens

* Fix frontend build

---------

Signed-off-by: Geometrically <18202329+Geometrically@users.noreply.github.com>
Co-authored-by: Jai A <jaiagr+gpg@pm.me>
Co-authored-by: Geometrically <18202329+Geometrically@users.noreply.github.com>
2024-12-11 19:54:18 -08:00
Jai A fb6b41630c Fix compose script 2024-10-19 10:07:22 -07:00
Jai A 98c050e7e9 Fix lint again 2024-10-18 17:33:15 -07:00
Jai A 86922c4547 Fix lint not working 2024-10-18 16:54:21 -07:00
Jai A b8ad22a6fb Fix checks 2024-10-18 16:23:32 -07:00
Jai A 8dd955563e Fix clippy errors + lint, use turbo CI 2024-10-18 16:07:35 -07:00
Jai A c143929b69 Move rust checks to own file 2024-10-18 15:43:30 -07:00