mirror of
https://github.com/modrinth/code.git
synced 2026-09-03 13:36:48 +00:00
Technical review queue (#4775)
* chore: fix typo in status message * feat(labrinth): overhaul malware scanner report storage and routes * chore: address some review comments * feat: add Delphi to Docker Compose `with-delphi` profile * chore: fix unused import Clippy lint * feat(labrinth/delphi): use PAT token authorization with project read scopes * chore: expose file IDs in version queries * fix: accept null decompiled source payloads from Delphi * tweak(labrinth): expose base62 file IDs more consistently for Delphi * feat(labrinth/delphi): support new Delphi report severity field * chore(labrinth): run `cargo sqlx prepare` to fix Docker build errors * tweak: add route for fetching Delphi issue type schema, abstract Labrinth away from issue types * chore: run `cargo sqlx prepare` * chore: fix typo on frontend generated state file message * feat: update to use new Delphi issue schema * wip: tech review endpoints * wip: add ToSchema for dependent types * wip: report issues return * wip * wip: returning more data * wip * Fix up db query * Delphi configuration to talk to Labrinth * Get Delphi working with Labrinth * Add Delphi dummy fixture * Better Delphi logging * Improve utoipa for tech review routes * Add more sorting options for tech review queue * Oops join * New routes for fetching issues and reports * Fix which kind of ID is returned in tech review endpoints * Deduplicate tech review report rows * Reduce info sent for projects * Fetch more thread info * Address PR comments * fix ci * fix postgres version mismatch * fix version creation * Implement routes * fix up tech review * Allow adding a moderation comment to Delphi rejections * fix up rebase * exclude rejected projects from tech review * add status change msg to tech review thread * cargo sqlx prepare * also ignore withheld projects * More filtering on issue search * wip: report routes * Fix up for build * cargo sqlx prepare * fix thread message privacy * New tech review search route * submit route * details have statuses now * add default to drid status * dedup issue details * fix sqlx query on empty files * fixes * Dedupe issue detail statuses and message on entering tech rev * Fix qa issues * Fix qa issues * fix review comments * typos * fix ci * feat: tech review frontend (#4781) * chore: fix typo in status message * feat(labrinth): overhaul malware scanner report storage and routes * chore: address some review comments * feat: add Delphi to Docker Compose `with-delphi` profile * chore: fix unused import Clippy lint * feat(labrinth/delphi): use PAT token authorization with project read scopes * chore: expose file IDs in version queries * fix: accept null decompiled source payloads from Delphi * tweak(labrinth): expose base62 file IDs more consistently for Delphi * feat(labrinth/delphi): support new Delphi report severity field * chore(labrinth): run `cargo sqlx prepare` to fix Docker build errors * tweak: add route for fetching Delphi issue type schema, abstract Labrinth away from issue types * chore: run `cargo sqlx prepare` * chore: fix typo on frontend generated state file message * feat: update to use new Delphi issue schema * wip: tech review endpoints * wip: add ToSchema for dependent types * wip: report issues return * wip * wip: returning more data * wip * Fix up db query * Delphi configuration to talk to Labrinth * Get Delphi working with Labrinth * Add Delphi dummy fixture * Better Delphi logging * Improve utoipa for tech review routes * Add more sorting options for tech review queue * Oops join * New routes for fetching issues and reports * Fix which kind of ID is returned in tech review endpoints * Deduplicate tech review report rows * Reduce info sent for projects * Fetch more thread info * Address PR comments * fix ci * fix ci * fix postgres version mismatch * fix version creation * Implement routes * feat: batch scan alert * feat: layout * feat: introduce surface variables * fix: theme selector * feat: rough draft of tech review card * feat: tab switcher * feat: batch scan btn * feat: api-client module for tech review * draft: impl * feat: auto icons * fix: layout issues * feat: fixes to code blocks + flag labels * feat: temp remove mock data * fix: search sort types * fix: intl & lint * chore: re-enable mock data * fix: flag badges + auto open first issue in file tab * feat: update for new routes * fix: more qa issues * feat: lazy load sources * fix: re-enable auth middleware * feat: impl threads * fix: lint & severity * feat: download btn + switch to using NavTabs with new local mode option * feat: re-add toplevel btns * feat: reports page consistency * fix: consistency on project queue * fix: icons + sizing * fix: colors and gaps * fix: impl endpoints * feat: load all flags on file tab * feat: thread generics changes * feat: more qa * feat: fix collapse * fix: qa * feat: msg modal * fix: ISO import * feat: qa fixes * fix: empty state basic * fix: collapsible region * fix: collapse thread by default * feat: rough draft of new process/flow * fix labrinth build * fix thread message privacy * New tech review search route * feat: qa fixes * feat: QA changes * fix: verdict on detail not whole issue * fix: lint + intl * fix: lint * fix: thread message for tech rev verdict * feat: use anim frames * fix: exports + typecheck * polish: qa changes * feat: qa * feat: qa polish * feat: fix malic modal * fix: lint * fix: qa + lint * fix: pagination * fix: lint * fix: qa * intl extract * fix ci --------- Signed-off-by: Calum H. <contact@cal.engineer> Co-authored-by: Alejandro González <me@alegon.dev> Co-authored-by: aecsocket <aecsocket@tutanota.com> --------- Signed-off-by: Calum H. <contact@cal.engineer> Co-authored-by: Alejandro González <me@alegon.dev> Co-authored-by: Calum H. <contact@cal.engineer>
This commit is contained in:
aecsocket
co-authored by
Alejandro González
Calum H.
Alejandro González
Calum H.
parent
1e9e13aebb
commit
39f2b0ecb6
@@ -0,0 +1,422 @@
|
||||
use super::ApiError;
|
||||
use crate::database;
|
||||
use crate::database::redis::RedisPool;
|
||||
use crate::models::ids::OrganizationId;
|
||||
use crate::models::projects::{Project, ProjectStatus};
|
||||
use crate::queue::moderation::{ApprovalType, IdentifiedFile, MissingMetadata};
|
||||
use crate::queue::session::AuthQueue;
|
||||
use crate::util::error::Context;
|
||||
use crate::{auth::check_is_moderator_from_headers, models::pats::Scopes};
|
||||
use actix_web::{HttpRequest, get, post, web};
|
||||
use ariadne::ids::{UserId, random_base62};
|
||||
use ownership::get_projects_ownership;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sqlx::PgPool;
|
||||
use std::collections::HashMap;
|
||||
|
||||
mod ownership;
|
||||
mod tech_review;
|
||||
|
||||
pub fn config(cfg: &mut utoipa_actix_web::service_config::ServiceConfig) {
|
||||
cfg.service(get_projects)
|
||||
.service(get_project_meta)
|
||||
.service(set_project_meta)
|
||||
.service(
|
||||
utoipa_actix_web::scope("/tech-review")
|
||||
.configure(tech_review::config),
|
||||
);
|
||||
}
|
||||
|
||||
#[derive(Deserialize, utoipa::ToSchema)]
|
||||
pub struct ProjectsRequestOptions {
|
||||
/// How many projects to fetch.
|
||||
#[serde(default = "default_count")]
|
||||
pub count: u16,
|
||||
/// How many projects to skip.
|
||||
#[serde(default)]
|
||||
pub offset: u32,
|
||||
}
|
||||
|
||||
fn default_count() -> u16 {
|
||||
100
|
||||
}
|
||||
|
||||
/// Project with extra information fetched from the database, to avoid having
|
||||
/// clients make more round trips.
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct FetchedProject {
|
||||
/// Project info.
|
||||
#[serde(flatten)]
|
||||
pub project: Project,
|
||||
/// Who owns the project.
|
||||
pub ownership: Ownership,
|
||||
}
|
||||
|
||||
/// Fetched information on who owns a project.
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema, Clone)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum Ownership {
|
||||
/// Project is owned by a team, and this is the team owner.
|
||||
User {
|
||||
/// ID of the team owner.
|
||||
id: UserId,
|
||||
/// Name of the team owner.
|
||||
name: String,
|
||||
/// URL of the team owner's icon.
|
||||
icon_url: Option<String>,
|
||||
},
|
||||
/// Project is owned by an organization.
|
||||
Organization {
|
||||
/// ID of the organization.
|
||||
id: OrganizationId,
|
||||
/// Name of the organization.
|
||||
name: String,
|
||||
/// URL of the organization's icon.
|
||||
icon_url: Option<String>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Fetch all projects which are in the moderation queue.
|
||||
#[utoipa::path(
|
||||
responses((status = OK, body = inline(Vec<FetchedProject>)))
|
||||
)]
|
||||
#[get("/projects")]
|
||||
async fn get_projects(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
request_opts: web::Query<ProjectsRequestOptions>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
) -> Result<web::Json<Vec<FetchedProject>>, ApiError> {
|
||||
get_projects_internal(req, pool, redis, request_opts, session_queue).await
|
||||
}
|
||||
|
||||
pub async fn get_projects_internal(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
request_opts: web::Query<ProjectsRequestOptions>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
) -> Result<web::Json<Vec<FetchedProject>>, ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_READ,
|
||||
)
|
||||
.await?;
|
||||
|
||||
use futures::stream::TryStreamExt;
|
||||
|
||||
let project_ids = sqlx::query!(
|
||||
"
|
||||
SELECT id
|
||||
FROM (
|
||||
SELECT DISTINCT ON (m.id)
|
||||
m.id,
|
||||
m.queued
|
||||
FROM mods m
|
||||
|
||||
-- exclude projects in tech review queue
|
||||
LEFT JOIN delphi_issue_details_with_statuses didws
|
||||
ON didws.project_id = m.id AND didws.status = 'pending'
|
||||
|
||||
WHERE
|
||||
m.status = $1
|
||||
AND didws.status IS NULL
|
||||
|
||||
GROUP BY m.id
|
||||
) t
|
||||
|
||||
ORDER BY queued ASC
|
||||
OFFSET $3
|
||||
LIMIT $2
|
||||
",
|
||||
ProjectStatus::Processing.as_str(),
|
||||
request_opts.count as i64,
|
||||
request_opts.offset as i64
|
||||
)
|
||||
.fetch(&**pool)
|
||||
.map_ok(|m| database::models::DBProjectId(m.id))
|
||||
.try_collect::<Vec<database::models::DBProjectId>>()
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch projects awaiting review")?;
|
||||
|
||||
let projects =
|
||||
database::DBProject::get_many_ids(&project_ids, &**pool, &redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch projects")?
|
||||
.into_iter()
|
||||
.map(crate::models::projects::Project::from)
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
let ownerships = get_projects_ownership(&projects, &pool, &redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch project ownerships")?;
|
||||
|
||||
let map_project =
|
||||
|(project, ownership): (Project, Ownership)| -> FetchedProject {
|
||||
FetchedProject { ownership, project }
|
||||
};
|
||||
|
||||
let projects = projects
|
||||
.into_iter()
|
||||
.zip(ownerships)
|
||||
.map(map_project)
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
Ok(web::Json(projects))
|
||||
}
|
||||
|
||||
/// Fetch moderation metadata for a specific project.
|
||||
#[utoipa::path(
|
||||
responses((status = OK, body = inline(Vec<Project>)))
|
||||
)]
|
||||
#[get("/project/{id}")]
|
||||
async fn get_project_meta(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
info: web::Path<(String,)>,
|
||||
) -> Result<web::Json<MissingMetadata>, ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_READ,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let project_id = info.into_inner().0;
|
||||
let project =
|
||||
database::models::DBProject::get(&project_id, &**pool, &redis).await?;
|
||||
|
||||
if let Some(project) = project {
|
||||
let rows = sqlx::query!(
|
||||
"
|
||||
SELECT
|
||||
f.metadata, v.id version_id
|
||||
FROM versions v
|
||||
INNER JOIN files f ON f.version_id = v.id
|
||||
WHERE v.mod_id = $1
|
||||
",
|
||||
project.inner.id.0
|
||||
)
|
||||
.fetch_all(&**pool)
|
||||
.await?;
|
||||
|
||||
let mut merged = MissingMetadata {
|
||||
identified: HashMap::new(),
|
||||
flame_files: HashMap::new(),
|
||||
unknown_files: HashMap::new(),
|
||||
};
|
||||
|
||||
let mut check_hashes = Vec::new();
|
||||
let mut check_flames = Vec::new();
|
||||
|
||||
for row in rows {
|
||||
if let Some(metadata) = row
|
||||
.metadata
|
||||
.and_then(|x| serde_json::from_value::<MissingMetadata>(x).ok())
|
||||
{
|
||||
merged.identified.extend(metadata.identified);
|
||||
merged.flame_files.extend(metadata.flame_files);
|
||||
merged.unknown_files.extend(metadata.unknown_files);
|
||||
|
||||
check_hashes.extend(merged.flame_files.keys().cloned());
|
||||
check_hashes.extend(merged.unknown_files.keys().cloned());
|
||||
check_flames
|
||||
.extend(merged.flame_files.values().map(|x| x.id as i32));
|
||||
}
|
||||
}
|
||||
|
||||
let rows = sqlx::query!(
|
||||
"
|
||||
SELECT encode(mef.sha1, 'escape') sha1, mel.status status
|
||||
FROM moderation_external_files mef
|
||||
INNER JOIN moderation_external_licenses mel ON mef.external_license_id = mel.id
|
||||
WHERE mef.sha1 = ANY($1)
|
||||
",
|
||||
&check_hashes
|
||||
.iter()
|
||||
.map(|x| x.as_bytes().to_vec())
|
||||
.collect::<Vec<_>>()
|
||||
)
|
||||
.fetch_all(&**pool)
|
||||
.await?;
|
||||
|
||||
for row in rows {
|
||||
if let Some(sha1) = row.sha1 {
|
||||
if let Some(val) = merged.flame_files.remove(&sha1) {
|
||||
merged.identified.insert(
|
||||
sha1,
|
||||
IdentifiedFile {
|
||||
file_name: val.file_name,
|
||||
status: ApprovalType::from_string(&row.status)
|
||||
.unwrap_or(ApprovalType::Unidentified),
|
||||
},
|
||||
);
|
||||
} else if let Some(val) = merged.unknown_files.remove(&sha1) {
|
||||
merged.identified.insert(
|
||||
sha1,
|
||||
IdentifiedFile {
|
||||
file_name: val,
|
||||
status: ApprovalType::from_string(&row.status)
|
||||
.unwrap_or(ApprovalType::Unidentified),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let rows = sqlx::query!(
|
||||
"
|
||||
SELECT mel.id, mel.flame_project_id, mel.status status
|
||||
FROM moderation_external_licenses mel
|
||||
WHERE mel.flame_project_id = ANY($1)
|
||||
",
|
||||
&check_flames,
|
||||
)
|
||||
.fetch_all(&**pool)
|
||||
.await?;
|
||||
|
||||
for row in rows {
|
||||
if let Some(sha1) = merged
|
||||
.flame_files
|
||||
.iter()
|
||||
.find(|x| Some(x.1.id as i32) == row.flame_project_id)
|
||||
.map(|x| x.0.clone())
|
||||
&& let Some(val) = merged.flame_files.remove(&sha1)
|
||||
{
|
||||
merged.identified.insert(
|
||||
sha1,
|
||||
IdentifiedFile {
|
||||
file_name: val.file_name.clone(),
|
||||
status: ApprovalType::from_string(&row.status)
|
||||
.unwrap_or(ApprovalType::Unidentified),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(web::Json(merged))
|
||||
} else {
|
||||
Err(ApiError::NotFound)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum Judgement {
|
||||
Flame {
|
||||
id: i32,
|
||||
status: ApprovalType,
|
||||
link: String,
|
||||
title: String,
|
||||
},
|
||||
Unknown {
|
||||
status: ApprovalType,
|
||||
proof: Option<String>,
|
||||
link: Option<String>,
|
||||
title: Option<String>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Update moderation judgements for projects in the review queue.
|
||||
#[utoipa::path]
|
||||
#[post("/project")]
|
||||
async fn set_project_meta(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
judgements: web::Json<HashMap<String, Judgement>>,
|
||||
) -> Result<(), ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_READ,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let mut transaction = pool.begin().await?;
|
||||
|
||||
let mut ids = Vec::new();
|
||||
let mut titles = Vec::new();
|
||||
let mut statuses = Vec::new();
|
||||
let mut links = Vec::new();
|
||||
let mut proofs = Vec::new();
|
||||
let mut flame_ids = Vec::new();
|
||||
|
||||
let mut file_hashes = Vec::new();
|
||||
|
||||
for (hash, judgement) in judgements.0 {
|
||||
let id = random_base62(8);
|
||||
|
||||
let (title, status, link, proof, flame_id) = match judgement {
|
||||
Judgement::Flame {
|
||||
id,
|
||||
status,
|
||||
link,
|
||||
title,
|
||||
} => (
|
||||
Some(title),
|
||||
status,
|
||||
Some(link),
|
||||
Some("See Flame page/license for permission".to_string()),
|
||||
Some(id),
|
||||
),
|
||||
Judgement::Unknown {
|
||||
status,
|
||||
proof,
|
||||
link,
|
||||
title,
|
||||
} => (title, status, link, proof, None),
|
||||
};
|
||||
|
||||
ids.push(id as i64);
|
||||
titles.push(title);
|
||||
statuses.push(status.as_str());
|
||||
links.push(link);
|
||||
proofs.push(proof);
|
||||
flame_ids.push(flame_id);
|
||||
file_hashes.push(hash);
|
||||
}
|
||||
|
||||
sqlx::query(
|
||||
"
|
||||
INSERT INTO moderation_external_licenses (id, title, status, link, proof, flame_project_id)
|
||||
SELECT * FROM UNNEST ($1::bigint[], $2::varchar[], $3::varchar[], $4::varchar[], $5::varchar[], $6::integer[])
|
||||
"
|
||||
)
|
||||
.bind(&ids[..])
|
||||
.bind(&titles[..])
|
||||
.bind(&statuses[..])
|
||||
.bind(&links[..])
|
||||
.bind(&proofs[..])
|
||||
.bind(&flame_ids[..])
|
||||
.execute(&mut *transaction)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"
|
||||
INSERT INTO moderation_external_files (sha1, external_license_id)
|
||||
SELECT * FROM UNNEST ($1::bytea[], $2::bigint[])
|
||||
ON CONFLICT (sha1)
|
||||
DO NOTHING
|
||||
",
|
||||
)
|
||||
.bind(&file_hashes[..])
|
||||
.bind(&ids[..])
|
||||
.execute(&mut *transaction)
|
||||
.await?;
|
||||
|
||||
transaction.commit().await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
use crate::database::models::{DBOrganization, DBTeamId, DBTeamMember, DBUser};
|
||||
use crate::database::redis::RedisPool;
|
||||
use crate::models::ids::OrganizationId;
|
||||
use crate::routes::internal::moderation::Ownership;
|
||||
use crate::util::error::Context;
|
||||
use ariadne::ids::UserId;
|
||||
use eyre::eyre;
|
||||
use sqlx::PgPool;
|
||||
|
||||
/// Fetches ownership information for multiple projects efficiently
|
||||
pub async fn get_projects_ownership(
|
||||
projects: &[crate::models::projects::Project],
|
||||
pool: &PgPool,
|
||||
redis: &RedisPool,
|
||||
) -> Result<Vec<Ownership>, crate::routes::ApiError> {
|
||||
let team_ids = projects
|
||||
.iter()
|
||||
.map(|project| project.team_id)
|
||||
.map(DBTeamId::from)
|
||||
.collect::<Vec<_>>();
|
||||
let org_ids = projects
|
||||
.iter()
|
||||
.filter_map(|project| project.organization)
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
let team_members =
|
||||
DBTeamMember::get_from_team_full_many(&team_ids, pool, redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch team members")?;
|
||||
let users = DBUser::get_many_ids(
|
||||
&team_members
|
||||
.iter()
|
||||
.map(|member| member.user_id)
|
||||
.collect::<Vec<_>>(),
|
||||
pool,
|
||||
redis,
|
||||
)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch user data of team members")?;
|
||||
let orgs = DBOrganization::get_many(&org_ids, pool, redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch organizations")?;
|
||||
|
||||
let mut ownerships = Vec::with_capacity(projects.len());
|
||||
|
||||
for project in projects {
|
||||
let project_id = project.id;
|
||||
let ownership = if let Some(org_id) = project.organization {
|
||||
let org = orgs
|
||||
.iter()
|
||||
.find(|org| OrganizationId::from(org.id) == org_id)
|
||||
.wrap_internal_err_with(|| {
|
||||
eyre!(
|
||||
"project {project_id} is owned by an invalid organization {org_id}"
|
||||
)
|
||||
})?;
|
||||
|
||||
Ownership::Organization {
|
||||
id: OrganizationId::from(org.id),
|
||||
name: org.name.clone(),
|
||||
icon_url: org.icon_url.clone(),
|
||||
}
|
||||
} else {
|
||||
let team_id = project.team_id;
|
||||
let team_owner = team_members.iter().find(|member| {
|
||||
crate::models::ids::TeamId::from(member.team_id) == team_id && member.is_owner
|
||||
})
|
||||
.wrap_internal_err_with(|| eyre!("project {project_id} is owned by a team {team_id} which has no valid owner"))?;
|
||||
let team_owner_id = team_owner.user_id;
|
||||
let user = users.iter().find(|user| user.id == team_owner_id)
|
||||
.wrap_internal_err_with(|| eyre!("project {project_id} is owned by a team {team_id} which has owner {} which does not exist", UserId::from(team_owner_id)))?;
|
||||
|
||||
Ownership::User {
|
||||
id: ariadne::ids::UserId::from(user.id),
|
||||
name: user.username.clone(),
|
||||
icon_url: user.avatar_url.clone(),
|
||||
}
|
||||
};
|
||||
|
||||
ownerships.push(ownership);
|
||||
}
|
||||
|
||||
Ok(ownerships)
|
||||
}
|
||||
@@ -0,0 +1,894 @@
|
||||
use std::{collections::HashMap, fmt};
|
||||
|
||||
use actix_web::{HttpRequest, get, patch, post, put, web};
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sqlx::PgPool;
|
||||
use tokio_stream::StreamExt;
|
||||
|
||||
use super::ownership::get_projects_ownership;
|
||||
use crate::{
|
||||
auth::check_is_moderator_from_headers,
|
||||
database::{
|
||||
DBProject,
|
||||
models::{
|
||||
DBFileId, DBProjectId, DBThread, DBThreadId, DBUser,
|
||||
DelphiReportId, DelphiReportIssueDetailsId, DelphiReportIssueId,
|
||||
ProjectTypeId,
|
||||
delphi_report_item::{
|
||||
DBDelphiReport, DelphiSeverity, DelphiStatus, DelphiVerdict,
|
||||
ReportIssueDetail,
|
||||
},
|
||||
thread_item::ThreadMessageBuilder,
|
||||
},
|
||||
redis::RedisPool,
|
||||
},
|
||||
models::{
|
||||
ids::{FileId, ProjectId, ThreadId, VersionId},
|
||||
pats::Scopes,
|
||||
projects::{Project, ProjectStatus},
|
||||
threads::{MessageBody, Thread},
|
||||
},
|
||||
queue::session::AuthQueue,
|
||||
routes::{ApiError, internal::moderation::Ownership},
|
||||
util::error::Context,
|
||||
};
|
||||
use eyre::eyre;
|
||||
|
||||
pub fn config(cfg: &mut utoipa_actix_web::service_config::ServiceConfig) {
|
||||
cfg.service(search_projects)
|
||||
.service(get_report)
|
||||
.service(get_issue)
|
||||
.service(submit_report)
|
||||
.service(update_issue_detail)
|
||||
.service(add_report);
|
||||
}
|
||||
|
||||
/// Arguments for searching project technical reviews.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct SearchProjects {
|
||||
#[serde(default = "default_limit")]
|
||||
#[schema(default = 20)]
|
||||
pub limit: u64,
|
||||
#[serde(default)]
|
||||
#[schema(default = 0)]
|
||||
pub page: u64,
|
||||
#[serde(default)]
|
||||
pub filter: SearchProjectsFilter,
|
||||
#[serde(default = "default_sort_by")]
|
||||
pub sort_by: SearchProjectsSort,
|
||||
}
|
||||
|
||||
fn default_limit() -> u64 {
|
||||
20
|
||||
}
|
||||
|
||||
fn default_sort_by() -> SearchProjectsSort {
|
||||
SearchProjectsSort::CreatedAsc
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct SearchProjectsFilter {
|
||||
pub project_type: Vec<ProjectTypeId>,
|
||||
}
|
||||
|
||||
#[derive(
|
||||
Debug,
|
||||
Clone,
|
||||
Copy,
|
||||
PartialEq,
|
||||
Eq,
|
||||
Hash,
|
||||
Serialize,
|
||||
Deserialize,
|
||||
utoipa::ToSchema,
|
||||
)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum SearchProjectsSort {
|
||||
CreatedAsc,
|
||||
CreatedDesc,
|
||||
SeverityAsc,
|
||||
SeverityDesc,
|
||||
}
|
||||
|
||||
impl fmt::Display for SearchProjectsSort {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
let s = serde_json::to_value(*self).unwrap();
|
||||
let s = s.as_str().unwrap();
|
||||
write!(f, "{s}")
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct FileReport {
|
||||
/// ID of this report.
|
||||
pub report_id: DelphiReportId,
|
||||
/// ID of the file that was scanned.
|
||||
pub file_id: FileId,
|
||||
/// When the report for this file was created.
|
||||
pub created: DateTime<Utc>,
|
||||
/// Why this project was flagged.
|
||||
pub flag_reason: FlagReason,
|
||||
/// According to this report, how likely is the project malicious?
|
||||
pub severity: DelphiSeverity,
|
||||
/// Name of the flagged file.
|
||||
pub file_name: String,
|
||||
/// Size of the flagged file, in bytes.
|
||||
pub file_size: i32,
|
||||
/// URL to download the flagged file.
|
||||
pub download_url: String,
|
||||
/// What issues appeared in the file.
|
||||
#[serde(default)]
|
||||
pub issues: Vec<FileIssue>,
|
||||
}
|
||||
|
||||
/// Issue raised by Delphi in a flagged file.
|
||||
///
|
||||
/// The issue is scoped to the JAR, not any specific class, but issues can be
|
||||
/// raised because they appeared in a class - see [`FileIssueDetails`].
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct FileIssue {
|
||||
/// ID of the issue.
|
||||
pub id: DelphiReportIssueId,
|
||||
/// ID of the report this issue is a part of.
|
||||
pub report_id: DelphiReportId,
|
||||
/// Delphi-determined kind of issue that this is, e.g. `OBFUSCATED_NAMES`.
|
||||
///
|
||||
/// Labrinth does not know the full set of kinds of issues, so this is kept
|
||||
/// as a string.
|
||||
pub issue_type: String,
|
||||
/// Details of why this issue might have been raised, such as what file it
|
||||
/// was found in.
|
||||
#[serde(default)]
|
||||
pub details: Vec<ReportIssueDetail>,
|
||||
}
|
||||
|
||||
/// Why a project was flagged for technical review.
|
||||
#[derive(
|
||||
Debug,
|
||||
Clone,
|
||||
Copy,
|
||||
PartialEq,
|
||||
Eq,
|
||||
Hash,
|
||||
Serialize,
|
||||
Deserialize,
|
||||
utoipa::ToSchema,
|
||||
)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum FlagReason {
|
||||
/// Delphi anti-malware scanner flagged a file in the project.
|
||||
Delphi,
|
||||
}
|
||||
|
||||
/// Get info on an issue in a Delphi report.
|
||||
#[utoipa::path(
|
||||
security(("bearer_auth" = [])),
|
||||
responses((status = OK, body = inline(FileIssue)))
|
||||
)]
|
||||
#[get("/issue/{issue_id}")]
|
||||
async fn get_issue(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
path: web::Path<(DelphiReportIssueId,)>,
|
||||
) -> Result<web::Json<FileIssue>, ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_READ,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let (issue_id,) = path.into_inner();
|
||||
let row = sqlx::query!(
|
||||
r#"
|
||||
SELECT
|
||||
to_jsonb(dri)
|
||||
|| jsonb_build_object(
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'details', (
|
||||
SELECT json_agg(to_jsonb(drid))
|
||||
FROM delphi_report_issue_details drid
|
||||
WHERE drid.issue_id = dri.id
|
||||
)
|
||||
) AS "data!: sqlx::types::Json<FileIssue>"
|
||||
FROM delphi_report_issues dri
|
||||
WHERE dri.id = $1
|
||||
"#,
|
||||
issue_id as DelphiReportIssueId,
|
||||
)
|
||||
.fetch_optional(&**pool)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch issue from database")?
|
||||
.ok_or(ApiError::NotFound)?;
|
||||
|
||||
Ok(web::Json(row.data.0))
|
||||
}
|
||||
|
||||
/// Get info on a specific report for a project.
|
||||
#[utoipa::path(
|
||||
security(("bearer_auth" = [])),
|
||||
responses((status = OK, body = inline(FileReport)))
|
||||
)]
|
||||
#[get("/report/{id}")]
|
||||
async fn get_report(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
path: web::Path<(DelphiReportId,)>,
|
||||
) -> Result<web::Json<FileReport>, ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_READ,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let (report_id,) = path.into_inner();
|
||||
|
||||
let row = sqlx::query!(
|
||||
r#"
|
||||
SELECT DISTINCT ON (dr.id)
|
||||
to_jsonb(dr)
|
||||
|| jsonb_build_object(
|
||||
'file_id', to_base62(f.id),
|
||||
'version_id', to_base62(v.id),
|
||||
'project_id', to_base62(v.mod_id),
|
||||
'file_name', f.filename,
|
||||
'file_size', f.size,
|
||||
'flag_reason', 'delphi',
|
||||
'download_url', f.url,
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'issues', (
|
||||
SELECT json_agg(
|
||||
to_jsonb(dri)
|
||||
|| jsonb_build_object(
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'details', (
|
||||
SELECT json_agg(to_jsonb(drid))
|
||||
FROM delphi_report_issue_details drid
|
||||
WHERE drid.issue_id = dri.id
|
||||
)
|
||||
)
|
||||
)
|
||||
FROM delphi_report_issues dri
|
||||
WHERE
|
||||
dri.report_id = dr.id
|
||||
-- see delphi.rs todo comment
|
||||
AND dri.issue_type != '__dummy'
|
||||
)
|
||||
) AS "data!: sqlx::types::Json<FileReport>"
|
||||
FROM delphi_reports dr
|
||||
INNER JOIN files f ON f.id = dr.file_id
|
||||
INNER JOIN versions v ON v.id = f.version_id
|
||||
WHERE dr.id = $1
|
||||
"#,
|
||||
report_id as DelphiReportId,
|
||||
)
|
||||
.fetch_optional(&**pool)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch report from database")?
|
||||
.ok_or(ApiError::NotFound)?;
|
||||
|
||||
Ok(web::Json(row.data.0))
|
||||
}
|
||||
|
||||
/// See [`search_projects`].
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct SearchResponse {
|
||||
/// List of reported projects returned, and their report data.
|
||||
pub project_reports: Vec<ProjectReport>,
|
||||
/// Fetched project information for projects in the returned reports.
|
||||
pub projects: HashMap<ProjectId, ProjectModerationInfo>,
|
||||
/// Fetched moderation threads for projects in the returned reports.
|
||||
pub threads: HashMap<ThreadId, Thread>,
|
||||
/// Fetched owner information for projects.
|
||||
pub ownership: HashMap<ProjectId, Ownership>,
|
||||
}
|
||||
|
||||
/// Single project's reports from a search response.
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct ProjectReport {
|
||||
/// ID of the project this report is for.
|
||||
pub project_id: ProjectId,
|
||||
/// Highest severity of any report of any file of any version under this
|
||||
/// project.
|
||||
pub max_severity: Option<DelphiSeverity>,
|
||||
/// Reports for this project's versions.
|
||||
#[serde(default)]
|
||||
pub versions: Vec<VersionReport>,
|
||||
}
|
||||
|
||||
/// Single project version's reports from a search response.
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct VersionReport {
|
||||
/// ID of the project version this report is for.
|
||||
pub version_id: VersionId,
|
||||
/// Reports for this version's files.
|
||||
#[serde(default)]
|
||||
pub files: Vec<FileReport>,
|
||||
}
|
||||
|
||||
/// Limited set of project information returned by [`search_projects`].
|
||||
#[derive(Debug, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct ProjectModerationInfo {
|
||||
/// Project ID.
|
||||
pub id: ProjectId,
|
||||
/// Project moderation thread ID.
|
||||
pub thread_id: ThreadId,
|
||||
/// Project name.
|
||||
pub name: String,
|
||||
/// The aggregated project typos of the versions of this project
|
||||
#[serde(default)]
|
||||
pub project_types: Vec<String>,
|
||||
/// The URL of the icon of the project
|
||||
pub icon_url: Option<String>,
|
||||
}
|
||||
|
||||
/// Searches all projects which are awaiting technical review.
|
||||
#[utoipa::path(
|
||||
security(("bearer_auth" = [])),
|
||||
responses((status = OK, body = inline(Vec<SearchResponse>)))
|
||||
)]
|
||||
#[post("/search")]
|
||||
async fn search_projects(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
search_req: web::Json<SearchProjects>,
|
||||
) -> Result<web::Json<SearchResponse>, ApiError> {
|
||||
let user = check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_READ,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let sort_by = search_req.sort_by.to_string();
|
||||
let limit = search_req.limit.max(50);
|
||||
let offset = limit.saturating_mul(search_req.page);
|
||||
|
||||
let limit =
|
||||
i64::try_from(limit).wrap_request_err("limit cannot fit into `i64`")?;
|
||||
let offset = i64::try_from(offset)
|
||||
.wrap_request_err("offset cannot fit into `i64`")?;
|
||||
|
||||
let mut project_reports = Vec::<ProjectReport>::new();
|
||||
let mut project_ids = Vec::<DBProjectId>::new();
|
||||
let mut thread_ids = Vec::<DBThreadId>::new();
|
||||
|
||||
let mut rows = sqlx::query!(
|
||||
r#"
|
||||
SELECT
|
||||
project_id AS "project_id: DBProjectId",
|
||||
project_thread_id AS "project_thread_id: DBThreadId",
|
||||
report AS "report!: sqlx::types::Json<ProjectReport>"
|
||||
FROM (
|
||||
SELECT DISTINCT ON (m.id)
|
||||
m.id AS project_id,
|
||||
t.id AS project_thread_id,
|
||||
MAX(dr.severity) AS severity,
|
||||
MIN(dr.created) AS earliest_report_created,
|
||||
MAX(dr.created) AS latest_report_created,
|
||||
|
||||
jsonb_build_object(
|
||||
'project_id', to_base62(m.id),
|
||||
'max_severity', MAX(dr.severity),
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'versions', (
|
||||
SELECT coalesce(jsonb_agg(jsonb_build_object(
|
||||
'version_id', to_base62(v.id),
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'files', (
|
||||
SELECT coalesce(jsonb_agg(jsonb_build_object(
|
||||
'report_id', dr.id,
|
||||
'file_id', to_base62(f.id),
|
||||
'created', dr.created,
|
||||
'flag_reason', 'delphi',
|
||||
'severity', dr.severity,
|
||||
'file_name', f.filename,
|
||||
'file_size', f.size,
|
||||
'download_url', f.url,
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'issues', (
|
||||
SELECT coalesce(jsonb_agg(
|
||||
to_jsonb(dri)
|
||||
|| jsonb_build_object(
|
||||
-- TODO: replace with `json_array` in Postgres 16
|
||||
'details', (
|
||||
SELECT coalesce(jsonb_agg(
|
||||
jsonb_build_object(
|
||||
'id', didws.id,
|
||||
'issue_id', didws.issue_id,
|
||||
'key', didws.key,
|
||||
'file_path', didws.file_path,
|
||||
-- ignore `decompiled_source`
|
||||
'data', didws.data,
|
||||
'severity', didws.severity,
|
||||
'status', didws.status
|
||||
)
|
||||
), '[]'::jsonb)
|
||||
FROM delphi_issue_details_with_statuses didws
|
||||
WHERE didws.issue_id = dri.id
|
||||
)
|
||||
)
|
||||
), '[]'::jsonb)
|
||||
FROM delphi_report_issues dri
|
||||
WHERE
|
||||
dri.report_id = dr.id
|
||||
-- see delphi.rs todo comment
|
||||
AND dri.issue_type != '__dummy'
|
||||
)
|
||||
)), '[]'::jsonb)
|
||||
FROM delphi_reports dr
|
||||
WHERE dr.file_id = f.id
|
||||
)
|
||||
)), '[]'::jsonb)
|
||||
FROM versions v
|
||||
INNER JOIN files f ON f.version_id = v.id
|
||||
WHERE v.mod_id = m.id
|
||||
)
|
||||
) AS report
|
||||
FROM mods m
|
||||
INNER JOIN threads t ON t.mod_id = m.id
|
||||
INNER JOIN versions v ON v.mod_id = m.id
|
||||
INNER JOIN files f ON f.version_id = v.id
|
||||
|
||||
-- only return projects with at least 1 pending drid
|
||||
INNER JOIN delphi_reports dr ON dr.file_id = f.id
|
||||
INNER JOIN delphi_issue_details_with_statuses didws
|
||||
ON didws.project_id = m.id AND didws.status = 'pending'
|
||||
|
||||
-- filtering
|
||||
LEFT JOIN mods_categories mc ON mc.joining_mod_id = m.id
|
||||
LEFT JOIN categories c ON c.id = mc.joining_category_id
|
||||
WHERE
|
||||
-- project type
|
||||
(cardinality($4::int[]) = 0 OR c.project_type = ANY($4::int[]))
|
||||
AND m.status NOT IN ('draft', 'rejected', 'withheld')
|
||||
|
||||
GROUP BY m.id, t.id
|
||||
) t
|
||||
|
||||
-- sorting
|
||||
ORDER BY
|
||||
CASE WHEN $3 = 'created_asc' THEN t.earliest_report_created ELSE TO_TIMESTAMP(0) END ASC,
|
||||
CASE WHEN $3 = 'created_desc' THEN t.latest_report_created ELSE TO_TIMESTAMP(0) END DESC,
|
||||
CASE WHEN $3 = 'severity_asc' THEN t.severity ELSE 'low'::delphi_severity END ASC,
|
||||
CASE WHEN $3 = 'severity_desc' THEN t.severity ELSE 'low'::delphi_severity END DESC
|
||||
|
||||
-- pagination
|
||||
LIMIT $1
|
||||
OFFSET $2
|
||||
"#,
|
||||
limit,
|
||||
offset,
|
||||
&sort_by,
|
||||
&search_req
|
||||
.filter
|
||||
.project_type
|
||||
.iter()
|
||||
.map(|ty| ty.0)
|
||||
.collect::<Vec<_>>(),
|
||||
)
|
||||
.fetch(&**pool);
|
||||
|
||||
while let Some(row) = rows
|
||||
.next()
|
||||
.await
|
||||
.transpose()
|
||||
.wrap_internal_err("failed to fetch reports")?
|
||||
{
|
||||
project_reports.push(row.report.0);
|
||||
project_ids.push(row.project_id);
|
||||
thread_ids.push(row.project_thread_id);
|
||||
}
|
||||
|
||||
let projects = DBProject::get_many_ids(&project_ids, &**pool, &redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch projects")?
|
||||
.into_iter()
|
||||
.map(|project| {
|
||||
(ProjectId::from(project.inner.id), Project::from(project))
|
||||
})
|
||||
.collect::<HashMap<_, _>>();
|
||||
let db_threads = DBThread::get_many(&thread_ids, &**pool)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch threads")?;
|
||||
let thread_author_ids = db_threads
|
||||
.iter()
|
||||
.flat_map(|thread| {
|
||||
thread
|
||||
.messages
|
||||
.iter()
|
||||
.filter_map(|message| message.author_id)
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
let thread_authors =
|
||||
DBUser::get_many_ids(&thread_author_ids, &**pool, &redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch thread authors")?
|
||||
.into_iter()
|
||||
.map(From::from)
|
||||
.collect::<Vec<_>>();
|
||||
let threads = db_threads
|
||||
.into_iter()
|
||||
.map(|thread| {
|
||||
let thread = Thread::from(thread, thread_authors.clone(), &user);
|
||||
(thread.id, thread)
|
||||
})
|
||||
.collect::<HashMap<_, _>>();
|
||||
|
||||
let project_list: Vec<Project> = projects.values().cloned().collect();
|
||||
|
||||
let ownerships = get_projects_ownership(&project_list, &pool, &redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch project ownerships")?;
|
||||
let ownership = projects
|
||||
.keys()
|
||||
.copied()
|
||||
.zip(ownerships)
|
||||
.collect::<HashMap<_, _>>();
|
||||
|
||||
Ok(web::Json(SearchResponse {
|
||||
project_reports,
|
||||
projects: projects
|
||||
.into_iter()
|
||||
.map(|(id, project)| {
|
||||
(
|
||||
id,
|
||||
ProjectModerationInfo {
|
||||
id,
|
||||
thread_id: project.thread_id,
|
||||
name: project.name,
|
||||
project_types: project.project_types,
|
||||
icon_url: project.icon_url,
|
||||
},
|
||||
)
|
||||
})
|
||||
.collect(),
|
||||
threads,
|
||||
ownership,
|
||||
}))
|
||||
}
|
||||
|
||||
/// See [`submit_report`].
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct SubmitReport {
|
||||
/// Does the moderator think this report shows that the project is safe or
|
||||
/// unsafe?
|
||||
pub verdict: DelphiVerdict,
|
||||
/// Moderator message to send to the thread when rejecting the project.
|
||||
pub message: Option<String>,
|
||||
}
|
||||
|
||||
/// Submits a verdict for a project based on its technical reports.
|
||||
///
|
||||
/// Before this is called, all issues for this project's reports must have been
|
||||
/// marked as either safe or unsafe. Otherwise, this will error with
|
||||
/// [`ApiError::TechReviewIssuesWithNoVerdict`], providing the issue IDs which
|
||||
/// are still unmarked.
|
||||
#[utoipa::path(
|
||||
security(("bearer_auth" = [])),
|
||||
responses((status = NO_CONTENT))
|
||||
)]
|
||||
#[post("/submit/{project_id}")]
|
||||
async fn submit_report(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
web::Json(submit_report): web::Json<SubmitReport>,
|
||||
path: web::Path<(ProjectId,)>,
|
||||
) -> Result<(), ApiError> {
|
||||
let user = check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_WRITE,
|
||||
)
|
||||
.await?;
|
||||
let (project_id,) = path.into_inner();
|
||||
let project_id = DBProjectId::from(project_id);
|
||||
|
||||
let mut txn = pool
|
||||
.begin()
|
||||
.await
|
||||
.wrap_internal_err("failed to begin transaction")?;
|
||||
|
||||
let pending_issue_details = sqlx::query!(
|
||||
r#"
|
||||
SELECT
|
||||
didws.id AS "issue_detail_id!"
|
||||
FROM mods m
|
||||
INNER JOIN versions v ON v.mod_id = m.id
|
||||
INNER JOIN files f ON f.version_id = v.id
|
||||
INNER JOIN delphi_reports dr ON dr.file_id = f.id
|
||||
INNER JOIN delphi_report_issues dri ON dri.report_id = dr.id
|
||||
INNER JOIN delphi_issue_details_with_statuses didws ON didws.issue_id = dri.id
|
||||
WHERE
|
||||
m.id = $1
|
||||
AND didws.status = 'pending'
|
||||
-- see delphi.rs todo comment
|
||||
AND dri.issue_type != '__dummy'
|
||||
"#,
|
||||
project_id as _,
|
||||
)
|
||||
.fetch_all(&mut *txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch pending issues")?;
|
||||
|
||||
if !pending_issue_details.is_empty() {
|
||||
return Err(ApiError::TechReviewDetailsWithNoVerdict {
|
||||
details: pending_issue_details
|
||||
.into_iter()
|
||||
.map(|record| {
|
||||
DelphiReportIssueDetailsId(record.issue_detail_id)
|
||||
})
|
||||
.collect(),
|
||||
});
|
||||
}
|
||||
|
||||
sqlx::query!(
|
||||
"
|
||||
DELETE FROM delphi_report_issue_details drid
|
||||
WHERE issue_id IN (
|
||||
SELECT dri.id
|
||||
FROM mods m
|
||||
INNER JOIN versions v ON v.mod_id = m.id
|
||||
INNER JOIN files f ON f.version_id = v.id
|
||||
INNER JOIN delphi_reports dr ON dr.file_id = f.id
|
||||
INNER JOIN delphi_report_issues dri ON dri.report_id = dr.id
|
||||
WHERE m.id = $1 AND dri.issue_type = '__dummy'
|
||||
)
|
||||
",
|
||||
project_id as _,
|
||||
)
|
||||
.execute(&mut *txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to delete dummy issue")?;
|
||||
|
||||
let record = sqlx::query!(
|
||||
r#"
|
||||
SELECT t.id AS "thread_id: DBThreadId"
|
||||
FROM mods m
|
||||
INNER JOIN threads t ON t.mod_id = m.id
|
||||
WHERE m.id = $1
|
||||
"#,
|
||||
project_id as _,
|
||||
)
|
||||
.fetch_one(&mut *txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to update reports")?;
|
||||
|
||||
if let Some(body) = submit_report.message {
|
||||
ThreadMessageBuilder {
|
||||
author_id: Some(user.id.into()),
|
||||
body: MessageBody::Text {
|
||||
body,
|
||||
private: true,
|
||||
replying_to: None,
|
||||
associated_images: Vec::new(),
|
||||
},
|
||||
thread_id: record.thread_id,
|
||||
hide_identity: user.role.is_mod(),
|
||||
}
|
||||
.insert(&mut txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to add moderator message")?;
|
||||
}
|
||||
|
||||
let verdict = submit_report.verdict;
|
||||
ThreadMessageBuilder {
|
||||
author_id: Some(user.id.into()),
|
||||
body: MessageBody::TechReview { verdict },
|
||||
thread_id: record.thread_id,
|
||||
hide_identity: user.role.is_mod(),
|
||||
}
|
||||
.insert(&mut txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to add tech review message")?;
|
||||
|
||||
if verdict == DelphiVerdict::Unsafe {
|
||||
let record = sqlx::query!(
|
||||
r#"
|
||||
UPDATE mods
|
||||
SET status = $1
|
||||
FROM mods m
|
||||
INNER JOIN threads t ON t.mod_id = m.id
|
||||
WHERE m.id = $2
|
||||
RETURNING
|
||||
t.id AS "thread_id: DBThreadId",
|
||||
(SELECT status FROM mods WHERE id = m.id) AS "old_status!"
|
||||
"#,
|
||||
ProjectStatus::Rejected.as_str(),
|
||||
project_id as _,
|
||||
)
|
||||
.fetch_one(&mut *txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to mark project as rejected")?;
|
||||
|
||||
ThreadMessageBuilder {
|
||||
author_id: Some(user.id.into()),
|
||||
body: MessageBody::StatusChange {
|
||||
new_status: ProjectStatus::Rejected,
|
||||
old_status: ProjectStatus::from_string(&record.old_status),
|
||||
},
|
||||
thread_id: record.thread_id,
|
||||
hide_identity: user.role.is_mod(),
|
||||
}
|
||||
.insert(&mut txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to add tech review message")?;
|
||||
|
||||
DBProject::clear_cache(project_id, None, None, &redis)
|
||||
.await
|
||||
.wrap_internal_err("failed to clear project cache")?;
|
||||
}
|
||||
|
||||
txn.commit()
|
||||
.await
|
||||
.wrap_internal_err("failed to commit transaction")?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// See [`update_issue`].
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct UpdateIssue {
|
||||
/// What the moderator has decided the outcome of this issue is.
|
||||
pub verdict: DelphiVerdict,
|
||||
}
|
||||
|
||||
/// Updates the state of a technical review issue detail.
|
||||
///
|
||||
/// This will not automatically reject the project for malware, but just flag
|
||||
/// this issue with a verdict.
|
||||
#[utoipa::path(
|
||||
security(("bearer_auth" = [])),
|
||||
responses((status = NO_CONTENT))
|
||||
)]
|
||||
#[patch("/issue-detail/{id}")]
|
||||
async fn update_issue_detail(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
update_req: web::Json<UpdateIssue>,
|
||||
path: web::Path<(DelphiReportIssueDetailsId,)>,
|
||||
) -> Result<(), ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_WRITE,
|
||||
)
|
||||
.await?;
|
||||
let (issue_detail_id,) = path.into_inner();
|
||||
|
||||
let mut txn = pool
|
||||
.begin()
|
||||
.await
|
||||
.wrap_internal_err("failed to start transaction")?;
|
||||
|
||||
let status = match update_req.verdict {
|
||||
DelphiVerdict::Safe => DelphiStatus::Safe,
|
||||
DelphiVerdict::Unsafe => DelphiStatus::Unsafe,
|
||||
};
|
||||
let results = sqlx::query!(
|
||||
r#"
|
||||
INSERT INTO delphi_issue_detail_verdicts (
|
||||
project_id,
|
||||
detail_key,
|
||||
verdict
|
||||
)
|
||||
SELECT
|
||||
didws.project_id,
|
||||
didws.key,
|
||||
$1
|
||||
FROM delphi_issue_details_with_statuses didws
|
||||
INNER JOIN delphi_report_issues dri ON dri.id = didws.issue_id
|
||||
WHERE
|
||||
didws.id = $2
|
||||
-- see delphi.rs todo comment
|
||||
AND dri.issue_type != '__dummy'
|
||||
"#,
|
||||
status as _,
|
||||
issue_detail_id as _,
|
||||
)
|
||||
.execute(&mut *txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to update issue detail")?;
|
||||
if results.rows_affected() == 0 {
|
||||
return Err(ApiError::Request(eyre!("issue detail does not exist")));
|
||||
}
|
||||
|
||||
txn.commit()
|
||||
.await
|
||||
.wrap_internal_err("failed to commit transaction")?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// See [`add_report`].
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, utoipa::ToSchema)]
|
||||
pub struct AddReport {
|
||||
pub file_id: FileId,
|
||||
}
|
||||
|
||||
/// Adds a file to the technical review queue by adding an empty report, if one
|
||||
/// does not already exist for it.
|
||||
#[utoipa::path]
|
||||
#[put("/report")]
|
||||
async fn add_report(
|
||||
req: HttpRequest,
|
||||
pool: web::Data<PgPool>,
|
||||
redis: web::Data<RedisPool>,
|
||||
session_queue: web::Data<AuthQueue>,
|
||||
web::Json(add_report): web::Json<AddReport>,
|
||||
) -> Result<web::Json<DelphiReportId>, ApiError> {
|
||||
check_is_moderator_from_headers(
|
||||
&req,
|
||||
&**pool,
|
||||
&redis,
|
||||
&session_queue,
|
||||
Scopes::PROJECT_WRITE,
|
||||
)
|
||||
.await?;
|
||||
let file_id = add_report.file_id;
|
||||
|
||||
let mut txn = pool
|
||||
.begin()
|
||||
.await
|
||||
.wrap_internal_err("failed to begin transaction")?;
|
||||
|
||||
let record = sqlx::query!(
|
||||
r#"
|
||||
SELECT
|
||||
f.url,
|
||||
COUNT(dr.id) AS "report_count!"
|
||||
FROM files f
|
||||
LEFT JOIN delphi_reports dr ON dr.file_id = f.id
|
||||
WHERE f.id = $1
|
||||
GROUP BY f.url
|
||||
"#,
|
||||
DBFileId::from(file_id) as _,
|
||||
)
|
||||
.fetch_one(&mut *txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to fetch file")?;
|
||||
|
||||
if record.report_count > 0 {
|
||||
return Err(ApiError::Request(eyre!("file already has reports")));
|
||||
}
|
||||
|
||||
let report_id = DBDelphiReport {
|
||||
id: DelphiReportId(0),
|
||||
file_id: Some(file_id.into()),
|
||||
delphi_version: -1, // TODO
|
||||
artifact_url: record.url,
|
||||
created: Utc::now(),
|
||||
severity: DelphiSeverity::Low, // TODO
|
||||
}
|
||||
.upsert(&mut txn)
|
||||
.await
|
||||
.wrap_internal_err("failed to insert report")?;
|
||||
|
||||
txn.commit()
|
||||
.await
|
||||
.wrap_internal_err("failed to commit transaction")?;
|
||||
|
||||
Ok(web::Json(report_id))
|
||||
}
|
||||
Reference in New Issue
Block a user