Right now, it appears that the iceshrimp.net database just grows forever. Is there a script I can run to trim any post that's over a certain age that a local user hasn't favourited, reposted, replied to, etc? My database is getting large, and it's making the admin pages take forever to load:
Total instances
16317
Active instances
16301
Total notes
21034392
Local notes
2814
Total notes, past 24h
54837
Local notes, past 24h
6
Interfree - Iceshrimp.NET v2025.1-beta5.patch3.security3+66bb0eaae2Authenticated as fastfinge
Unfortunately, if you have a super large notes table, that just fills up all of your shared memory and dies. However, you put me on the right track. This does the trick for anyone who comes hear via search with this problem:
DO $$
DECLARE
cursor_date DATE := (CURRENT_DATE - interval '183 days')::date;
stop_date DATE := (CURRENT_DATE - interval '10 years')::date;
rows_deleted INT;
BEGIN
RAISE NOTICE 'Starting Time Walker deletion...';
CREATE TEMP TABLE IF NOT EXISTS temp_local_users AS
SELECT id FROM "user" WHERE "host" IS NULL;
CREATE INDEX IF NOT EXISTS idx_temp_local_users ON temp_local_users(id);
WHILE cursor_date > stop_date LOOP
DELETE FROM "note" n
WHERE n."createdAt" >= cursor_date
AND n."createdAt" < (cursor_date + interval '1 day')
AND n."userHost" IS NOT NULL
AND NOT EXISTS (
SELECT 1 FROM "following" f
WHERE n."userId" = f."followeeId" OR n."userId" = f."followerId"
OR n."renoteId" = f."followeeId"
OR n."renoteId" = f."followerId"
)
AND NOT EXISTS (
SELECT 1 FROM "note_like" nl
WHERE nl."noteId" = n.id
AND EXISTS (SELECT 1 FROM temp_local_users u WHERE u.id = nl."userId")
)
AND NOT EXISTS (
SELECT 1 FROM "note_reaction" nr
WHERE nr."noteId" = n.id
AND EXISTS (SELECT 1 FROM temp_local_users u WHERE u.id = nr."userId")
)
AND NOT EXISTS (
SELECT 1 FROM "note_watching" nw
WHERE nw."noteId" = n.id
AND EXISTS (SELECT 1 FROM temp_local_users u WHERE u.id = nw."userId")
);
GET DIAGNOSTICS rows_deleted = ROW_COUNT;
IF rows_deleted > 0 THEN
RAISE NOTICE 'Cleared %: deleted % rows', cursor_date, rows_deleted;
END IF;
COMMIT;
cursor_date := cursor_date - interval '1 day';
PERFORM pg_sleep(0.05);
END LOOP;
DROP TABLE IF EXISTS temp_local_users;
RAISE NOTICE 'Cleanup complete.';
END $$;
Nice. I would like to see something official eventually but this works well for me too.
Manual workaround below.
This gets rid of REMOTE posts older than a year, except those that
Season to taste
DELETEFROM "note" nWHERE n."userHost" IS NOT NULLAND n."createdAt"::date <= (CURRENT_DATE - interval '365' DAY)AND NOT EXISTS(SELECT idFROM "following" fWHERE n."userId" = f."followeeId"OR n."userId" = f."followerId"OR n."renoteId" = f."followeeId"OR n."renoteId" = f."followerId")AND NOT EXISTS(SELECT idFROM "note_like" nfWHERE nf."userId" IN(SELECT idFROM "user" uWHERE u."host" IS NULLAND nf."noteId" = n.id))AND NOT EXISTS(SELECT idFROM "note_reaction" nrWHERE nr."userId" IN(SELECT idFROM "user" uWHERE u."host" IS NULLAND nr."noteId" = n.id))AND NOT EXISTS(SELECT idFROM "note_watching" nwWHERE nw."userId" IN(SELECT idFROM "user" uWHERE u."host" IS NULLAND nw."noteId" = n.id));